Data preprocessing modules

Kenneth French 49 Industry Portfolios

Preprocessing utilities for Kenneth French 49 Industry Portfolios.

This module provides helper functions to convert monthly returns into synthetic price indices and to download the datasets used in the experiments.

tda_finance.data_preprocessing.preprocess_kf49.load_kf49_prices_from_returns(rets: DataFrame, base: float = 100.0, require_complete_panel: bool = True) → DataFrame[source]

Build synthetic prices from Kenneth French 49 industry returns.

Parameters:

  • rets (pandas.DataFrame) – Monthly industry returns in decimal form.

  • base (float, default=100.0) – Initial value used for each synthetic price index.

  • require_complete_panel (bool, default=True) – If True, industries with any missing value are removed before building prices.

Returns:

Synthetic monthly price matrix sorted by date.

Return type:

pandas.DataFrame

tda_finance.data_preprocessing.preprocess_kf49.main() → None[source]

Download KF49 industry returns and Fama-French 3 factors.

tda_finance.data_preprocessing.preprocess_kf49.returns_to_price_index(rets: DataFrame, base: float = 100.0) → DataFrame[source]

Convert return series into synthetic price indices.

Each column is treated independently. If an asset has missing values at the beginning of the sample, its synthetic price index starts at the first valid return observation.

Parameters:

  • rets (pandas.DataFrame) – Return matrix indexed by date, with one column per asset. Returns must be expressed in decimal form.

  • base (float, default=100.0) – Initial value used for each synthetic price index.

Returns:

Synthetic price matrix with the same index and columns as the input.

Return type:

pandas.DataFrame

S&P 500 CRSP data

Preprocessing utilities for S&P 500 CRSP monthly data.

This module loads the cleaned CRSP/WRDS monthly file used in the experiments and converts monthly stock returns into synthetic price indices.

tda_finance.data_preprocessing.preprocess_sp500_crsp.load_sp500_prices_from_monthly_returns(path: str = 'data/sp500_crsp_monthly_clean.parquet', file_format: str = 'parquet', start_price: float = 100.0, min_price: float = 1e-06) → DataFrame[source]

Load monthly S&P 500 returns and convert them into synthetic prices.

The expected input file contains at least the following columns: PERMNO, Month and MonthlyRet. PERMNO is used as the asset identifier because it is more stable than the ticker.

Parameters:

  • path (str, default="data/sp500_crsp_monthly_clean.parquet") – Path to the cleaned CRSP monthly file.

  • file_format ({"parquet", "csv"}, default="parquet") – File format used to read the input data.

  • start_price (float, default=100.0) – Initial value used for each synthetic price index.

  • min_price (float, default=1e-6) – Lower bound applied to gross returns to avoid zero synthetic prices.

Returns:

Synthetic price matrix indexed by month-end date, with one column per PERMNO.

Return type:

pandas.DataFrame

Raises:

ValueError – If the file format is unsupported, required columns are missing, or duplicated PERMNO-month observations are found.

tda_finance.data_preprocessing.preprocess_sp500_crsp.load_sp500_returns_matrix(path: str = 'data/sp500_crsp_monthly_clean.parquet', file_format: str = 'parquet') → DataFrame[source]

Load monthly S&P 500 returns as a Month x PERMNO matrix.

Parameters:

  • path (str, default="data/sp500_crsp_monthly_clean.parquet") – Path to the cleaned CRSP monthly file.

  • file_format ({"parquet", "csv"}, default="parquet") – File format used to read the input data.

Returns:

Monthly return matrix indexed by month-end date, with one column per PERMNO.

Return type:

pandas.DataFrame

Raises:

ValueError – If the file format is unsupported.