Hello Forecasting World!
New series on time series algorithms for fundamental analysis
For fundamental analysts, revenues and earnings are, well, fundamental. They are the key to building financial accounting models, developing forecasts, and ultimately valuing public and private corporate debt and equity. While a lot of thought may go into building such models and forecasts, the models themselves are not overly sophisticated compared to even simple linear regressions let alone auto-regressive, decision tree, sequential neural network, and other algorithms used in statistical or machine learning forecasting. This is not to say that judgmental forecasting—that is, forecasts generated from a fundamental analyst's model—is less accurate than sophisticated algorithms. Prof. Rob Hyndman notes that judgmental forecasting improves when the forecaster has "important domain knowledge."1 Yet, the late, great Daniel Kahneman notes that even experts can differ in forecasts by up to 70%.2 Nonetheless, it is a surprise that few, if any time series models are used (to the best of our knowledge) by fundamental analysts, despite being part of the core curriculum for legions of aspiring CFA charterholders.3
Our focus, however, is not to debate the pros and cons of judgmental vs. algorithmic forecasting. Rather, we're interested in finding out what algorithms work well if we were to use them to forecast company financial statement items. We took a stab at this using neural networks in a previous post. But this time we'd like to approach this methodically by testing the main time series models in use today. Inspiration for this series comes in part from the timeseries-notebooks found at Microprediction. As noted in the README there, "There are many time series packages...Surprisingly few contain the most basic "hello world" univariate example: predicting the (k+1)th number in a sequence." For time series enthusiasts, this a great resource. Unfortunately, we can't do it justice here.
We will focus on using (most of) the popular time series models today and applying them to forecasting income statement items. The idea is to see if one particular algorithm performs better at forecasting revenue than the others. If one does, it can be used to produce scalable forecasts for a large universe of companies under consideration. This begs the question as to what companies and how we'll judge the accuracy of these models. Our plan is to use five companies from the Top 10 holdings in each of the ten of the S&P 500 sectors, which are financials, industrials, energy, technology, healthcare, consumer discretionary, consumer staples, basic materials, utilities, and communications. We use the holdings as found in the Select Sector SPDRs® ETFs, under the tickers XLF, XLI, XLE, XLK, XLV, XLY, XLP, XLB, XLU, and XLC. Note: this is not affiliate marketing. We use these ETFs precisely because they are meant to track the major S&P 500 sectors and information on their holdings is relatively easy to procure. We also omit the real estate sector mainly because real estate is a different asset class, even if its wrapped in an equity vehicle.
We'll explain a little more in our next post, but for now we'll present a quick teaser graph below of the indexed revenues of the companies we'll use in our forthcoming posts. Stay tuned!
The code follows. Note that it is not the full code to produce the graph above. We will post a detailed guide in Code Walk-Throughs to show how we found and selected these tickers and then built up the base dataset. This was relatively lengthy process, so there will be multiple posts on this topic, which we separate from this main series to preserve a smooth narrative.
# Load packages
import os
import requests
import pandas as pd
import numpy as np
from datetime import datetime, timedelta
import yfinance as yf
import statsmodels.api as sm
import matplotlib.pyplot as plt
from matplotlib.ticker import FuncFormatter
import matplotlib as mpl
import time
import re
from dotenv import load_dotenv
import pickle
# Assign chart style
plt.style.use('seaborn-v0_8')
plt.rcParams["figure.figsize"] = (12,6)
# Symbols used
# Will explain how chose these symbols
etf_symbols = ['XLF', 'XLI', 'XLE', 'XLK', 'XLV', 'XLY', 'XLP', 'XLB', 'XLU', 'XLC']
ticker_list = ["SHW", "LIN", "ECL", "FCX", "VMC",
"XOM", "CVX", "COP", "WMB", "SLB",
"JPM", "V", "MA", "BAC", "GS",
"CAT", "RTX", "DE", "UNP", "BA",
"AAPL", "MSFT", "NVDA", "ORCL", "CRM",
"COST", "WMT", "PG", "KO", "PEP",
"NEE", "D", "DUK", "VST", "SRE",
"LLY", "UNH", "JNJ", "PFE", "MRK",
"AMZN", "SBUX", "HD", "BKNG", "MCD",
"META", "GOOG", "NFLX", "T", "DIS"
]
xlb = ["SHW", "LIN", "ECL", "FCX", "VMC"]
xle = ["XOM", "CVX", "COP", "WMB", "SLB"]
xlf = ["JPM", "V", "MA", "BAC", "GS"]
xli = ["CAT", "RTX", "DE", "UNP", "BA"]
xlk = ["AAPL", "MSFT", "NVDA", "ORCL", "CRM"]
xlp = ["COST", "WMT", "PG", "KO", "PEP"]
xlu = ["NEE", "D", "DUK", "VST", "SRE"]
xlv = ["LLY", "UNH", "JNJ", "PFE", "MRK"]
xly = ["AMZN", "SBUX", "HD", "BKNG", "MCD"]
xlc = ["META", "GOOG", "NFLX", "T", "DIS"]
sectors = [xlf, xli, xle, xlk, xlv, xly, xlp, xlb, xlu, xlc]
sector_dict = {name: obj for name, obj in globals().items() if isinstance(obj, list) and name.startswith('x')}
# Function to load dictionary
def load_dict_from_file(filename):
with open(filename, 'rb') as f:
return pickle.load(f)
df_sector_dict = load_dict_from_file("path/to/data/df_sector_dict.pkl")
# Create functions for indexing
def create_index(series):
if series.iloc[0] > 0:
return series/series.iloc[0] * 100
else:
return (series - series.iloc[0])/-series.iloc[0] * 100
# Clean dataframes
df_rev_index_dict = {}
for key in sector_dict:
temp_df = df_sector_dict[key].copy()
col_1 = temp_df.columns[0]
temp_df = temp_df[[col_1] + [x for x in temp_df.columns if 'revenue' in x]]
temp_df.columns = ['date'] + [x.replace('revenue_', '').lower() for x in temp_df.columns[1:]]
temp_idx = temp_df.copy()
temp_idx[[x for x in temp_idx if 'date' not in x]] = temp_idx[[x for x in temp_idx if 'date' not in x]].apply(create_index)
df_rev_index_dict[key] = temp_idx
# Print dataframes for quick spot checking
for key in sector_dict:
print(key)
print(df_rev_index_dict[key].head())
print('')
# Create train/test dataframes
df_rev_train_dict = {}
df_rev_test_dict = {}
for key in df_rev_index_dict:
df_out = df_rev_index_dict[key]
df_rev_train_dict[key] = df_out.loc[df_out['date'] < "2023-01-01"]
df_rev_test_dict[key] = df_out.loc[df_out['date'] >= "2023-01-01"]
# Plot dataframes
fig, axes = plt.subplots(2, 5, sharey=False, sharex=True, figsize=(14,8))
for idx, ax in enumerate(fig.axes):
etf = etf_symbols[idx].lower()
stocks = [x.lower() for x in sector_dict[etf.lower()]]
df_plot = df_rev_train_dict[etf]
ax.plot(df_plot['date'], df_plot[stocks])
ax.set_xlabel('')
ax.tick_params(axis='x', rotation=45)
ax.legend([x.upper() for x in stocks], fontsize=6, loc='upper left')
if idx == 3:
ax.set_ylabel("Index")
ax.yaxis.set_major_formatter(FuncFormatter(lambda x, p: f'{int(x):,}'))
ax.set_title(etf.upper())
plt.tight_layout()
plt.show()
Hyndman, R.J., & Athanasopoulos, G. (2021) Forecasting: principles and practice, 3rd edition, OTexts: Melbourne, Australia. OTexts.com/fpp3. Accessed on 2024-12-27
Kahneman, D., Sibony, O., & Sunstein, C.R. (2021) Noise, Little, Brown Spark: New York, USA, pg. 6
CFA stands for Chartered Financial Analyst, which should always be used as an adjective, never a noun, according to the CFA Institute, despite legions of charterholders using it as such!