Redefining Financial Forecasting: A CAPE-Based Approach to S&P 500 Analysis and Portfolio Construction

Historically, the S&P 500 Index has been the object of numerous efforts by scholars and investment professionals seeking to deploy statistical and quantitative techniques in forecasting attempts. To this extent, a wide range of macroeconomic and financial variables have been studied to understand their potential influence on the Index’s performance, primarily focusing on price-based fundamental and technical financial metrics. This study diverges from the conventional approach by centring its analysis on the Cyclically-Adjusted Price-to-Earnings Ratio (CAPE), a concept made famous by Robert Shiller and John Campbell. Specifically, it implements linear regression models combined with ARIMA processes and the Newey–West estimator, to examine the extent to which behavioural and macroeconomic variables, such as investor sentiment and economic indicators, may carry explanatory power in forecasting CAPE fluctuations. Accordingly, this research argues that CAPE represents a more appropriate object of analysis rather than the raw Index price and explores the possibility of leveraging the evidence produced by statistical modelling to achieve superior portfolio returns.