Ahmer Nadeem Khan
Return to research

Equity Portfolio Optimization with James–Stein Shrinkage and Data Extraction Pipelines

Empirical research project analyzing covariance shrinkage estimators for mean–variance portfolio optimization on two years of daily data for S&P 500 constituents.

2% Reduction in out-of-sample portfolio variance
2 years/daily Data from online sources
502 Equity constituents analyzed

Abstract

This empirical research project evaluates covariance shrinkage techniques for mean–variance portfolio construction. Using two years of daily data extracted from Yahoo Finance for S&P 500 constituents and Treasury rates, we compare classical, Ledoit–Wolf, and James–Stein covariance estimators in constructing both global minimum-variance (GMV) and maximum Sharpe ratio (MSR) portfolios.

The study demonstrates that shrinkage estimators achieve up to 2% reduction in out-of-sample portfolio variance relative to sample covariance methods. The project emphasizes practical implementation including automated data extraction, robust numerical computation, and comprehensive out-of-sample evaluation metrics.

Note: The GitHub repository README provides detailed instructions for data collection and employing the extraction pipeline.

Main contributions

  • Empirical research project analyzing covariance shrinkage estimators for mean–variance portfolio optimization.
  • Automated large-scale data extraction from Yahoo Finance, Slickcharts, Wikipedia, and US Dept of Treasury website for two years of daily S&P 500 constituents and Treasury rates.
  • Implemented and compared global minimum-variance (GMV) and maximum Sharpe ratio (MSR) portfolios under classical, Ledoit–Wolf and James–Stein estimators.
  • Achieved up to 2% reduction in out-of-sample portfolio variance.

Conclusions

Shrinkage-based covariance estimators, particularly Ledoit–Wolf and James–Stein methods, deliver measurable improvements in out-of-sample portfolio stability. Our empirical analysis demonstrates approximately 2% reduction in portfolio variance relative to classical sample covariance. The automated data extraction pipeline enables rapid re-evaluation on updated market data, making this framework practical for dynamic portfolio construction. These results confirm that structured covariance estimation is essential for robust mean–variance portfolio optimization in practice.