Professor David Donoho
Optimal Shrinkage of Singular Values and Eigenvalues under 'Spiked' Big-Data Asymptotics
In the 1950's Charles Stein had the revolutionary insight that in estimating high-dimensional covariance matrices, the empirical eigenvalues ought to be dramatically outperformed by nonlinear shrinkage of the eigenvalues. This insight inspired dozens of papers in mathematical statistics over the next 6 decades.
In the last decade, mathematical analysts working in Random Matrix Theory made a great deal of progress on the so-called spiked covariance model introduced by Johnstone (2001).
This talk will show how this recent progress makes it now possible to elegantly derive the unique asymptotically admissible shrinkage rules in many problems of matrix de-noising and covariance matrix estimation.
The new rules are very concrete and simple, and they dramatically outperform heuristics such as scree plot truncation and bulk edge truncation which have been around for decades and are used in thousands of papers across all of science.
Joint Work with Matan Gavish and Iain Johnstone.
Visit the Department of Math website