Professor David Donoho, Stanford University
What’s the big deal about 'big data'? Emergent Phenomena in high-dimensional data analysis
In classical statistical analysis, one assumed that the number of variables of interest is mall but the number of of observations is large. In the big-data era the number of variables is almost as large as, or even larger than the number of observations. In this new regime, several fascinating phenomena arise which both complicate life, but also render it more interesting.
I will illustrate the emergent new phenomena with vignettes showing how the big-data asymptotic overturns traditional statistics, such as covariance estimation, and its applications in signal processing and finance; from high-dimensional robust estimation of linear models, and its use for outlier detection. For example, traditional optimal procedures are no longer optimal.
We will show how the emergent new high-dimensional phenomena offer exciting new opportunities in science and technology, for example in compressed sensing.
Visit the Department of Math website