ORIE Colloquium: Yash Deshpande (MIT / Microsoft Research) - Towards fixed SNR theory and practice of statistical estimation and inference

Description

Much of modern statistical theory and methods operate in a “high signal-noise-ratio (SNR)” regime wherein the sample size exceeds a measure of intrinsic dimension, such as sparsity, rank or number of parameters. Qualitatively different phenomena appear when we focus on a regime where the sample size is proportional to the dimension. These have implications for modeling and algorithm design for statistical estimation and inference. In this talk, I want to focus on two parts of my work that exemplify this phenomenon. The first part of this talk will center on contextual stochastic block models, an example of a high-dimensional estimation problem. I will demonstrate how ideas from statistical physics allow obtaining optimal estimation and testing results. In the second part of the talk, I will focus a model for batched bandit data, as exemplar of adaptively collected data. Here, the “fixed SNR” regime clarifies the difficulties with adaptively collected data, where bias may dominate the estimation error. I will discuss online debiasing, an algorithmic procedure that “debiases” estimators computed on adaptively collected data. This allows for near-optimal results for standard measures of uncertainty in inference, like confidence intervals and p-values.

Bio:
Yash Deshpande received a B.Tech in electrical engineering from the Indian Institute of Technology, Bombay in 2011, and an M.S. and Ph.D. in electrical engineering in 2016 from Stanford University. He is presently a Schramm Fellow at the Department of Mathematics at MIT and Microsoft Research, New England. His research interests are in statistical inference, graphical models, message passing algorithms and applied probability.