ORIE Colloquium: Henry Lam (Columbia)

Description

Towards Universal Optimality on Perturbation Choice for Zeroth-Order Stochastic Gradient and Hessian Estimation

We consider the problem of estimating the gradient with respect to many parameters using only noisy black-box function evaluations, which arises prominently in applications such as stochastic optimization and sensitivity analysis. Commonly used approaches evaluate the noisy function at randomly perturbed values and "project" to obtain the gradient estimate via multiplying by some "score function". In this talk, we present a general framework to characterize the optimal choice of perturbation and score function under maximal freedom. Our framework is universal, i.e., independent of unknown characterizing constants on the target function, by turning the perturbation-score-function selection problem into a constant-free moment problem that controls bias-variance tradeoff. Our results reveal the (sub)optimality of a range of existing perturbation proposals. Lastly, our framework also applies to Hessian estimation and provides some performance comparisons for stochastic approximation.

Bio:
Henry Lam is an Associate Professor in the Department of Industrial Engineering and Operations Research at Columbia University. His research interests include Monte Carlo simulation, uncertainty and risk quantification, and data-driven optimization. His research has been recognized by several venues such as the NSF Career Award, JPMorgan Chase Faculty Research Award and Adobe Faculty Research Award. He serves on the editorial boards of Operations Research, INFORMS Journal on Computing, Applied Probability Journals, Stochastic Models, Manufacturing and Service Operations Management, and Operations Research Letters. He received his Ph.D. degree in statistics from Harvard University in 2011 and was on the faculty of Boston University and the University of Michigan before joining Columbia in 2017.