ORIE Colloquium: Shie Mannor (Technion) - Risk and robustness in RL: Nothing ventured, nothing gained

Description

In this talk I will start from giving a broad overview of my research, focusing on the essential elements needed for scaling reinforcement learning to real-world problems. I will present a scheme called "extended intelligence" that concerns the design of systems that participate as responsible, aware, and robust elements of more complex systems. I will then deep dive into the question of how to create control policies from existing historical data and how to sample trajectories so that future control policies would have a less uncertain return. This question has been central in reinforcement learning in the last decade, if not more, and involves methods from statistics, optimization, and control theory. We will focus on one of the possible remedies to uncertainty in sequential decision problems: using risk measures such as the conditional value-at-risk as the objective to be optimized rather than the ubiquitous expected reward. We consider the complexity and efficiency of evaluating and optimizing risk measures. Our main theme is that considering risk is essential to obtain resilience to model uncertainty and model mismatch. We then turn our attention to online approaches that adapt on the fly to the level of uncertainty of a given trajectory, thus achieving robustness without being overly conservative. If time permits, I will shortly discuss a couple of real-world applications my group has been working on—one in energy management and one in healthcare.

Bio:
Shie Mannor is a professor at the Andrew and Erna Viterbi Faculty of Electrical Engineering at the Technion. He is a member of the Technion Machine Learning Center and the Grand Technion Energy Program, and the head of the Technion and Intel Corporation’s Center for Artificial Intelligence. Prof. Mannor’s research interests include machine learning and pattern recognition, planning and control, multi-agent systems, computer network security, and communications. He is currently leading a new initiative by Ford to develop a decision-making system in Israel for driving autonomous cars.