Friday, October 14, 2022
10:20 – 11:10 a.m. (CST)
Virtual via Zoom: https://tamu.zoom.us/j/93347193479 (password in emails or syllabus)
Dr. Aditya Mahajan,
Associate Professor in Electrical and Computer Engineering
McGill University – Montreal, Canada
Title: “Approximate Planning and Learning for Partially Observed Systems ”
Talking Points:
* Propose a theoretical framework–based on the fundamental notion of information state–for approximate planning and learning in partially observed systems.
* Define an approximate information state (AIS) and a corresponding approximate dynamic program (ADP).
* Bound the error in using the policy obtained by the solution of the AIS-based ADP.
* Develop RL algorithms based on these bounds and illustrate that they perform well in high-dimensional partially observed grid-world environments.
Abstract
Reinforcement learning (RL) provides a conceptual framework for designing agents which learn to act optimally in unknown environments. RL has been successfully used in various applications ranging from robotics, industrial automation, finance, healthcare, and natural language processing. The success of RL is based on a solid foundation of combining the theory of exact and approximate Markov decision processes (MDPs) with iterative algorithms that are guaranteed to learn an exact or approximate action-value function and/or an approximately optimal policy. However, for the most part, the research on RL theory is focused on systems with full state observations.
In various applications including robotics, finance, and healthcare, the agent only gets a partial observation of the state of the environment. In this talk, I will describe a new framework for approximate planning and learning for partially observed systems based on the notion of approximate information state. The talk will highlight the strong theoretical foundations of this framework, illustrate how many of the existing approximation results can be viewed as a special case of approximate information state, and provide empirical evidence which suggests that this approach works well in practice.
Joint work with Jayakumar Subramanian, Amit Sinha, Raihan Seraj, and Erfan Seyedsalehi
Biography
Dr. Aditya Mahajan is Associate Professor of Electrical and Computer Engineering at McGill University, Montreal, Canada. He is affiliated with the McGill Center of Intelligent Machines (CIM), Montreal Institute of Learning Algorithms (Mila), and Group for research in decision analysis (GERAD). He received the B.Tech degree in Electrical Engineering from the Indian Institute of Technology, Kanpur, India in 2003 and the MS and PhD degrees in Electrical Engineering and Computer Science from the University of Michigan, Ann Arbor, USA in 2006 and 2008.
He is the recipient of the 2015 George Axelby Outstanding Paper Award, the 2016 NSERC Discovery Accelerator Award, the 2014 CDC Best Student Paper Award (as supervisor), and the 2016 NecSys Best Student Paper Award (as supervisor). His principal research interests are learning and control of decentralized stochastic system.
More information on Dr. Mahajan HERE.
More info. on past and future CESG Seminars at CESG Seminars (tamu.edu)
* Friday, 10/14/22 at 10:20 a.m. via Zoom *