Friday, November 8, 2024 
10:20 – 11:10 a.m. (CST)
ETB 1020
Dr. I-Hong Hou
Professor of CE in ECE
Texas A&M University
Title: “Distributed No-Regret Learning for Multi-Stage Systems with End-to-End Bandit Feedback”
Abstract
This talk focuses on multi-stage systems with end-to-end bandit feedback. In such systems, each job needs to go through multiple stages, each managed by a different agent, before generating an outcome. Each agent can only control its own action and learn the outcome of the job. It has neither knowledge nor control on actions taken by agents in the next stage. The goal of this paper is to develop distributed online learning algorithms that achieve sublinear regret in adversarial environments.
The setting of this paper significantly expands the traditional multi-armed bandit problem, which considers only one agent and one stage. In addition to the exploration-exploitation dilemma in the traditional multi-armed bandit problem, we show that the consideration of multiple stages introduces a third component, education, where an agent needs to choose its actions to facilitate the learning of agents in the next stage. To solve this newly introduced exploration-exploitation-education trilemma, we propose a simple distributed online learning algorithm, ϵ−EXP3. We theoretically prove that the ϵ−EXP3 algorithm is a no-regret policy that achieves sublinear regret. Simulation results show that the ϵ−EXP3 algorithm significantly outperforms existing no-regret online learning algorithms for the traditional multi-armed bandit problem.
Biography
Dr. Hou is a Professor in the ECE Department of Texas A&M University. His research interests include cloud/edge computing, wireless networks, and machine learning. Dr. Hou received the B.S. in Electrical Engineering from National Taiwan University in 2004 and his M.S. and Ph.D. in Computer Science from the University of Illinois, Urbana-Champaign in 2008 and 2011, respectively. He received the Best Paper Awards in ACM MobiHoc 2020 and ACM MobiHoc 2017, the Best Student Paper Award in WiOpt 2017, and the C.W. Gear Outstanding Graduate Student Award from the University of Illinois at Urbana-Champaign.
To learn more about Dr. Hou, visit our webpage at https://cesg.tamu.edu/faculty/i-hong-hou/.