• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • News
  • Seminars
    • CESG Seminars
    • Fishbowl Seminar Series
    • Computer Engineering Eminent Scholar Seminar Series
    • Topics In Systems Seminar
    • Related Seminars
  • People
    • Faculty
    • Staff
    • Current Visitors
    • Students
  • Research
  • Academics
    • Graduate
    • Undergraduate
  • All Courses
  • Contact
  • Information
    • Information
    • Technology Resources
    • Directions

Computer Engineering and Systems Group

Texas A&M University College of Engineering

CESG Seminar – Sanjay Shakkottai

Posted on November 2, 2022 by Vickie Winston

Friday, November 18, 2022
10:20 – 11:10 a.m. (CST)
Zoom (Links and PW in syllabus or email)

Dr. Sanjay Shakkottai
Professor, Department of Electrical and Computer Engineering
University of Texas at Austin

Title: “The Power of Adaptivity in Representation Learning: from Meta-Learning to Federated Learning”

Talking Points

  • Algorithms for multi-task learning that learn representation
  • Understanding the training dynamics of meta-learning and federated averaging with fine tuning

Abstract
A central problem in machine learning is as follows: How should we train models using data generated from a collection of clients/environments, if we know that these models will be deployed in a new and unseen environment? In the setting of few-shot learning, two prominent approaches are: (a) develop a modeling framework that is “primed” to adapt, such as Model Adaptive Meta Learning (MAML), or (b) develop a common model using federated learning (such as FedAvg), and then fine tune the model for the deployment environment. We study both these approaches in the multi-task linear representation setting. We show that the reason behind generalizability of the models in new environments trained through either of these approaches is that the dynamics of training induces the models to evolve toward the common data representation among the clients’ tasks. In both cases, the structure of the bi-level update at each iteration (an inner and outer update with MAML, and a local and global update with FedAvg) holds the key — the diversity among client data distributions are exploited via inner/local updates, and induces the outer/global updates to bring the representation closer to the ground-truth. In both these settings, these are the first results that formally show representation learning, and derive exponentially fast convergence to the ground-truth representation. Based on joint work with Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sewoong Oh. Papers: https://arxiv.org/abs/2202.03483 , https://arxiv.org/abs/2205.13692

Biography
Dr. Sanjay Shakkottaireceived his Ph.D. from the ECE Department at the University of Illinois at Urbana-Champaign in 2002. He is with The University of Texas at Austin, where he is a Professor in the Department of Electrical and Computer Engineering, and holds the Cockrell Family Chair in Engineering #15. He received the NSF CAREER award in 2004 and was elected as an IEEE Fellow in 2014. He was a co-recipient of the IEEE Communications Society William R. Bennett Prize in 2021. He is currently the Editor in Chief of IEEE/ACM Transactions on Networking. His research interests lie at the intersection of algorithms for resource allocation, statistical learning and networks, with applications to wireless communication networks and online platforms.

Webpage to learn more about Dr. Shakkottai: HERE

More on CESG Seminars: HERE

Please join on Friday, 11/18/22 at 10:20 a.m. in ETB 1020.

Filed Under: Uncategorized

Recent NEWS

  • CESG Seminar: Peipei Zhou March 1, 2023
  • CESG Seminar – Desik Rengarajan February 21, 2023
  • CESG Seminar – Manoranjan Majji February 13, 2023
  • CESG Seminar – Jiang Hu February 2, 2023
  • CESG Seminar – Sabit Ekin January 24, 2023
  • Congratulations Dr. Hu! January 13, 2023
  • Congratulations Fall 2022 Graduates! December 12, 2022

© 2016–2023 Computer Engineering and Systems Group Log in

Texas A&M Engineering Experiment Station Logo
  • State of Texas
  • Open Records
  • Risk, Fraud & Misconduct Hotline
  • Statewide Search
  • Site Links & Policies
  • Accommodations
  • Environmental Health, Safety & Security
  • Employment