Seminars

Friday, March 24, 2023
3:50 – 4:50 p.m. (CST)
Zoom (see syllabus or email list for link)

Desik Rengarajan 
PhD Candidate, Spring 2023
Dept. of Electrical and Computer Engineering
Texas A&M University

Title: “Enhancing Reinforcement Learning Using Data and Structure”

Talking Points

• Challenges in learning in sparse reward environments
• Developing RL algorithms that take advantage of sub-optimal demonstration data to learn in sparse reward environments
• Developing meta-RL algorithms that take advantage of sub-optimal demonstration data and structure to learn in sparse reward environments

Abstract
In reinforcement learning, reward functions serve as an indirect method of defining the goal of the algorithm. Designing a reward function that accurately captures the task at hand while effectively guiding the learning process can be a difficult challenge, requiring expert domain knowledge and manual fine-tuning. To overcome this, it is often easier to rely on sparse rewards that merely indicate partial or complete task completion. However, this leads to RL algorithms failing to learn an optimal policy in a timely manner due to the lack of fine-grained feedback. During this talk, I will delve into the impact of sparse rewards on reinforcement and meta reinforcement learning and present algorithms that leverage sub-optimal demonstration data to overcome these challenges.

Biography
Desik Rengarajan is a PhD candidate at Texas A&M University’s Department of Electrical and Computer Engineering, where he specializes in reinforcement learning. His research centers on the development of reinforcement learning algorithms that take advantage of side information, such as demonstration data and structure, to enhance the learning process and overcome challenges that arise when implementing RL in real-world situations.

More on Desik Rengarajan: HERE

More on CESG Seminars: HERE

Please join on Friday, 3/24/22 at 3:50 p.m. via Zoom.
Zoom option: Links and PW in syllabus or email announcement.

Friday, February 24, 2023
3:50 – 4:50 p.m. (CST)
ETB 1020 

Dr. Manoranjan Majji
Associate Professor
Dept. of Aerospace Engineering
Texas A&M University

Title: “Advances in Computer Engineering: Impact on Aerospace Applications ”

Talking Points

• Revolutions in computing continue to advance a wide variety of aerospace vehicle navigation and control problems. Three broad applications are discussed to demonstrate this tangible impact.
• Recent research advances in space manufacturing and assembly automation at LASR lab.
• Novel velocimeter LIDAR and interferometric rate sensing technologies developed by Prof. Majji and his students are discussed.
• New embedded processing pipelines developed by Prof. Majji’s students to estimate the forces sensed by optomechanical accelerometers developed by Prof. Guzman are elaborated.

Abstract

Recent advances in aerospace vehicle guidance, navigation and control furthered by emerging computer engineering technologies are elaborated in the lecture. Novel approaches for relative navigation using doppler sensing technologies are outlined with applications to terrain relative navigation and ship landing. Approaches to automate space systems and manufacture elements of swarm satellites in space are demonstrated using proximity operation emulation robots developed at the Land, Air and Space Robotics (LASR) laboratory. Embedded compute elements to process sensor data in order to realize an advanced optomechanical accelerometer are described to showcase advances in space avionics. The new accelerometer technology developed in collaboration with Prof. Felipe Guzman is discussed, which is found to enable spacecraft autonomy.

Biography
Dr. Manoranjan Majji is an Associate Professor of Aerospace Engineering and is the Director of the Land, Air and Space Robotics (LASR) Laboratory at Texas A&M University. He has a diverse background in several aspects of dynamics and control of aerospace vehicles with expertise spanning the whole spectrum of analysis, modeling, computations and experiments. In the areas of astrodynamics, estimation and system identification, he has made fundamental contributions documented in over 170 publications (including 45 journal articles) in the areas of guidance, navigation and control. Working with a team of 20 graduate students and 6 undergraduate researchers at the LASR lab, he works on a variety of research projects sponsored by NGA, NASA, JPL, AFRL, AFOSR, ONR, DARPA, JHTO, and the IC, in addition to various industrial partners, including BlackSky Geospatial, Dezyne Technologies, and VectorNav Technologies. His 10 PhD graduates are making valuable contributions in the academic, national laboratory, and industrial research establishments. In addition to being a scholar, Majji has a great deal of engineering experience developing software systems and embedded systems from OEM products. He holds a provisional patent on a simultaneous location and mapping software suite and was awarded a patent for developing a novel omni directional robot. He has disclosed various sensor inventions in the past decade. Manoranjan is the recipient of the 2021 Dean of Engineering Excellence Award at Texas A&M and the 2021 Texas A&M Institute of Data Science Career Initiation Fellowship. He is an Associate Fellow of the American Institute of Aeronautics and Astronautics (AIAA), a senior member of the Institute of Electrical and Electronics Engineers (IEEE), and is a Fellow of the American Astronautical Society (AAS).

More at https://cesg.tamu.edu/people-2/faculty/jiang-hu/

More on CESG Seminars: HERE

Please join on Friday, 2/24/22 at 3:50 p.m. in ETB 1020.
Zoom option: Links and PW in syllabus or email announcement.

Friday, February 10, 2023
3:50 – 4:50 p.m. (CST)
ETB 1020 

Dr. Jiang Hu
Professor
Dept. of Electrical and Computer Engineering
Affiliate of Computer Science Electrical and Computer Engineering
Texas A&M University

Title: “Machine Learning for EDA and EDA for Machine Learning”

Talking Points

• A stochastic approach to handling noisy labels in machine learning models for chip design automation
• An analytical approach to co-optimization of CNN hardware and dataflow mapping

Abstract
The wave of machine learning splashes to almost every corner of the world due to its unprecedented success. The first part of this talk will be focused on how to leverage machine learning for EDA (Electronic Design Automation). Specifically, a machine learning-based early routability prediction technique will be introduced. This technique provides a stochastic approach to handling non-deterministic data labels, which may exist in other machine learning applications. In the second part, an EDA technique for ML hardware acceleration will be presented. This is the first analytical approach to CNN hardware and dataflow co-optimization, and outperforms state-of-the-art methods in terms of both solution quality and computation runtime.

Biography
Dr. Jiang Hu is a professor in the Department of Electrical and Computer Engineering at Texas A&M University. His research interests include design automation of VLSI circuits and systems, computer architecture optimization and hardware security. He has published over 240 technical papers. He received best paper awards at DAC 2001, ICCAD 2011, MICRO 2021 and ASPDAC 2023. He was the technical program chair and the general chair of the ACM International Symposium on Physical Design (ISPD) in 2011 and 2012, respectively. He was named an IEEE fellow in 2016. He will serve as the program co-chair for ACM/IEEE Workshop on Machine Learning for CAD 2023.

More at https://cesg.tamu.edu/people-2/faculty/jiang-hu/

More on CESG Seminars: HERE

Please join on Friday, 2/10/22 at 4:10 p.m. in ETB 1020.
Zoom option: Links and PW in syllabus or email announcement.

Friday, February 3, 2023
3:50 – 4:50 p.m. (CST)
ETB 1020  (Zoom option; Links and PW in syllabus or email)

Dr. Sabit Ekin
Associate Professor
Affiliate of Electrical and Computer Engineering
Department of Engineering Technology & Industrial Distribution
Texas A&M University

Title: “An Overview of Wireless Communication, Sensing and IoT Research Projects at Texas Wireless Lab (TWL)”

Talking Points

• mmWave/Terahertz wireless communication systems for 5G, 6G and Beyond technologies
• Hybrid RF/Optical communication system design
• UAV-assisted wireless communications
• Satellite and space communications

Abstract
Wireless communication and sensing constitute two of the most critical technological advances that broadly impact myriad aspects of the evolving digital society and support the burgeoning era of smart & connected communities and the Internet of Things (IoT). In this talk, I will provide an overview of our state-of-the-art research projects that tackles new fundamental scientific questions and addresses the challenges in three main synergistic research thrusts: (i) Wireless Communication, (ii) Wireless Sensing, and (iii) Wireless IoT. Example wireless communication technologies and applications include mmWave/Terahertz wireless communication systems for 5G, 6G and Beyond technologies to support the ever-increasing demand for higher data rates, UAV-assisted wireless communications, satellite and space communications. Wireless sensing projects include gesture recognition for human-computer interaction (HCI) applications and vital signs monitoring such as respiration, heart rate, and glucose level for healthcare applications. Finally, the projects on wireless IoT applications include remote control and monitoring applications such as livestock monitoring, soil monitoring, and localization.

Biography
Dr. Sabit Ekin is a wireless system design researcher and engineer. He received his Ph.D. in Electrical and Computer Engineering from Texas A&M University (TAMU) in 2012. In January 2023, he joined TAMU as an Associate Professor of Engineering Technology & Industrial Distribution, and Electrical & Computer Engineering (affiliated faculty).  He has 11+ years (post Ph.D.) of successful track records, including 4 years of industry experience as a Wireless System Engineer at Qualcomm Inc—a world leader in wireless technologies—where he has received numerous awards for his achievements on cellular modem designs for Apple, Samsung, Google, Nokia, etc. Prior to joining TAMU, he was an Assosciate Professor of ECE at Oklahoma State University, where he worked for 6 years. He was the Director/Co-founder of Oklahoma CubeSat Initiative (OKSat)—the first CubeSat program in the state of Oklahoma. He received the Department of Energy (DOE) 2022 Early Career Award—one of the 83 scientists selected from across the nation. He is awarded with OSU PSO/Albrecht Naeter Endowed Professor of ECE (2022), and Jack H. Graham Endowed Fellow of Engineering (2021). His research focuses on design and analysis of mmWave/Terahertz wireless communication systems for 5G-6G and Beyond technologies and wireless sensing systems.  His research is sponsored by major agencies, including NSF(5), NASA(2), DOE-CAREER(1), DOD(4), DOT(2), Qatar Foundation(1), and U.S. corporations(2).

More at www.sabitekin.com

More on CESG Seminars: HERE

Please join on Friday, 2/3/22 at 4:10 p.m. in ETB 1020.
Zoom option: Links and PW in syllabus or email announcement.

Friday, November 18, 2022
10:20 – 11:10 a.m. (CST)
Zoom (Links and PW in syllabus or email)

Dr. Sanjay Shakkottai
Professor, Department of Electrical and Computer Engineering
University of Texas at Austin

Title: “The Power of Adaptivity in Representation Learning: from Meta-Learning to Federated Learning”

Talking Points

• Algorithms for multi-task learning that learn representation
• Understanding the training dynamics of meta-learning and federated averaging with fine tuning

Abstract
A central problem in machine learning is as follows: How should we train models using data generated from a collection of clients/environments, if we know that these models will be deployed in a new and unseen environment? In the setting of few-shot learning, two prominent approaches are: (a) develop a modeling framework that is “primed” to adapt, such as Model Adaptive Meta Learning (MAML), or (b) develop a common model using federated learning (such as FedAvg), and then fine tune the model for the deployment environment. We study both these approaches in the multi-task linear representation setting. We show that the reason behind generalizability of the models in new environments trained through either of these approaches is that the dynamics of training induces the models to evolve toward the common data representation among the clients’ tasks. In both cases, the structure of the bi-level update at each iteration (an inner and outer update with MAML, and a local and global update with FedAvg) holds the key — the diversity among client data distributions are exploited via inner/local updates, and induces the outer/global updates to bring the representation closer to the ground-truth. In both these settings, these are the first results that formally show representation learning, and derive exponentially fast convergence to the ground-truth representation. Based on joint work with Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sewoong Oh. Papers: https://arxiv.org/abs/2202.03483 , https://arxiv.org/abs/2205.13692

Biography
Dr. Sanjay Shakkottaireceived his Ph.D. from the ECE Department at the University of Illinois at Urbana-Champaign in 2002. He is with The University of Texas at Austin, where he is a Professor in the Department of Electrical and Computer Engineering, and holds the Cockrell Family Chair in Engineering #15. He received the NSF CAREER award in 2004 and was elected as an IEEE Fellow in 2014. He was a co-recipient of the IEEE Communications Society William R. Bennett Prize in 2021. He is currently the Editor in Chief of IEEE/ACM Transactions on Networking. His research interests lie at the intersection of algorithms for resource allocation, statistical learning and networks, with applications to wireless communication networks and online platforms.

Webpage to learn more about Dr. Shakkottai: HERE

More on CESG Seminars: HERE

Please join on Friday, 11/18/22 at 10:20 a.m. in ETB 1020.

Friday, November 11, 2022
10:20 – 11:10 a.m. (CST)
Virtual via Zoom: https://tamu.zoom.us/j/93347193479 (password in emails or syllabus)

Dr.  Mohammad Ghavamzadeh
Senior Staff Research Scientist
Google

Title: “Mitigating the Risk Associated with Epistemic and Aleatory Uncertainties in MDPs”

Abstract
Prior work on safe reinforcement learning (RL) has studied risk-aversion to randomness in dynamics (aleatory) and to model uncertainty (epistemic) in isolation. We propose and analyze a new framework to jointly model the risk associated with epistemic and aleatory uncertainties in finite-horizon and discounted infinite-horizon MDPs. We call this framework that combines risk-averse and soft-robust methods RASR. We show that when the risk-aversion is defined using either the entropic value-at-risk (EVaR) or the entropic risk measure (ERM), the optimal policy in RASR can be computed efficiently using a new dynamic program formulation with a time-dependent risk level. As a result, the optimal risk-averse policies are deterministic but time-dependent, even in the infinite-horizon discounted setting. We also show that particular RASR objectives reduce to risk-averse RL with mean posterior transition probabilities. Our empirical results show that our new algorithms consistently mitigate uncertainty as measured by EVaR and other standard risk measures.

Biography 
Dr. Mohammad Ghavamzadeh received a Ph.D. degree from UMass Amherst in 2005. He was a postdoctoral fellow at UAlberta from 2005 to 2008. He was a permanent researcher at INRIA from 2008 to 2013. He was the recipient of the “INRIA award for scientific excellence” in 2011, and obtained his Habilitation in 2014. Since 2013, he has been a senior researcher at Adobe and FAIR, and now a senior staff research scientist at Google. He has published over 100 refereed papers in major machine learning, AI, and control journals and conferences. He has co-chaired more than 10 workshops and tutorials at NeurIPS, ICML, and AAAI. His research has been mainly focused on the areas of reinforcement learning, bandit algorithms, and recommendation systems.

More information on Dr. Ghavamzadeh can be found at
https://mohammadghavamzadeh.github.io/
https://scholar.google.ca/citations?user=Bo-wyrkAAAAJ&hl=en

More info. on past and future CESG Seminars at CESG Seminars (tamu.edu)

* Friday, 11/11/22 at 10:20 a.m. via Zoom *

Friday, November 4, 2022
10:20 – 11:10 a.m. (CST)
ETB 1020 – **In-person** (Zoom option; Links and PW in syllabus or email)

Dr. Alan Kuhnle
Assistant Professor, Computer Science and Engineering
Texas A&M University

Title: “Scalable and Learned Algorithms for Discrete Optimization”

Talking Points

• Linear-time algorithms for subset selection problems
• RL for learning local search algorithm

Abstract
In this talk, I present the work of the Optimization and Learning Systems Lab on the design of scalable algorithms for optimization problems on big data. In particular, I describe our work on linear-time, parallelizable algorithms for combinatorial optimization problems arising from online social networks. Finally, I give an overview of future directions of the lab, which include augmenting algorithms with learned components to improve practical performance; optimization with incomplete information; and submodular planning.

Biography
Dr. Alan Kuhnle is an Assistant Professor of Computer Science & Engineering at Texas A&M University, where he directs the Optimization and Learning Systems Lab. His work focuses on the design and analysis of scalable algorithms for ubiquitous combinatorial optimization problems arising in data science applications, such as vehicle routing and marketing on social networks. He is the recipient of the First Year Assistant Professor Award at Florida State University in 2020 and his work has led to 34 publications in leading academic journals and conferences. He has served on the program committee of machine learning conferences and is Associate Editor of Journal of Combinatorial Optimization.

Recent TAMU article on Dr. Kuhnle: HERE

More on CESG Seminars: HERE

Please join on Friday, 11/4/22 at 10:20 a.m. in ETB 1020.

Friday, October 28, 2022
10:20 – 11:10 a.m. (CST)
Virtual via Zoom: https://tamu.zoom.us/j/93347193479 (password in emails or syllabus)

Dr.  Dileep Kalathil
Assistant Professor in the Dept. of Electrical and Computer Engineering
Texas A&M University

Title: “Reinforcement Learning with Robustness and Safety Guarantees”

Talking Points
*How do we develop reinforcement learning algorithms that can overcome the simulation-to-reality gap?
*How do we develop reinforcement learning algorithms that maintain safety constraints during learning?

Abstract
Reinforcement Learning (RL) is the class of machine learning that addresses the problem of learning to control unknown dynamical systems. RL has achieved remarkable success recently in applications like playing games and robotics. However, most of these successes are limited to very structured or simulated environments. When applied to real-world systems, RL algorithms face two fundamental sources of fragility. First, the real-world system parameters can be very different from that of the nominal values used for training RL algorithms. Second, the control policy for any real-world system is required to maintain some necessary safety criteria to avoid undesirable outcomes. Most deep RL algorithms overlook these fundamental challenges which often results in learned policies that perform poorly in the real-world settings. In this talk, I will present two approaches to overcome these challenges. First, I will present an RL algorithm that is robust against the parameter mismatches between the simulation system and the real-world system. Second, I will discuss a safe RL algorithm to learn policies such that the frequency of visiting undesirable states and expensive actions satisfies the safety constraints. I will also briefly discuss some practical challenges due to the sparse reward feedback and the need for rapid real-time adaptation in real-world systems, and the approaches to overcome these challenges.
Robust RL papers: R-P1, R-P2, R-P3
Safe RL papers: S-P1, S-P2, S-P3, S-P4

Biography 
Dr. Dileep Kalathil  is an Assistant Professor in the Department of Electrical and Computer Engineering here at Texas A&M University (TAMU). His main research area is reinforcement learning theory and algorithms, and their applications in communication networks and power systems. Before joining TAMU, he was a postdoctoral researcher in the EECS department at UC Berkeley. He received his Ph.D. from University of Southern California (USC) in 2014, where he won the best Ph.D. Dissertation Prize in the Department of Electrical Engineering. He received his M. Tech. from IIT Madras, where he won the award for the best academic performance in the Electrical Engineering Department. He received the NSF CRII Award in 2019 and the NSF CAREER award in 2021. He is a senior member of IEEE.

More information on Dr. Kalathil can be found HERE.

More info. on past and future CESG Seminars at CESG Seminars (tamu.edu)

* Friday, 10/28/22 at 10:20 a.m. via Zoom *

Friday, ????  2023
10:20 – 11:10 a.m. (CST)
Virtual via Zoom: https://tamu.zoom.us/j/93347193479 (password in emails or syllabus)

Dr.  Jiantao Jiao
Assistant Professor in the Dept. of EECS and the Dept. of Statistics 
University of California, Berkeley

Title: “Optimal Offline RL with General Function Approximation via Augmented Lagrangian”

Talking Points
*Statistically optimal offline RL algorithm with general function approximators such as neural networks
*Bypassing the need for Uncertainty quantification in handling partial data coverage through marginalized importance sampling (MIS)
*Augmented Lagrangian Method (ALM) combined with MIS gives optimal conservative offline RL without any hand-designed behavior regularization  or oracle uncertainty quantification

Abstract
Offline reinforcement learning (RL), which refers to decision-making from a previously-collected dataset of interactions, has received significant attention over the past years. Much effort has focused on improving offline RL practicality by addressing the prevalent issue of partial data coverage through various forms of conservative policy learning. While the majority of algorithms do not have finite-sample guarantees, several provable conservative offline RL algorithms are designed and analyzed within the single-policy concentrability framework that handles partial coverage. Yet, in the nonlinear function approximation setting where confidence intervals are difficult to obtain, existing provable algorithms suffer from computational intractability, prohibitively strong assumptions, and suboptimal statistical rates. In this paper, we leverage the marginalized importance sampling (MIS) formulation of RL and present the first set of offline RL algorithms that are statistically optimal and practical under general function approximation and single-policy concentrability, bypassing the need for uncertainty quantification. We identify that the key to successfully solving the sample-based approximation of the MIS problem is ensuring that certain state occupancy validity constraints are nearly satisfied. We enforce these constraints by a novel application of the augmented Lagrangian method and prove the following result: with MIS formulation, augmented Lagrangian is enough for statistically optimal offline RL. In stark contrast to prior algorithms that induce additional conservatism through methods such as behavior regularization, our approach provably eliminates this need and reinterprets regularizers as “enforcers of state occupancy validity” than “promoters of conservatism”.

Biography 
Dr. Jiantao Jiao is an Assistant Professor in the Department of Electrical Engineering and Computer Sciences and Department of Statistics at the University of California, Berkeley. He co-directs the Center for the Theoretical Foundations of Learning, Inference, Information, Intelligence, Mathematics, and Microeconomics at Berkeley (CLIMB), is a member of the Berkeley Artificial Intelligence Research (BAIR) Lab, and the Berkeley Laboratory for Information and System Sciences (BLISS). He received his Ph.D. from Stanford University in 2018. He is a recipient of the Presidential Award of Tsinghua University and the Stanford Graduate Fellowship. He was a semi-plenary speaker at ISIT 2015 and a co-recipient of the MobiHoc 2019 best paper award.

More information on Dr. Jiao can be found HERE.

More info. on past and future CESG Seminars at CESG Seminars (tamu.edu)

* Friday, 10/28/22 at 10:20 a.m. via Zoom *

Friday, October 7, 2022
10:20 – 11:10 a.m. (CST)
ETB 1020 – **In-person** (Zoom option; Links and PW in syllabus or email)

Dr. George T. Ligler
Professor, Multidisciplinary Engineering
Texas A&M University

Title: “Automatic Dependent Surveillance—Broadcast (ADS-B): A Major Aviation System’s Conception, Development, and Operational Implementation”

Talking Points

• In order to be successful, computer systems engineers need to learn to work with the full set of project stakeholders.
• The “best” technical solution to a problem is often not the one employed or that should be employed–broaden your view of he “trade space”!
• Computer system engineers have the opportunity to constantly learn while being instrumental in “getting things done”.
• We have a worldwide shortage of interdisciplinary computer systems engineers.

Abstract
ADS-B provides the largest improvement in air traffic surveillance since the advent of radar in the 1940’s. This presentation discusses the 27-year journey of the ADS-B System from concept development to operational implementation on over 100,000 aircraft in the U.S. National Airspace System alone. The role of interdisciplinary computer system engineering in the development and implementation of this high-impact system is highlighted.

Biography
Dr. George T. Ligler is a Professor and Dean’s Excellence Chair of Multidisciplinary Engineering at Texas A&M University. He is also the proprietor of GTL Associates, a consultancy which has provided systems integration/engineering and product management services in multiple fields to over 40 clients on three continents. An elected member of the U.S. National Academy of Engineering, Dr. Ligler is a member and past Chair of the Academy’s Section 12, Special Fields and Interdisciplinary Engineering.  With regard to aviation, he has served on one or more aviation standards development committees since 1992 and since 2005 has been the Co-Chair of RTCA Special Committee 159, Navigation Equipment Using the Global Navigation Satellite System (GNSS). He has been recognized with RTCA’s highest award, the Achievement Award, in 2006 and 2017 and was a co-recipient of the Air Traffic Control Association’s 2015 Chairman’s Citation Award of Merit to the NextGen Institute’s Equip 2020 Initiative for ADS-B.  Dr. Ligler has also participated in nine National Academies of Sciences, Engineering, and Medicine committees advising the Departments of Transportation, Treasury, and Commerce, and is a current member of the Academies’ Aeronautics and Space Engineering Board. He holds a doctorate in mathematics from the University of Oxford, with his studies supported by a Rhodes Scholarship.

Recent TAMU article on Dr. Ligler: HERE

More on CESG Seminars: HERE

Please join on Friday, 10/7/22 at 10:20 a.m. in ETB 1020.