[Abstract]

I will talk about how Vision-Language-Action (VLA) models can continuously adapt to changing environments through Lifelong Machine Learning (LML). To overcome the limitations of traditional disjoint task setups and catastrophic forgetting, I will talk about a few continual learning configurations we have been proposing. In addition, I will talk about an efficient online continual learning we have proposed, it selectively freezes neural network layers based on the informativeness of incoming data batches to save computation and prevent overfitting. I will extend these concepts to Embodied AI by proposing the CL-ALFRED framework, which allows interactive instruction-following agents to sequentially learn new behaviors and environments. Additionally, I will talk about some of our attempts in VLA domain, which we have been pursuing since 2020.

[Biography]

I received my BS, MS degree from Seoul National University, and my Ph.D. from the University of Maryland, College Park. Currently, I am an associate professor at the Department of Electrical and Computer Engineering at Seoul National University. During my doctoral studies, I was a research intern at Microsoft Research, Disney Research, and Adobe Research. Following graduation, I worked at the Allen Institute for AI (AI2) in the US (2016-2018), was an Assistant Professor at the Gwangju Institute of Science and Technology (GIST) (2018-2022), and was an Associate Professor at Yonsei University (2022-2024). My research primarily focuses on efficient and accurate multimodal cognitive models, taking into consideration computational complexity and labeling costs. The main application areas of my work include robotics, as well as video and language understanding.