Learning a Lot from a Little: Human-Aligned Robot Learning from Structured, Imperfect Feedback
Abstract: Human-in-the-loop robot learning faces a fundamental data challenge that general machine learning doesn't: unlike settings where we can collect massive offline datasets, robots must learn from limited, real-time human interactions. This creates a critical bottleneck: we need methods that can make the most of limited human input, or, in other words, that can learn a lot from a little. The challenge is that humans are imperfect communicators of their own intent: language instructions are often ambiguous, demonstrations can be incomplete or overfit to a specific setting, and physical corrections have multiple valid interpretations. Our key insight is that human feedback is shaped by context in predictable ways, and that modeling that context turns ambiguous, incomplete feedback into a rich signal about underlying intent. In this talk, I will discuss three sources of such structure: 1. human feedback modalities are incomplete in complementary ways, and that complementarity is itself a source of signal; 2. human feedback is only interpretable relative to how humans represent the world, and learning that representation is as important as learning the reward; and 3. behind any specific human input lies a higher-level intent that generalizes far beyond the situation in which it was expressed. Together, these directions show that understanding the structure of human communication — rather than simply collecting more of it — is the key to efficient, generalizable, human-aligned robot learning.
Bio: Andreea Bobu is an Assistant Professor at MIT in AeroAstro and CSAIL. She leads the Collaborative Learning and Autonomy Research Lab (CLEAR Lab), where they develop autonomous agents that learn to do tasks for, with, and around people. Her goal is to ensure that these agents' behavior is consistent with human expectations, whether they interact with expert designers or novice users. She obtained her Ph.D. in Electrical Engineering and Computer Science at UC Berkeley with Anca Dragan in 2023. Prior to her Ph.D. she earned her Bachelor’s degree in Computer Science and Engineering from MIT in 2017. She was the recipient of the Apple AI/ML Ph.D. fellowship, is a Rising Star in EECS and an R:SS and HRI Pioneer, and has won best paper award at HRI 2020 and the Emerging Research Award at the International Symposium on the Mathematics of Neuroscience 2023. Before MIT, she was also a Research Scientist at the AI Institute and an intern at NVIDIA in the Robotics Lab.