Unifying the Stack: A Principled Structuralist Approach to Intelligent Robot Control
Abstract: There are two dominant approaches to designing intelligent robots. One, typified by language behavior models, leverages unstructured deep neural networks and learning from demonstration to generate behavior. These approaches have had several impressive successes but face scaling, trust, and explainability challenges. The second approach seeks to integrate, rather than discard, technologies from existing subfields (like motion planning and SLAM) into a coherent control architecture that retains their favorable properties while accessing the strengths of deep network. The primary challenge here is that there is no unifying theoretical framework for all of robotics: each subfield was designed and studied largely in isolation. I will propose a unifying framework that models the control stack as layers of increasingly abstract decision processes. Each layer combines perceptual and action abstractions, to generate a more tractable decision process by exploiting structure in the world or the robot. Existing technologies fit naturally into this stack as observation or action abstractions. The result is a natural hierarchy with a few missing technologies. I will discuss my group's recent results on both filling in these missing technologies, and more generally in learning decision process abstractions from pixel-level data.
Bio: George Konidaris is an Associate Professor of Computer Science at Brown, where he directs the Intelligent Robot Lab. George is also the co-founder of two technology startups: Realtime Robotics, which commercializes his work on hardware-accelerated motion planning, and Lelapa AI, which is based in his home country of South Africa and develops African language models. George is the recent recipient of an NSF CAREER award, young faculty awards from DARPA and the AFOSR, and the IJCAI-JAIR Best Paper Prize.