Beyond Scaling: Exploration, Guidance, and Symmetry in Robot Perception
Recent generalist robot systems rely on vision-language-action models
without making any use of perception capabilities like 3D or 4D
representations encoded in vision foundation models. They increasingly
rely on scaling up the number of examples needed for behavior cloning,
not only to capture the distribution of tasks but also basic
perceptual skills. We argue that a robot should be an active observer
that selects the best views required for scene representation and the
affordances involved in the task at hand. Such an exploration can rely
on information-theoretic principles that guide the robot towards
unpredictable views. Moreover, test-time geometric reasoning can adapt
to arbitrary environments, enabling collision-free planning and
one-shot adaptation. Last, symmetry enables better generalization and
learning dynamics. We propose an equivariant canonicalization
framework with applications in trajectory planning and odometry.
Bio:
Kostas Daniilidis is the Ruth Yalom Stone Professor of Computer and Information Science at the University of Pennsylvania, where he has been a faculty member since 1998. He is an IEEE Fellow. He was the director of the GRASP laboratory from 2008 to 2013, and Associate Dean for Graduate Education from 2012 to 2016. He obtained his undergraduate degree in Electrical Engineering from the National Technical University of Athens in 1986 and his PhD in Computer Science from the University of Karlsruhe in 1992. He received the Best Conference Paper Award at the 2017 IEEE International Conference on Robotics and Automation (ICRA 2017). He is the recipient of the 2025 Provost’s Award for Distinguished PhD Mentoring and Teaching. He was Program co-Chair at ECCV 2010 and 3DPVT (now 3DV) 2006. His most cited works are on event-based vision, equivariant geometric learning, 3D human and object pose, and hand-eye calibration.