Skip to main content

ECE Graduate Seminar Lecture by Kaiyu Zheng, Ph.D. Student, CS Department, Brown University, via Zoom

Wednesday, November 30, 2022
1:30 pm to 2:30 pm
Floor/Room #: 
via Zoom (


Generalized Object Search



Future collaborative robots must be capable of finding objects. As such a fundamental skill, we expect object search to eventually become an off-the-shelf capability for any robot, similar to e.g. object detection, SLAM, and motion planning.  However, existing approaches either make unrealistic compromises (e.g. reduce the problem from 3D to 2D), resort to ad-hoc, greedy search strategies, or attempt to learn end-to-end policies in simulation that are yet to generalize across real robots and environments. In this talk, we argue that through using Partially Observable Markov Decision Processes (POMDPs) to model object search while exploiting structures in the real world (e.g., octrees, correlations) and in human-robot interaction (e.g., spatial language), a practical and effective system for generalized object search can be achieved. In support of this argument, we develop methods and systems for (multi-)object search in 3D environments under uncertainty due to limited field-of-view, occlusion, noisy, unreliable detectors, spatial correlations between objects, and possibly ambiguous spatial language (e.g. "The red car is behind Chase Bank"). In this talk, I will present my research projects behind these efforts.



Kaiyu Zheng

Ph.D. Student, CS Department, Brown University



Kaiyu is a Ph.D. student in Computer Science at Brown University and a member of the Humans to Robots Lab, advised by Prof. Stefanie Tellex.  He received a B.S. and M.S. from the University of Washington, Seattle. He won the IROS RoboCup Best Paper Award in 2021 for his 3D multi-object search work and was selected as an RSS Pioneer in 2022.


     Host: Professor Xinming Huang