Learning Semantic Maps from Natural Language Descriptions

Matthew Walter
Friday, April 25th, 2014

Matthew Walter
Research Scientist
Computer Science and Artificial Intelligence Lab

Friday, April 25th, 2014

Abstract: Whether they are providing personalized care, assisting people with cognitive or physical impairments, or carrying out household chores, robots have the potential to improve our quality of life in revolutionary ways. In order to realize this potential, we must develop robots that people can efficiently command and naturally interact with. This interaction demands that robots be able to reason over models of their environments as rich as those of their human partners. However, today's robots understand their environment through representations that are either limited to low-level metric properties or that require domain experts to hard-code higher-level semantic knowledge.

In this talk, I will describe an algorithm that I developed to enable robots to efficiently learn shared cognitive models of their surroundings from a user's natural language descriptions. The novelty lies in inferring spatial and semantic knowledge from these descriptions and fusing this information with the metric measurements from the robot's sensor streams. The method maintains a joint distribution over a hybrid metric, topological, and semantic representation of the environment, which provides a common framework in which to integrate these disparate sources of information. I will demonstrate that the algorithm allows people to share meaningful, human-centric properties of their environment simply by speaking to the robot. I will conclude by describing ongoing efforts in human-robot dialog and planning that build upon this semantic mapping algorithm to enable a voice-commandable wheelchair and other robots to follow free-form spoken instructions.

Bio:  Matthew Walter is a research scientist in the Computer Science and Artificial Intelligent Laboratory at the Massachusetts Institute of Technology. His research focuses on probabilistic approaches to perception and natural language understanding that make it possible for robots to work effectively alongside humans. Matthew received his Ph.D. in Mechanical Engineering from the Massachusetts Institute of Technology and the Woods Hole Oceanographic Institution.

April 25, 2014

  • Email a Friend
  • Bookmark this Page
  • Share this Page