This is a research seminar on how robots can learn from their own autonomous experience the structure of the sensorimotor system; control laws for effective action; foundational concepts such as Space, Objects, and Actions; and causal and taxonomic theories that help them make sense of the world around them. We will draw on current research in AI, machine learning, robotics, and developmental psychology. The course will require student presentations and a substantial term project.
The physical environment of a human or robot is unboundedly complex, changing continuously in time and space. An embodied robot, embedded in the physical world, will receive a high bandwidth stream of sensory information, and may have multiple effectors with continuous control signals. In addition to dynamic change in the world, the properties of the robot itself -- its sensors and effectors -- change over time. How can it cope with this complexity?
We can get valuable inspiration by looking at how children develop in their ability to perceive the world, to act in it, and to represent increasingly sophisticated knowledge about it. Ideally, some of our computational modeling work will shed new light on questions about human and animal development and cognition as well..
A successful robot will need to be a learning agent, learning the properties of its sensors, effectors, and environment from its own experience, and adapting over time. Inspired by human developmental learning, we believe that foundational concepts such as Space, Object, Action, etc., are essential for such a learning agent to abstract and control the complexity of its world. To bridge the gap between continuous interaction with the physical environment, and discrete symbolic descriptions that support effective planning, the agent will need multiple representations for these foundational domains, linked by abstraction relations.
In previous work, we have developed the Spatial Semantic Hierarchy (SSH), a hierarchy of representations for large-scale and small-scale space describing how a mobile learning agent (human or robot) can learn a cognitive map from exploration experience in its environment. The SSH shows how a local metrical map can be abstracted to local topological representations, which can be linked over time to construct a global topological map, which in turn can be used as the skeleton for a global metrical map. The robustness of human knowledge of space comes in part from the simultaneous availability of all of these representations.
Building on this approach, we are developing the Object Semantic Hierarchy (OSH), which shows how a learning agent can create a hierarchy of representations for objects it interacts with. The OSH shows how the ``object abstraction'' factors the uncertainty in the sensor stream into object models and object trajectories. These object models then support the creation of action models, abstracting from low-level motor signals.
In simple, short-lived robotic experiments on performing actions and recognizing objects, it is feasible to build perceptual features and motor control laws by hand. However, to cope with the complexity of the real world, robots will need richer sensory systems and more complex motor systems, capable of adapting to extensive changes. Learning will start with developmental learning to acquire and ground high-level concepts in the first place, and then will continue with life-long learning to adapt to changes in the world and in the robot's own capabilities.
Recent progress in AI, machine learning, and developmental psychology provide new methods for autonomous learning, which we will study and discuss.
This is a research seminar, intended first to bring you to the state of the art, and then to help you do a research project and write a paper of publishable quality. After some introductory lectures, most of the course will consist of reading and discussion of recent research papers that will be handed out.
The requirements of the course will be:
Each class member will do a term project. You may replicate one of the methods we are studying, and evaluate it by applying it to a robot learning problem. Or you may extend an existing method, or develop a new method to solve a problem. Ideally, your term project will extend the state of the art, and will be suitable for submission to a relevant research conference such as IJCAI, AAAI, ICRA, RSS, IROS, or ICDL.
A separate document will be handed out, describing the major research threads we will be investigating. You should select one of those threads (some of them have plenty of room for several people), and consult with me to define a project topic in that area. You are encouraged to select a topic that fits well with your background and other research interests.
All of these projects will be related in some ways, and some of them will be closely related, perhaps building on each other. Collaboration is an important part of modern research. Work with others in the seminar, explaining your own ideas, making suggestions to others, and generally helping to bring the overall problem to a solution.
Each class member will be responsible for one or more formal presentations to the entire class. Select a topic and an important paper from your research thread. The entire class will be responsible for reading the important paper you assign, but you are responsible for becoming an expert in this area, finding and reading additional papers in the area, creating an illuminating example to present, and leading a discussion on the strengths and limitations of this method.
The handout will have many references that can serve as starting points for your literature search. Use Citeseer, Google Scholar, and other online indexes to search for relevant papers. With one paper as a starting point, look at the papers that it cites, and the papers that cite it. Find the authors' web pages, and look at other publications by those authors and others at their lab. Many papers are accessible online, often through the authors' web pages or through the UM Library. In some cases, I will post copies of papers on the CTools site.
Be prepared to give a 45 minute presentation, followed by specific questions and a discussion of the value and importance of the material presented. This will often lead to a more general discussion of the research approaches this suggests.
Here is a thematic outline for your presentation (and for any presentation you make to a technical audience). You don't need to cover the points in exactly this order, but try to address these needs for your audience.
Prepare PowerPoint (or PDF or Keynote or whatever) slides for your presentation. Send me a copy of your slides two or three days before your presentation, and I will give you feedback as quickly as I can. Make paper copies of your slides (2-4 slides/page) to hand out to the class before your presentation.
To get a sense of our own work, and where this topic fits into it, read my essay, An intellectual history of the Spatial Semantic Hierarchy.
The Spatial Semantic Hierarchy [Kuipers, AIJ, 2000] is a multi-level representation for knowledge of large-scale space (the ``cognitive map''), grounded in the concept of a distinctive state. We have since developed the Hybrid Spatial Semantic Hierarchy [Beeson, Modayil and Kuipers, IJRR, 2009], grounding a similar hierarchy of representations in local perceptual maps of small-scale space.
Our first major paper on foundational robot learning was Map learning with uninterpreted sensors and effectors [Pierce and Kuipers, AIJ, 1997], which showed how a robot with very little prior knowledge (but a lot of statistical tools) could learn enough about its sensors and effectors to buil thed control laws at the foundation of the basic Spatial Semantic Hierarchy. It would be helpful to read this paper in advance.
Max Lungarella, Giorgio Metta, Rolf Pfeiffer and Giulio Sandini, Developmental robotics: a survey. Connection Science 15(4): 151-190, 2003.
RobotCub Project. A Roadmap for the Development of Cognitive Capabilities in Humanoid Robots. Report D2.1, Revision 6.3, October 2008.
M. Asada, K. Hosoda, Y. Kuniyoshi, H. Ishiguro, T. Inui, Y. Yoshikawa, M. Ogino, and C. Yoshida. Cognitive developmental robotics: a survey. IEEE Trans. Autonomous Mental Development 1(1): 12-34, 2009.
One of the best sources of insight into robot foundational learning is the study of foundational knowledge in humans, and especially how that knowledge is learned by human children. The following books have valuable insights related to this course, and are well worth reading.
The following are some useful books that you should consider including in your professional library, and that are related to this course.