"Evaluation of Robots for Human-Robot Interaction"
Henrik I. Christensen
KUKA Chair of Robotics
Robotics & Intelligent Machines (RIM@GT)
College of Computing

12:00 Noon on Thursday, November 2, 2006
TSRB 132

Robotics is gradually maturing as a discipline which also implies an increased need for comparative R&D. At the same time, robots are more and more deployed to serve as assistants to humans be it for search and rescue or as part of normal daily chores in the home. To enable evaluation of progress in research, it is essential that rigorous methodologies for evaluation and performance characterization are adopted. Often a number of objections are put forward as to why such rigorous experimental protocols are not well suited for robotics. Some of the typical objections will be presented and discussed in the presentation. To illustrate the value and strategy of experimental evaluation, two example applications will be presented. Both applications are closely tied to robots that serve as assistants to people as part of daily operations.

A Wizard-of-Oz study has been used for the design of a strategy for joint human-robot mapping of domestic setting as part of creation of a cognitive robots for assistance to people in their homes. The Wizard-of-Oz study clearly illustrates some of the challenges that a robot has to cope with and indicate a number of important issues to be considered in the design of dialog behaviors and associated autonomous functionality. We will discuss here the use of initial exploratory system designs to ensure early integration of the end-user.

Another study has considered the deployment of iRobot PackBot systems for assistance to soldiers as part of urban intervention. Integration of robots into a unit of soldiers poses a number of interesting challenges. There is a need to consider the impact of the system on the robot operator and to re-consider the entire strategy for urban intervention when a unit is equipped with a light-weight robot for scouting. Results from a study with the international brigade from the Swedish military are presented and a number of important lessons from a long-term (12 month) study is reported.

Experience from prior studies clearly illustrates the value of a careful design for evaluation and characterization of systems, which goes beyond the simple verification of theoretical models. Observations and lessons from an extensive set of studies are summarized.