USC Viterbi computer science researchers have developed a system to teach robots how to predict human preferences in assembly tasks. The system, which was a finalist for the best paper award at the ACM/IEEE International Conference on Human-Robot Interaction (HRI), could help robots become more collaborative helpers in manufacturing and everyday life.
The ability to understand others’ goals, desires, and beliefs, known as “theory of mind,” comes naturally to humans, but is still a challenging skill for robots. “When working with people, a robot needs to constantly guess what the person will do next,” said lead author Heramb Nemlekar, a USC computer science PhD student working under the supervision of assistant professor of computer science, Stefanos Nikolaidis.
To teach the robot to predict human preferences, researchers created a small assembly task called a “canonical” task that people can easily and quickly perform. In this case, they used parts of a simple model airplane, such as the wings, tail, and propeller. The robot “watched” the human complete the task using a camera placed directly above the assembly area, looking down. To detect the parts operated by the human, the system used AprilTags, similar to QR codes, attached to the parts.
The system then used machine learning to learn a person’s preference based on their sequence of actions in the canonical task. “Based on how a person performs the small assembly, the robot predicts what that person will do in the larger assembly,” said Nemlekar. “For example, if the robot sees that a person likes to start the small assembly with the easiest part, it will predict that they will start with the easiest part in the large assembly as well.”
In the researchers’ user study, their system was able to predict the actions that humans will take with around 82% accuracy. The researchers hope that their research will lead to significant improvements in the safety and productivity of assembly workers in human-robot hybrid factories, as robots can perform non-value-added or ergonomically challenging tasks that are currently being performed by workers.
The technology could also help people with disabilities or limited mobility to more easily assemble products and maintain independence. The researchers plan to develop a method to automatically design canonical tasks for different types of assembly tasks and evaluate the benefit of learning human preferences from short tasks and predicting their actions in complex tasks in different contexts, such as personal assistance in homes.
Hd student Heramb Hemlekar (left) and assistant professor Stefanos Nikolaidis. Credit: Keith Wang
