A new system, developed by a team from The University of Texas at Dallas (UTD), is enhancing robots’ capabilities to recognize and differentiate objects. This development promises to pave the way for robots to interact more effectively with their environments, bringing us closer to a future where they can adeptly navigate and perform tasks within everyday settings like homes.
In the Intelligent Robotics and Vision Lab at UTD, experiments show robots moving toy packages, such as butter, around tables. This simple act of pushing enables the robot to gather a series of images that, when processed, help it recognize the object. Traditional methods required robots to understand objects from just a single push or grasp, but this new system facilitates multiple interactions to gather a broader understanding.
Dr. Yu Xiang, the senior author of the research paper presented at the Robotics: Science and Systems conference in South Korea, explained the importance of the development: “If you ask a robot to pick up the mug or bring you a bottle of water, the robot needs to recognize those objects.” He emphasized the need for robots to identify everyday objects and to distinguish between similar items, like various brands and designs of water bottles.
The lab’s robot, named Ramp, stands around 4 feet tall and possesses an elongated mechanical arm, capped with a two-fingered “hand” to grasp objects. Ramp’s training involves interaction with toy food packages, from spaghetti to ketchup. Describing the learning process, Xiang said, “After pushing the object, the robot learns to recognize it. By the second time it sees the object, it will just pick it up.”
The novelty in this approach lies in the repetition. Instead of one push, the robot engages with each item 15 to 20 times, taking multiple photos with its RGB-D camera. This comprehensive interaction ensures a detailed understanding, thus minimizing mistakes.
Object segmentation, the skill of recognizing and distinguishing between objects, remains vital for robotic functionality. “To the best of our knowledge, this is the first system that leverages long-term robot interaction for object segmentation,” Xiang proudly stated.
Ninad Khargonkar, a doctoral student on the team, shared his excitement about the practical application of their algorithm, noting the stark difference between testing in theoretical settings and real-world environments.
With this leap in robotic object recognition, the team’s future endeavors will focus on honing other robot capabilities, including planning and task execution, such as sorting recyclables.
The research received backing from the Defense Advanced Research Projects Agency, emphasizing the development of AI technologies that enhance human skills and minimize errors through augmented reality and task guidance.
From left: Computer science doctoral students Sai Haneesh Allu and Ninad Khargonkar with Dr. Yu Xiang, assistant professor of computer science, are shown with Ramp, a robot they are training to recognize and manipulate common objects. Credit: University of Texas at Dallas
