Researchers at the University of Virginia School of Engineering and Applied Science are advancing the integration of human language and reasoning into semi-autonomous vehicles. Led by Yen-Ling Kuo, assistant professor of computer science and Anita Jones Faculty Fellow, the project aims to enhance the interaction between drivers and autonomous technology. The research is supported by a two-year Young Faculty Researcher grant from the Toyota Research Institute.
Kuo’s work focuses on enabling semi-autonomous cars to communicate their actions to drivers, such as explaining sudden braking. Additionally, this technology could assist new drivers in learning to drive more safely. Kuo emphasizes that the collaboration between humans and machines can lead to better driving outcomes, as neither is perfect on their own.
The project seeks to develop machine learning models that provide robots with generalizable reasoning skills. This approach contrasts with the traditional method of collecting extensive datasets for every possible scenario, which is costly and impractical. Kuo is working with a team at the Toyota Research Institute to create language representations of driving behavior. These representations allow a robot to understand the meaning of words by observing human interactions with the environment or through its own experiences.
This new intelligence is particularly useful in unusual driving conditions, such as icy roads or complex situations. For example, the AI could guide a driver to slow down while turning to avoid skidding and make adjustments if the driver’s actions are insufficient.
Kuo’s work involves developing these language representations from various data sources, including a driving simulator she is building for her lab. Her research has gained recognition, including an invited talk at the Association for the Advancement of Artificial Intelligence’s New Faculty Highlights 2024 program and a forthcoming paper in AI Magazine.
Kuo’s research aligns with the goals of the Toyota Research Institute, which aims to advance human-centered AI and improve human-vehicle interaction. Guy Rosman, a co-investigator and manager of the institute’s Human Aware Interaction and Learning team, notes that language-based reasoning can enhance driver-vehicle interactions by integrating common sense into the AI’s operations.
Ultimately, Kuo’s research aims to build trust between drivers and autonomous systems, ensuring safer and more intuitive driving experiences.
Image: credit Graeme Jenvey/University of Virginia School of Engineering and Applied Science
