A research team from the Institut de Ciències del Mar (ICM-CSIC) in Barcelona, in partnership with the Monterey Bay Aquarium Research Institute (MBARI) in California, the Universitat Politècnica de Catalunya (UPC), and the Universitat de Girona (UdG), has successfully demonstrated that reinforcement learning can be used to enable underwater robots to accurately track marine objects and animals.
The details of this groundbreaking study were published in the leading scientific journal in the field of robotics, Science Robotics.
Underwater robotics, with vehicles able to descend to depths of up to 4,000 meters, are playing an increasingly vital role in our understanding of the oceans. These autonomous vehicles provide valuable in-situ data that complements information obtained from satellites, thereby facilitating the study of small-scale phenomena, such as CO2 capture by marine organisms that helps regulate climate change.
In a significant breakthrough, the team discovered that reinforcement learning, a method widely employed in robotics and control as well as in the development of natural language processing tools, can equip underwater robots with the ability to optimize actions to attain a specific objective. “We have been able to demonstrate that it is possible to optimize the trajectory of a vehicle to locate and track objects moving underwater”, said Ivan Masmitjà, the lead author of the study.
The development will notably aid in the study of marine migration and movement on both small and large scales. Additionally, the innovation is set to improve real-time monitoring of other oceanographic instruments via a network of robots, some of which can report the actions of underwater robotic platforms via satellite.
The success of the project hinged on the use of range acoustic techniques and artificial intelligence. Reinforcement learning played a crucial role in determining the optimal trajectory for the robot, based on the most accurate points for acoustic range measurements.
Researchers trained neural networks using the high-performance supercomputer located at the Barcelona Supercomputing Center (BSC-CNS), which significantly reduced the time required to adjust different algorithm parameters.
The trained algorithms were then tested on various autonomous vehicles, including the AUV Sparus II developed by VICOROB, in a series of experimental missions conducted in Spain and the US.
The team plans to apply the same algorithms to more complex missions in the future, such as using multiple vehicles to locate objects or detect thermoclines, and cooperative algae upwelling.
The research was supported by the European Marie Curie Individual Fellowship, awarded to Ivan Masmitjà in 2020, and the BITER project, funded by Spain’s Ministry of Science and Innovation.
Image: ICM
