COOPERAMOS - COOPErative Resident robots for Autonomous ManipulatiOn Subsea -- Vi-SMART Visual Sensing for Multi-iAUV opeRaTions.

COOPERAMOS is a coordinated project whose consortium includes 3 Spanish Universities: University Jaume I of Castellón (UJI), University of Girona (UdG) and the University of the Balearic Islands (UIB). The coordinator is Prof. Pedro Sanz from the University Jaume I. The project aims at using of at least 3 I-AUVs, cooperating for enabling complex underwater intervention tasks, such as bulky load transport and cooperative complex structure assembly, in a priori unknown area, including obstacles. This intervention will be done with high autonomy never demonstrated before. The project objectives include the design of multimodal user interfaces, advanced control strategies, multi-vehicle localization and mapping, target detection and tracking, underwater wireless communication and cooperative mobile manipulation of 3 I-AUVs. The complexity of the proposed system requires a multidisciplinary team including mechanical, electronics and computer engineers as well as experts in artificial intelligence, mathematics and telecommunications. COOPERAMOS project proposes the use of an scalable team of resident single/dual arm I-AUVs for complex structure assembly. To implement the resident I-AUV concept a new cage-type docking station will be designed and developed which, together with new homing and docking algorithms will enable the long term deployment of the robot. Assembling 2 objects requires solving the peg-into-hole problem with an accuracy hard to achieve underwater. The use of a dual-arm system will alleviate this problem since achieving high accuracy between both end-effectors is feasible through a good calibration process. The assembly of complex structures requires the cooperation of several robots. Planning the mission with several robots to obtain the sequence of operations required for assembling complex structures is also one of the challenges included in the project. The main effors of the SRV Group framed in the VI-SMART Subproject, will be mostly focused on a twofold objective: to supply and analyse visual information, subsequently used by several tasks of the global procedure and to provide vision-based multi agent localization and mapping tools to the vehicles. More specifically: a) a new design of a stereo rig will be developed to improve the vision systems available; critical specifications to improve it include resolution, frame-rate, dynamic range and overlapping area at short distances, b) mapping the a priori unknown operation scenes and developing multi-robot visual localization for all the targets that participate in the mission; the cooperation architecture will take into account the communication constraints, due to signal attenuation underwater, to guarantee the data exchange between the vehicles for multi agent localization, c) the research will be centred on innovating and adapting AI techniques (CNNs) for 3D object recognition underwater and grasping guidance, c) the SRV is also involved in the experimental activity at local and at consortium level.

X. Rovira, A. Burguera. Automated Mask Generation for Efficient YOLO-based Instance Segmentation in Marine Environments for Fish Detection. In International WorkShop on Marine Technologies (MARTECH) , Palma (Spain), 2024.

A. Burguera. Combining Progressive Hierarchical Image Encoding and YOLO to Detect Fish in their Natural Habitat. In 19th International Conference on Computer Vision Theory and Applications (VISAPP), Rome, pp. 333-340, 2024.

F. Bonin-Font. Microplankton discrimination in FlowCAM images using Deep Learning. In 19 th International Conference on Computer Vision, Theory and Applications. Complete List of Authors: Francisco Bonin-Font (SRV), Gorka Buenvaron (IFISC), Mary K. Kane and Idan Tuval (IMEDEA), 2024.

A. Martorell, J. Guerrero, G. Oliver. Coordination of marine multi robot systems with communication constraints. In Applied Ocean Research, vol. 142, January, 2024.

B. M. Nordfeldt-Fiol, F. Bonin-Font, G. Oliver. Evolving Real-time Stereo Odometry for AUV Navigation in Challenging Marine Environments. In Journal of Intelligent & Robotic Systems, Springer, vol. 108, no. 83, pp. 3-20, August, 2023.

M. Martín, Y. González. Efficient Implementation of Deep Nets for Video Processing to Preserve Marine Ecosystem Services. University of the Balearic Islands, Ph. D. Thesis, 2023.

A. Burguera, F. Bonin-Font. Mesopelagic Crustacean Habitat Identification and Analysis Using Deep Learning. In International WorkShop on Marine Technologies (MARTECH) and Instrumentation ViewPoint 2023 Journal, 2023.

F. Bonin-Font. Towards Hash Similarity and Geometric-based Image Filtering to Lighten Underwater Photo-Mosaicing. In International WorkShop on Marine Technologies (MARTECH) - Instrumentation ViewPoint 2023 Journal - Complete list of Authors: Pol Baños Castelló, Francisco Bonin Font, 2023.

R. Fos, A. Burguera. Using Deep Neural Networks to Detect and Track Fish in Underwater Video Sequences. In MTS/IEEE Oceans conference, Limerick (Ireland), 2023.

C. Muntaner, B. M. Nordfeldt-Fiol, M. Martín, A. Martorell, F. Bonin-Font, F. Molina, R. Marín, G. Oliver, P. J. Sanz. Towards Underwater Robust Image Transmission Using Acoustic Communications. In IEEE Oceans, 2023.

F. Bonin-Font, A. Burguera. Image Compression for Underwater Multi-Robot Loop Closing. In IEEE Oceans , 2023.

A. Burguera. Hierarchical Color Encoding for Progressive Image Transmission in Underwater Environments. In IEEE Robotics and Automation Letters , IEEE, IEEE, vol. 8, no. 5, pp. 2970-2975, May, 2023.

A. Martorell, E. Guerrero, J. Guerrero, G. Oliver. Xiroi II, an Evolved ASV Platform for Marine Multirobot Operations. In Sensors, MDPI, vol. 23, January, 2023.

M. Martín, G. Oliver, Y. González. Real-Time Pipe and Valve Characterisation and Mapping for Autonomous Underwater Intervention Tasks. In Sensors, MDPI, vol. 22, no. 8141, 2022.

A. Burguera, F. Bonin-Font. Progressive Hierarchical Encoding for Image Transmission in Underwater Environments. In Oceans, Hampton Roads (USA, Virtual), 2022.

A. Burguera. A Loop Selection Front-end for Underwater Visual GraphSLAM. In Oceans, Hampton Roads (USA, Virtual), 2022.

B. M. Nordfeldt-Fiol, F. Bonin-Font, Y. González, G. Oliver. Evolving Visual Odometry for Autonomous Underwater Vehicles. In 14th IFAC Conference on Control Applications in Marine Systems, Robotics and Vehicles, Lyngby, Denmark, pp. 381-386, 2022.

A. Burguera, F. Bonin-Font, E. Guerrero, A. Martorell. Combining Deep Learning and Robust Estimation for Outlier-Resilient Underwater Visual Graph SLAM. In Journal of Marine Science and Engineering, Special Issue Advances in Autonomous Underwater Robotics Based on Machine Learning, MDPI, vol. 10, no. 4:511, pp. 30, april, 2022.