Automatic detection, identification and counting of deep-water snappers on underwater baited video using deep learning

Florian Baletaud ^1,2,3* Sébastien Villon ¹ Antoine Gilbert ² Jean-Marie Côme ³ Sylvie Fiat ¹ Corina Iovan ¹ Laurent Vigliola ¹

• ¹ ENTROPIE, Institut de Recherche pour le Développement (IRD), UR, UNC, IFREMER, CNRS, Centre IRD de Nouméa, 98000 Noumea, New Caledonia
• ² Ginger Soproner, Noumea, New Caledonia
• ³ Burgeap, groupe GINGER, Lyon, France

Deep-sea demersal fisheries in the Pacific have strong commercial, cultural, and recreational value, especially snappers (Lutjanidae) which make the bulk of catches. Yet, managing these fisheries is challenging due to the scarcity of data. Stereo-Baited Remote Underwater Video Stations (BRUVS) can provide valuable quantitative information on fish stocks, but manually processing large amounts of videos is time-consuming and sometimes unrealistic. To address this issue, we used a Regionbased Convolutional Neural Network (Faster R-CNN), a deep learning architecture to automatically detect, identify and count deep-water snappers in BRUVS. Videos were collected in New Caledonia (South Pacific) at depths ranging from 47 to 552 m. Using a dataset of 12,100 an-notations from 11 deep-water snapper species observed in 6,364 images extracted from BRUVS, we obtained good model performance (F-measures >0.7, up to 0.87) for the 6 species with sufficient annotations (Fmeasures >0.7, up to 0.87). The correlation between automatic and manual estimates of fish MaxN abundance in videos was high (0.72 -0.9), but the Faster R-CNN showed an underestimation bias at higher abundances. A semi-automatic protocol where our model supported manual observers in processing BRUVS footage improved performance with a correlation of 0.96 with manual counts and a perfect match (R=1) for some key species.