Development and operation of a novel non-invasive opto-acoustic underwater fish observatory in Kiel Bight, Southwestern Baltic Sea

Gröger, Joachim P.; Cisewski, Boris; Badri-Hoeher, Sabah; Böer, Gordon; Boos, Karin; Clemmesen, Catriona; Cojocaru, Ala; Dauben, Verena; Hoeher, Peter A.; Lehmann, Andreas; Matz, Sebastian; Mehrtens, Hela; Mittermayer, Felix; Renkewitz, Helge; Schramm, Hauke; Strickmann, Tobias; Westphalen, Jonni; Wilts, Thomas; Winkler, Julian; Wolf, Dennis; Zenk, Oliver

doi:10.3389/fmars.2024.1425259

ORIGINAL RESEARCH article

Front. Mar. Sci., 20 August 2024

Sec. Ocean Observation

Volume 11 - 2024 | https://doi.org/10.3389/fmars.2024.1425259

Development and operation of a novel non-invasive opto-acoustic underwater fish observatory in Kiel Bight, Southwestern Baltic Sea

Joachim P. Gröger^1,2†

Boris Cisewski^1*†

Sabah Badri-Hoeher³

Gordon Böer⁴

Karin Boos^1,5

Catriona Clemmesen²

Ala Cojocaru⁶

Verena Dauben⁷

Peter A. Hoeher⁸

Andreas Lehmann²

Sebastian Matz⁹

Hela Mehrtens²

Felix Mittermayer²

Helge Renkewitz⁹

Hauke Schramm^4,8

Tobias Strickmann²

Jonni Westphalen³

Thomas Wilts³

Julian Winkler³

Dennis Wolf⁷

Oliver Zenk⁷

¹Thünen Institute of Sea Fisheries, Bremerhaven, Germany
²GEOMAR Helmholtz Centre for Ocean Research, Kiel, Germany
³Institute of Communications Technology and Microelectronics, University of Applied Sciences, Kiel, Germany
⁴Institute of Applied Computer Science, Kiel University of Applied Sciences, Kiel, Germany
⁵City University of Applied Science, Bremen, Germany
⁶Phi-Stone AG, Kiel, Germany
⁷MacArtney Germany GmbH, Kiel, Germany
⁸Faculty of Engineering, Kiel University, Kiel, Germany
⁹Fraunhofer IOSB, IOSB-AST Ilmenau, Fraunhofer Institute of Optronics, System Technologies and Image Exploitation, Ilmenau, Germany

This study presents a trilateral test array of new opto-acoustic Underwater Fish Observatories (UFOs) that were operated and tested in Kiel Bight as part of the “UFOTriNet” project. While hydroacoustic and optical techniques have so far been used individually to observe and monitor fish stocks, we present a coupled hybrid system consisting of an optical device intended to scan the near-field as a subsample of a spatially larger medium-to-far-field, scanned by an acoustical device. The optical device consists of two residual light amplifying camera modules able to detect and classify various marine species at a high resolution in the range of at max 4 meters in the study area. To compensate for this spatial limitation, the acoustical component consists of a 2D imaging sonar with a maximum range of 50 m, albeit with a lower resolution. Species affiliation, morphometric characteristics of fish and other marine organisms were stereo-optically detected and classified in the nearfield, blended with acoustical activity in medium to far range, and projected onto the entire insonified area using a hybrid algorithm. Through the synchronous acquisition of multiparametric abiotic and biotic data, UFO allows an automatic, continuous, and non-invasive long-term monitoring of various fish and other marine species and their habitats at regional hotspots. An 86-day multiparametric sample revealing an abrupt shift from a clupeid fish to a gelatinous plankton dominated regime in summer/autumn 2021 in Kiel Fjord is used to demonstrate the potential of UFO for various applications.

1 Introduction

Marine ecosystems are very dynamic systems that change continuously under the influence of a wide range of external factors. This is particularly true for marine ecosystems, which - like the Baltic Sea - usually suffer from multiple pressures due to e.g., fisheries, climate changes and discharges of pollutants and nutrients as well as the introduction of non-indigenous species (HELCOM, 2018). The potential cumulative effects of overexploitation on the one hand and the simultaneous manifestation of anthropogenic climate change on the other hand, suggest that commercially exploited fish stocks are vulnerable components of marine ecosystems. Especially fish, shellfish and other food from the aquatic environment make a fundamental contribution to human nutrition and health, while they also play an important economic role in almost all EU countries. In 2021, consumption amounted to 10,60 million tons, corresponding to 24 kg per capita. The internal demand of fishery and aquaculture products in the EU was mostly met through imports, as they covered around 70% of the total supply.¹ Increases in demand can create many incentives for production and hence fishing pressure on almost all EU fish resources. Consequently, the EU fishing sector suffers from the overarching problem, that since years many fish stocks are fully or overexploited in almost all marine EU waters including the Baltic Sea, the North Sea or the Mediterranean Sea. The critical level of wild fish harvesting does expose the entire value chain of the fishing sector to economic and ecological sustainability risks. Indeed the “weakest link of the <EU seafood value> chain” for wild fish, is the maintenance of a sufficient biomass generated by healthy fish populations, simply with the consequence: no fish, no fishing.

Unfortunately, the present situation and the currently used methods and technologies in the fishing sector, are in fact counteracting and undermining EU policy measures, as most fishing activities are not environmentally friendly but largely invasive and prone to reduce stock reproductive potential, damaging habitats, destroying food chains and wasting biomass by producing bycatch of unwanted and/or protected species. This has major effects on the entire EU fishing sector (and thus the whole EU seafood value chain) composed of the direct fishing industry (wild fisheries and aquaculture) but also all indirect sectors, which includes the subordinate fish processing and marketing industry, coastal tourism and game fishing, small to medium size enterprises (SME) that develop and commercialize fishing gear and technology (e.g. fish detection devices), and also all governmental, national or university based fisheries research institutes, as well as other intergovernmental or international scientific bodies (e.g., International Council for the Exploration of the Sea, ICES).

Part of the problem is that, although required for an adequately performed Ecosystem Based Management (EBM) approach, the scientific community is not providing sufficient high-quality data. “Ecosystem-based management is an environmental management approach that recognizes the full array of interactions within an ecosystem, including humans, rather than considering single issues, species, or ecosystem services in isolation” (e.g., Christensen et al., 1996; McLeod et al., 2005). This requires the provision of adequate, accurate and timely data on the diversity and abundance of species and the functioning of ecosystems. To this end, the situation in the EU fishing sector is unsatisfactory from an ecological and economical point of view and it is also clearly in contrast with European policy goals, including the EU Common Fisheries Policy (CFP)² and the Marine Strategy Framework Directive (MSFD)³. Article 14 of the EU CFP claims: It is important for the management of the CFP to be guided by principles of good governance. Those principles include decision-making based on best available scientific advice, broad stakeholder involvement and a long-term perspective. Moreover, the EU has formulated a “Community framework for the collection, management and use of data in the fisheries sector and support for scientific advice regarding the Common Fisheries Policy” to setup rules for data collection, management, and use to ensure high-quality data for strengthening the advice as a major basis for sustainability. Given this, Fisheries sciences act in an area of tension between exploitation and conservation. Although the overarching goal regarding the quality of the information required as input for a sustainable fishery management is setting high standards, one major reason for the distressing situation in the fishing sector is that data collection and surveillance procedures are critically under-performing.

Traditional methods fail to observe reality on an appropriate time scale as demanded for deriving unbiased descriptors and indicators of good environmental health. Often the reason for insufficient understanding of the underlying processes is due to sub-optimal, i.e., sporadic or under-sampling. The underlying physical processes in the ocean occur on horizontal and vertical scales from millimeters to thousands of kilometers and on time scales from milliseconds to several decades and beyond (Dickey, 1991). Due to the complexity and the interactions between organisms and their environment, and the non-linear environmental dynamics, most of the underlying interacting processes and mechanisms are poorly understood. To study the processes shaping the structure and dynamics of marine ecosystems often requires coupled investigations of biology and physical oceanography. Integrative techniques that allow a synchronous collection of physical, chemical, and biological parameters, including higher trophic levels, are therefore particularly useful. Moreover, it is necessary to select survey designs and investigation methods that allow for spatial and temporal resolution of the relevant scales. Thus, continuous, and synchronous high-resolution multiparametric time series data makes long-term fixed-point observatories uniquely insightful in resolving decadal environmental trends required to understand the impacts of environmental and climate changes.

The standard practice of marine monitoring in fisheries research is performing hydroacoustic and/or trawl surveys using research vessels. On predefined hydroacoustic transects in the survey area, data on echo distribution and total echo strength are collected. The basic instrument is a vertically oriented, single beam echosounder with frequencies operating between 38 and 200 kHz. The basic data, calibration and analysis procedures for determining fish abundances are described by Foote et al. (1987) and Simmonds and MacLennan (2005). Since hydroacoustics are neither yet capable of identifying the species of individual fish (or groups of fish) nor morphometric fish characteristics given a too low resolution, traditional fishing hauls must be used to identify the relative species composition in the area and the inter-calibration of size frequencies of the individual fish species. These data are further used in internationally coordinated assessments of the status and sustainable use of the corresponding stocks to derive appropriate international stock-based quotas. Hydroacoustic methods are widely used in fisheries research since the 1930s (see review of Fernandes et al., 2002) because the various hydroacoustic instruments have the advantage of transmitting and receiving sound waves to detect fish or other objects non-invasively throughout the whole water column, while the penetration of light is restricted on the upper few meters of the surface layer (Simmonds and MacLennan, 2005). Depending on the purpose, different systems are used: single-beam sonar, dual-beam echosounder, split-beam echosounders, and multi-beam sonars (Simmonds and MacLennan, 2005). While the single-beam sonar represents a basic type of sonar system that emits a single acoustic beam of sound waves in a narrow cone, the dual-beam echosounder uses two beams at different frequencies (high and low) to provide more detailed information about the underwater environment. These systems can measure the depth and composition of the seafloor or water column more accurately than single-beam sonar. Split-beam echosounders divide the emitted sound cone into four sectors. This configuration allows for detailed spatial information about objects within the sound cone. They are frequently used in fisheries research and fish biomass surveys to study the distribution and abundance of fish in different parts of the water column. By analyzing the echoes returned from these four sectors, scientists can gain insights into fish distribution. Multi-beam sonars, such as Dual-Frequency Identification Sonar (DIDSON) and Adaptive Resolution Imaging Sonar (ARIS), are advanced systems that provide high-resolution, video-like images of the underwater environment. They are particularly valuable for studying fish behavior, tracking their movements, and understanding the underwater ecosystem. Wei et al. (2022) and Munnelly et al. (2023) have recently studied the current literature to investigate the suitability of imaging for identifying different fish species, while also providing information on length and abundance. The authors conclude that the imaging sonars they analyzed are generally suitable for monitoring fish in their environment, but that there are still major challenges involved in identifying different species and small fish in dense aggregations, and that the associated data processing is very time consuming. To overcome the limitations of species identification, Wei et al. (2022) recommend combining sonar data with image data and trawl data and using AI methods for automated image analysis in the future. The trade-off of this type of echosounders is – as with optical systems – their limited spatial detection range, given their high frequencies at reversely short-wave lengths.

Camera systems have already been used for several years at several cabled long-term observatories to observe fish and other macrobiota in their habitats, where the temporal variation in counted individuals can be considered as a proxy of variations in local abundance (Aguzzi et al., 2015, 2020). Bicknell et al. (2016) reviewed the development of camera imagery and its implementation for examining human impact on marine ecosystems and biodiversity. Depending on the purpose, different systems are used: (1) remote under-water cameras, deep-sea cameras, baited camera systems, animal-born cameras, and TV and stereo-video cameras mounted on towed and autonomous operating vehicles (Bicknell et al., 2016). In these adaptations a variety of hardware such as TV cameras (e.g., Aguzzi et al., 2011) or specially adapted stereo camera systems (e.g., Fischer et al., 2017) are used. The evaluation ranges from simple manual annotation of individual images to fully automated image recognition using AI methods (e.g., Bonofiglio et al., 2022; Tarling et al., 2022). However, the operation of underwater camera systems is strongly limited by reduced visibility in the water, reduced light in deeper layers and biofouling on the camera housings.

There is still a need for long-term ocean observations, as coastal and deep-sea observatories are key to improving the understanding of the underlying processes within the marine ecosystem (Lantéri et al., 2022). For this reason, several international initiatives have promoted their use through investments in infrastructure development and dedicated scientific meetings on how to coordinate the acquisition, analysis and dissemination of ocean data. Among these initiatives, the European Multidisciplinary Seafloor and Water Column Observatory (EMSO) is a pan-European distributed research infrastructure with the major scientific objective of long-term monitoring, mainly in real-time, of environmental processes related to the interaction between the geosphere, biosphere, and hydrosphere, including natural hazards. Several deep-sea nodes e.g., MARS (Monterey Accelerated Research System) (Dawe et al., 2005), NEPTUNE (North-East Pacific Time-Series Undersea Network experiments) (Best et al., 2007) and VENUS (Victoria Experimental Network Under the Sea) (Dewey and Tunnicliffe, 2003) and shallow water nodes e.g., COSYNA (Coastal Observing System for Northern and Arctic Seas (Baschek et al., 2017; Fischer et al., 2020), OBSEA (Aguzzi et al., 2011) and SmartBay (Smart Bay Cabled Observatory) (Cullen et al., 2015) have been deployed at key locations in the world’s oceans, from the Arctic to the Atlantic, and via the Mediterranean to the Black Sea. For a more detailed overview of long-term observatories operating worldwide compare Zielinski et al. (2022).

While existing coastal observatories mainly use optical or hydroacoustic methods to study the occurrence and behavior of specific fish and shellfish species (e.g., Aguzzi et al., 2011; Fischer et al., 2017; Pelletier et al., 2021; Francisco et al., 2022), this study presents three different device variants of a new concept of non-invasive stationary opto-acoustic Underwater Fish Observatories (UFO). UFO’s core feature is a high-resolution multibeam imaging sonar (900 kHz), whose acoustic signal in the medium-to-far field will be synchronously calibrated by a stereo-video signal (using highly sensitive low light stereo-cameras) in the overlapping near-field. To fulfil requirements of assessing state-of-the-art size-based population dynamic models for fish the acoustic signal for instance is targeting the density (biomass, abundance) level of important fish species in the coastal zone, while the stereo-video signal inter alia is aiming at detecting the type of fish species and the size of the individuals within the video scanned zone. For this objective, we developed first a fully equipped stationary underwater fish observatory (UFO) as part of an initial project which was initially deployed in the North and Baltic Sea. In a subsequent project (“UFOTriNet”) it followed a less equipped miniaturized UFO (UFO portable) plus a mobile UFO (UFO mobile), both being coupled with the stationary one from the previous project in a trilateral test network, to develop standardized automatic pattern recognition algorithms for all three devices and to optimize them in such a way that a species, size and weight classification can be achieved with a high statistical confidence (> 92%) (Böer et al., 2023).

UFO is intended to allow for continuous all-season collection of data to assess a broad range of indicators. This is especially relevant to evaluate the status of various components of the Baltic Sea ecosystem with regard to achieving Good Ecological and Environmental Status (GES) in the context of both, the Water Framework Directive (WFD, European Commission, 2000) and the Marine Strategy Framework Directive (MSFD, European Commission, 2008). As all UFOs enable smart identification, classification and quantification of fish, this project will make a highly valuable contribution to existing and future coastal observation programs by developing autonomous technology solutions that are fully compatible and therefore combinable with other external common underwater platforms.

In the current manuscript, we introduce a novel trilateral test array of new non-invasive opto-acoustic Underwater Fish Observatories (UFOs) that were operated and tested in Kiel Bight. We present the basic design features of the three variants as well as data handling procedures with focus on the hybridization of the opto-acoustic data. Based on an 86-day multiparametric sample obtained by the stationary UFO in Kiel Fjord, we demonstrate and discuss the potential of the UFO for various applications.

2 Material and methods

2.1 Study site

Kiel Fjord is an approximately 17 km long inlet of the Baltic Sea at the eastern coast of Schleswig-Holstein, Germany, and reveals daily and seasonal variations, e.g., in temperature, salinity and primary productivity (Ory et al., 2020). It is characterized by a broad access to the western part of the Baltic Sea and hence by a discontinuous, weather-dependent saline water inflow from the North Sea via Skagerrak and Kattegat. Due to its proximity to an estuarine subsystem, it is influenced by a continuous freshwater inflow from the Schwentine river as well as brackish water from the Kiel Canal. The Baltic Sea is a semi-enclosed only 8,000 years old brackish water system with a strong salinity gradient being influenced by a series of multiple stressors like eutrophication, warming, decrease in oxygen and increase in acidification. Due to its unique geographical, climatological, and oceanographic characteristics, the Baltic Sea is highly sensitive to environmental impacts of human activities not only in its sea areas but also in its catchment areas. Since the Baltic Sea is rather young in age, it only has few endemic species with locally adapted populations (Reusch et al., 2018). Therefore, it is an ideal study area to look at the effects of environmental variables on fish species using the UFO system.

Dominant larger fish species of the Western Baltic Sea are cod, herring and sprat as well as some flatfish species (Aro, 1989). Herring is one of the ecologically and previous to its collapse commercially most important species in European northern seas and the Western Baltic Sea, especially in the study area. The Kiel Fjord and the Kiel Canal as part of the Western Baltic Sea are important spawning areas for Western Baltic herring (Weber, 1971). In the western Baltic Sea, autumn spawning herring were the dominant commercial fishery stock during the first half of the 19th century and supporting a large fishery until the sixties of the last century (Rechlin, 1991). However, this stock has significantly decreased since the 1970s (Rechlin, 2000). Today spring spawning herring are the dominating stock in the western Baltic Sea, but autumn spawning herring are also found in the Kiel Fjord area with increasing numbers in recent years (pers. comm. from fishermen). Genetic analyses performed on herring from the Western Baltic Sea have shown spring- and autumn spawning populations of herring locally coexisting (Bekkevold et al., 2016). Analyses of ichthyoplankton Bongo samples from the inner Kiel Fjord and the Kiel Canal ongoing since 2005 on a weekly to biweekly basis have shown the occurrence of herring larvae and confirmed spawning events in spring (Paulsen et al., 2014, 2017). Additionally, irregular catches of larger larvae in January and February indicate the possibility of autumn spawning events also happening in the vicinity (Clemmesen pers. comm.). European sprat is a small pelagic clupeid schooling fish (up to 16 cm, Rechlin, 1975; Stepputtis, 2006). Although sprat predominantly spawn in more offshore basin areas, nearshore and coastal spawning events have been observed as well (Voss et al., 2012). Analyses of ichthyoplankton- Bongo samples from the inner Kiel Fjord confirm the occurrence of sprat eggs and larvae from May to July in line with results from Parmanne et al. (1994) and Baumann et al. (2006). Juvenile herring and sprat form mixed-species schools in the coastal waters of the Baltic and North Seas and are therefore strongly associated with each other (Arrhenius and Hansson, 1993; Maes and Ollevier, 2002). Dipnet fishery in the Kiel Fjord has confirmed the occurrence of these mixed schools during the investigated period. While the stock of Western Baltic Cod has declined dramatically since the beginning of this decade, the stocks for the most common flatfish in the Western Baltic (plaice, flounder and dab) have increased⁴. In addition, the Baltic Sea harbours a number of seasonally visiting species such as mackerel, horse mackerel, garfish and mullet that can be found in variable numbers during the summer months.

2.2 UFOTriNet system architecture

The test array of “UFOTriNet” integrated two fixed monitoring systems (stationary and portable UFO) with a mobile ROV/AUV monitoring system (mobile UFO). Here we present a brief description of the three different applications. Study area and locations are presented in Figure 1 and types and technical specification of sensors installed on the different UFOs are listed in Table 1.

Figure 1

Figure 1. Deployment stations (stationary and portable UFO) and connecting path (mobile UFO) of the Underwater Fish Observatories (UFO) in the Southwestern Baltic Sea (redrawn after Hoeher et al., 2023, licensed CC-BY-4.0).

Table 1

Table 1. Types and technical specification of sensors installed on the UFO.

2.2.1 Stationary UFO

The stationary UFO (Figures 2A, B) was located at 54°20.4375’N, 10°10.4463’E between 2021/03/05 and 2022/05/18, about 50 m from the eastern shore of the Kiel Fjord, at a depth of 6 m and was directly connected to a land station by a fiber optic (FO) cable. The data management system at the land station consists of two servers storing time series of abiotic, camera and sonar data from the UFO and makes these data remotely accessible to web clients. The core unit of the UFO underwater system features two Photonis Nocturn XL camera modules which amplify residual light, each with a resolution of 1280x1024 pixels, alongside a Teledyne Blueview M900–130 2D imaging sonar. It offers remote adjustability for tilting up or down, allowing angle alterations relative to the seabed to minimize bottom reflection or to alter its vertical viewpoint. From the technological approach the variety and number of sensors ranging from optical cameras and acoustic observation devices to sensors of physical and chemical parameters result in an individual multiplexer design. The multi-channel multiplexer (MacArtney NEXUS MK-C) used as integrative core unit is a highspeed device being compatible with almost all sensors and probes currently on the market. The customized multiplexer solution consists of three Gigabit-Ethernet and eight RS232/RS485 interfaces and is therefore capable to interface a vast variety of sensors available on the market. Providing the fibre optic link between the UFO and the surface, the multiplexing device uses state-of-the art telemetric technologies that are widely used for subsea applications. In addition, the UFO is equipped with a central motor control system responsible for operating the camera window wiper, maneuvering the gimbal’s two motors, and activating the recovery buoy’s releaser. This system ensures regular cleaning (3 h) of the stereo camera lenses through the wiper to maintain clarity by removing any fouling. The gimbal, housing the cameras and sonar, can be adjusted by two additional motors. This arrangement, in combination with a position sensor that monitors the gimbal’s roll and tilt, allows for the automatic adjustment to uneven seabed terrain once the lander is positioned. The fourth motor can be used to release the recovery buoy to which the recovery rope is attached. The abiotic sensor instrumentation consists of an upward looking acoustic Doppler current profiler (ADCP; Teledyne Workhorse Sentinel 1200 kHz), a pumped conductivity, temperature, and depth sonde (CTD; Sea Bird Scientific MicroCAT) and a combined fluorometer and turbidity sensor (Sea Bird Scientific ECO FLNTUS).

Figure 2

Figure 2. (A) The stationary UFO ready for deployment. (B) An annotated sketch showing the major components of the stationary UFO: (a) stereo camera system, (b) sonar, (c) multiplexer, (d) conductivity, temperature, depth sensor, (e) fluorometer, (f) acoustic Doppler current profiler, (g) recovery buoy, (h) wiper, (i) position sensor.

2.2.2 Portable UFO

The FO-cabled portable UFO (Figures 3A, B) was located at 54°28.406’N; 9°50.618’E between 2021/07/23 and 2022/03/17 and attached to the outer sheet pile wall of the town harbor in Eckernfoerde at a depth of 7 meters, using a ladder construction for height adjustment and easy recovery of the portable UFO. In contrast to the stationary UFO, the portable UFO is a miniaturized version, in which the core unit consists of two residual light amplifying Photonis Nocturn XL camera modules and a 2D imaging Teledyne BlueView P450–130. Signal and data flow were controlled by an EMO Mini Series-T Multiplexer. The portable UFO is designed for stationary use in areas where a rather flexible small unit is required, such as for tethering to bridge pylons, rocks, or lighthouses, or, e.g., as part of continuous monitoring in harder-to-reach areas.

Figure 3

Figure 3. (A) The portable UFO attached to an underwater ladder assembly. (B) An annotated sketch showing the major components of the portable UFO: (a) stereo camera system, (b) sonar, (c) multiplexer, (d) ladder assembly.

2.2.3 Mobile UFO

The mobile UFO (Figures 4A, B) combines the capabilities of a remotely operated vehicle (ROV) with those of an autonomous underwater vehicle (AUV). It can be operated remotely at short distances of up to 800 meters and autonomously for missions with longer ranges. While the stationary and portable UFO continuously observe the fish habitat at a certain position (e.g., in hotspots, marine protected areas, wind farms etc.), the mobile version extends the geographical radius of monitoring through up-scaling, meaning by either inspecting the transect between two or more different stationary UFO platforms or by inspecting the perimeter of a single stationary UFO platform within a certain radius. For this purpose, the mobile UFO carries the same core unit consisting of two residual light amplifying Photonis Nocturn XL camera modules and a 2D imaging Teledyne BlueView M900–130 sonar. Real-time evaluation and fusion of the stereo camera and sonar data takes place on an embedded computing system designed for AI Edge Computing (Up Xtreme aided by an UP Vision Plus X). Due to the heat generation of these components, a special cooling concept using heat pipes was developed and implemented. The video and sonar data are stored synchronously. To determine the absolute position on the water surface, several GNSS systems (GPS, GLONASS, Galileo and BeiDou) are interrogated simultaneously and transmitted to the navigation system with an accuracy of up to 0.01 m. Furthermore, in the AUV mode a WiFi system is used for communication with the control station at the surface. Underwater, an ultra-short baseline (USBL) system is used for data transmission and relative position determination. This has a range of ~3500 m with a data rate of up to 13.9 kbit/s. In addition to the fiber optic gyro as the main sensor, the navigation Kalman filter also uses data from the Doppler velocity log (DVL), pressure sensor, GNSS system and USBL system in real time. To correct the data from the acoustic sensors (Sonar, DVL), a sound velocity sensor is used to determine the speed of sound at the current position of the vehicle with an accuracy of 0.02 ms^-1. For the remotely operated vehicle (ROV) mode, a separate camera is used. Since this does not have residual light amplification, travel path illumination is provided by four controllable LEDs at the front of the carrier system. If the autonomous driving mode is selected, the use of the driving camera is omitted. The vehicle has a manual trim system with weights and foam to adapt the appropriate density of the water in the field. Dimensions (LxWxH) 0.94x0.63x0.6 m with a mass between 65 kg or 95 kg depending on the payload of the system. In ROV mode, ~1.25 Gbit/s can be transmitted over the ~800 m single mode fibre. Additional sensor systems can be easily integrated via existing free slots. The battery management system (BMS) monitors the voltage of the 7 LiPo single cells as well as the current and temperature in the battery and reacts to its current state. Two different sized batteries were developed for different application scenarios. The smaller one has a capacity of ~700 Wh and weighs 6.5 kg while the larger one has ~1400 Wh and weighs ~11 kg. To control the system, a control station was set up with two monitors and a computer. Using game controllers and additional controllers, the vehicle can be operated autonomously, semi-autonomously or remotely via the in-house developed software.

Figure 4

Figure 4. (A) The mobile UFO ready for deployment onboard the RV Littorina. (B) An annotated sketch showing its major components: (a) stereo camera system, (b) sonar, (c) main thrusters, (d) payload computer for real-time AI applications, (e) underwater navigation sensor with high accuracy, (f) communication module with wifi and USBL.

2.2.4 Maintenance and antifouling strategies

Maintaining underwater observatories is a complex and ongoing task, as these systems are often placed in challenging and rough marine environments. To ensure the observatories’ continued functionality and the collection of scientific data for monitoring the marine ecosystems, the UFO lander and the UFO components, including sensors, camera and sonar systems, data loggers, cables, and power systems were regularly inspected for signs of malfunction, corrosion, or damage. Since the data recording, power management and communication between the underwater unit and the land station of the UFO can be monitored via VPN access at any time during operation, failures, erroneous values such as outliers and sensor drifts can already indicate errors that require inspection. For a first visual inspection, scientific divers manually checked and mechanically cleaned affected sensors, connectors, and cables from fouling.

Biofouling poses a severe problem for long term deployments of marine instruments, because the sensor surfaces can become overgrown by e. g. barnacles Balanus improvisus and blue mussels Mytilus edulis which are amongst the most common species found in epibenthic communities in the Baltic Sea. Especially in summer, the overall warming of the upper layers of the Baltic Sea leads to an increased survival and accelerated larval development of barnacles (Nasrolahi et al., 2013). Different methods were used and further developed to reduce biofouling. To keep the camera lenses free from fouling, a mechanical wiper system was designed to clean the lenses at intervals determined by the user. However, due to intermittent failures of the mechanical wiper, Hoeher et al. (2023) developed an energy-efficient and environmentally friendly antifouling concept based on UVC irradiation. LEDs are known for their energy efficiency and versatility in control, making them ideal for precise and targeted UVC irradiation applications, such as the disinfection or sterilization of water or surfaces. The most effective results during the experiment were achieved by positioning the UVC irradiation source inside the camera’s pressure housing. This approach offers several benefits, such as protecting the UVC light source from harsh external conditions (e.g., high pressure, corrosive water, physical damage). Moreover, installing the source internally prevents the attenuation of LED light intensity caused by turbid and/or turbulent seawater. Notably, the UVC irradiation intervals selected were considerably shorter than those typically reported in literature. With this setup the system can deliver rapid and frequent UVC treatment, which could be crucial for efficiently eliminating or controlling microorganisms and pathogens in the water column. Moreover, employing shorter intervals could potentially lower energy consumption, thereby extending the overall operation time of the system. By integrating such advanced sterilization methods, it is possible to significantly mitigate the risk of biofouling through monitoring equipment.

Copper-nickel alloys or other metals with antifouling properties can be used in the construction of underwater structures to deter fouling. Thus, the inlet of the Seabird CTD and the bottom plate of the fluorometer, where the two optical fluorescence and turbidity sensors are located, were made of copper to minimize fouling. In this project, we also used a mechanically robust polymer composite called CSPC (Core-Spike-Particle Composite) as a coating material on the frames of the stationary and portable UFO as well as on most sensor housings. This composite is primarily composed of polythiourethane, with the inclusion of tetrapodal-shaped ZnO (t-ZnO) particles to enhance its mechanical strength. The CSPC-coating offers an optimal alternative to the present biocide-containing antifouling systems. Remarkably, this composite is entirely free from volatile substances and biocides, making it an environmentally responsible choice. Its exceptional mechanical properties make it easy to clean, effectively reducing and preventing the accumulation of biofilms and other adhesives. When keeping the cleaning-intervals, a residue-free removal of growth is easily possible. Additionally, the composite demonstrates UV stability and a remarkable resistance to the harsh conditions prevalent in marine environments. This innovative coating also offers robust protection against biocorrosion, safeguarding the integrity of the hull. More details about this material can be found in references (Hölken et al., 2016; Qiu et al., 2018). During the UFO deployment, it became apparent that the application of the antifouling coating, particularly on the landing gear, inhibited the fouling to such an extent that it could remain in the water longer between maintenance operations and the cleaning effort was significantly reduced.

2.3 Multiparametric time series acquisition

2.3.1 Abiotic measurements

The UFO instrumentation additionally is equipped with an acoustic Doppler current profiler, temperature, salinity, oxygen, fluorescence, and turbidity sensors to enable analysis and monitoring of the physical and chemical environment. All sensors were placed ~1.0 m above the seafloor. Time series data for ADCP, CTD and fluorometer were downloaded and preprocessed to obtain average estimates at a frequency of 1 h. To guarantee the high quality of the measurements and the data output, the sensors are regularly serviced. After each survey, back in the laboratories, the raw data collected at sea are first checked for consistency, and then processed and calibrated according to international standards. To verify and calibrate the T, S, O₂, CHL, and Turb data, a vertical CTD profile was measured fortnightly close to the two UFO sites. Additional water samples were taken to determine oxygen levels using the Winkler method and to determine Chlorophyll levels to calibrate the fluorometer. These data were also used to validate the regional results of the Baltic Sea Ice-Ocean Model (BSIOM, Lehmann and Hinrichsen, 2000; Lennartz et al., 2014). BSIOM is a fully validated model system which described the abiotic physical environment of the entire Baltic Sea including Kattegat and Skagerrak. With this approach, temporally high-resolution but spatially low-resolution observations are integrated into a spatially and temporally high resolved environment of the entire Baltic Sea. The realistic atmospheric forcing data will be used to force the model system but also to detect extreme events e.g., marine heat waves.

2.3.2 Video measurements

A comprehensive processing pipeline, that leverages state-of-the-art computer vision algorithms, was developed for the automatic detection, quantification, and size estimation of marine organisms using the recorded underwater videos (Böer et al., 2023). The pipeline utilizes an activity detection module, which identifies sequences displaying movement, using an adaptive background estimation algorithm. A detection algorithm based on a YOLOv5 model (Jocher et al., 2022) is then applied to detect marine organisms in each video frame. Finally, a stereo processing module calculates the size and distance of each detected organism. The employed stereo-camera-system is critical for the detection of marine organisms and for calculating the real-world distances and sizes of detected organisms.

The YOLOv5 model, utilized in the detection module, was trained to detect marine organisms using transfer learning on a curated dataset, with the available pretrained weights for the YOLOv5l6 model configuration. The training was performed on four NVidia Titan RTX graphics processing units, with a batch size of 32, utilizing the stochastic gradient descent optimization algorithm. Since the training did not converge after the default upper limit of 200 epochs, it was continued from the best checkpoint for an additional 129 epochs, until no further improvement was observed. The inference time, required to process the image data with the selected model size is estimated to be around 20 ms per image, but note that the overall processing time is further impacted by the post-processing procedures. Particularly, the non-maximum suppression (NMS) processing time may fluctuate considerably, ranging from 1 to 20 ms, depending on the number of detected objects.

The training and validation data consisted of 60,144 and 13,000 images respectively, with 76,658 and 16,241 bounding box annotations across 10 different categories of marine species. The performance of the model was evaluated using common metrics in the field of object detection, including the F1 score and the mean average precision (mAP). On validation data, the model achieved a mean detection accuracy of 92.4%, a mean average precision of 94.8%, and a F1 score of 93% (Böer et al., 2023).

The final step of the pipeline is the stereo processing module, which is responsible for calculating the size of each animal and its distance to the camera. This step is based on a matching scheme between the detected bounding boxes on each of the synchronized stereo frames. The stereo matching process involves checking the angle of the horizontal line connecting the center points of the bounding boxes in the left and right images, the Intersection over Union (IoU) between the bounding boxes, and the species assigned to each bounding box. The matching score between two bounding boxes is calculated as the IoU if the angle between the center points is below a certain threshold, the IoU is above a certain threshold, and the predicted species for the two boxes correspond. Otherwise, the matching score is set to zero. If many fishes have been detected on the same epipolar line, it may be possible that a single bounding box is matched to multiple other bounding boxes; in this case, the established match is chosen based on the highest matching score.

Upon establishing the correlation between bounding boxes, the 3D coordinates of any image point simultaneously observed in both cameras can be triangulated, leveraging the pre-calibrated camera parameters. This triangulation enables the computation of the distance between any of these points, which can subsequently be utilized to measure the dimensions of a detected organism. Specifically, the central point of the bounding boxes is employed to calculate the distance, while an estimate of an animal’s length is derived from the diagonal line connecting the upper-left and lower-right corners of the box.

2.3.3 Sonar measurements

For the continuous and concurrent recording of acoustic data, a Teledyne BlueView M900–130 imaging sonar was utilized. This sonar operates at 900 kHz frequency and has a horizontal field of view of 130° as it emits 768 beams with 0.18° beam spacing and a vertical beam width of 20° (Wolff and Badri-Hoeher, 2014). Because of its high update rate of up to 5 Hz it is possible to view movie-like imagery. The sonar was mounted on the UFO lander in a water depth of 5 m enabling a maximum range of 50 m. While a BlueView imaging sonar is advantageous for its long signal propagation distance and strong penetrating ability, the sonar images are generally suffered from structural noise due to interference and reverberation (Zhou et al, 2021). Following Wolff and Badri-Hoeher (2014) and Winkler et al. (2023) the captured sonar images are processed in different steps including (i) filtering of background and unwanted moving particles, (ii) segmentation of moving objects using a Gaussian model and (iii) fish tracking using Kalman Filter.

2.4 Hybrid algorithm

In the development of a novel method for the abundance estimation of fish, the strengths of the optical and acoustic systems were combined, a process referred to as hybridization. The camera images allow for the recognition of various marine species, including fish and jellyfish and comb jellies, enabling not only the counting of fish but also their specification. However, the effective viewing distance of the camera systems is constrained by underwater conditions to a realistic maximum of approximately 4 m. To overcome this limitation, sonar systems were employed. While sonar images do not provide the detail of optical cameras, they significantly expand the observation area. However, the resolution of sonar imagery is insufficient for species identification.

To bridge this gap, a process of hybridization was developed (Figure 5). The core concept involves leveraging the precise detections and classifications obtained from the camera system to inform the analysis of the broader area covered by sonar. This process assumes that the optical detections and classifications are accurate while acoustic signals primarily indicate the presence of activity, rather than specifying the types of marine life. Here, “activities” refer to sonar detections distinct from static elements like the seabed, encompassing fish and any other moving objects within the sonar’s detection range.

Figure 5

Figure 5. The processing pipeline from the recording of underwater videos to the final abundance estimation.

For the extrapolation, it is hypothesized that the number of sonar activities in the short-range corresponds to the detections and classifications of the cameras in the same vicinity. These observations are then linearly extrapolated based on the quantity of sonar-detected activities from the long-range. The number of extrapolated animals in the area observed by the sonar is calculated as follows:

C_{h y b r i d} = O_{s} + O_{s} \cdot \frac{A_{l}}{A_{s}}

Where C_hybrid represents the number of extrapolated animals in the area observed by the sonar, O_s is the number of detections provided by the camera system, A_s is the number of sonar activities in the short-range and A_l is the number of sonar activities in the long-range.

2.4.1 Evaluation of the hybrid algorithm and considerations

The evaluation of the UFO sensor data presents significant challenges owing to the absence of ground truth data, necessitating a predominantly manual enumeration of fish occurrences within sonar and camera imagery. Given the intermittent low visibility of fish in both imaging modalities, manual counting stands as the highest attainable standard in terms of accuracy. To assess the performance of our approach, we manually enumerated occurrences in two dozen randomly selected time frames, spanning scenarios ranging from solitary fish to aggregations numbering several hundred individuals. Additionally, we explored the correlation between the detected fish counts in the short-range sonar and the camera system, hypothesizing that these counts would exhibit an equivalent or at least linearly proportional relationship.

Our investigation revealed that, for a small number of fish, the camera and sonar data indeed exhibited a correlation and linear relationship. However, as larger fish schools appeared, this correlation diminished. This phenomenon can be attributed to the inherent nature of the blob detection process in the sonar imagery. In the sonar image, a school of fish manifests as a single large blob, with discrete intensity peaks corresponding to individual fish. Our methodology does not account for local intensity maxima within a blob, and the opening and closing operations merge closely spaced individual fish appearances.

To mitigate these issues, an alternative approach could involve counting the cumulative pixel values within a blob. Assuming that the pixel sizes for each fish within a swarm are relatively uniform, the ratio of pixel sums, denoted as $\frac{A_{l}^{p}}{A_{s}^{p}}$ , would also exhibit similarity to $\frac{A_{l}}{A_{s}}$ , where $A_{r}^{p}$ signifies the sum of pixels within a blob of area r.

Further examination of the sonar data uncovered a persistent bias in the activity counts. This bias arises from the initial assumption that only fish movements are present in the sonar image, allowing for the elimination of static background. However, it was observed that sediment clouds, induced by water currents, were consistently present in the sonar image, leading to false positives in the activity detection. Moreover, the sonar exhibited signs of defects, characterized by frequent flickering in certain segments of the sonar imagery. This issue was mitigated by considering the typical duration of fish appearances, which generally ranged from 2 to 300 seconds. Consequently, the bias in the activity data was rectified by applying a band-pass filter to remove undesirable low-frequency currents and high-frequency flickering from the activity counts.

Another source of errors emerged from a detailed examination of the field of view for both, the camera and the sonar systems, visualized in Figure 6. The sonar boasted a wider horizontal viewing angle, whereas the camera system had a broader vertical field of view. Examples illustrating the occurrence of one single fish and a school of clupeids are given in Figures 7A, B, respectively. Aside from that, the minimum detectable range of the sonar resulted in false negatives when fish were near the system, where the sonar could not detect any activities. Additionally, the effectiveness of extrapolation relied on the accurate estimation of the maximum detection range in the camera system. Unfortunately, camera failures precluded the measurement of this parameter, leading to the assumption of a fixed maximum viewing range for the entire dataset.

Figure 6

Figure 6. UFO imagery sensor setup (only one camera is depicted). Left: Horizontal alignment (and scan angles) in top-view; right: vertical alignment (and scan angles) in side-view.

Figure 7

Figure 7. Comparison between sonar (upper panel) and video data (lower panel) showing (A) a single fish and (B) a school of clupeids.

2.5 Data management

The Ocean Science Information System (OSIS), permanently operated at GEOMAR was used as central infrastructure, allowing scientists to upload and share data files in the context of expeditions, numerical models or experiments with allowance for any file format and content structure. Thus, OSIS serves for the collection of high-quality metadata and at the same time as a central versioned repository for the generated research data, as an information and exchange platform as well as for the administration and traceability of the data to be delivered and the already existing data. The presented multiparametric data will be published in thematically suitable, trustworthy long-term archives (World Data Center) and at the same time made citable by assigning DOIs. This will be done primarily in collaboration with the World Data Center PANGAEA.

3 Results

Basic abiotic parameters (temperature, salinity, oxygen, chlorophyll, and turbidity) are shown in Figures 8A-E, respectively. Water temperature varied in the normal range for the time of year and decreased from 17.39°C on 6 September 2021 to 11.09°C on 9 November 2021 (Figure 8A). During this period, salinity revealed an opposite trend and increased from 14.02 on 3 September 2021 to 20.77 on 9 November 2021 (Figure 8B). The oxygen concentration varied between 1.38 and 6.01 ml/l and revealed anoxic conditions on 2 October 2021 (Figure 8C). The chlorophyll a concentration ranged between 0.3 ml/l and 13.5 ml/l and reveals an autumn algae bloom with peak concentration of 13.5 ml/l occurring between October 8 and October 13 (Figure 8D).

Figure 8

Figure 8. 86-day sample (2021/08/22 to 2021/11/16) of (from top to bottom) (A-C) temperature, salinity and oxygen observed (black) and modelled (red), (D) chlorophyll a concentration, (E) turbidity, (F) abundance of Clupeidae, (G) abundance of Ctenophora derived from the Underwater Fish Observatory. While the abiotic factors are given as hourly averages, the calculated abundances for clupeids and ctenophores are illustrated as maximum numbers per day.

Based on the hybrid algorithm out of a total of 85867 observations in the selected period (22.08.2021 – 16.11.2021), 62,383 were fish observations (including unspecified fishes) and 23,467 were jellyfish (Ctenophores and Scyphozoa); 9 were bird (cormorant) and 8 were crab (Carcinus maenas) observations (Figure 9). Fish represented 72,7% of all observations, 27.3% were detected as jellyfish. 16,142 observations were of clupeid fish (representing 25.9% of all fishes), other fishes detected were cod (1,215 individuals, 1.9% of all fishes), mackerel (100 individuals, 0.2%), salmonids (56 individuals, 0.09%) and 14 other fish observations (eels, unspecified flatfish and horse mackerel representing 0.02% of all fishes). Unspecified fish are mostly Gobiids, which occur occasionally in high numbers and yielded 44,856 individuals (71.9% of all fishes detected) in total. Out of the total observations for gelatinous zooplankton (N= 23,467), 27 were identified as Aurelia aurita (moon jelly, 0.12% of total jellies), 385 as Cyanea capilatta (lion’s mane jellyfish, 1.6%), 339 individuals (1.4%) could not be automatically identified to species level. The most observed jellyfish was the non-native comb jelly (Mnemiopsis leidyi) with 22,716 individuals, accounting for 96.8% of all observed jellies. Overall, the dominant species detected in the inner Kiel Fjord were clupeid fish and the invasive comb jelly. Since herring (Clupea harengus) and sprat (Sprattus sprattus) are the main clupeid fish occurring in the catchment area of Kiel Fjord, where the stationary UFO was deployed, these two species are the fish species detected the most. Further differentiation into the two species cannot be done using the camera observations, since fish smaller than 16 cm (maximum size of sprat) are indistinguishable on the images taken. Species identification can only be performed through the analysis of fin positions on caught fish. The temporal abundance analysis shows a switch in occurrence between the dominant fishes (clupeids) and comb jellies (ctenophores). Herring and sprat dominate between August and beginning of September and are absent after the end of October, with comb jellies becoming the dominant species in the study area around that time (Figures 8F, G). Due to their general low biomass and ground dwelling behaviour, respectively, the detection of other species such as cod and flatfish were very few.

Figure 9

Figure 9. The distribution of different organisms among the 85867 observations in the 86-day sample (2021/08/22 to 2021/11/16).

4 Discussion

Western Baltic herring are pelagically schooling clupeids occurring in a size range of 20 – 32 cm in the Kiel Fjord area (Rudnick, 2022) and were the most detected fish species in the study area. The absence of herring and sprat from the end of October through November can be explained by the fish migrating to the overwintering areas, which are assumed to be in the region of the Kattegat and Øresund (Nielsen et al., 2001; Kvamme et al., 2003). Decreasing temperatures below 8–9°C also cause the young herring to move to deeper waters (Klinkhardt, 1996). The dominance of the ctenophores (Mnemiopsis leidyi) matches observations from previous plankton samplings showing a peak in plankton abundance during these months (Javidpour et al., 2006; 2009; Riisgård et al., 2012). The comb jelly was observed in the western Baltic Sea by Javidpour et al. (2006) for the first time in autumn 2006. As Mnemiopsis leidyi feeds primarily on zooplankton and to a lesser extent on ichthyoplankton of planktivorous fish such as anchovies, sardines and herring during their spawning period (e.g. Purcell et al., 2001), it mostly represents a potential food competitor of planktivorous fish and their larval stages. There is a fundamental risk that the intrusion of invasive species may permanently change the structure of marine ecosystems. For example, the introduction and mass spread of the predatory East American comb jelly Mnemiopsis leidyi significantly impacted the local zooplankton communities and led to the collapse of the anchovy stocks in the Black Sea and the Caspian Sea at the end of the 1980s (Zaitsev and Mamaev, 1997).

Concerns regarding the accuracy of the proposed hybrid optical-acoustic system are valid, particularly in terms of the potential for errors when extrapolating data from the detailed visual range of cameras to the broader range covered by sonar. It is acknowledged that validating the correctness of the algorithm in unconstrained field conditions, as in the current study, poses significant challenges due to the absence of known ground truth. Without a definitive benchmark, the precision of the extrapolation method cannot be unequivocally determined. However, to counteract these concerns and offer a robust validation strategy, a potential scenario has been envisioned. An offshore investigation site, encircled by net barriers, could serve as a controlled yet realistic environment for concrete experimental setups. This approach would allow for the precise enumeration and identification of marine species within the enclosed area, thereby providing a verifiable ground truth against which the accuracy of the hybrid system can be rigorously tested. Such a setup not only addresses the critical need for validation in the face of inherent uncertainties but also underscores our commitment to refining the system’s reliability and efficacy in real-world applications.

Many marine habitats and their biodiversity are under threat of climate change, discharges of pollutants, overfishing, habitat destruction, and invasive species (e.g., Halpern et al., 2008). Since the conservation and management of biodiversity requires knowledge of its components, status and trends, the temporally and spatially high-resolution UFO time series on the occurrence of individual key species can greatly improve the understanding of the status and trends of biodiversity components. This high-resolution UFO concept allows to monitor and analyse the development of marine biodiversity and the impact of environmental factors. With this novel observatory, we will not only focus on commercial but explicitly include non-commercial fish and other marine species in coastal areas, which are generally overlooked, even though they are known to play important roles in the ecosystem and also because they directly interact with commercial species, either as prey or predators. The species spectrum captured by the stationary UFO ranged from jellyfish, comb jellies, crustaceans and cephalopods to pelagic and demersal fish species, diving seabirds and marine mammals such as seals, porpoises and dolphins (Figure 10). In addition, biodiversity (Descriptor 1) is listed as one of the eleven descriptors of the Marine Strategy Framework Directive (MSFD).

Figure 10

Figure 10. Range of visually observed marine species.

The UFO also enables the identification and monitoring of non-indigenous species (Descriptor 2) in the Baltic Sea: e.g., the ctenophore Mnemiopsis leidyi or the round goby Neogobius melanostomus. The UFO can also be used as an early warning system for the appearance of invasive species like comb jelly (Mnemiopsis leidyi) or endangered species like the harbour porpoises and thus substantially diminish bycatch for instance of small cetaceans. Descriptor 3 specifically addresses the impact of fishing activities on target species, and the objective of Good Environmental Status (GES) is achieved when “Populations of all commercially exploited fish and shellfish are within safe biological limits and have an age and size distribution indicative of a healthy stock”. As our hybrid algorithm has been trained to recognize, identify and quantify different fish species, and at the same time to calculate and classify the lengths of their individuals, the size distribution of commercially exploited fish stocks could be used to generate data for D3. Descriptor 5 is defined as “Eutrophication is minimised: Human-induced eutrophication should be minimized, especially in its adverse effects thereof, such as losses in biodiversity, ecosystem degradation, harmful algal blooms and oxygen deficiency in bottom waters”. The abiotic sensor setup of the stationary UFO allows Chl-a, turbidity and oxygen detection. Descriptor 7 focuses on permanently altered hydrographic conditions resulting mainly from coastal activities that cause topographic changes (e.g. land reclamation, dams, sea defences) and coastal and offshore infrastructure (e.g., ports, wind farms, oil rigs, pipelines, heat and brine discharges). The UFO employs CTD (Conductivity, Temperature, Depth) and ADCP (Acoustic Doppler Current Profiler) to automatically observe and measure the hydrographical conditions (e.g., changes in currents, salinity, temperature). Hence, the UFO concept addresses a wide range of MSFD descriptors. With the addition of acoustic sensors, the UFO could contribute to monitoring ambient noise levels in the ocean, addressing MSFD Descriptor 11 defined as “Introduction of energy, including underwater noise, is at levels that do not adversely affect the marine environment”. This would be particularly relevant for assessing the impacts of shipping, construction, and other anthropogenic activities on marine life, especially on sensitive species such as cetaceans.

5 Conclusions

“UFOTriNet” enables a continuous long-term monitoring of different fish stocks and their habitats at regional hotspots. Our suggested technology provides methods of monitoring fish resources that cause no mortality of fish and/or other organisms. We are able to observe and identify individuals and schools of fish in relation to natural or anthropogenic changes in their environment. Thus, the biotic and abiotic time series derived from the UFO contribute to the assessment of climate-induced changes to fish stocks. Due to the outstanding ecological and economic importance of coastal marine areas, there will be considerable scientific and social interest in the longer term in being able to better predict future developments in these regions in order to identify possible tipping points or state changes in this ecosystem and to develop options for action.

Modularity, compactness, structural integrity and multiple deployment options are key advantages of the trilateral UFO array. The most important difference of the UFO concept compared to all other systems worldwide is that the UFO platform will be available in three different variants, including a stationary, a portable and a mobile UFO, all three modularly equipped with the same core set of opto-acoustic sensors. The portable UFO is a miniaturized version of the stationary UFO. While the stationary UFO will enable continuous (high resolution) fixed point measurements and control of key areas (sensitive hotspots such as marine sanctuaries, spawning, nursery or recruitment areas, wind farms, etc.), the portable applications are intended for stationary use in areas where a rather flexible small unit is required (for example, where it tethers to bridge pylons, rocks, lighthouses, fishing nets, etc.). In contrast, the use of mobile systems is mission-based. The mobile UFO allows for spatial upscaling or expansion as well as flexible application in space and time, e.g., between fixed points or as accompanying support for fishing vessels or research ships.

“UFOTriNet” represents a significant technological and methodological advancement in marine science, offering a model for future marine monitoring systems worldwide. By integrating opto-acoustic sensors across stationary, mobile, and portable platforms, it facilitates a more nuanced understanding of marine environments and their inhabitants. This innovation not only enriches scientific research but also enhances educational opportunities, fostering a deeper appreciation and understanding of marine ecosystems among the scientific community and the public alike. The comprehensive data gathered by the “UFOTriNet” system can significantly inform policy-making and regulatory frameworks aimed at sustainable fisheries management and marine conservation. The insights gained from continuous monitoring can help in the development of more effective, science-based management practices and policies that ensure the long-term sustainability of fish stocks and their habitats, aligning with international conservation goals and agreements.

In summary, our future vision is the idea of a heterogeneously distributed but integrated network or array of stationary coastal UFOs interfacing and interactively communicating with mobile UFOs plus other external buoys and observatory platforms, thus collecting harmonised (standardised) data from our target species and key areas. We see this as part of an overarching Decision Support System for the sustainable management of exploited ecosystems and the implementation of the MSFD.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

Ethical approval was not required for the study involving animals in accordance with the local legislation and institutional requirements because the study was purely observational.

Author contributions

JG: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Writing – original draft, Writing – review & editing. BC: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing. SB: Investigation, Resources, Supervision, Writing – review & editing. GB: Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Writing – review & editing. KB: Investigation, Project administration, Writing – review & editing. CC: Investigation, Resources, Supervision, Writing – review & editing. AC: Investigation, Writing – review & editing. VD: Methodology, Resources, Supervision, Writing – review & editing. PH: Investigation, Methodology, Validation, Writing – review & editing, Formal analysis. AL: Data curation, Formal analysis, Investigation, Resources, Supervision, Writing – review & editing. SM: Investigation, Writing – review & editing. HM: Data curation, Writing – review & editing. FM: Data curation, Investigation, Validation, Writing – review & editing. HR: Investigation, Resources, Supervision, Writing – review & editing. HS: Investigation, Resources, Supervision, Writing – review & editing. TS: Investigation, Methodology, Validation, Writing – review & editing. JoW: Data curation, Formal analysis, Investigation, Methodology, Validation, Writing – review & editing. TW: Data curation, Formal analysis, Investigation, Methodology, Validation, Writing – review & editing. JuW: Formal analysis, Investigation, Methodology, Validation, Writing – review & editing. DW: Methodology, Software, Writing – review & editing. OZ: Methodology, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was funded as part of the project “UFOTriNet” being supported by the German Federal Ministry of Food and Agriculture (BMEL) based on a decision of the Parliament of the Federal Republic of Germany via the Federal Office for Agriculture and Food (BLE) under the innovation support programme, grant number 2819111518. The UFO system and concept has been patented under the registration code DE 10 2018 217 164 B4 (GEOMAR, Gröger).

Acknowledgments

We are grateful to Yvonne Rößner and Sophie Bodenstein (Aquakulturgesellschaft Ostseeforelle, Kiel, Germany) for their support of the UFOTriNet project by providing electric power, accommodations for the electronic equipment and assistance with deployment and recovery of the UFO and the experimental equipment. Many thanks also to Marko Knaup (Ostsee Info-Center Eckernförde) and Christian Hüttner (Schleitaucher) for their technical and logistic support in Eckernförde as well as the Research Divers from Kiel University and the fisher Björn Fischer of Möltenort for their support and good collaboration. We also thank the captain and the crew of the FRV Littorina as well as the divers from the Kiel University diving group for their excellent work during the maintenance missions. We would also like to thank the reviewers and the editor, as their comments provided valuable contributions to improve the manuscript.

Conflict of interest

Authors VD, DW and OZ were employed by MacArtney Germany GmbH. Author AC was employed by company Phi-Stone AG.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

^ www.eumofa.eu.
^ CFP, EU Regulation 1380/2013/EU, http://eur-lex.europa.eu/legal-content/EN/TXT/?uri= celex%3A32013R1380.
^ EC Directive 2008/56/ EC, https://eur-lex.europa.eu/eli/dir/2008/56/oj.
^ See the 2024 ICES Report of the Baltic Fisheries Assessment Working Group under ices-library.figshare.com/articles/report/Baltic_Fisheries_Assessment_Working_Group_WGBFAS_/23123768.

References

Aguzzi J., Chatzievangelou D., Company J. B., Thomsen L., Marini S., Bonofiglio F., et al. (2020). The potential of video imagery from worldwide cabled observatory networks to provide information supporting fish-stock and biodiversity assessment. ICES J. Mar. Sci. 77, 2396–2410. doi: 10.1093/icesjms/fsaa169