A Review of Holography in the Aquatic Sciences: In situ Characterization of Particles, Plankton, and Small Scale Biophysical Interactions

Nayak, Aditya R.; Malkiel, Ed; McFarland, Malcolm N.; Twardowski, Michael S.; Sullivan, James M.

doi:10.3389/fmars.2020.572147

REVIEW article

Front. Mar. Sci., 22 January 2021

Sec. Marine Ecosystem Ecology

Volume 7 - 2020 | https://doi.org/10.3389/fmars.2020.572147

This article is part of the Research Topic Small Scale Spatial and Temporal Patterns in Particles, Plankton, and Other Organisms View all 24 articles

A Review of Holography in the Aquatic Sciences: In situ Characterization of Particles, Plankton, and Small Scale Biophysical Interactions

$\r\nAditya R. Nayak,*$ Aditya R. Nayak^1,2*

Ed Malkiel²

Malcolm N. McFarland²

Michael S. Twardowski^1,2

James M. Sullivan²

¹Department of Ocean and Mechanical Engineering, Florida Atlantic University, Boca Raton, FL, United States
²Harbor Branch Oceanographic Institute, Florida Atlantic University, Fort Pierce, FL, United States

The characterization of particle and plankton populations, as well as microscale biophysical interactions, is critical to several important research areas in oceanography and limnology. A growing number of aquatic researchers are turning to holography as a tool of choice to quantify particle fields in diverse environments, including but not limited to, studies on particle orientation, thin layers, phytoplankton blooms, and zooplankton distributions and behavior. Holography provides a non-intrusive, free-stream approach to imaging and characterizing aquatic particles, organisms, and behavior in situ at high resolution through a 3-D sampling volume. Compared to other imaging techniques, e.g., flow cytometry, much larger volumes of water can be processed over the same duration, resolving particle sizes ranging from a few microns to a few centimeters. Modern holographic imaging systems are compact enough to be deployed through various modes, including profiling/towed platforms, buoys, gliders, long-term observatories, or benthic landers. Limitations of the technique include the data-intensive hologram acquisition process, computationally expensive image reconstruction, and coherent noise associated with the holograms that can make post-processing challenging. However, continued processing refinements, rapid advancements in computing power, and development of powerful machine learning algorithms for particle/organism classification are paving the way for holography to be used ubiquitously across different disciplines in the aquatic sciences. This review aims to provide a comprehensive overview of holography in the context of aquatic studies, including historical developments, prior research applications, as well as advantages and limitations of the technique. Ongoing technological developments that can facilitate larger employment of this technique toward in situ measurements in the future, as well as potential applications in emerging research areas in the aquatic sciences are also discussed.

Introduction

Since the introduction of holography to the scientific community in a pair of seminal papers more than seven decades ago (Gabor, 1948, 1949), it has become an indispensable technique used across a range of disciplines, including but not limited to, environmental and applied fluid mechanics, interferometric studies, metrology, and medical imaging. While the utility of holography in oceanographic and limnological field research has been evident since the 1970s, the last decade and a half has brought significant technological advances and an increase in off-the-shelf availability of holographic imaging systems, leading to a surge in its use by the aquatic sciences community.

This review endeavors to provide a brief background on the history of in situ holographic instrumentation in the aquatic sciences, predominantly focusing on the advances made since the advent of digital holography at the turn of the millennium. While a brief introduction to the technique and processing methods is provided, an exhaustive discussion is beyond the scope of this review and can be found elsewhere (Schnars and Jüptner, 2002, 2005; Katz and Sheng, 2010). An overview of the diverse applications of in situ holography that demonstrate its versatility is provided. Finally, existing limitations, scope for further development of the technique to enable widespread utility in the aquatic sciences community, and future avenues of applications in emerging research areas are discussed. For additional information about the development of holography, the readers are referred to other publications (Watson, 2005, 2011), where an overview of the technique, with a focus on different systems developed at the University of Aberdeen over the decades is provided. While these reviews highlight developments in holography for aquatic applications prior to 2011, their main focus is on the system design and processing approaches. Furthermore, marked improvements in the technology and the development of several commercial holographic systems have led to its increased use in the last decade for a variety of scientific applications, which will be covered here. Lab-based studies (e.g., Sheng et al., 2007; Hong et al., 2012) as well as flow through based approaches (e.g., Yourassowsky and Dubois, 2014; Zetsche et al., 2014) characterizing aquatic particles/plankton find a fleeting mention but are generally out of the purview of this article. Hologrammetry used as a tool for underwater inspections of offshore installations (e.g., Foster and Watson, 1997) is also not covered here. Historically speaking, holography in the aquatic sciences has been predominantly used for marine applications, and this fact is reflected in the material covered in the review. While all these applications are easily transferable to limnological research, prior field studies in lake environments have been scarce and are only touched upon briefly.

Aquatic Particles and in situ Measurement Techniques

A diverse range of planktonic species, marine snow, detrital matter, sediments, microplastics, and fecal pellets constitute the particle population in the world’s water bodies (Lal, 1977; Turner, 2015). Together, these particles and their interactions with the local environment, affect a range of processes across vast spatial and temporal scales, with consequence to marine ecology, human health, climate change, coastal engineering, ocean optics, acoustics, and remote sensing. For example, healthy phytoplankton and zooplankton populations are critical to sustaining aquatic ecosystems (Tett et al., 2008); conversely, abnormal and drastic increases in plankton abundance can lead to massive harmful algal blooms (HABs), which can severely impact aquatic ecosystems, human health and local economies (Anderson et al., 2002; Carmichael and Boyer, 2016; Glibert et al., 2018). Under favorable conditions, plankton can accumulate in enhanced concentrations to form “thin layers,” which can enhance foraging success rates for fish and other species (Sullivan et al., 2010a). The presence (or absence) of particles, and their composition, concentration and orientation, can alter optical and acoustic propagation through the ocean (Sullivan et al., 2005; Holliday et al., 2009; Basterretxea et al., 2020). Sinking particles are major pathways of carbon transport into the ocean’s interior and important to understanding the ocean carbon budget, and consequently, climate change research (Turner, 2015; Briggs et al., 2020). Sediment suspension and transport influence key coastal processes (Conley et al., 2012). Thus, developing and utilizing methods and instrumentation to quantify particle composition, distribution and particle-flow interactions have been an important focus of the aquatic sciences community for decades.

Among these methods, in situ imaging provides a direct way to record particle characteristics in their natural environment. A more comprehensive review on the field of underwater optical imaging can be found elsewhere (Dahms and Hwang, 2010; Jaffe, 2014). To date, different imaging techniques used by aquatic scientists include, but are not limited to, lidar imaging (Churnside and Wilson, 2004; Busck, 2005; McKenzie et al., 2020), bulk and planar laser-induced fluorescence (Prairie et al., 2011; Jaffe et al., 2013), laser sheet reflective imaging, e.g., Laser Optical Plankton Recorder (Herman et al., 2004; Checkley et al., 2008), underwater light microscopy (Mullen et al., 2016), imaging-in-flow cytometry (Olson and Sosik, 2007), silhouette photography (Milligan, 1996), and holography (Katz et al., 1999; Watson, 2011). Among these, holography can provide 3-D spatial distributions of particles in the micron to centimeter range, within a freestream sample volume, thus enabling mapping of particle characteristics at high resolution without fragmenting them. Furthermore, as opposed to methods like flow cytometry, much larger volumes can be sampled.

Broadly speaking, the applications of holography toward particle characterization can provide the following information: (a) Particle counts and size distributions; (b) Particle shape metrics, including cross-sectional area, major and minor axis lengths, and aspect ratio; (c) 3-D spatial structure of the particle field, e.g., nearest neighbor distances (NNDs) and particle orientation; (d) Particle identification and classification; (e) Particle interactions, including aggregation, settling speed, predator-prey behavior, and organism swimming trajectories; and (f) 3-D velocity distributions by integrating particle image velocimetry with holography (HPIV).

Holography

Briefly, holography involves the recording of an interference pattern (hologram) of the diffracted light field from particles in a given volume illuminated by a coherent light source (e.g., laser) and the reference beam (Schnars and Jüptner, 2002; Katz and Sheng, 2010). The hologram encapsulates the information about the phase and intensity of the diffracted light field. Direct visual observations of a recorded hologram would typically not produce any useful information, unless a particle is very close to the hologram plane; rather, optical or numerical reconstruction techniques are required to elicit information about in-focus particles in different 2-D cross-sections within the 3-D sample volume. Based on the geometric configuration of the object and reference beams, holography can be further classified as inline and off-axis holography. In the inline configuration, the reference and object beams are parallel to each other. Here, the sample volume is illuminated by a coherent beam of light, and the interference pattern recorded is caused by the portion of the beam scattered by the particles and the undisturbed part of the same beam. In the off-axis configuration, the object and reference beams are at an angle to each other. Inline geometric configurations are relatively simple, especially in the context of field instrumentation design; thus, most (but not all) in situ holographic systems have opted to use the inline setup. Readers are referred to other review articles or texts on holography for a more detailed explanation on different geometrical configurations (Vikram, 1992; Murata and Yasuda, 2000; Pan and Meng, 2003; Katz and Sheng, 2010; Picart and Montresor, 2020).

History of Development of Holography for in situ Aquatic Applications

The history of holographic development and in situ applications for marine sciences can be broadly divided into the pre-digital (e.g., recording on photographic plates) and the digital eras (Figure 1). Two related technological developments (lasers and digital cameras) have led to separate inflection points four decades apart, each leading to a subsequent surge in diversifying scientific applications of holography. While holography was introduced in 1948, it was only in the 1960s, that the emergence of lasers as a source of coherent light led to significant progress and expansion of the scope of holography. To the best of our knowledge, the potential of holography toward oceanographic applications was first demonstrated by Knox (1966), wherein a lab-based inline setup was used to image planktonic organisms (also see Beers et al., 1970). This was later extended to recording high-speed holographic movies of plankton (70 frames per second), where all particles >10 μm were resolved (Knox and Brooks, 1969). Stewart et al. (1973) developed and deployed the first known submersible in situ holographic imaging system, which weighed ∼1000 kg and could sample approximately 100 L of water per hologram. Further modifications to this system followed, with an off-axis configuration developed by Heflinger et al. (1978). Over the next decade and a half, several groups led other early efforts toward developing holographic imaging systems for diverse applications, including plankton characterization, studying cavitation nuclei and particle size, and settling rates in the ocean (Carder, 1979, Carder et al., 1982; Katz et al., 1984; O’Hern et al., 1988; Costello et al., 1989). A remotely operable inline submersible holocamera with an adaptable optical configuration (i.e., capability to switch between in-line and off-axis modes) was developed by Katz et al. (1999), with multiple deployments of the same instrument reported in other studies (Malkiel et al., 1999, 2006). Submersible holocameras which recorded both inline and off-axis holograms simultaneously were reported by Watson et al. (2001) and Hobson and Watson (2002). The readers are also referred to Watson (2005) and Hobson and Watson (2002) for further details on historical film-based holographic imaging systems. While film-based holographic systems used powerful, pulsed lasers that could sample very large volumes of water at high resolution, the systems themselves were bulky, physically unwieldy and challenging to deploy (e.g., Katz et al., 1999). Also, hologram processing was done manually, which is exceedingly cumbersome; for reference, processing one hologram could take several days, an unimaginable scenario in today’s big data world. Malkiel et al. (2004) reported improvements in processing methods with the development of an automated approach; however, this still required 5 h to process a 500 mL sample volume over one hologram. Furthermore, film-based approaches severely constrained the number of holograms that could be recorded, with these data limitations oftentimes leading to a paucity of data to draw conclusions from. For example, Katz et al. (1999) reported the recording of 300 holograms during each deployment of the submersible system. To put this in context of typical sampling rates of 15 Hz in digital holographic systems today, 300 holograms correspond to data recorded across a 20 s time interval. An instrument can record 9,000 holograms over a 10 min vertical profile, albeit with much smaller sample volumes for each hologram as opposed to film-based systems (between 1–10%).

FIGURE 1

Figure 1. A timeline of the historical developments in holography in the context of in situ aquatic applications.

Around the turn of the century, with the development of digital cameras as well as increasing miniaturization of lasers, the second burst in advancement in underwater holographic imaging systems occurred. The first known digital in situ holographic system was developed by Owen and Zozulya (2000), where they employed a 10 mW diode laser to image over a ∼25 cm deep depth of field, resolving particulate sizes down to 5 μm. Jericho et al. (2006) developed a holographic microscope that used point source illumination to record marine organisms in situ. Bochdansky et al. (2013) developed the first deep-sea holographic microscope rated to 6000 m water depth. Other successfully deployed in situ holographic sensors with different optical setups and varying sampling parameters include those by Pfitsch et al. (2005, 2007), Sun et al. (2008), Graham et al. (2012), Talapatra et al. (2012, 2013), Dyomin et al. (2019, 2020), and Nayak et al. (2020). Table 1 provides further details on some selected systems. The last decade has also seen the development of a handful of commercial holographic imaging systems, including Sequoia Scientific’s LISST-HOLO (Davies and Nepstad, 2017; Ouillon, 2018), 4Deep Imaging’s holographic microscope (Rotermund et al., 2016) and the WET Labs HOLOCAM (Moore et al., 2017, 2019; Nayak et al., 2018a, b). These developments leave the scientific community poised to take further advantage of the holographic technique in the near future.

TABLE 1

Table 1. Sampling characteristics of selected free-stream, digital holographic imaging systems reported in literature since 2000.

Holographic Data Processing

Digital holographic data processing can be a computationally expensive process, which is a function of several parameters. Briefly, the image processing steps for each hologram can be sub-divided as follows: (a) Image pre-processing; (b) Hologram reconstruction; (c) 3-D segmentation and/or image plane consolidation; and (d) Particle feature extraction. Pre-processing of raw holograms is required to eliminate any nonuniformities associated with uneven laser beam illumination and dust or other unwanted particles on the windows. Typically, this is achieved by computing the average background intensity calculated from a sequence of holograms and subtracting this from each single one (Talapatra et al., 2013; Nayak et al., 2018a). Alternately, a reference background hologram can be recorded just prior to data acquisition and then subtracted from each of the holograms to generate a “contrast hologram” (Bochdansky et al., 2013). A further pre-processing step could include grayscale histogram equalization (Toloui and Hong, 2015). After suitable pre-processing, the hologram reconstruction is carried out using numerical techniques, usually the Kirchoff-Fresnel transform, which is mathematically described elsewhere (Owen and Zozulya, 2000; Katz and Sheng, 2010). Reconstruction at different depths (or planes) brings all particles in each particular plane in focus over the entire 3-D sample volume. Typically, hologram reconstruction is the most memory intensive step in the entire processing routine. After reconstruction, the task is to isolate and label in-focus particles. 3-D segmentation can be directly implemented on the reconstructed stack to achieve this, providing in-focus locations of particles (Katz and Sheng, 2010). In cases where the particle field is sparse and the objects are compact, the stack can first be consolidated into a single composite image, because the in-focus images do not overlap one another, thus expediting processing. Further operations could include median filtering, image thresholding and segmentation and defining regions of interest (Jericho et al., 2006; Graham and Nimmo Smith, 2010; Bochdansky et al., 2013; Talapatra et al., 2013; Nayak et al., 2018a). All these steps are then repeated for each hologram in the dataset to generate a distribution of particles for further analysis. Figure 2 shows the steps involved in the holographic processing routine. Figure 3 shows a collage of planktonic images from processed holograms collected in diverse environments using the WET Labs HOLOCAM.

FIGURE 2

Figure 2. An illustrative example showing the different steps in holographic data processing: (A) A sample raw unprocessed hologram; (B) The same image after background subtraction, with regions of interest highlighted, and showing only interference patterns; (C) The in-focus diatom chains at different reconstruction depths (z) within the sample volume in the highlighted region; and (D) The 2-D composite image, where all in-focus particles are collapsed onto the same plane for further analysis.

FIGURE 3

Figure 3. A collage of different plankton recorded from diverse environments on field cruises in the Atlantic Ocean (Delaware shelf), the Gulf of Mexico and East Sound, Washington, a coastal fjord in the Pacific Northwest. (A) Ditylum brightwelli; (B) Thalassiosira sp.; (C)Thalassionema sp.; (D) Tripos cf. fusus; (E) Unidentified colony; (F) Tripos cf. furca; (G) Tripos cf. furca, dividing cell; (H) Tripos muelleri; (I) Tripos sp. dividing; (J) Chaetoceros sp.; (K) Chaetoceros cf. debilis; (L) Appendicularian; (M) Coscinodiscus sp.; (N) cf. Strombidium sp.; (O) Calanoid copepod with egg sac; (P) Calanoid copepod; (Q) Cyclopoid copepod with paired egg sacs; and (R) Unidentified large copepod, potentially feeding on cf. Strombidium sp.

Sample Applications of Holography in Aquatic Research

Aquatic Particle Characterization

Particle Size Distributions

The characterization of spatial and temporal particle size distributions (PSDs) is vital to many oceanographic research topics. For example, PSDs are an important parameter in sediment transport studies (Mikkelsen and Pejrup, 2001; Conley et al., 2012). Physical processes such as molecular diffusion, turbulence, and settling, lead to aggregation and fragmentation of particles which are reflected in variations in the PSDs; this is important in the quantification of particle fluxes in the ocean (Jackson et al., 1997; Sullivan et al., 2005). PSDs can also affect light absorption and scattering in the ocean (Ulloa et al., 1994; Jonasz and Fournier, 2007). Phytoplankton functional type variability, of relevance to global biogeochemical models, are assessed using modeled PSDs (Kostadinov et al., 2010; Stemmann and Boss, 2012). While some of the seminal work on oceanic PSDs have employed an in situ imaging system (Underwater Video Profiler), sometimes in conjunction with a shipboard particle counter, to resolve a broad size spectrum (Stemmann et al., 2000, 2002, 2008), PSDs are usually estimated from direct samples collected and processed in the laboratory using instruments such as Coulter counters and flow cytometers (Jackson et al., 1997; Boss et al., 2001; McFarland et al., 2015). Inherent biases exist in inferring PSDs from discrete samples, primarily because particles get fragmented during the collection, sampling, and analysis process, potentially leading to an enhanced concentration of smaller particles. Indirect methods focus on employing models to invert acoustic or optical scattering to infer PSDs (e.g., Sequoia Scientific’s LISST series of instruments), and may require some a priori knowledge of particle shape and scattering properties. Thus, validation by direct imaging is recommended to ensure fidelity in measurements (Graham et al., 2012).

Aquatic PSDs are most commonly described using a simple power law, also referred to as a “Junge type” distribution, because such a relationship is generally representative in diverse environments for smaller particles <100 μm that are important for aquatic optical properties (Kitchen et al., 1982; Buonassissi and Dierssen, 2010). Here, the PSD shape is embodied in a single parameter, i.e., the exponent, which is convenient for modeling and inversions (Fournier and Forand, 1994; Boss et al., 2001; Twardowski et al., 2001). While particles <100 μm control optical properties such as scattering and absorption, larger particles can affect diffraction at very near-forward angles. More complex models such as the two-component gamma (Risović, 1993) have also been developed to describe oceanic PSDs. The wide particle size range that can be mapped by holography (few microns to cm) makes it an attractive technique for direct in situ PSD measurements.

A few studies have compared PSDs obtained from holographic systems and other instruments. For example, O’Hern et al. (1988) evaluated PSDs using holographic and Coulter counter measurements. It was reported that the Coulter counter consistently under-estimated the PSDs, with the authors concluding that this most likely occurred due to assumptions inherent in the Coulter counter calibration procedures. Kumar et al. (2020) compared PSDs estimated using a laser diffraction based approach and holography, with a lab-based setup to characterize spray droplets. Here, it was found that the PSDs were under-estimated in the laser diffraction based approach for droplet sizes higher than 1 mm. Graham et al. (2012) co-deployed a LISST-100X alongside a holographic system, where they reported that in the overlapping size ranges, the LISST-100X showed higher particle concentrations, especially in smaller size bins. They suggest that in particle fields dominated by non-spherical particles, the LISST-100X might report a single large particle as several individual particles, thus skewing the PSD. While the above studies use holography as a standard to compare other instrumentation, a study by Walcutt et al. (2020) is also pertinent to this discussion. Here, a 4Deep holographic microscope was simultaneously used along with a FlowCam, Imaging Flow Cytobot (IFCB), and standard microscope, to compare PSDs. Ocean water samples, lab-grown cultures and microspheres provided diverse datasets to characterize PSDs across these methods. After appropriate post processing corrections, it was found that holographic measurements under-estimated the PSD by about 3–10%. However, the slopes of the distributions remained comparable. The authors list several potential factors, e.g., nonuniform illumination of the sample volume, which might possibly need to be accounted for to gain higher statistical confidence in inferring particle concentrations and distributions from holographic data.

Bochdansky et al. (2016) reported the particle distributions in bathy- and abyssopelagic waters (up to 5,500 m depth) at several stations in the North Atlantic Ocean, using a digital inline holographic microscope. A majority of the particles seen in the holograms consisted of marine snow, fecal pellets, phytoplankton, and other detrital matter. When all other particles excluding marine snow were considered, they fit a typical oceanic PSD, with a systematic decrease in particle counts in the larger size bins. However, when only marine snow was considered, the particle counts across the size bins did not exhibit any specific trends, such as a skew toward the smaller sizes. For all particles, including marine snow, the PSD slope deviated significantly from an expected Junge distribution above a certain size threshold (∼380 μm). The authors termed particles that were significantly more frequent than expected from the frequency distribution of smaller particles as “dragon kings.” Aggregation is an important process that can skew PSDs and cause deviations from a simple power law (Boss et al., 2009; Hill et al., 2011).

Similar trends in deviations of the PSDs from expected profiles have been reported by other studies in diverse aquatic environments. Nayak et al. (2018a) analyzed the small-scale PSD variations with depth in a coastal fjord, where a holographic system was slowly profiled through a ∼25 m deep water column. Here, the particle field was dominated by diatoms, and size was represented by considering the equivalent size diameter ( $ESD = \sqrt{4 A / π}$ , where A represents the measured area of the particle). At shallow depths, a broad peak in the PSD was observed, centered around 300 μm, which corresponded to an enhanced abundance in diatom chains in that size range (Figure 4A). Across stations and depths, PSD slopes presented two linear ranges, with a shallower slope (−1.7 to −1.9) fitting the 50–250 μm range. For particles above 250 μm, slopes were much steeper (−5.7 to −6.1). Moore et al. (2017) reported the deployment of a holographic system to characterize PSDs across different stations in western Lake Erie during a cyanobacterial bloom. In locations dominated by detrital particles, the mean PSD slopes were ∼−3.45 across depths. At stations where the particle field was dominated by the cyanobacteria Microcystis aeruginosa which forms amorphous colonies, the PSDs show a pronounced bump, broadly between 200 and 500 μm, with overall mean slope ∼−2.3 (Figure 4B). Modeling studies characterizing colony size and associated vertical migrations have previously shown that a colony ∼250 μm in size corresponded to the highest migration ranges (Visser et al., 1997). Finally, at stations dominated by a second, morphologically distinct (elongate) cyanobacteria, Planktothrix aghardii, similar PSD slopes to the M. aeruginosa dominant waters were observed, albeit with the absence of the bumps (Figure 4C). These different studies in diverse conditions demonstrate the feasibility of using holography to characterize fine-scale PSDs in the water column and highlight trends which other methods might miss. It should be noted that while oceanic bubbles do not fall under the general ambit of particles being discussed in most of this paper, there is a substantial community focusing research efforts in this direction. Acoustics and optics remain the preferred method of characterization for large and small bubbles, respectively (Czerski et al., 2011; Twardowski et al., 2012). Holography has been successfully demonstrated to be applicable to in situ bubble research by at least two previous studies (O’Hern et al., 1988; Talapatra et al., 2012).

FIGURE 4

Figure 4. Depth-binned and depth-averaged particle size distributions (PSDs) obtained from vertical profiles of a holographic imaging system during two field experiments at East Sound, Washington, in September 2015 and at Lake Erie in August 2014. Figure adapted from data presented in Moore et al. (2017) and Nayak et al. (2018a; 2018b). (A) The East Sound profile was recorded during the presence of a “thin layer” of diatoms, dominated by Ditylum brightwelli; (B) Data recorded during a Microcystis aeruginosa bloom in western Lake Erie; and (C) PSDs at Lake Erie in a water mass dominated by the cyanobacteria Planktothrix aghardii.

Particle Settling Velocities

Particle size and density directly affects sinking rates and transport in the water column (McDonnell and Buesseler, 2010; Bach et al., 2012). In the open ocean, particle sinking is an important pathway for carbon sequestration to the ocean’s interior (Trull et al., 2008; Fischer and Karakas, 2009; Nayak and Twardowski, 2020), while settling rates are critical to inform coastal studies involving sediment suspension and transport (Agrawal and Pottsmith, 2000). Carder et al. (1982) integrated a holographic setup into a free-floating sediment trap to quantify settling rates of individual particles in the water column, with holograms recorded at discrete intervals. The processed data provided size distributions of particles >15 μm. Some of the faster sinking particles were segregated to compute attributes: observed particle densities ranged from 1370–5100 kg/m³, with associated settling velocities corresponding to daily excursions of 16–199 m across the particle spectrum. This was the first demonstration of holography for oceanic particle settling analysis, with results corroborated from microscopic analysis of the particles collected in the sediment trap. Graham and Nimmo Smith (2010) demonstrated the applicability of holography to study suspended sediment size distributions and settling velocities in the coastal ocean. Automated tracking software, incorporating parameters such as particle area and major and minor axis lengths was used to identify a particle across different frames, thus facilitating computation of trajectories and consequently velocities. Cross et al. (2013) studied particle resuspension in the context of the relation between turbulence and particle size. Holographic data was used to identify suspended particle matter and characterize variations in their size distribution over different tidal cycles. These data were compared to turbulence measurements obtained from a microstructure profiler; based on their observations, the authors postulated that the growth and size of flocs are dependent not only on turbulence, but also biological controls.

Particle Orientation

In atmospheric studies, preferential (or non-random) orientation of particles and ice crystals in clouds has been relatively well-documented (Ono, 1969; Sassen and Takano, 2000; Noel and Chepfer, 2004). Only fairly recently has this area of research gained traction in oceanography. The consequences of particle orientation are significant to optical oceanographers as most models assume that particles are randomly oriented in the ocean (Bohren and Huffman, 1983; Jonasz and Fournier, 2007). In aquatic ecology, preferential horizontal orientation of phytoplankton chains could potentially increase light capture and thus enhance primary productivity (Sun et al., 2016; McFarland et al., 2020). Several laboratory and modeling studies have shown that particle and plankton orientation is a function of the small-scale flow physics (Karp-Boss and Jumars, 1998; Karp-Boss et al., 2000; Marcos et al., 2011) and influences light propagation (Marcos et al., 2011; McFarland et al., 2020). Basterretxea et al. (2020) provide a useful overview of phytoplankton orientation in the ocean, laying it in the context of microscale plankton-flow interactions. While recent innovative approaches at inferring in situ particle orientation from indirect scattering measurements have been reported (Font-Muñoz et al., 2019, 2020), direct imaging still provides the best means to do so.

Most reports to date on particle orientation in aquatic environments have used holographic systems. To the best of our knowledge, the first, albeit limited, direct observations on non-random particle orientation in the ocean were reported by Malkiel et al. (1999). From a sample of 300 holograms recorded during a vertical scan of the water column, the majority of diatom chains present (Chaetoceros spp. and Ditylum sp.) at a certain depth (15.3 m) were found to be preferentially oriented within ±15° to the horizontal. Talapatra et al. (2013) documented the occurrence of enhanced preferential horizontal orientation of colonial diatom species (primarily Chaetoceros debilis and Chaetoceros radicans) at several distinct depths in two vertical profiles. A pulsed laser acting as the illumination source allowed for particles to be tracked over successive frames, enabling co-located small-scale flow and turbulence measurements, which indicated that regions of enhanced preferential particle concentrations coincided with regions of low turbulent kinetic energy (TKE) dissipation. A comprehensive set of measurements of oceanic particle orientation were later provided by Nayak et al. (2018a). A bio-optical profiling package consisting of a holographic system, an acoustic Doppler velocimeter, and a CTD, among other instruments, was gently profiled through the water column at several stations to simultaneously quantify particle orientation, small-scale shear, and TKE dissipation rates with depth. Across all stations, preferential horizontal orientation was observed at several depths, each time corresponding to regions of low shear and TKE dissipation rates. Further analysis highlighted the relation between instantaneous orientation of a particle and its aspect ratio (ratio of the major axis length to the minor axis length of a particle). In simple 2-D laminar shear flow, Jeffrey (1922) derived a set of relations for spheroidal particles, showing that they exhibit periodic flipping motions, where higher aspect ratio particles spend more time oriented in the horizontal direction; in situ data presented in Nayak et al. (2018a) were in reasonable agreement with Jeffrey’s theory.

Plankton Distributions and Patchiness

In situ observations of plankton distributions, behavior and interactions with the natural environment provide valuable insights into the functioning of aquatic ecosystems. It has been well-established that plankton distributions in aquatic systems tend to be incredibly “patchy,” i.e., non-homogeneous (Durham and Stocker, 2012; McManus and Woodson, 2012; Prairie et al., 2012; Graff and Menden-Deuer, 2016). There exists a plethora of possible mechanisms for the formation of these patches across different spatial scales, including small-scale shear, horizontal and vertical mixing, and enhanced nutrient availability (Powell and Okubo, 1994; Abraham, 1998; Sullivan et al., 2010a; Breier et al., 2018). Examples of plankton patchiness include algal blooms (harmful or otherwise) and phytoplankton and zooplankton “thin layers,” which are discussed further below.

Phytoplankton Blooms and Vertical Distributions

Large phytoplankton blooms covering several hundreds of kilometers occur annually in different gyres, sometimes tied with the formation and transport of mesoscale eddies (Guidi et al., 2012). Anderson et al. (2018) deployed Sequoia Scientific’s LISST-HOLO on a neutrally buoyant towed body from an autonomous vehicle to characterize a summer phytoplankton bloom in the North Pacific subtropical gyre, which was dominated by several species of colonial diatoms. Holographic images were processed to obtain cell and aggregate counts of Hemiaulus and Rhizosolenia sp. over the course of the campaign. These observations showed that the overall bloom was comprized of two distinct phases, where either species dominated; simultaneous large abundances of both taxa seldom occurred. Similarly, Bochdansky et al. (2017) reported the particle composition in the Ross Sea in Antarctica, where massive phytoplankton blooms occur seasonally, connecting the distribution of important phytoplankton species to carbon export. In particular, large scale sinking of Phaeocystis antarctica colonies from the surface to the deep layer below the pycnocline was noted. These observations were made during the weakening of the pycnocline, thus establishing the significant contribution of Phaeocystis colonies to total carbon export, even during the non-bloom season.

In a limnological study, Moore et al. (2017, 2019) used holography to characterize phytoplankton community composition during a HAB event in western Lake Erie, where the particle field was dominated by two cyanobacterial species, M. aeruginosa and P. aghardii. The profiling system enabled characterization of vertical distributions of colony and cell counts of either species. A majority of Microcystis colonies were found to be located near the surface, while Planktothrix tended to be located in larger numbers deeper in the water column, and the vertical variability was correlated to the light availability and tolerance of either species. All these studies further highlight the ability of in situ holography (and direct imaging in general) to tease out small-scale patterns and variations between distinct particle and plankton populations which could help scientists to better understand ecosystem dynamics as well as inform robust modeling efforts.

Thin Layers and Phytoplankton in Turbulence

The term “thin layers” refers to vertically limited (few cm to meters) dense aggregations of plankton which are temporally coherent, i.e., last for several hours to days (Sullivan et al., 2010a; Durham and Stocker, 2012). Manual sample collections at discrete depths oftentimes miss these finescale structures in the water column; high frequency in situ observations provide the best means to characterize them (Donaghay et al., 1992; Twardowski et al., 1999; Rines et al., 2002). Previous field studies have shown that thin layers are frequently present in coastal waters, with their vertical location controlled by the physical structure and/or vertical migration of the organisms (Sullivan et al., 2010a). For a comprehensive study of thin layers, the readers are directed to Sullivan et al. (2010b) and articles within that special issue.

In a constrained coastal fjord at East Sound, Washington, where thin layers have been previously well-documented (Dekshenieks et al., 2001; McManus et al., 2003), Talapatra et al. (2013) deployed a holographic system to characterize the small-scale biophysical interactions and particle populations. A thin layer comprising of small, non-motile particles as well as larger colonies of the diatom Chaetoceros socialis were observed. Associated CTD profiles indicated that this layer was present at the base of a strong pycnocline. Zooplankton distributions showed that they seemed to avoid the thin layer. At the same geographical location, other studies reported the presence of a strong thin layer, predominantly containing the diatom Ditylum brightwellii, again coincident with the location of a near-surface pycnocline (Figure 5; Nayak et al., 2018a, b; McFarland et al., 2020). In all these reports, the water column was stably stratified, and the presence of the thin layer was associated with regions of low shear and TKE dissipation.

FIGURE 5

Figure 5. Holographic data acquired on September 22, 2015 during a vertical profile through a diatom thin layer at East Sound, Washington. Background subtracted holograms showing the representative particle fields (A) above the thin layer at 1.58 m depth; and (B) within the thin layer at 3.12 m depth; (C) Chl a and density profiles acquired during the same profile, showing the thin layer occurred in a stably stratified region within the pycnocline; and (D) Particle distribution with depth, where only particles >200 μm in length and aspect ratio >5 are considered, consisting of different elongate diatom species, and dominated primarily by Ditylum brightwelli. Figure generated from a dataset presented in Nayak et al. (2018a; 2018b).

Turbulence has long been known to play an important role in predator-prey encounters, enhancing phytoplankton nutrient uptake, as well as affecting growth rates (Karp-Boss et al., 1996; Sullivan and Swift, 2003; Durham et al., 2013); thus, there is a lot of interest in simultaneously quantifying plankton populations and turbulence in situ. Cross et al. (2014) used integrated holographic imaging and microstructure turbulence profiler measurements to study the effect of episodic and enhanced turbulent mixing on vertical distributions of colonial diatoms. Although limited to a single set of observations over a tidal cycle, enhanced mixing seemed to correlate with shorter chain lengths, with diatoms being transported to lower depths, where they were found to aggregate with other suspended matter. Nayak et al. (2018a; 2018b), albeit qualitatively, also report similar observations with chain lengths, demonstrating the importance of slowly profiling the water column with large instrument packages used in particle characterization studies. Vigorous mixing during fast profiles can lead to bias in observations, with enhanced concentrations of particles at lower sizes, possibly due to breakage of chains due to enhanced turbulence.

Nearest Neighbor Distances

Yet another interesting application of holography is the ability to characterize instantaneous 3-D spatial distributions across each individual hologram in a time series, thus allowing computation of vital statistics such as NNDs. These parameters allow for quantification of predator-prey interactions and population dynamics in the local community, which is of great interest to biological oceanographers and ecological modelers.

Malkiel et al. (1999, 2006) computed NNDs between particles detected in holograms by employing two-tailed hypothesis testing (Freedman et al., 2007). Qualitatively, differences were represented by looking at the histogram of NNDs of the sample and a random distribution. Quantitatively, a “z-score” was computed as $z = \frac{\bar{x} - {\bar{x}}_{R}}{σ_{R} / N^{1 / 2}},$ where $\bar{x}$ and N are the mean NND and the size of the sample, respectively, while ${\bar{x}}_{R}$ and σ_R represent the mean and standard deviation of the NND obtained by a random distribution. If |z| > 2, the sample data are deemed different from a random distribution. Using this method, statistics were computed to study NNDs between individual particles (or plankton) within the same class, as well as different classes, to come up with the following observations: (a) Detrital particles exhibited clustering at certain depths below the pycnocline, while appearing to be randomly distributed elsewhere; (b) No significant clustering was found among dinoflagellates; and (c) No significant correlation between the location of copepods with respect to small particle clusters were detected. To the best of our knowledge, these are the only in situ holographic studies characterizing NNDs to date; further observations in different environments to better understand population dynamics are warranted.

Bacterial and Zooplankton Behavioral Studies

Holographic cinematography provides a way to visualize 3-D trajectories of multiple motile organisms, including zooplankton, simultaneously, as well as observe their natural behavior in situ. While this ability has been leveraged by several laboratory studies, far too many to list here in detail (e.g., Sheng et al., 2007; Hong et al., 2012), very few studies report this in the field. During deployment in a Lagrangian, drifting mode, Pfitsch et al. (2005) observed the behavior of swimming medusa jellyfish in situ, in the Ria de Pontevedra in coastal Spain. The same authors also recorded the swimming behavior of an appendicularian (Pfitsch et al., 2007). Appendicularians (or larvaceans) are filter feeders that generate a mucous “house” around them to trap particles. The long tail, flapping at a particular frequency, helps pump water through the house and facilitates their feeding (Alldredge, 1981; Selander and Tiselius, 2003). In this instance, the flapping frequency of the tail was found to be in the range of those observed in controlled laboratory measurements (Pfitsch et al., 2007). Other observations with the same instrument included interactions between copepods and dinoflagellates, which can provide pertinent data to modelers such as perception distances by predator/prey species in the turbulent environment. These observations include instances of copepods initiating jumps and orienting their feeding currents after prey and dinoflagellate behavioral changes (jumps and quick spiraling) to avoid a cyclopoid copepod that was cruising through their neighborhood (Malkiel et al., 2007). Jericho et al. (2006) also reported observations of the trajectory of organisms during a field deployment; however, the organisms were not clearly identifiable as they were at the edge of the detection limit of the instrument.

While not a free stream approach, Lindensmith et al. (2016) developed a flow-through based off-axis submersible holographic microscope, geared toward studying microbial motility in extreme, icy environments. During a deployment in Greenland over winter, brine samples were collected and inserted into the sampling chamber of the microscope, to characterize microbial motility. Analyzed samples recorded both prokaryotes and eukaryotes; motility was observed in only a few instances in either case. Swimming trajectories clearly recorded the motion of organisms, at swimming rates varying from 5–50 μm/s between different individuals. This highlights the ability of holographic microscopy for use in extreme environments, including in potential future extraterrestrial missions (Bedrossian et al., 2017).

Discussion

The previous sections have shown the applicability of holography toward diverse research areas in the aquatic sciences. Here, existing limitations, future potential applications and advancements of holographic technology in emergent aquatic research areas are outlined.

Limitations

The main limitation associated with holography is the depth-of-focus (DOF) problem, which refers to the depth over which a particle in the image tends to appear in focus in the axial direction (Vikram, 1992; Katz et al., 1999; Katz and Sheng, 2010). The DOF can be approximated based on the smallest particle diameter (d) resolvable and the wavelength of light used (λ), being proportional to d²/λ (Yang et al., 2005; Katz and Sheng, 2010). For example, an optical system capable of resolving particles sized 10 and 50 μm and above, will have a depth of focus of ∼0.16 and 4.17 mm, respectively, assuming the illumination source is red light (660 nm). Thus, while the particle location in the image plane can be detected fairly accurately, there is a certain degree of uncertainty associated with detecting the axial location (Gao et al., 2013).

The quality of recorded holograms (as with other optical methods) also depends, to a certain extent, on the turbidity and particle size and abundance within the water column. In-line systems in particular are susceptible to loss of image quality for particle concentrations beyond certain thresholds (Katz et al., 1999). Furthermore, the presence of a large number of sub-micron particles below the resolution limit of typical holographic systems can lead to loss of coherency in the reference laser beam, thus leading to bad quality recordings. A related and important problem in holography is the inherent coherent noise, which includes (a) parasitic interference fringes, caused by multiple reflections or scattering (Pan et al., 2017); and (b) speckle noise, resulting from scattering by particles larger than the wavelength of the laser in dense particle fields (Meng et al., 1993; Garcia-Sucerquia, 2013). This can lead to serious degradation in image quality and holographic reconstructions. Typically, these noise reduction techniques involve engineering the coherent source (Garcia-Sucerquia, 2013; Pan et al., 2017), or optimizing numerical reconstruction schemes (e.g., Kosmeier et al., 2012; Leo et al., 2014). While a detailed discussion is beyond the scope, readers are also referred to Bianco et al. (2018), where different approaches to tackle the speckle noise problem in digital holography are reviewed.

The data-intensive nature of hologram acquisition and the extensive processing routines involved have so far limited the use of in situ holographic systems in long-term and/or real-time monitoring networks. For near real-time observations, data transmission is one of the main bottlenecks due to the large file sizes; rapid onboard processing to provide simplified data (e.g., organism presence and counts) in a compressed format, instead of transmitting entire images could help alleviate this problem. For this to work, is it key to develop not only fast and efficient automated classification algorithms, but also those feasible to be developed in a light-weight architecture (Guo et al., in press). Previous and ongoing efforts toward automated classification of detected particles using various machine learning techniques, including convolutional neural networks, still need holograms to be reconstructed and processed (Davies et al., 2015; Bianco et al., 2020). Recent work has focused on the application of deep learning techniques to extract features from the interference patterns recorded on the raw holograms (Guo et al., in press; Shao et al., 2020). Successful translation to in situ applications would help revolutionize the field, as this would enable skipping the holographic reconstruction step, greatly reducing processing times and facilitating near real-time data dissemination. To continue improving these algorithms, building enormous labeled datasets of hundreds of different particle and plankton types and morphologies from in situ deployments is critical.

Current Research and Future Frontiers

Holographic Particle Image Velocimetry (HPIV)

Different methods of particle tracking as well as 2-D and stereo PIV have been used to characterize small-scale 2-D and 3-D velocity distributions in aquatic environments (Nimmo Smith, 2008; Steinbuck et al., 2010; Nayak et al., 2015). Talapatra et al. (2013) used holographic data from in-focus particles at a particular plane to perform PIV and calculate the 2-D velocity fields (not 3-D measurements). The ability to simultaneously quantify particles and flow distributions within the same volume holds enormous appeal and would greatly help elucidate particle-flow interaction studies. In laboratory studies, HPIV has been successfully established as a valuable tool to characterize three-dimensional velocity measurements within a sample volume (e.g., Barnhart et al., 1994; Hinsch, 2002; Katz and Sheng, 2010). Translating this to field applications, while challenging, would be the next step.

Other potential directions include integrating holography with other methodologies to develop new techniques that could significantly enhance particle or plankton characterization capabilities. For example, the ability to quantify the refractive index (RI) of individual particles is valuable to optical oceanographers. Typically, the particle of interest is immersed in fluids of varying RIs and studied under a microscope (Hodgson and Newkirk, 1975). When the RI of the particle is the same as the fluid in which it is immersed, the particle becomes nearly invisible (or least opaque). This is a highly tedious process; thus, previous efforts have estimated the bulk RI of a water sample from scattering measurements (Carder et al., 1972; Twardowski et al., 2001). Holotomography has recently emerged as a tool to map high-resolution 3-D refractive indices of cells, including diatoms, in laboratory measurements (Charrière et al., 2006; Merola et al., 2017; Umemura et al., 2020), and has high potential to be transformed to in situ aquatic applications (Poulin and Zhang, 2020).

Microplastics

Microplastics have emerged as a primary source of pollutants in aquatic systems in recent times, leading to an increasing number of studies focusing on their presence, fate and transport in aquatic environments, and effects on aquatic ecosystems (Cózar et al., 2014; Zhao et al., 2018; Granek et al., 2020). Early estimates have shown that only a fraction of the known anthropogenic microplastic inputs into the oceans have been accounted for, leading to questions about the fate of the remaining portion. Consequently, improvements in monitoring and detection techniques of microplastics have become crucial to understanding this issue (Garaba and Dierssen, 2018; Mai et al., 2018). Recent laboratory studies have shown that holography can be a valuable tool in the detection of microplastics in the ocean (Merola et al., 2018; Bianco et al., 2020). Future efforts geared toward testing these in field environments are needed.

Long-Term and Real-Time Observation Networks

The democratization of oceanographic data access has received significant attention in the past decade; consequently, data collected from networks of floats, gliders, buoys, and other observation and monitoring platforms are freely available to scientists and the public. Examples include National Oceanic and Atmospheric Administration’s Integrated Ocean Observing System (IOOS) and Marine Biodiversity Observation Networks (MBONs), National Science Foundation’s Ocean Observatories Initiative (OOI), the Argo global network of profiling floats, etc. Other in situ imaging systems, such as the IFCB, are now being routinely used for real-time monitoring of plankton community composition as well as HAB events (Kudela et al., 2015). The versatile nature of holography allows for enough flexibility to be used in a variety of purposes, with appropriate modifications or trade-offs in resolution and image quality. For example, efforts to develop holographic imaging systems for long-term aquatic deployments with a wide particle resolution range (Nayak et al., 2020) can be complemented with low-cost robotic holographic samplers for environmental monitoring (Mallery et al., 2019). Future work should be geared toward integrating holographic systems into aquatic monitoring networks, e.g., MBONs and/or Argo floats. Continuing improvements in image processing methods, technological improvements in data transmission and storage, and an increasing focus on machine learning for automated particle and plankton classification are promising indicators of the aquatic sciences community being poised to embrace holographic systems on a large scale in the near future.

Author Contributions

AN conceived the manuscript and compiled the literature survey. AN, EM, MM, MT, and JS prepared the manuscript. All authors contributed to the article and approved the submitted version.

Funding

AN, MM, and JS were primarily supported through United States National Science Foundation awards OCE-1634053 and OCE-1657332. AN and MT were partially supported by the Harbor Branch Oceanographic Institute Foundation (HBOIF). AN was also supported by an Early Career Research Fellowship from the Gulf Research Program of the National Academies of Sciences, Engineering, and Medicine.

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the Gulf Research Program of the National Academies of Sciences, Engineering, and Medicine.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors are grateful to Lisa Nyman for her help in generating some of the figures used in this manuscript and Ranjoy Barua for useful inputs.

References

Abraham, E. R. (1998). The generation of plankton patchiness by turbulent stirring. Nature 391, 577–580. doi: 10.1038/35361