Social Sensors for Wildlife: Ecological Opportunities in the Era of Camera Ubiquity

Borowicz, Alex; Lynch, Heather J.; Estro, Tyler; Foley, Catherine; Gonçalves, Bento; Herman, Katelyn B.; Adamczak, Stephanie K.; Stirling, Ian; Thorne, Lesley

doi:10.3389/fmars.2021.645288

ORIGINAL RESEARCH article

Front. Mar. Sci., 26 May 2021

Sec. Marine Megafauna

Volume 8 - 2021 | https://doi.org/10.3389/fmars.2021.645288

Social Sensors for Wildlife: Ecological Opportunities in the Era of Camera Ubiquity

$\r\nAlex Borowicz*$ Alex Borowicz^1*

Heather J. Lynch^1,2

Tyler Estro³

Catherine Foley⁴

Bento Gonçalves¹

Katelyn B. Herman⁵

Stephanie K. Adamczak⁶

Ian Stirling⁷

Lesley Thorne⁸

¹Department of Ecology and Evolution, Stony Brook University, Stony Brook, NY, United States
²Institute for Advanced Computational Science, Stony Brook University, Stony Brook, NY, United States
³Department of Computer Science, Stony Brook University, Stony Brook, NY, United States
⁴Hawai‘i Institute of Marine Biology, University of Hawai‘i, Kāne‘ohe, HI, United States
⁵Georgia Aquarium, Atlanta, GA, United States
⁶Department of Ecology and Evolutionary Biology, University of California, Santa Cruz, Santa Cruz, CA, United States
⁷Department of Biological Sciences, University of Alberta, Edmonton, AB, Canada
⁸School of Marine and Atmospheric Sciences, Stony Brook University, Stony Brook, NY, United States

Expansive study areas, such as those used by highly-mobile species, provide numerous logistical challenges for researchers. Community science initiatives have been proposed as a means of overcoming some of these challenges but often suffer from low uptake or limited long-term participation rates. Nevertheless, there are many places where the public has a much higher visitation rate than do field researchers. Here we demonstrate a passive means of collecting community science data by sourcing ecological image data from the digital public, who act as “eco-social sensors,” via a public photo-sharing platform—Flickr. To achieve this, we use freely-available Python packages and simple applications of convolutional neural networks. Using the Weddell seal (Leptonychotes weddellii) on the Antarctic Peninsula as an example, we use these data with field survey data to demonstrate the viability of photo-identification for this species, supplement traditional field studies to better understand patterns of habitat use, describe spatial and sex-specific signals in molt phenology, and examine behavioral differences between the Antarctic Peninsula’s Weddell seal population and better-studied populations in the species’ more southerly fast-ice habitat. While our analyses are unavoidably limited by the relatively small volume of imagery currently available, this pilot study demonstrates the utility an eco-social sensors approach, the value of ad hoc wildlife photography, the role of geographic metadata for the incorporation of such imagery into ecological analyses, the remaining challenges of computer vision for ecological applications, and the viability of pelage patterns for use in individual recognition for this species.

Introduction

Marine mammals are highly mobile species that are inherently difficult to study, both because of the large spatial scales over which they interact with their environment, and because they spend the majority of their life underwater (Kaschner et al., 2012). Tracking studies of marine mammal movement are typically limited in sample size and this unavoidably constrains the types of questions that can be addressed (Sequeira et al., 2019). Surveys across vast landscapes are often expensive, requiring intensive dedicated field studies and large-scale collaborations, and frequently have lengthy intervals between visits (Calambokidis et al., 2001; e.g., Smith et al., 1999). Despite these challenges, understanding movement, habitat use, and connectivity at the scale of an individual’s range is important for making conservation decisions (Runge et al., 2014), and thus there is substantial value in developing new approaches to solve this long-standing problem in marine ecology.

The distribution and migratory pathways of numerous species of marine mammal are poorly understood, and marine species inhabit a complex three-dimensional space that is difficult to observe. Community science (also called citizen science or participatory science) has been proposed as a means of crowd-sourcing the collection or analysis of data on marine mammal abundance, distribution, and habitat use (e.g., Cheeseman et al., 2017; LaRue et al., 2020), but this method also has some well-known challenges. Data quality is often raised as a shortcoming, though community-science projects can produce accurate datasets and should be assessed based on the specific details of each project’s design and goals (Kosmala et al., 2016). Arguably, a greater challenge to community science’s impact lies in the tiny fraction of the relevant population that formally participate in such projects; many more people encounter wildlife than can be motivated or trained to report or document them. This motivation gap makes it difficult to recruit community scientists and to keep them engaged over time (Nov et al., 2011) and creates an organizational barrier that can dissuade many professional researchers from setting up new projects. Here we explore the use of passively sourced images from the internet as a complement to both traditional field research and more formal community science efforts involving the active solicitation of data or analysis from the community. While passive collection of data misses the opportunity for sustained engagement with dedicated observers, it sidesteps the aforementioned motivation gap by capitalizing on people’s desire to share information with their peers online, requires no setup, and leverages the ubiquity of high quality GPS-enabled cameras in most feasible environments.

We illustrate the potential for “eco-social sensors” whose opportunistic photographs, posted online, can provide useful ecological inference. To this end, we develop a “web crawler” to locate and acquire images from online platforms, using the Weddell seal (Leptonychotes weddellii), a species with a circumpolar Antarctic distribution, as a case study. While Weddell seals have been well studied in other parts of their range (Stirling, 1969; Testa and Siniff, 1987; Castellini et al., 1992; Burns et al., 1999; Hückstädt et al., 2017), Weddell seal life history and spatial ecology on the Antarctic Peninsula have received little attention to date. In continental Antarctica, Weddell seals are generally site-faithful, likely due to their reliance on holes in the sea ice that they use to enter the water to forage, mate, and breathe and their use of the ice surface for resting, nursing, and molting (Stirling, 1969; Croxall and Hiby, 1983; Cameron and Siniff, 2004). It is unclear whether Weddell seals on the Antarctic Peninsula and sub-Antarctic islands demonstrate site-fidelity when there is no breathing hole to maintain, though in winter, sea ice does accumulate in bays and fjords. The large and increasing numbers of tourists visiting the Antarctic Peninsula (68,000 in 2019/20; (International Association of Antarctica Tour Operators) IAATO, 2020) provides an opportunity to capitalize on photographs collected and posted online by Antarctic visitors as a means of better understanding Weddell seal biology in this region. Photo-identification (photo-ID), the use of natural markings to identify individual animals, is a widely-used technique for studying the biology and habitat use of a wide range of both marine (e.g., Katona and Whitehead, 1981; Bigg, 1982; Yochem et al., 1990; Speed et al., 2007) and terrestrial species (e.g., Kelly, 2001). Several species of phocid seal have been shown to retain the patterns of spots or streaks on their pelage between years (e.g., Forcada and Robinson, 2006; Patterson and Redman, 2013; Koivuniemi et al., 2016), and to date no work has yet confirmed this occurrence in Weddell seals.

By building an image catalog, photographs of Weddell seals can be used to assess both basic Weddell seal phenology and life-history diversity and their patterns of spatial habitat use. Here, we demonstrate the viability of Weddell seal pelage patterns for photo-ID and assess the utility of passively collecting community science data to supplement traditional field studies of Weddell seals. Further, we focus on three applications for images collected by biologists conducting field studies in the Antarctic Peninsula and internet-based photographic data from tourists visiting the region: (1) annual molting phenology and the potential for latitudinal differences, (2) haul-out site fidelity, and (3) spatial sex segregation. Together these applications demonstrate an alternative and cost-effective method for crowdsourcing previously existing data on social media platforms and harnesses the widespread use of high-resolution digital cameras with GPS.

Materials and Methods

Study Species

We demonstrate the applicability of this method using photographs of Weddell seals, a species frequently found in the regions of the Antarctic coast with fast ice, that is, permanent sea ice attached to land. Fast-ice regions are considered their primary habitat (Stirling, 1969, 1977; LaRue et al., 2019), though Weddell seals are also found outside of fast-ice habitat. The most notable examples are the Antarctic Peninsula (Figure 1), which has only seasonal sea ice—often as pack ice (loosely aggregated sea ice)—and as far north as Larsen Harbor, South Georgia (55° S; Vaughan, 1968; Burton, 2015), which has no regular sea ice.

FIGURE 1

Figure 1. Locations and extent of web-crawler geographic queries on the Antarctic Peninsula.

Web Crawler and Automated Detection

We used an automated data-collection pipeline comprized of three stages: a web-crawler to locate and acquire publicly-shared images with relevant metadata such as location and date, a machine learning-based seal detector algorithm to remove images that are not seals and identify species, and a metadata scraper that isolates geo-tagged images (Figure 2). Once manually verified, these images were added to photographs collected by scientists or actively submitted by community scientists to create a photo catalog ready for subsequent analysis.

FIGURE 2

Figure 2. The workflow. Text and search terms are fed to the web-crawler, which interacts with the platform API (in this case, Flickr) to find images. Once downloaded, images are fed to the trained classifier which separates images by species. A manual verification step fixes misclassifications, and manual annotations such as molt and sex can then be added to the output dataset.

We implemented the web-crawler, written with the Python package icrawler, to scrape data from Flickr (Chen, 2017). This web-crawler uses text-based search terms associated with image metadata, limiting the volume of imagery to analyze from all images to only those tagged, titled, or otherwise annotated with search terms specific to the questions of interest. To create a manageable test set of images and given the limitations of the Flickr application programming interface (API), we selected 20 sites spread across the western Antarctic Peninsula as search centroids (Figure 1 and Supplementary Table 1). For each site, the web crawler searched for images tagged with the word “seal” with geotags situated within a 32-km radius of the site (the maximum search radius within the Flickr API), and within the date range of Oct. 1, 2004 to Apr. 1, 2019, which roughly corresponded to our field-collected dataset. While the search regions overlapped (Figure 1), images were recorded with their unique filenames from Flickr, such that duplicate photographs from overlapping searches were not included twice. The workflow also includes capability to search for all images using only text tags and to eliminate those images without a geotag, suitable for a broad survey that would be difficult to accomplish by specifying a geographic search (e.g., “antarctica” + “seal”). We used the search term “seal,” which would capture seals of many species, rather than a more specific search term such as “Weddell seal” so as to cast a wide net to avoid losing data due to misidentifications.

The web-crawler application locates and downloads images. At this stage, a Python-based image classifier determines whether the image contains a seal and then attempts to identify the species of seal. To build the classifier, we implemented two common convolutional neural network (CNN) model architectures, Resnet-50 (He et al., 2016) and Densenet-121 (Huang et al., 2017) within the PyTorch framework (Paszke et al., 2019) to isolate the best model for this application. CNNs are computer vision models that use an objective function (e.g., cross-entropy between predicted and ground-truth labels) to learn how to extract high-level representations of images that highlight important features. They then map those representations to class labels by learning from a set of labeled training images. Each of our models was trained on a training set of images annotated by experienced Antarctic seal researchers; images in which the species identification was ambiguous were excluded. A set of 4,261 images was used to train and validate the models. Further details of the CNN implementation can be found in Supplementary Material 1.

The output of the classification stage is a pared-down dataset ideally free of irrelevant images, but we included a final visual verification step to ensure the integrity of the data and to remove any irrelevant images the classifier may have missed. Throughout this process, the individual images are not changed, and all EXIF metadata (image file metadata) remains associated with the image. The optional final stage of dataset creation looks through the remaining images to isolate those images with geographic metadata, information that is automatically recorded by a smartphone and often included by more traditional cameras as well. The remaining dataset consists of directory of images of seals for each category with geographic metadata, and a.csv file listing the filename, date, and coordinates of the image.

Study Area

Our analyses are based on two streams of Weddell seal photographs: (1) images taken by trained observers during field work, and (2) “web-crawled” images taken by other Antarctic visitors. All images were collected along the portions of the Antarctic Peninsula (Figure 1) visited most frequently by commercial cruise vessels, notably its western coastline and associated islands. Most of this region has only seasonal fast-ice formation and has variable and drifting concentrations of pack ice during the summer. Pack- and fast-ice provide potential haul-out locations for Weddell seals, but individuals also haul out on pebble, sand, and cobble beaches and on low-lying rock outcrops.

Field Collection of Weddell Seal Images

From Nov. 2015 to Mar. 2018, field biologists working on monitoring and research projects, along with a small number of Antarctic expedition guides, were asked to contribute photographs of Weddell seals, including images from prior field seasons. All researchers and guides were based on commercial tour vessels or occasionally on research expeditions around the Antarctic Peninsula and South Shetland Islands. Images were either associated with precise coordinates or assigned to a “site,” following the framework of other biodiversity mapping efforts on the Antarctic Peninsula that map seabird colonies to particular islands or beaches (Humphries et al., 2017). While this method of differentiating sites is not ideal for seals, which might be hauled out on ice floes or on islands or beaches without seabird colonies, most images mapped neatly onto named visitor sites or sites of continued penguin population monitoring (see Humphries et al., 2017) since Antarctic tourism is tightly regulated. For images with only site-level geographic data, we grouped images under a single set of coordinates for the site, and for those with more precise geographic information, we retained those finer coordinates.

Photo-Identification

Photographs from both the web crawler and field collection were processed to enhance the natural patterns on the pelage, primarily by altering brightness and contrast. We focused photo-identification efforts on the ventral region, as seals often lie on their side to sleep when hauled out (Figure 3). While coat patterns are retained through molt, molt stage can impact the distinctiveness and visibility of the coat patterns, as old, worn fur can appear as one uniform gray or tan color that makes it harder to match a photograph to other photographs in the catalog. For images in which the pre-molt stage (Figure 3A) substantially diminished the visible pattern, we made an effort to find a match but, in the absence of a match, did not include the image in the catalog as a new individual. To ensure that images in the catalog could be reliably used as “type specimens,” we assigned images ratings of quality and distinctiveness (Supplementary Material 2). Images that exceeded a score threshold of 8 out of 15 when summed across ratings were not included in the catalog. A lower standard was considered for images to be matched against the catalog in order to locate additional observations of a previously-identified seal (i.e., a individual already in the catalog). If no match was found, such lower-quality images were retained separately in order to compare against the catalog again after new images were added. Images were excluded from all analyses if the ventral region was obstructed, the angle was extreme enough to prevent a view of the pattern, or resolution or irretrievable glare/shadows made the pattern unclear (Supplementary Material 2).

FIGURE 3

Figure 3. (A) A pre-molt individual showing heavy wear on the ventral surface. (B) A seal mid-molt. (C) A recently-molted individual.

Matching was completed manually by trained observers who either had prior photo-identification experience from other taxa or who had completed a training program and test-set of images. Initial matching was completed without any software support but was later migrated to the Discovery program (Gailey and Karczmarski, 2012) which provided database integration and a means of including relevant metadata. Each image was compared to the catalog by two observers unless a match was found, when it was then verified by A. Borowicz. In every case, matches were verified by two people to ensure agreement.

Site Fidelity

Weddell seals in the fast-ice regions of their range frequently show site fidelity (Stirling, 1969; Croxall and Hiby, 1983; Cameron and Siniff, 2004), though far less is known about site fidelity in areas without extensive or permanent fast ice. While our dataset does not yet contain enough records to map widespread movement along the Antarctic Peninsula, we were able to focus on two frequently-visited sites, Half Moon Island in the South Shetland Islands and Mikkelsen Harbor on Trinity Island (Figure 1 Sites 1 and 7, respectively), given the popularity of these sites among tour operators. Together, these two sites hosted 24,517 passengers ashore in the 2018–19 season alone [(International Association of Antarctica Tour Operators) IAATO, 2020]. We examined images collected by scientists, sent directly to us by passengers, and those obtained from the web-crawler to examine the degree of site fidelity exhibited in this region.

We also used the combined image catalog to track individual seals as a means of understanding their movement over time. Re-sightings of individuals provide important information about the use of haul-out sites in the Antarctic Peninsula by Weddell seals through the austral summer, particularly since the vast majority of haul-out sites in the Antarctic Peninsula have never been documented. This region is of particular interest, as it was excluded from a continent-wide remote-sensing survey for Weddell seals (LaRue et al., 2020), and thus remains poorly documented. Moreover, the potential prevalence of Weddell seals in non-fast-ice habitats is important to the understanding of population-level changes that might occur as ice habitats diminish with changing Southern Ocean climate dynamics (Siniff et al., 2008).

Spatial Sex-Segregation

As a simple example of the potential value of broad-scale image data, we also investigated spatial patterns of sex-segregation among Weddell seals. We annotated seal images based on sex, including an “unknown” category for those seals we were unable to rate. For this preliminary evaluation of differential sex distribution, we considered overall patterns based on latitude as well as the potential for individual haul-out sites to host different sex ratios of seals. We included all images of Weddell seals in this analysis, regardless of their suitability for photo-identification.

Molt Phenology

To examine molt phenology in Weddell seal photographs, each image from both the photo-identification catalog and the database of web-crawled images was qualitatively rated for molt progression as being (1) unmolted (having not yet started the molt process), (2) molting, or (3) fully molted (Figure 3). Unmolted individuals were characterized by typically tan-brown (bleached) fur and indistinct patterning and occasionally showed heavy wear on their fur. Molting typically begins with the head, followed by a dorsal stripe, and the flanks (Beltran et al., 2019). Fully molted individuals displayed clear patterning with more gray and white coloration. Actively-molting individuals were those with some characteristics of both unmolted and fully molted seals. We excluded any image lacking a date or location, and also removed images for which molt stage was unclear or the individual was young-of-the-year. Given that one of the first areas to molt is generally a dorsal stripe (Beltran et al., 2019) which is frequently out of view in our photographs, it is possible that some individuals were categorized as unmolted, but had in fact begun the process. The overall pattern of timing, however, should be unaffected.

As a preliminary means of exploring the potential drivers of molt phenology patterns, we considered all images (which allowed images with a timestamp but no geographic information to be used) and further separated images by sex and location. We then compared molt phenology within two regions of the Antarctic Peninsula, as a basic means of examining any link between latitude and phenology as has been indicated for other species within the Antarctic and without (Bonenfant et al., 2004; Herfindal et al., 2006; Lynch et al., 2012; McLean and Guralnick, 2021). We subdivided the region into a northern component, including the South Shetland Islands and the Peninsula north of Trinity Island, and a Southern component, including the Peninsula and islands south of 63.5° S latitude. Seals in the northern region have ready access to the deep water of the Bransfield Strait and, for seals in the South Shetland Islands, both the Strait and the relatively close shelf break to the north of the islands. In the southern region, the shelf break is considerably more distant, and the region is more likely to contain some sea ice into the summer.

Results

Image Collection

Across the 21 sites on the Antarctic Peninsula, our web search returned 978 images tagged “seal” (Supplementary Material 3). While all of these images were returned from a search that explicitly required a geo-tag, only 458 images contained coordinates, suggesting that the other photographs were tagged with a location within Flickr and that this information was not written to the image metadata. Of those geo-tagged images, 149 were verified as containing a Weddell seal and were used in subsequent analyses (Figure 4A). From our field-based efforts, we collected 148 Weddell seal images around the Antarctic Peninsula (Figure 4A).

FIGURE 4

Figure 4. (A) Locations of field-collected and web-scraped Weddell seal images. (B) Longest within-season travel for an individual, from Jougla Point in the south to Mikkelsen Harbor in the north (167-km straight-line distance).

Of the two classifiers evaluated, the ResNet-50-based classifier performed best, but required substantial verification. The overall recall of the model [true positives ÷ (true positives + false negatives)] was 48.7%, while the precision (percent true positive out of all images labeled as positive) was 52.6% (Table 1). As a result, nearly half of all web-scraped images, in nearly all classes, had to be reclassified manually.

TABLE 1

Table 1. Precision and recall outcomes for different image classes from both CNN classifiers, and the overall weighted macro precision and recall for each model.

Photo-Identification

From the 148 Weddell seal images collected during our field effort, we amassed a catalog of 96 individuals, 16 of which were resighted at least once. After the manual verification of the CNN classifier, we examined the resulting 149 web-scraped Weddell seal images; seventy-two did not meet the quality standards for photo-identification. Of the remaining 79 usable images, we determined that there were 41 unique seal sightings, increasing the size of our photo-ID catalog by 28% to 189 images. In several cases, multiple photographs had been taken concurrently of the same seal, and in others multiple seals appeared in a single photograph. In this scheme, two individuals within the same photograph are considered two sightings.

Site Fidelity

Our dataset from Half Moon Island (Figure 1, site 1) comprised 15 seal sightings (Table 2) collected between 2005 and 2015. At Mikkelsen Harbor (Figure 1, site 7) our dataset consisted of 33 sightings (Table 1) from 2015 to 2019. At both sites, some individuals were re-sighted in the same season and some in different years. At Half Moon Island, seven individuals were seen only once, while the remaining eight images were re-sightings at either Half Moon Island or at another site. Only seven individuals at Mikkelsen Harbor were seen only once.

TABLE 2

Table 2. Images from two sites of focus for photo-identification.

While broad inference about travel and site fidelity within and among years at a large spatial scale will require further data collection, the data collected thus far provide an interesting preliminary view into Weddell seal movement. More than half of all individuals (Flickr and catalog; n = 93 individuals) were photographed only once (65%; n = 115), even at frequently visited sites. Eighteen individuals (13% of all individuals) accounted for all 51 re-sightings (i.e., 29% of photographs were re-sightings). Of those re-sightings, 13 individuals were resighted across different research seasons and 13 were resighted within the same research season. Overall, 16 were resighted at the same site and only eight individuals at different sites; only one individual was only resighted at a different site.

Aside from re-sightings at the same location, the shortest within-season distance observed was 2.5 km (straight-line distance) from Mikkelsen Harbor to the Tetrad Islands, which occurred over 4 days. The longest within-season distance was 167 km over 5 days in 2019 between Jougla Point and Mikkelsen Harbor (Figure 4B). More interesting, perhaps, is another seal that made a 150-km trip between Mikkelsen Harbor (26 Dec 2015) and Half Moon Island (27 Dec 2015) over the course of 1 day (Supplementary Material 4). This is notable both for the distance and speed, but also because a suggestive (but low-quality) image from Georges Point was taken the day before (25 Dec 2015). If the same individual, this seal would have made a further 125-km trip from Georges Point to Mikkelsen Harbor. While that image does not meet our quality standards, it is a striking possibility because another seal was positively identified at both Georges Point and Mikkelsen Harbor on 25 and 26 Dec 2015, respectively (Supplementary Material 4). The confirmed Mikkelsen-Half Moon sighting is also notable in that it is of the most frequently sighted individual in the catalog, seen seven times between 2017 and 2019, though all other sightings are from Mikkelsen Harbor.

Mikkelsen Harbor, a small bay on the southern shore of Trinity Island, was commonly used by resighted seals. Of seals seen in at least two different locations, all but one was observed at Mikkelsen Harbor at least once. While this could be an artifact of the popularity of this site with tour operators, and the resulting high number of photographs obtained from this site, resighting rates were much lower at Half Moon Island, another frequently-visited site at which Weddell seals are nearly always present during the tourist season (Naveen and Lynch, 2011).

Sex Segregation

Of all 232 geo-tagged Weddell seal images from all sources (inclusive of images unsuitable for identification), 127 could be sexed (60 males, 67 females), while 105 could not be sexed (Figure 5A). Broadly, seals were distributed evenly across a latitudinal gradient (Figure 5B). Our results suggest a pattern of site-specific sex-segregation, though substantially more data are required to investigate this trend in more detail (Figure 5C). Whaler’s Bay (Figure 1, site 3) and Half Moon Bay (Figure 1, site 1), for example, showed a roughly 2:1 female-to-male ratio, while Mikkelsen Harbor and Neko Harbor (Figure 1, sites 7 and 17) showed more even sex ratios, though we did not investigate the potential for different patterns to emerge in different years.

FIGURE 5

Figure 5. (A) Male/female distribution across the region. (B) Latitudinal distribution by sex. (C) Site-level sex ratios.

Molt Phenology

We were able to rate molt stage for 169, or 73%, of the 232 geo-tagged images. The dataset included roughly similar numbers of images for each molt stage, with 57 unmolted seals, 66 molting, and 46 fully molted. Their distribution across the season showed a clear seasonality to molt phenology (Figure 6). While our observations of both the unmolted and fully-molted categories are necessarily curtailed by the challenges of navigating in early- and late-season sea ice and the bounds of the field season, the summary statistics from this work demonstrate seasonal patterns. The mean date of observations of seals in the molting stage was January 15 (s.d. = 22 days), and the earliest date that a molting seal was observed was December 14. While the earliest date of an unmolted sighting and the latest day of a fully-molted sighting are artifacts of the start and end of the research and tourism seasons, the latest unmolted sighting (March 16) and earliest fully-molted sighting (January 25) give us some insight into temporal patterns of molt. The latest unmolted individual falls almost three standard deviations from the mean date of a molting sighting, which suggests that this individual may have had some underlying condition—whether related to disease or body condition—that prevented molt from beginning or delayed initiation.

FIGURE 6

Figure 6. The seasonal progression of Weddell seal molt. Histogram data in 12-day bins, and kernel density estimates for each stage. The northern region lies north of 63.5° S while the southern region lies south of this latitude.

The northern region included 49 images in which molt stage could be rated, while the southern region included 120 images. The mean date of observations for molting seals was January 17 in the north, and January 14 in the south. The aforementioned late molter doesn’t allow us to make any meaningful inference about differences in timing for starting molt, but there was a gap of 11 days between the earliest date a seal was observed as fully-molted in the North (Jan 14; mean date of being molted = Feb 26, s.d. = 13 days) and South (Jan 25, mean = Mar 4, s.d. = 13 days), though the relatively fewer images associated with the northern sector essentially make this gap between dates unreliable (Student’s t-test for difference in mean date: t = 1.721,df = 40.05,p = 0.093).

Discussion

“Traditional” community science has produced valuable insights by radically expanding the scope of data collection in some applications (e.g., Lodi and Tardin, 2018; Poisson et al., 2020) and has, as an additional benefit, promoted science knowledge and data literacy among participants (Cronje et al., 2011; Jordan et al., 2011; Haywood, 2015; Dean et al., 2018). However, the active solicitation of data from the public is not well-suited to all applications and is only one possible means of engaging non-scientists in data collection. For better or worse, members of the public are consistently pressed into service as unwitting sensors, providing passive data on traffic conditions, patterns of movement, and commercial behaviors (Baruh and Popescu, 2015). Meanwhile, they intentionally share other information with their friends, family, and the public at large, at the scale of four million Flickr posts and more than 100 million Instagram posts per day (Thornley-Brown, 2019; Aslam, 2020). While the former data stream is fraught with complexities surrounding privacy and choice, the latter provides an opportunity for the public and planet to benefit from eco-social sensor data that are already being intentionally and willingly shared online.

Here we have used Flickr, a photography-sharing website hosting tens of billions of photographs, to gain insight into one species, the Weddell seal, at its range limits and in one of the most remote regions of the world. Using freely-available Python packages, we have collected image data from the Antarctic Peninsula, classified images to the species level, and extracted basic geographic and temporal information. While the images often do not meet the standards we typically set for fieldwork, they provide an ancillary data stream beyond what our field team is able to produce.

Even with a limited dataset from both field expeditions and tourist photographs, we have demonstrated the viability of photo-identification in this species for the first time and have identified several patterns of interest for further, targeted investigation. While Weddell seals in more southerly fast-ice habitats are site-faithful to breathing holes (Stirling, 1969; Croxall and Hiby, 1983; Cameron and Siniff, 2004), there is less obvious need for such a behavior in the perennially ice-free regions of the Antarctic Peninsula. Nevertheless, we have demonstrated that these seals are regularly seen both within and among seasons at the same site. At the same time, we have documented long-distance travel in the same species. The degree to which site-fidelity is plastic among or within seals is worth further investigation. Even aside from these patterns, leveraging photo-sharing platforms has allowed us to create a rough distributional map of all seal species on the Antarctic Peninsula (Supplementary Material 5). A field program to collect even these rough presence-only data with the level of detail demonstrated here would be logistically infeasible.

As for sex-segregation, there is evidence for substantial segregation among other Antarctic pinnipeds (Santora, 2013). Physiological needs for a post-partum female and a post-mating male are likely different, which could drive sex-specific habitat use during and after the mating season (Beltran, 2018). Our preliminary examination of sex-based patterns of distribution did not suggest a robust latitudinal trend, though the small sample size limits our inference. We were unable to sex nearly half of the seal images, though our inability was largely due to physical obstructions or photography than to anything specific to the seal being photographed. As a result, unsexed individuals are, given the evidence available, missing at random. At the scale of individual sites, however, there is some evidence that males and females prefer different haul-outs, and perhaps different foraging grounds. More dedicated effort monitoring these sites within seasons would demonstrate whether patterns of sex segregation are random, permanent, or shift over the season.

Notably, January 15 was the mean of the dates in which we recorded a visibly-molting seal, the same date that Beltran et al. (2019) assess as the mean molt initiation date in the McMurdo Sound area, a region at a much higher latitude and with a more extreme fast-ice climate. Because the period over which one can observe a molting seal is longer than the period over which one can observe the initiation of molt, these findings suggest that molt may initiate earlier for Weddell seals on the Antarctic Peninsula than for those further south, which would align with the suggestion that molt timing is linked to ambient conditions as seals’ thermoregulatory costs associated with molting are roughly double those prior to the onset of molting (Walcott et al., 2020). The climate of the Antarctic Peninsula is notably warmer than that of the Ross Sea, however, where Beltran et al. (2019) and Walcott et al. (2020) based their research, so it is unclear the degree to which thermoregulatory cost-saving is an important driver of phenology in the Antarctic Peninsula region. Our first observation of a molting individual, however, December 14, is close to but later than the observed earliest molt-initiation date from Beltran et al. (2019). It is possible that this is an artifact of the small sample size and inconsistent effort, but this could also suggest that phenology is less clearly defined on the Peninsula than in other regions. More consistent sampling, especially in areas with consistent Weddell seal presence, would resolve this question.

This underlying question of the linkage between behavioral and phenological responses both in extreme environments (such as the fast-ice habitat of McMurdo Sound and the Ross Sea) and in less-extreme environments (such as the ice-free regions of the Antarctic Peninsula), deserves somewhat more attention, given the possibility of warming temperatures in many polar environments due to anthropogenic global climate change. The phenological and behavioral differences between McMurdo Sound Weddell seals and their more northerly Antarctic Peninsula cousins could provide insight into the ability of Weddell seals and other Antarctic pinnipeds to adapt to changing physical conditions in their respective environments. What little evidence we have suggests that diminishing sea ice in the Antarctic Peninsula region could result in migration or population decline (Siniff et al., 2008).

While our implementation of a ResNet CNN was minimally helpful, requiring substantial manual intervention to achieve a reliable dataset, such classification algorithms would likely be more successful with other study species. Phocid seals have broadly similar body plans, which we expect made differentiating between species challenging. More training data would likely alleviate some of this problem and improve the model performance, though the classification problem is challenging and an area ripe with opportunities for the computer vision community. Images from tourists come from many different angles, distances, and lighting conditions, and the cues used by biologists and naturalists to identify a seal on an ice floe from a distance may be subtle. The characteristic body shape of a seal species may be visible to an observer who can deduce how a seal is angled and rolled, but an algorithm is naïve to the third-dimensional characteristics of an image; body position, roll, and the blubber layer combine to create a deformable surfaces problem. To encourage further work on the automated identification of seals, we have included a set of photographs in Supplementary Material 6.

ResNet correctly classified roughly half of our images but also missed half. Those mis-classified images were fairly evenly distributed across the major classes of image with some exceptions. Antarctic fur seals and Southern elephant seals are both abundant in the South Shetland Islands (Hucke-Gaete et al., 2004; Gil-Delgado et al., 2013), especially in several well-visited areas with unique orange-tinted rocks. The misclassification of several landscape photographs as these seals, and of fur seals as elephant seals, suggests that the algorithm may have cued on the color values more than the shapes of the seals. On the other hand, of Antarctic seals, elephant and fur seals are the only two species to regularly position their bodies upright, another potential area of confusion. In a two-dimensional sense, a Weddell seal occupies a form that many of the other seal species included could adopt and this could explain why the Weddell seal class had the lowest precision among the classes (Table 1). Where CNN performance is concerned, the answer is nearly always more training data.

For species with more distinct shapes or coloration, a paired web-crawler/classification algorithm approach will likely be very successful at differentiating a study species from the other photographs of a region available on photo-sharing websites. Areas such as national parks have far more visitors than biologists and many tourists are focused on wildlife at these locations. For example, Flickr hosts over 10,000 geo-tagged images labeled “elk,” and over 29,000 labeled “buffalo” or “bison.” While small, timid, or nocturnal species may not be a good fit for this approach, megafauna would be ideal candidates. A combined approach of fieldwork and the shared data of wildlife enthusiasts as passive “eco-social sensors” has the opportunity to dramatically increase the size of ecological datasets, at the expense of some data quality. We are heartened by the fact that such an approach can work in the Antarctic, which suggests areas with higher volumes of visitation may have even more success with such a method.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The animal study was reviewed and approved by the Stony Brook University Institutional Animal Care and Use Committee.

Author Contributions

AB conceived of the project, wrote the manuscript, analyzed the data, contributed to the code, prepared the figures, and collected data. HL contributed to the manuscript, collected data, and supervised the analysis. TE prepared the web crawler code and contributed to the manuscript. CF conceived of the project, analyzed the data, and contributed to the manuscript. BG prepared the classifier and contributed to the manuscript. KH conceived of the project and contributed to the manuscript. SA participated in analysis and contributed to the manuscript. IS conceived of the project and provided advice throughout the project. LT conceived of the project, supervised the analysis, and contributed to the manuscript. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We gratefully acknowledge the assistance of members of the Oceanites field team for collecting photographs; numerous expedition guides for submitting photographs; Nicole Cassale, Emily Enzinger, and Yanbing Gu for matching assistance; Ted Cheeseman for gathering photographs in the run-up to HappyWhale; and computational time from the SeaWulf cluster at the Institute of Advanced Computational Science (NSF award #1531492).

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmars.2021.645288/full#supplementary-material

References

Aslam, S. (2020). Instagram by the Numbers: Stats, Demographics & Fun Facts. 2020. Available online at: https://www.omnicoreagency.com/instagram-statistics/ (accessed September 4, 2020).

Google Scholar

Baruh, L., and Popescu, M. (2015). Big data analytics and the limits of privacy self-management. New Media Soc. 19, 579–596. doi: 10.1177/1461444815614001