Recruiting, paying, and evaluating the experiences of civic scientists studying urban park usage during the beginning of the COVID-19 pandemic

This paper describes an attempt to utilize paid citizen science in a research project that documented urban park usage during the early stages of the COVID-19 pandemic in two U.S. cities. Strategies used by the research team to recruit, pay, and evaluate the experiences of the 43 citizen scientists are discussed alongside key challenges in contemporary citizen science. A literature review suggests that successful citizen science projects foster diverse and inclusive participation; develop appropriate ways to compensate citizen scientists for their work; maximize opportunities for participant learning; and ensure high standards for data quality. In this case study, the selection process proved successful in employing economically vulnerable individuals, though the citizen scientist participants were disproportionately female, young, White, non-Hispanic, single, and college educated relative to the communities studied. The participants reported that the financial compensation provided by the study, similar in amount to the economic stimulus checks distributed simultaneously by the Federal government, were reasonable given the workload, and many used it to cover basic household needs. Though the study took place in a period of high economic risk, and more than 80% of the participants had never participated in a scientific study, the experience was rated overwhelmingly positive. Participants reported that the work provided stress relief, indicated they would consider participating in similar research in the future. Despite the vast majority never having engaged in most park stewardship activities, they expressed interest in learning more about park usage, mask usage in public spaces, and socio-economic trends in relation to COVID-19. Though there were some minor challenges in data collection, data quality was sufficient to publish the topical results in a peer-reviewed companion paper. Key insights on the logistical constraints faced by the research team are highlighted throughout the paper to advance the case for paid citizen science.

Introduction

Citizen science, the collection and analysis of data relating to the natural world by members of the general public, typically as part of a collaborative project with professional scientists (Oxford Dictionary), is broadly recognized as a strategy for expanding knowledge in a wide variety of scientific disciplines (Vohland et al., 2021). The practice can help increase distributed data collection while engaging the public in societal issues and enabling them to participate meaningfully in the scientific process (Turrini et al., 2018; Peter et al., 2019). The impact that citizen science has had on scientific discovery has been widely documented and discussed, with current research focusing on how to maximize the impact that these programs can have on the participants themselves (Jordan et al., 2012; Kieslinger et al., 2018; Phillips et al., 2018; Walker et al., 2021).

In citizen science projects, a key goal is to develop study designs and standards that maximize data accuracy and participant satisfaction (Walker et al., 2021). In-depth empirical documentation of the successes and failures of specific projects can be useful in achieving this goal, specifically by associating intended or unintended outcomes with specific aspects of the study design (Schaefer et al., 2021), thereby helping to inform the design of future studies (Conrad and Hilchey, 2011; Heiss and Matthes, 2017; Peter et al., 2019).

This paper focuses on the incorporation of citizen science in a research project that documented urban park usage during the early stages of the COVID-19 pandemic. Two overarching goals motivated the study. The first research goal was to study the potentially opposing roles that urban parks in residential neighborhoods of Philadelphia and New York City may play in mitigating the impacts of COVID-19, and/or facilitating its spread. We found no strong correlation between park visits and COVID cases and, as described in detail in Alizadehtazi et al. (2020), to the contrary saw evidence that park visits provided respite and relief during the early phase of the pandemic—an example of what Tidball (2012) calls “urgent biophilia.” The second research goal, addressed in this paper, was to determine whether resident populations could be converted into paid officers of distributed data collection, promoting economic resilience in times of crisis. Specifically, we analyze the use of citizen science to achieve the project goals. To note, the term “civic scientist” (as opposed to “citizen scientist”) was used by the research team so as not to exclude participants who are not citizens of the United States, with “citizen science” used in this paper only when describing the broader literature.

After a review of challenges to citizen science documented in the literature, the approaches taken for recruitment and financial compensation of study participants are described and critiqued, incorporating feedback provided by the civic scientists through a formal evaluation process. We also include descriptions of the experience as reported by the civic scientists, providing insights regarding execution of this study amidst a pandemic and lockdown, and recommendations for future studies.

Background literature review

A synthesis of relevant literature published between 2010 and 2021 was used to identify four “Key Challenges” in Citizen Science: Diversity and Inclusion, Financial Compensation, Participant Learning and Attitudes, and Data Quality.

Key Challenge #1: Achieving diversity and inclusion

A key goal of citizen science is to democratize science by fostering inclusivity in many dimensions (Bonney et al., 2014). However, in practice, individuals who participate in citizen science programs are often disproportionately middle-class, educated, and White (National Academies of Sciences, Engineering, and Medicine, 2018; Paleco et al., 2021; Walker et al., 2021). Challenges to diverse participation in citizen science projects include the tendency of the research team to rely on pre-existing networks for recruitment and the requirement for technological literacy among participants (Foster et al., 2017; Møller et al., 2019; Rall et al., 2019). Bela et al. (2016) reported that a lack of diversity in public data collection efforts can cause the views of certain groups to be overlooked, reinforcing social inequalities. Paleco et al. (2021) underscored the importance of tailoring recruitment strategies to the target study participants. Addressing Key Challenge #1 involves development of strategies that foster diverse and inclusive participation within the citizen science team.

Key Challenge #2: Financial compensation

There has been much debate about financially compensating citizen scientists. Informal interviews conducted over several years by the research team suggest that the architects of some stewardship programs assume that citizen scientists volunteer without the expectation of payment, and are motivated principally by a desire to learn, to contribute to a cause, and/or to experience personal enjoyment/leisure. From this perspective, the opportunity to participate is viewed as a privilege, a pleasure, and/or a civic duty for which financial rewards are inappropriate. Riesch and Potter (2014) related that some scientists justify the lack of financial payment as long as the participants are receiving free learning materials and an opportunity at scientific research. Non-payment for citizen science may also be simply because the research team neglects to budget funds explicitly for this purpose (Long et al., 2016).

However, critics of unpaid citizen science claim it is exploitative, especially when the rest of the research team is paid to participate in the study (Tauginiene et al., 2021). Others argued in favor of financial compensation because of a belief that it creates sustained motivation and work ethics leading to higher quality data (Alabri and Hunter, 2010; Resnik et al., 2015). Payments have also been described as important in attracting citizen scientists with economic needs and/or limited leisure time (Lave, 2015; Cieslik et al., 2019; Walker et al., 2021), advancing diversity. Key Challenge #2 thus focuses on the need to develop the right financial incentive structure that recognizes the monetary value of the work associated with data collection, attracts diverse participants, but which also recognize the highly varied entry points of individuals into citizen science projects.

Key Challenge #3: Participant learning and attitudes

Citizen science is often lauded for the numerous potential benefits it brings to participants, including improved research skills, increased content knowledge, and heightened incentive toward environmental stewardship (Krasny and Bonney, 2005; Jordan et al., 2012; Riesch and Potter, 2014; Bela et al., 2016; Peter et al., 2019). But as Phillips et al. (2018) and Walker et al. (2021) pointed out, actual participant outcomes are largely unstudied. Additionally, there can be a discrepancy between the goals of the researchers, on the one hand, and the experiences of the participants, on the other. A poorly executed project can cause participants to feel bored, overburdened, or unsafe, ultimately dissuading them from engaging in future projects (Resnik et al., 2015). To bolster participant satisfaction and avoid potential negative outcomes, researchers are urged to assess the motivations, learning outcomes, and general experiences of potential citizen scientists through surveys and interviews conducted before, during, and after the study (Jordan et al., 2012; Kieslinger et al., 2018). Participant feedback is also useful in adapting the structure of citizen science programs to better align intentions with outcomes (Phillips et al., 2018). Key Challenge #3 emphasizes the importance of developing study designs that maximize opportunities for participant learning while simultaneously achieving the research objectives.

Key Challenge #4: Data quality

Citizen science is a strategy for gathering data that might otherwise be difficult or impossible to obtain using traditional research methods. However, several concerns about data accuracy have been reported (Riesch and Potter, 2014; Theobald et al., 2015). Because citizen scientists lack the skills and incentives of trained professionals, the reliability and consistency of the data they collect has been called into question (Resnik et al., 2015). While collection of high-quality data has been documented, some citizen scientist projects include non-standard sampling protocols, feature poor spatial or temporal representation, and/or small sample sizes (Anhalt-Depies et al., 2019; Balázs et al., 2021). Data quality is typically greatest when participants are properly trained, communication is maintained throughout the study, and random errors considered in data analysis (Resnik et al., 2015; Kosmala et al., 2016). Key Challenge #4 highlights the need to develop data quality standards that ensure high quality data is collected throughout the project.

Case study materials and methods

In 2020, a cohort of civic scientists was mobilized to gather data about park usage in Philadelphia, PA and New York City (NYC), NY—two East Coast cities in the United States that were, at the time, subject to stay-at-home orders (New York State, 2020; City of Philadelphia, 2020a). All non-essential businesses were closed, and residents were urged not to leave their homes unless necessary (including travel to universities). Despite these restrictions, people continued to use parks and other public spaces (Insider, 2020), and the research team became interested in whether communities with highly visited parks would ultimately present higher COVID-19 infection rates. Given the abrupt nature of the stay-at-home orders, the diffused locations of the parks of interest, the ephemeral nature of the solicited data, and other logistical constraints on research introduced by the university, the research team proposed a rapid-response citizen science project to the National Science Foundation (NSF).

A detailed description of the methodology and study findings are beyond the scope of this paper and are provided in a companion paper (Alizadehtazi et al., 2020). In brief, the team recruited and paid 43 civic scientists to document park usage patterns in 22 parks selected to represent low and high social vulnerability, and low, medium, and high population density in both cities. A strong correlation between the number of confirmed COVID-19 cases in adjacent zip codes and the number of park users was ultimately not found. Moreover, no significant differences in park usage were detected between parks in high and low vulnerability neighborhoods. The study found no evidence that park visits posed measurable risk of COVID-19 infection in the surrounding communities and, to the contrary, may actually have provided palliative value to residents during this early phase of the pandemic.

As a complement to Alizadehtazi et al. (2020), this paper focuses on the unique strategy for collecting data using civic scientists. The procedures used to recruit, select, hire, and survey the civic scientists are described, contrasting individuals who initially expressed interest in the project (i.e., applicants) with those who ultimately participated (i.e., participants). We also describe the civic scientists' experiences and discuss how each of the four Key Challenges identified in the literature review were addressed by the study.

Civic scientist recruitment and selection

Separate plans were made for recruitment and selection of study participants. The research team initially sought to study 10 pre-selected parks in neighborhoods of varying population density and social vulnerability characteristics in each city, with the goal of hiring two civic scientists to enumerate different kinds of visitors in each park. The project budget included financial compensation for ~40 civic scientists to make two data entries per day, at a rate of $10 per entry, over the 8-week study. The researchers were obliged to adhere to university and city guidance regarding park visits, and to devise a civic scientist recruitment strategy that satisfied Drexel University's Institutional Research Board (IRB). A snowball sampling strategy was adopted wherein initial potential study participants identified through outreach to community-based organizations (City of Philadelphia, 2020b) geographically situated near the candidate parks and other environmental networks were contacted by email and asked to identify potential study participants. All interactions between the research team, potential participants, applicants, and actual participants remained anonymous because the civic scientists were viewed by the university as research subjects.

A website was developed to introduce the study, with interactive maps highlighting the pre-selected parks. A Qualtrics application form was made available on the website, through which applicants could submit anonymized demographic information (e.g., gender, age, race and ethnicity, income, etc.) and an anonymous email address with no identifiable information for communication purposes (the anonymized email addresses were also used to pay the participants for their work using Paypal, as described in greater detail below). Through the Qualtrics form applicants were also invited to select one or more of the pre-selected parks they wished to research, or to propose another park, and then to answer specific descriptive questions about those parks. To characterize prior experience working in parks, the application form also included a series of questions about prior parks stewardship activities. The goal of these questions was to determine whether applicants had been previously (i.e., pre-COVID-19) involved in conservation, management, monitoring, education, advocacy, and transformation activities in their local park (Table 1).

TABLE 1

Table 1. Stewardship practices adapted from Landau et al. (2019).

Through the snowball sampling method, 300 applications were received for study of 85 different parks across the two cities. Applications were not received for all the pre-selected parks and more responses were received in Philadelphia than in New York City. To boost the response rate in New York City, the research team did more targeted outreach to “Friends of…” park groups and other organizations located near selected parks.

To maximize the economic benefits of the study amidst the layoffs and furloughs that occurred at the early phase of the pandemic, the research team developed a selection process that prioritized hiring unemployed applicants. Among applicants with the same employment status, the research team sought to diversify the participants based on gender, income, and/or race and ethnicity. No attempt was made in the selection process to diversify the entire cohort of civic scientists.

Applicants selected for participation in the study were sent an acceptance email, to which some did not ultimately respond. The research team learned later that some acceptance emails had been lost to the applicants' spam folders. Additionally, some applicants who initially agreed to participate in the study opted out. To fill these gaps, substitutions were made from the original pool of applicants. The total number of civic scientists who participated, including both those who opted out and those who were later selected as replacements, was 43.

Data about the civic scientists

In addition to the required park-specific observations (described in the companion paper), the civic scientists were asked to complete two surveys: (1) a personal conditions survey administered at three points during the study period; and (2) a post-study evaluation survey. Each of these surveys is described below.

Personal conditions survey

At three points during the project, roughly during Weeks 1, 4, and 8 of the study, civic scientists were asked to fill out the same survey that included personal conditions questions regarding their levels of stress, financial situation, and experiences around the COVID-19 pandemic. The intention of this survey was to demonstrate how/if the civic scientist's responses evolved over the course of the study. This survey is included in its entirety in Supplementary Section 2.

Post-study evaluation survey

At the end of the study, the civic scientists were asked to evaluate their overall experience in the study in a second survey (Supplementary Section 3). Specifically, this survey asked whether participation in the project increased their interest in this type of research and solicited feedback regarding communication with the research team and financial compensation provided by the project. Civic scientists were also invited to provide open-ended comments and recommendations for the research team regarding whether the project could have been implemented differently (Question 10) and whether they had other general feedback (Question 11).

Data analysis

Three different analyses of the survey responses were performed using R, version 3.6.3 (R Core Team, 2020). First, the 300 applicants were compared to the 43 participants to examine whether the selection process was successful in diversifying participation and prioritizing the economically vulnerable. The racial profiles of the applicants and participants were also compared to the racial profile of the population regarding in all zip codes within 400 m (1/4 mile) of each park, using American Community Survey for 2016–2020. Second, responses to the three personal surveys were analyzed to investigate trends over the course of the study. Finally, responses to the post-study evaluation were analyzed to profile the overall experience of the civic scientists.

Results

Comparison of applicants, participants and the population of the surrounding communities

The gender, age, race/ethnicity and marital status characteristics of the applicants and civic scientists are shown in Table 2. There were more female (n = 174, 58%) than male (n = 118, 39.4%) applicants for this study. The final cohort of civic scientists was comprised of 22 females (51.2%) and 19 males (44.2%). Most applicants and civic scientists were between the age of 25–34, followed by individuals in the 35–44 age bracket. Most applicants and civic scientists were not of Hispanic, Latin, or Spanish origin (82.3 and 86%, respectively) and were White (57.5 and 60.5%, respectively). Both applicants and civic scientists were mostly single (64.3 and 60.5%, respectively).

TABLE 2

Table 2. Demographic characteristics of applicants and civic scientists.

The highest level of education, employment status, and household income of the applicants and civic scientists are displayed in Figure 1. Most of the applicants and civic scientists held bachelor's degrees (41.7 and 37.2%, respectively) and were employed (45 and 37.2%, respectively). There was a notable difference between applicants (19.7%) and civic scientists (27.9%) who were recently unemployed, a direct result of the selection process. After “prefer not to disclose” (19 and 25.6%, respectively), the next highest category of household income was $50,000–$75,000 (18 and 23.3%, respectively).

FIGURE 1

Figure 1. Comparison of applicants and civic scientists: (A) education level, (B) employment status, and (C) household income. In (B), “Other” refers to individuals who are either students or retired.

The racial makeup of the applicants and participants differs from the population of the communities surrounding the parks. While most applicants and civic scientists were White (57.5 and 60.5%, respectively), the racial makeup of the population surrounding the parks was predominantly non-White. In aggregate, the populations of the zip codes surrounding the parks were 38.6% White, 29.8% Black or African American, 8.2% Asian, 5.9% mixed race, 0.5% American Indian or Alaska Native, and 16.9% other race(s). The applicants and civic scientists were also more likely not to be of Hispanic, Latin, or Spanish origin (82.3 and 86%, respectively) compared to the surrounding population, 68.8% of which was not Hispanic/Latinx.

Figure 2 compares applicant and civic scientists' prior involvement in stewardship practices. More than 70% of the applicants and participants had never been involved in any forms of the stewardship practices mentioned above (with advocacy as an exception at >60% never having been involved).

FIGURE 2

Figure 2. Prior stewardship experiences of the applicants and civic scientist.

Overall, the pools of applicants and participants were similar in terms of age, race, marital status, educational level, and background experience in stewardship work. The applicants had a slightly higher percentage of females and were less likely to be unemployed than the participants, a direct result of the criteria underlying the selection process. The applicants and participants were, however, more likely to be White and not Hispanic, Latin, or Spanish than the residents of the surrounding communities.

Responses to the personal surveys

Of the 43 civic scientists, 24 individuals completed all three personal surveys. Figures 3A,B display the evolving employment statuses and financial situations, respectively, of the participants throughout the study period. The results are color-coded by survey number (1: beginning, 2: middle, and 3: end, respectively). Most of the participants were employed, and only 20.8–25% remained unemployed throughout the study period (Figure 3A). A gradual increase in “earning more than I am spending” was noted over the study period. Between survey 1 and 3 there was a parallel decrease in “spending more than I am earning.” During the first survey more participants reported the “spending more than I am earning,” while during the third survey the opposite was true. By the end of the study, the number of participants who reported living paycheck to paycheck had dropped by one individual.

FIGURE 3

Figure 3. Civic scientists' evolving: (A) employment status and (B) financial situation. 1, 2, and 3 represent survey numbers (1: beginning, 2: middle, and 3: end). In (A), “Other” represents people who are students or retired.

Figure 4 presents reported use of the financial compensation derived from the project. The most common response was for basic household needs and expenses, followed by paying off debt, with no clear temporal trends evident in the data.

FIGURE 4

Figure 4. Civic scientists' evolving spending of financial compensation derived from the study, where “n” represents the amount of time that the answer was selected.

Figures 5A,B describe levels of financial and general stress. Over the study period, participants who reported “some” or “a lot” of financial and general stress outnumbered those with lower stress.

FIGURE 5

Figure 5. Civic scientists' evolving: (A) financial and (B) general stress.

Because the goal was to determine whether paid participation in the study could reduce economic stress, the participants were asked questions regarding the risk level that COVID-19 poses to the local economy, and their assessment of the Federal government's response to the pandemic (which because of the timing of the study included distribution of the stimulus checks). The results, shown in Figure 6, indicate that throughout the study the participants rated the economic risks to their community as “high” and “extremely high,” while rating the governmental response as resoundingly “poor.”

FIGURE 6

Figure 6. Civic scientists' evolving rating of: (A) risk level that the COVID-19 pose to the community/financial situation in their community and (B) the Federal Government's latest actions and responses to COVID-19.

Overall, the pandemic appears to have presented significant perceived risks to the civic scientists and to their communities. However, the financial compensation provided by the project appears to have been helpful in covering basic household—and other-expenses.

Post-study evaluation

A portion of the results of the post-study evaluation are presented in Table 3 (complete results are provided in Supplementary Table 1). Of the 43 civic scientists, 24 individuals completed the post-study evaluation (though some did not answer all questions). Although 83.3% of study participants reported that they had not previously participated in a scientific study, all respondents indicated that after this experience they would consider participating in other scientific studies. A total of 54.2% of the civic scientists reported that their views on scientific studies had changed for the better (i.e., “yes, positively”); and none reported less favorable views (i.e., “yes, negatively”). When asked whether involvement in this study increased their interest in related research, 12.5% respondents answered “yes, very much so,” 50% said “yes, a little,” 4.2% stated “yes,” and 33.3% said “maybe in the future.” Additionally, 45.8% of respondents reported that the compensation provided by this study “very much” impacted their financial situation during this time of economic crisis, and a total of 95.8% indicated that the tasks asked of them were reasonable given the compensation. Finally, about 67% of the civic scientists found the application process, data regimen, and communication with the research team to be “easy” (see Supplementary Table 1).

TABLE 3

Table 3. Post-study evaluation.

Besides general expressions of “thanks,” only five open-ended responses were submitted: two responses regarding how the study could have been conducted differently (Question 10), and three responses to the general feedback inquiry (Question 11). As presented in greater detail in the Discussion, the civic scientists reported that it was “nice being out every day and having the sun” and that the project had a “great impact” on family and that it made them feel “useful” and “anchored” in a difficult time. They also acknowledged “racial and social tension” between the park users and the civic scientists and recommended potentially providing future civic scientists with an “official pin/insignia” to clarify their role as a researcher. The full quotes are integrated in the Discussion.

Discussion

The discussion analyzes lessons learned from the application process, personal survey responses, and the post-study evaluations in terms of their relevance to the four Key Challenges revealed during the literature review.

Diversity and inclusion

The snowball sampling method was effective in attracting a large pool of 300 applicants spanning the two cities. The number of applicants accumulated more quickly in Philadelphia, possibly due to institutional name recognition (Drexel University is in Philadelphia) and fewer COVID-19 cases at the time of the study, relative to NYC. In Philadelphia, 135 applications received within a few days of publicizing the study, and in NYC 165 applications were received over a 2-week period.

White people made up a higher percentage of both the applicants and civic scientists than found in the population of the surrounding zip codes. This discrepancy is noteworthy given the intention of the research team to diversify the two civic scientists servicing each park in each density/vulnerability category. In part, the shortcoming was due to the goal of assigning exactly two civic scientists to each park. Parks receiving only one applicant were removed from the study, reducing the overall pool of applicants (and potentially some non-White applicants). If a particular park had exactly two applicants, both applicants were automatically included in the study, independent of demographic profile. It was only when multiple applicants applied to study the same park that the research team had any ability to diversify participation, and in those cases, priority went first to unemployed applicants. In such instances, once the research team made a selection both accepted applicants needed to respond to the acceptance email in a timely manner and agree to accept the position. If those conditions were not met, other individuals were selected from the pool of applicants.

If diverse participation was hindered by the limited pools of applicants to each park, the study's rigid schedule and privacy considerations may have also been partly to blame. With more time, the research team could have extended and broadened the recruitment process until a more diverse team of civic scientists were assembled. However, the scientific goals driving the research effort required collection of ephemeral data at this unique, early phase of the pandemic when stay-at-home orders were in effect. This urgency created pressure on the research team to expedite the recruitment process.

The rigid conditions imposed by the study's IRB protocol may also have worked against the research team's goal of reflecting the community demographics in the participant pool. Had in-person recruitment through flyers and tabling at the parks of interest not been prohibited, it is possible that recruitment strategy could have been better tailored to recruit individuals residing in the vicinity of each park over those identified through the network of NGOs known by the research team. Recent research suggests that greater diversity can be achieved by expanding networks beyond direct institutional affiliation, offering multiple ways to participate at different levels (Paleco et al., 2021), and relating project goals to potential participant values and interests (Whitmarsh et al., 2013; Merenlender et al., 2016).

In summary, the selection process did achieve the goal of creating jobs for economically vulnerable, i.e., unemployed individuals. However, by extending and diversifying the recruitment period and process, a larger pool of applicants to study each park could have been generated. This larger pool could have presented the research team with more options for diversifying study participation. Such changes to the recruitment duration and process would perhaps be more feasible on citizen science projects that do not require anonymous participation and does not seek to collect ephemeral data at a time when direct interpersonal contact is discouraged for public health reasons.

Financial compensation

The project appears to have been successful in providing some meaningful financial relief to the participants. At a rate of $10 per entry, with two entries possible per day, civic scientists who completed all of the requested activities would have earned a total of $1,120 over the course of the 8-week study, approximately the amount of the government-sponsored stimulus checks being distributed at the time (Clifford Colby, 2021). The personal surveys revealed that although the employment status of the civic scientists did not change significantly throughout the study (i.e., most of them remained employed throughout), there was an increase in the number of participants who reported earning more than they were spending, perhaps due somewhat to the payments provided by the study.

The timing and scale of financial compensation also appear to have been appropriate. Positive feedback regarding the study's financial compensation is not surprising. Izraeli and Murphy (2003) suggest that in post-disaster periods, the creation of new employment opportunities that leverage the skills and availability of the local labor force can foster a positive community response to disruptions. COVID-19 was both an economic and public health crisis, and the civic scientist feedback suggests that the compensation provided by the study helped to alleviate some of the background financial burden.

Though the payments appear to have been appreciated and impactful, the process of delivering them to the civic scientists was onerous and plagued with institutional barriers. Distribution of gift cards, often a default strategy for universities to financially reward study participants, was not logistically feasible in this case, both because the payment amount needed to be scaled to the number of observations made by each civic scientist (i.e., more observations, more payment), and because the research team promised to compensate the civic scientists on a weekly basis. The possibility of formally hiring the civic scientists as temporary employees of the university was initially explored but ultimately abandoned because of the need to hide participant identities from the research team. The delivery mechanism ultimately implemented involved the use of an online money transfer application (PayPal). Each civic scientists linked their PayPal accounts to an email address that did not reveal their identity (the same email address they used to interact with the research team). Funds were disbursed weekly from a university account after verifying data submissions made by each civic scientist.

Although the PayPal approach was ultimately successful, several challenges emerged as the study proceeded. Administering payments to 43 civic scientists every Friday afternoon required a non-trivial level of collaboration between the university accounting office and the research team. Additionally, at the outset of the study, it appeared to the research team that the burden of collecting tax forms (W-9s) for participants earning over $600 (the threshold for which payments are taxable) would fall on PayPal. Ultimately, the university determined that it was responsible for collecting W-9s and issuing 1099s. Because the civic scientists needed to remain anonymous to the research team, these fiduciary responsibilities also had to be managed by the university's accounting office, adding significant complexity to the process. When it was revealed that the civic scientists needed to pay taxes on the compensation, several individuals became concerned that their participation would render them ineligible for unemployment or other forms of public assistance. Through additional consultation with the accounting office, it was, however, ultimately determined that while they did indeed need to pay taxes, the civic scientists were not required to report the compensation as income since it was for participation in a research study.

This case study demonstrates the significant logistical complexity faced by one university seeking to provide financial compensation to citizen scientists. This project suggests that if such barriers can be overcome, financial compensation for this type of work is appreciated, can be scaled to work completed, and can reduce economic hardship in times of crisis, a key finding of the study.

Participant learnings and attitudes

The post-study evaluation yielded mostly positive feedback. Most civic scientists had never participated in a scientific study before, yet all respondents indicated they would consider participating in similar efforts in the future. Similarly, more than half of the participants confirmed that the study encouraged them to investigate topics of research related to the study. This observation is critical, considering that the public perception on science has grown increasingly partisan (Kirchner, 2017). Civic scientists shared that they were specifically interested in learning about park usage, mask usage in public spaces, and socio-economic trends in relation to COVID-19.

Although most civic scientists reported “some” or “a lot” of stress throughout the duration of the study, comments submitted through the post-study evaluation form suggested that the data gathering activity provided some relief. In general, the pandemic triggered feelings of helplessness and loss of control (BBC, 2020; The New York Times, 2021). Yet, one civic scientist stated in their post-study evaluation:

“This was actually a great impact to my family. I was able to get my kids out and walk with me every day. I did all the pictures and note taking, but they helped and were paid. I think we all look back fondly on participating in the study.”

In a similar response, another participant reported:

“I felt somehow ‘useful’ during a time when I am actually quite powerless to change the course of events, in several ways: I could help further knowledge; it ‘anchored’ my day during a time when my regular schedule has been disrupted; it encouraged me to walk from 3-5 miles a day; and the money has been used for charitable donations, something that makes me feel good.”

During this unique time when daily life changed abruptly and dramatically, this study provided participants with structure to their days. Other researchers (Pocock et al., 2019) have shown that participation in research aimed at understanding the crisis at hand can help to combat feelings of uselessness and provide a sense of belonging. This crucial affirmation of one's value at a difficult time is key in helping individuals adapt and respond. The project created linkages between the urban environment, individual behavior, and social information, creating a social-ecological feedback loop that has been shown to build resilience in disaster contexts (Tidball and Aktipis, 2018).

It is worth noting that the reported stress relief is not surprising given the nature of the specific work required of these civic scientists. The data collection regimen for this study had the participants visit their local park two times a day for the duration of May–July 2020. Visits to urban parks have been shown to improve physical and psychological health (Jennings and Bamkole, 2019; Ma et al., 2019; Zuniga-Teran et al., 2020). More specifically, advocates have emphasized the important role urban parks have had on morale boosting and stress relief amidst societal disruption throughout the pandemic (Kleinschroth and Kowarik, 2020; Uchiyama and Kohsaka, 2020; Ugolini et al., 2020).

Although most civic scientists reported positive experiences with the observation process, such as getting exercise and getting to know their park better, one civic scientist had concerns with awkward encounters while collecting data and suggested:

“…possibly providing the surveyors with an official pin/insignia so that if they are approached during such an observation, they can refer to that title.”

Another described this discomfort further, stating:

“There is a racial and social tension that I felt. White male walking around in a very ethnic/African American/and Mexican neighborhood with a lot of homeless. I usually had my kids and I was raised in NYC otherwise I dunno, it would not be pleasant. Several times people approached me too close and started a convo and wanted to see if I was fascist or something, but I am a true NYer and that is never the case… but I felt more uncomfortable with the social distance factor… but it was nice being out every day and having the sun, fresh air, and in the middle of the study I was reading most cases were from long island people who have been quarantined in their home, so I dunno. I hope this study sheds some light to the virus and homeless and so on…”

The comment illustrates the relationship between participant diversity and experiences. It could be that there would have been fewer feelings of “racial and social tension” had the racial and ethnic identities of the participants better represented the surrounding community.

Data quality

Research, like many other activities, was complicated in the early phases of the COVID-19 pandemic. Although most civic scientists reported no problems navigating the instructions and communicating with the research team in their post-study evaluations, of the 43 civic scientists, only 24 completed the personal survey all three times, and completed the post-study evaluation. However, as described in our companion paper, the data gathered by the civic scientists in this study was more than adequate to publish the study findings in a peer-reviewed academic journal (Alizadehtazi et al., 2020).

This said, the requirement of anonymity posed some challenges in training and supervising the civic scientists, specifically with respect to describing the required activities. In retrospect, though the civic scientists were provided a digital memo on Drexel University letterhead indicating that they were part of a research study, interactions with the public might have been smoother if they could have been provided with a more formal badge, though mailing these out was not possible, again because of the requirement of anonymity. The application form requested that applicants demonstrate that they take photographs with a mobile phone and blur out any faces, since the actual study required this task. Instructions were provided on the application form but inability to complete this task may have inadvertently reduced the pool of applicants.

Roughly 20% of the participants reported difficulty with the initial application process and problems comprehending the tasks requested of them. Referring to one of the enumeration duties of counting the number of people wearing masks and the number of homeless people, one civic scientist elaborated in their post study evaluation,

“I think it should have been clearer what proper use of the mask is. Additionally, I would have liked more clarification on the definition of those ‘seeking refuge’ in the park.”

Because of the abrupt nature of the stay-at-home orders, the diffuse locations of the parks of interest, the ephemeral nature of the solicited data, and other logistical constraints on research introduced by the university, the participation of citizen scientists in this research project was essential to its successful completion. Indeed, this data could not have easily been collected any other way.

Conclusion

This paper analyzes the incorporation of civic scientists into a research study conducted at a unique, early stage of the COVID-19 pandemic. In-depth scrutiny of the recruitment, payment, and evaluation processes undertaken by the research team yielded useful insights into some of the key contemporary challenges associated with citizen science. It also helped to develop recommendations for how to maximize the benefits of these projects on the participants, while achieving the intended scientific outcomes (Bonney et al., 2016; Schaefer et al., 2021).

Citizen scientists collected data of sufficient quality to produce at least one article for publication in a peer-reviewed journal (besides this one), and the study's policy-relevant conclusions were picked up by the media (Philly Voice, 2021). Through their data gathering activities, the civic scientists became more interested in the specific focus of the study, and in research in general. They also reported that the research itself helped to relieve general and economic stress they were experiencing at this early phase of the pandemic.

Though administration of weekly payments by the university to the citizen scientists was logistically cumbersome, financial compensation provided by the project was used by the citizen scientists to cover household, and other expenses in this time of high economic risk.

In these ways, the study represents what Riesch and Potter (2014) describe as a win-win. However, while the research team was successful in recruiting economically vulnerable (i.e., unemployed) participants, the pool of participants did not racially and ethnically reflect the communities surrounding the parks of interest. Notably, the participants were disproportionately female, young, White, non-Hispanic, single, and college educated. In this way, the diversity outcomes were similar to many other citizen science projects in which participants are often found to be disproportionately middle-class, educated, and White (National Academies of Sciences, Engineering, and Medicine, 2018; Paleco et al., 2021; Walker et al., 2021).

The IRB protocol requirement that the civic scientists remain anonymous to the research team introduced a wide range of logistical obstacles for the research team, from diverse recruitment, to communication, to training, to administration of payments. The requirement of anonymity was driven by the desire of the research team to evaluate the experiences of the civic scientists themselves. Ironically, if the team had not sought to survey the civic scientists' experiences, it might have been easier to recruit a diverse cohort, train, communicate, and pay the citizen scientists for their work. One possible solution could have been to hire two cohorts of civic scientists, only one of which would have been personally surveyed.

The co-production of knowledge in the midst or aftermath of a crisis is essential in gaining different perspectives from a wide range of those affected, and can increase the legitimacy of the study findings while providing more evidence for decision making and solution implementation. The case study illustrates that paid civic science can be successful and efficient, even in a time of crisis and stress.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Drexel Institutional Research Board (IRB). The patients/participants provided their written informed consent to participate in this study.

Author contributions

BA, KT, NM, and FM: conception and design. BA, SW, KT, NM, and FM: methodology. BA, SW, KT, and AG: data curation. BA: software, analysis, and visualization. BA and SW: manuscript drafting with the contributions from all other authors. KT and FM: project administration. FM: principal investigator and supervision. All authors contributed to the article and approved the submitted version.

Funding

This research was funded by the National Science Foundation through CBET award #2027600 RAPID: park usage during mandated social distancing.

Acknowledgments

The authors wish to acknowledge contributions by the study Advisory Committee: Novem Auyeung, NYC Parks; Lindsey Campbell, USDA Forest Service; Stephanie Craighead, Philadelphia Parks & Recreation; Danielle Denk, Trust for Public Land; Christopher Gantz, Jefferson University; Bram Gunther, The Nature of Cities; Shima Hamidi, Johns Hopkins University; Michelle Johnson, USDA Forest Service; David Maddox, The Nature of Cities; Marit Larson, NYC Parks; Sarah Charlop-Powers, Natural Areas Conservancy; Christina Rosan, Temple University; Mimi Sheller, Drexel University; and Erika Svendsen, USDA Forest Service. The authors are grateful for the participation of the Civic scientists, as this research would have not been possible without their contributions.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frsc.2022.709968/full#supplementary-material

References

Alabri, A., and Hunter, J. (2010). “Enhancing the quality and trust of citizen science data,” in 2010 IEEE Sixth International Conference on E-science (Brisbane: IEEE), 81–88.

Google Scholar

Alizadehtazi, B., Tangtrakul, K., Woerdeman, S., Gussenhoven, A., Mostafavi, N., and Montalto, F. A. (2020). Urban park usage during the COVID-19 pandemic. J. Extreme Events 7, 2150008. doi: 10.1142/S2345737621500081

CrossRef Full Text | Google Scholar

Anhalt-Depies, C., Stenglein, J. L., Zuckerberg, B., Townsend, P. A., and Rissman, A. R. (2019). Tradeoffs and tools for data quality, privacy, transparency, and trust in citizen science. Biol. Conserv. 238, 108195. doi: 10.1016/j.biocon.2019.108195

CrossRef Full Text | Google Scholar

Balázs, B., Mooney, P., Nováková, E., Bastin, L., and Arsanjani, J. J. (2021). “Data quality in citizen science,” in The Science of Citizen Science, eds K. Vohland, A. Land-Zandstra, L. Ceccaroni, R. Lemmens, J. Perelló, M. Ponti, R. Samson, and K. Wagenknecht (Cham: Springer International Publishing), 139–157.

Google Scholar

BBC (2020). How to Restore Your Sense of Control When You Feel Powerless. Available online at: https://www.bbc.com/worklife/article/20201209-how-to-restore-your-sense-of-control-when-you-feel-powerless (accessed April 01, 2021).

Google Scholar

Bela, G., Peltola, T., Young, J. C., Balázs, B., Arpin, I., Pataki, G., et al. (2016). Learning and the transformative potential of citizen science. Conserv. Biol. 30, 990–999. doi: 10.1111/cobi.12762

PubMed Abstract | CrossRef Full Text | Google Scholar

Bonney, R., Phillips, T. B., Ballard, H. L., and Enck, J. W. (2016). Can citizen science enhance public understanding of science? Public Understand. Sci. 25, 2–16. doi: 10.1177/0963662515607406

PubMed Abstract | CrossRef Full Text | Google Scholar

Bonney, R., Shirk, J. L., Phillips, T. B., Wiggins, A., Ballard, H. L., Miller-Rushing, A. J., et al. (2014). Next steps for citizen science. Science 343, 1436–1437. doi: 10.1126/science.1251554

PubMed Abstract | CrossRef Full Text | Google Scholar

Cieslik, K., Shakya, P., Uprety, M., Dewulf, A., Russell, C., Clark, J., et al. (2019). Building resilience to chronic landslide hazard through citizen science. Front. Earth Sci. 7, 278. doi: 10.3389/feart.2019.00278

CrossRef Full Text | Google Scholar

City of Philadelphia (2020a). City Issues Stay at Home Order Clarifying Restrictions on Business Activity in Philadelphia. Board of Health, Managing Director's Office, Department of Public Health, Department of Commerce, Office of the Mayor, Office of Emergency Management. Available online at: https://www.phila.gov/2020-03-22-city-issues-stay-at-home-order-clarifying-restrictions-on-business-activity-in-philadelphia/#:~:text=All%2520Philadelphia%2520residents%2520must%2520remainfood%2520or%2520seeking%2520medical%2520attention (accessed November 01, 2020).

Google Scholar

City of Philadelphia (2020b). Registered Community Organizations (RCOs). Available online at: https://www.phila.gov/programs/registered-community-organizations-rcos/ (accessed April 01, 2020).

Google Scholar

Clifford Colby, K. L. (2021). How Much Was the First Stimulus Check? Your Tax Return May Need That Total. Available online at: https://www.cnet.com/personal-finance/taxes/how-much-was-the-first-stimulus-check-your-tax-return-2020-may-need-that-total/ (accessed May 1, 2021).

Google Scholar

Conrad, C. C., and Hilchey, K. G. (2011). A review of citizen science and community-based environmental monitoring: issues and opportunities. Environ. Monit. Assess. 176, 273–291. doi: 10.1007/s10661-010-1582-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Foster, A., Dunham, I. M., and Kaylor, C. (2017). Citizen science for urban forest management? Predicting the data density and richness of urban forest volunteered geographic information. Urban Sci. 1, 30. doi: 10.3390/urbansci1030030

CrossRef Full Text | Google Scholar

Heiss, R., and Matthes, J. (2017). Citizen science in the social sciences: a call for more evidence. GAIA Ecol. Perspect. Sci. Soc. 26, 22–26. doi: 10.14512/gaia.26.1.7

CrossRef Full Text | Google Scholar

Insider (2020). The Pandemic Is Transforming How Americans Use Public Libraries, Parks, and Streets - and It's Depriving Vulnerable People of Space When They Need it Most. Available online at: https://www.insider.com/how-public-space-will-change-after-pandemic-class-race-inequality-2020-9 (accessed November 01, 2020).

Google Scholar

Izraeli, O., and Murphy, K. J. (2003). The effect of industrial diversity on state unemployment rate and per capita income. Ann. Reg. Sci. 37, 1–14. doi: 10.1007/s001680200100

CrossRef Full Text | Google Scholar

Jennings, V., and Bamkole, O. (2019). The relationship between social cohesion and urban green space: an avenue for health promotion. Int. J. Environ. Res. Public Health 16, 452. doi: 10.3390/ijerph16030452

PubMed Abstract | CrossRef Full Text | Google Scholar

Jordan, R. C., Ballard, H. L., and Phillips, T. B. (2012). Key issues and new approaches for evaluating citizen-science learning outcomes. Front. Ecol. Environ. 10, 307–309. doi: 10.1890/110280

CrossRef Full Text | Google Scholar

Kieslinger, B., Schäfer, T., Heigl, F., Dörler, D., Richter, A., and Bonn, A. (2018). Evaluating Citizen Science-Towards an Open Framework. London: JSTOR.

Google Scholar

Kirchner, J. W. (2017). Science, politics, and rationality in a partisan era. Water Resour. Res. 53, 3545–3549. doi: 10.1002/2017WR020882

CrossRef Full Text | Google Scholar

Kleinschroth, F., and Kowarik, I. (2020). COVID-19 crisis demonstrates the urgent need for urban greenspaces. Front. Ecol. Environ. 18, 318. doi: 10.1002/fee.2230

PubMed Abstract | CrossRef Full Text | Google Scholar

Kosmala, M., Wiggins, A., Swanson, A., and Simmons, B. (2016). Assessing data quality in citizen science. Front. Ecol. Environ. 14, 551–560. doi: 10.1002/fee.1436

CrossRef Full Text | Google Scholar

Krasny, M. E., and Bonney, R. (2005). “Environmental education through citizen science and participatory action research,” in Environmental Education and Advocacy: Changing Perspectives of Ecology and Education, eds E. A. Johnson and M. Mappin (Cambridge: Cambridge University Press), 292–319.

Google Scholar

Landau, L., Campbell, L. K., Johnson, M., Svendsen, E., and Berman, H. (2019). STEW-MAP in the New York City Region: Survey Results of the Stewardship Mapping and Assessment Project, Vol 189. General Technical Reports, NRS-189. Newtown Square, PA: US Department of Agriculture, Forest Service, Northern Research Station, 1–69.

Google Scholar

Lave, R. (2015). The future of environmental expertise. Ann. Assoc. Am. Geogr. 105, 244–252. doi: 10.1080/00045608.2014.988099

CrossRef Full Text | Google Scholar

Long, J. W., Ballard, H. L., Fisher, L. A., and Belsky, J. M. (2016). Questions that won't go away in participatory research. Soc. Nat. Resour. 29, 250–263. doi: 10.1080/08941920.2015.1024368

CrossRef Full Text | Google Scholar

Ma, B., Zhou, T., Lei, S., Wen, Y., and Htun, T. T. (2019). Effects of urban green spaces on residents' well-being. Environ. Dev. Sustain. 21, 2793–2809. doi: 10.1007/s10668-018-0161-8

CrossRef Full Text | Google Scholar

Merenlender, A. M., Crall, A. W., Drill, S., Prysby, M., and Ballard, H. (2016). Evaluating environmental education, citizen science, and stewardship through naturalist programs. Conserv. Biol. 30, 1255–1265. doi: 10.1111/cobi.12737

PubMed Abstract | CrossRef Full Text | Google Scholar

Møller, M. S., Olafsson, A. S., Vierikko, K., Sehested, K., Elands, B., Buijs, A., et al. (2019). Participation through place-based e-tools: a valuable resource for urban green infrastructure governance? Urban Forestry Urban Green. 40, 245–253. doi: 10.1016/j.ufug.2018.09.003

CrossRef Full Text | Google Scholar

National Academies of Sciences Engineering, and Medicine. (2018). Learning Through Citizen Science: Enhancing Opportunities by Design. Washington DC: The National Academies Press.

Google Scholar

New York State (2020). Governor Cuomo Signs the'New York State on PAUSE'Executive Order. Pressroom, NYS Governor's Press Office. Available online at: https://www.governor.ny.gov/news/governor-cuomo-signs-new-york-state-pause-executive-order (accessed November 01, 2020).

Google Scholar

Paleco, C., Peter, S. G., Seoane, N. S., Kaufmann, J., and Argyri, P. (2021). “Inclusiveness and diversity in citizen science,” in The Science of Citizen Science, eds K. Vohland, A. Land-Zandstra, L. Ceccaroni, R. Lemmens, J. Perelló, M. Ponti, R. Samson, and K. Wagenknecht (Cham: Springer International Publishing), 261–281.

PubMed Abstract | Google Scholar

Peter, M., Diekötter, T., and Kremer, K. (2019). Participant outcomes of biodiversity citizen science projects: a systematic literature review. Sustainability 11, 2780. doi: 10.3390/su11102780

CrossRef Full Text | Google Scholar

Phillips, T., Porticella, N., Constas, M., and Bonney, R. (2018). A framework for articulating and measuring individual learning outcomes from participation in citizen science. Citizen Sci. Theory Prac. 3:1–19. doi: 10.5334/cstp.126

CrossRef Full Text | Google Scholar

Philly Voice (2021). Outdoor Recreation Remains Popular and Can Help Provide a Big Health Boost. Available online at: https://www.phillyvoice.com/nature-health-benefits-outdoor-recreation-vacation-destinations-philadelphia-parks/ (accessed May 13, 2021).

Google Scholar

Pocock, M. J., Roy, H. E., August, T., Kuria, A., Barasa, F., Bett, J., et al. (2019). Developing the global potential of citizen science: assessing opportunities that benefit people, society and the environment in East Africa. J. Appl. Ecol. 56, 274–281. doi: 10.1111/1365-2664.13279

CrossRef Full Text | Google Scholar

R Core Team (2020). The R Foundation for Statistical Computing. Vienna. Available online at: https://www.R-project.org/ (accessed March 1, 2020).

Google Scholar

Rall, E., Hansen, R., and Pauleit, S. (2019). The added value of public participation GIS (PPGIS) for urban green infrastructure planning. Urban Forestry Urban Green. 40, 264–274. doi: 10.1016/j.ufug.2018.06.016

CrossRef Full Text | Google Scholar

Resnik, D. B., Elliott, K. C., and Miller, A. K. (2015). A framework for addressing ethical issues in citizen science. Environ. Sci. Policy 54, 475–481. doi: 10.1016/j.envsci.2015.05.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Riesch, H., and Potter, C. (2014). Citizen science as seen by scientists: methodological, epistemological and ethical dimensions. Public Understand. Sci. 23, 107–120. doi: 10.1177/0963662513497324

PubMed Abstract | CrossRef Full Text | Google Scholar

Schaefer, T., Kieslinger, B., Brandt, M., and van den Bogaert, V. (2021). “Evaluation in citizen science: The art of tracing a moving target,” in The Science of Citizen Science, eds K. Vohland, A. Land-Zandstra, L. Ceccaroni, R. Lemmens, J. Perelló, M. Ponti, R. Samson, and K. Wagenknecht (Cham: Springer International Publishing), 495–514.

Google Scholar

Tauginiene, L., Hummer, P., Albert, A., Cigarini, A., and Vohland, K. (2021). “Ethical challenges and dynamic informed consent,” in The Science of Citizen Science, eds K. Vohland, A. Land-Zandstra, L. Ceccaroni, R. Lemmens, J. Perelló, M. Ponti, R. Samson, and K. Wagenknecht (Cham: Springer International Publishing), 397–416.

Google Scholar

The New York Times (2021). The Pandemic Brought Depression and Anxiety. Reaching Out Helped. Available online at: https://www.nytimes.com/2021/02/09/well/mind/covid-depression-anxiety.html (accessed April 01, 2021).

Google Scholar

Theobald, E. J., Ettinger, A. K., Burgess, H. K., DeBey, L. B., Schmidt, N. R., Froehlich, H. E., et al. (2015). Global change and local solutions: tapping the unrealized potential of citizen science for biodiversity research. Biol. Conserv. 181, 236–244. doi: 10.1016/j.biocon.2014.10.021

CrossRef Full Text | Google Scholar

Tidball, K. G. (2012). Urgent biophilia: human-nature interactions and biological attractions in disaster resilience. Ecol. Soc. 17, 5. doi: 10.5751/ES-04596-170205

CrossRef Full Text | Google Scholar

Tidball, K. G., and Aktipis, A. (2018). Feedback enhances greening during disaster recovery: A model of social and ecological processes in neighborhood scale investment. Urban Forest. Urban Green. 34, 269–280. doi: 10.1016/j.ufug.2018.07.005

CrossRef Full Text | Google Scholar

Turrini, T., Dörler, D., Richter, A., Heigl, F., and Bonn, A. (2018). The threefold potential of environmental citizen science - generating knowledge, creating learning opportunities and enabling civic participation. Biol. Conserv. 225, 176–186. doi: 10.1016/j.biocon.2018.03.024

CrossRef Full Text | Google Scholar

Uchiyama, Y., and Kohsaka, R. (2020). Access and use of green areas during the COVID-19 pandemic: green infrastructure management in the ”new normal“. Sustainability 12, 9842. doi: 10.3390/su12239842

CrossRef Full Text | Google Scholar

Ugolini, F., Massetti, L., Calaza-Martínez, P., Cariñanos, P., Dobbs, C., Ostoić, S. K., et al. (2020). Effects of the COVID-19 pandemic on the use and perceptions of urban green space: an international exploratory study. Urban Forestry Urban Green. 56, 126888. doi: 10.1016/j.ufug.2020.126888

PubMed Abstract | CrossRef Full Text | Google Scholar

Vohland, K., Land-Zandstra, A., Ceccaroni, L., Lemmens, R., Perelló, J., Ponti, M., et al. (2021). The Science of Citizen Science. Cham: Springer Nature.

Google Scholar

Walker, D. W., Smigaj, M., and Tani, M. (2021). The benefits and negative impacts of citizen science applications to water as experienced by participants and communities. Wiley Interdiscipl. Rev. Water 8, e1488. doi: 10.1002/wat2.1488

CrossRef Full Text | Google Scholar

Whitmarsh, L., O'Neill, S., and Lorenzoni, I. (2013). Public engagement with climate change: what do we know and where do we go from here? Int. J. Media Cultural Politics 9, 7–25. doi: 10.1386/macp.9.1.7_1

CrossRef Full Text | Google Scholar

Zuniga-Teran, A. A., Staddon, C., de Vito, L., Gerlak, A. K., Ward, S., Schoeman, Y., et al. (2020). Challenges of mainstreaming green infrastructure in built environment professions. J. Environ. Plann. Manage. 63, 710–732. doi: 10.1080/09640568.2019.1605890

CrossRef Full Text | Google Scholar