The Good, the Bad, and the Uncertain: Contributions of Volunteered Geographic Information to Community Disaster Resilience

Haworth, Billy Tusker; Bruce, Eleanor; Whittaker, Joshua; Read, Róisín

doi:10.3389/feart.2018.00183

REVIEW article

Front. Earth Sci., 01 November 2018

Sec. Geohazards and Georisks

Volume 6 - 2018 | https://doi.org/10.3389/feart.2018.00183

This article is part of the Research Topic Citizen Science: Reducing Risk and Building Resilience to Natural Hazards View all 14 articles

The Good, the Bad, and the Uncertain: Contributions of Volunteered Geographic Information to Community Disaster Resilience

$\r\nBilly Tusker Haworth*$ Billy Tusker Haworth^1*

Eleanor Bruce²

Joshua Whittaker³

Róisín Read¹

¹Humanitarian and Conflict Response Institute, School of Arts, Languages and Cultures, University of Manchester, Manchester, United Kingdom
²School of Geosciences, University of Sydney, Sydney, NSW, Australia
³Centre for Environmental Risk Management of Bushfires, University of Wollongong, Wollongong, NSW, Australia

The adoption of location-based information sharing technologies, and the emergence of volunteered geographic information (VGI), has seen changes to community involvement in disaster management. The concept of resilience, and recognition of the capacity for renewal, re-organization, and societal development, has gained currency in disaster management. However, the opportunities presented by spatially referenced data for sourcing contextual information for understanding processes of social–ecological resilience and fostering local inclusion has not been examined. We examine how web 2.0 platforms, including VGI and social media, can support resilience building, and critically evaluate how these technologies potentially undermine resilience. We concentrate our analysis on factors deemed important for community disaster resilience through review of recent literature, policy documents, and author experience. Establishing which elements of VGI in disaster management should be emphasized, such as increased flexibility or individual empowerment, and which require careful management, such as compromised privacy or data quality, will enable VGI to become less opportunistic, data-centric, disruptive, and exclusionary, and allow for more reliable, community-centric, complementary, and socially inclusive practices. Incorporating awareness and training on collaborative geoweb technologies into disaster preparedness programs will equip individuals to make informed judgments on VGI content and reduce unintended consequences of social media initiatives.

Introduction

The adoption of social media and location-based information-sharing technologies, and the emergence of volunteered geographic information (VGI), has seen a shift in the spatio-temporal scales of community involvement in disaster management. Often associated with the related concept of citizen science¹ (e.g., Haklay, 2013), VGI is widely understood to involve the growing practices of private citizens voluntarily generating (Goodchild, 2007) and subsequently disseminating (Ricker et al., 2014) geographic information, predominantly through dedicated online platforms. VGI contributors may explicitly volunteer information through focused mapping activities using platforms such as OpenStreetMap (OSM) or generate data that are implicitly associated with geographic location such as geotagged photographs (shared via Flickr) or microblogs (e.g., Tweets) (Craglia et al., 2012; Senaratne et al., 2017). The increasingly widespread use of such technologies by private individuals for participation in emergency and disaster management has been termed digital volunteering (Whittaker et al., 2015). However, social media and VGI efforts tend to focus on reactive response initiatives, such as the use of Facebook to connect community members during bushfires (wildfires) in Australia (Irons et al., 2015), rather than preparedness and participatory practices for promoting community resilience. The concept of resilience, and the recognition of the capacity for renewal, re-organization, and societal development, following system disturbance has gained currency in disaster management research. This paper examines how resilience thinking can better inform the development of public participation geoweb platforms and shape understandings of the motivations and requirements of such initiatives. The resilience perspective is adopted here to provide a contextual framework for critical evaluation of the role of VGI practices and digital volunteering in disaster management.

The aim of many current emergency management policies is to use the Prevention, Preparedness, Response, and Recovery (PPRR) model to work toward a more disaster-resilient population, that is, one that is able to recognize current and future risk, reduce and manage those risks, and is better-able to recover from disasters (Council of Australian Governments [COAG], 2011; Prosser and Peters, 2011). Increased emphasis on resilience requires emergency practitioners to shift focus from a top-down “command-and-control” model to one more strategic, participatory, and dialogic with communities and stakeholders; where value is increasingly recognized in both authoritative and citizen information and practices (Burnside-Lawry et al., 2013). This has led to a focus on implementing initiatives centered on community engagement (e.g., Frandsen et al., 2011) and innovations in information and communication technologies, such as social media, that can empower citizens in disaster response (e.g., Taylor et al., 2012).

Allowing ordinary citizens to voluntarily create and share geographic information – through technologies such as the Internet and Web 2.0, global positioning systems, personal locational devices such as smartphones, inexpensive map-making platforms open to public contributions (e.g., OSM, Ushahidi Crowdmap), cloud storage, and broadband communication – has transformed the traditional model of authoritative production of geographic information, with particular pertinence to disaster management (see for examples Zook et al., 2010; Haworth and Bruce, 2015). The production of geographic knowledge is no longer exclusive to experts, such as geographers (Elwood et al., 2012). The central discourse of Web 2.0 technology and practices like VGI is public participation and interactivity – a key element in crisis literature aimed at building resilience and increasing the involvement of the general public in disaster management (Bittner et al., 2016).

Although recent work has argued for the role of Web 2.0 information sharing platforms in building community disaster resilience (Dufty, 2012; Taylor et al., 2012), this is often focused on disaster response or a single platform in isolation, such as a social-media site like Facebook or Twitter, at the exclusion of other VGI practices, including crowdsourced web-mapping. The opportunities presented by spatially referenced (geotagged) data for sourcing contextual information at geographical and temporal scales relevant in understanding processes of resilience and fostering local inclusion have not been critically examined. We consider Web 2.0 information sharing platforms more broadly in community disaster resilience to include social media-based exchange of information and resources as well as more participatory enterprises such as crowdsourced disaster risk mapping. We also include here the concept of participatory mapping, as conceptualized by Brown and Fagerholm (2015) to collectively denote any process whereby citizens contribute to the creation of maps, which includes VGI. Further, while there are exceptions, much existing work on VGI and disasters has tended to be somewhat boosterish, at times exuding “techno-optimism” (Read et al., 2016). The optimism and promise of mapping and big data have been critiqued in the context of digital humanitarianism (Read et al., 2016), but similar analyses in relation to disaster resilience are lacking.

Thus, the intent of this paper is to examine how Web 2.0 information sharing platforms for VGI generation can support resilience-building in disaster management and, critically, evaluate how these technologies potentially undermine resilience building. We include social media beyond strict VGI in our analysis. While we recognize some social media information is not strictly VGI (i.e., data may not be geographical), commonalities such as the nature of large-scale online networking, information sharing by private citizens, and disruption to traditional systems catalyzed by technological advancements and associated practices warrant inclusion of this literature. The novelty of our approach lies in the explicit linking of VGI data and practices, including their promise, limitations, and implications, with conceptual understandings of resilience – an important endeavor as resilience remains high on international disaster management agendas.

Resilience

Social–Ecological Resilience and Emergence in Disaster Management

A resilience concept has been embraced by various disciplines, including anthropology, engineering, urban planning, and geography (Weichselgartner and Kelman, 2015), with origins in physics (Van der Leeuw and Aschan-Leygonie, 2005), development psychology, and ecology (Manyena, 2006). The resilience perspective emerged from the ecology literature in the 1960–1970s with the discovery of multi-stable states in natural systems and non-linear forms of functional response (Folke, 2006). In challenging the dominant assumption of static equilibrium, Holling (1973) introduced the concept of multiple basins of attraction in which the size of stability domains (basin) or amount of disturbance a system can tolerate before shifting into another regime provides a measure of resilience (Folke, 2006). The recognized importance of process dynamics across multiple and interacting spatio-temporal scales later placed the resilience perspective in the context of complex adaptive systems (Folke, 2006). Integration of the social dimension in resilience thinking led to the concept of social–ecological resilience (SER). SER considers the amount of disturbance the system can absorb, the system’s ability for self-organization, and the degree to which the system can build and increase capacity for learning and adaptation (transformability) (Carpenter and Gunderson, 2001; Folke, 2006). Rather than focusing on ecosystems or societies as separated entities, recognition of the dynamic interplay between the social and ecological components is critical in understanding system resilience (Gallopín, 2006).

In ecology, resilience emphasizes efficiency, control, constancy, and predictability as attributes of reliable systems (Holling and Gunderson, 2002). Another perspective considers the persistence, adaptability, variability, and unpredictability of ecological systems (Holling, 1973). The first definition is a measure of the speed at which a system “bounces back” after disturbance (Pimm, 1984); while the second measures the degree of disturbance that can be absorbed by a system before undergoing structural change (Holling and Gunderson, 2002). More recently, interpretations of resilience have encompassed elements such as capacity and capability, moving from the early engineering-based “bounce back” perspective to something that suggests doing better than before, or “bouncing forward” (Manyena et al., 2011; Weichselgartner and Kelman, 2015).

The term resilience has come into vogue in disaster management in recent decades with concern the language of “vulnerability” in disaster management was disempowering. Often resilience is treated as an antonym of vulnerability, a more positive way to talk about the same problem, but there are important differences (Whittaker, 2008). Vulnerability is constructed in the social and economic circumstances of everyday living and describes the ways people are differentially exposed to hazards and have varying capacities to reduce risks and withstand potential impacts (Morrow, 1999; Whittaker, 2008). Recognizing the terms are related, Zhou et al. (2010) attempt to contrast vulnerability and resilience, placing the emphasis of resilience on the process of enhancing capacity to respond and recover from disaster impacts in the shortest possible time with minimal outside assistance. Disaster resilience considered from a geographer’s perspective is “the capacity of hazard-affected bodies to resist loss during disaster and to regenerate and reorganize after disaster in a specific area in a given period” (Zhou et al., 2010, p. 28).

Understandings and management of disasters since the 1970s has shifted focus from hazard identification and response to the drivers of vulnerability that turn hazards into disasters (Collier et al., 2009). Promotion of the Disaster Risk Reduction (DRR) framework by the practitioner community, both internationally (Hyogo Framework for Action and Sendai Framework) and at national levels, has highlighted the importance of SER thinking, the need for indicators of effective disaster preparedness, and participatory approaches that ensure local inclusion (Collier et al., 2009). Emergency management is shifting away from models of “command and control” and is beginning to diversify into a more collaborative activity and dynamic enterprise that facilitates multi-organizational, intergovernmental, and intersectoral co-operation (Waugh and Streib, 2006). This shift has led to philosophies such as “shared responsibility” (McLennan and Handmer, 2012). Shared responsibility itself has evolved from the notion of community responsibility and self-reliance (Elsworth et al., 2009) to a principle that implies increased responsibility for all concerned (being the state, municipal councils, individuals, household members, and the broader community) and a focus on community safety.

Despite its prevalence, the concept of resilience has undergone considerable critique. A basic criticism of the social–ecological systems approach is the assumption that the ecological and social domains of such systems can be addressed in a common conceptual, theoretical, and modeling framework (Welsh, 2014). A key problem here relates to the issue of defining the parameters of “the system” and locating human action within it. MacKinnon and Derickson (2012) note that resilience privileges spatial sites and scales such as cities, regions, and local communities, which are implicitly equated with ecosystems, and are viewed as autonomous and subject to the same principles of self-organization. They argue that a focus on the local scale neglects consideration of more powerful, global scale processes that enable and constrain action in specific places. Another criticism of resilience in disaster management concerns the transferral of responsibility for risk from the state to communities, households, and individuals. Welsh (2014) notes that resilience approaches assume that communities can and should self-organize to manage risk, and that the role of government is limited to facilitating and supporting, rather than funding, these processes. Much resilience research has ignored questions of politics, governance, and the unequal distribution of power and resources in disaster preparedness and response (Cretney, 2014). Insufficient attention has also been paid to the value judgments that underpin resilience: what elements of the system should be protected, and for whom? (Handmer and Dovers, 1992; Davoudi, 2012). Weichselgartner and Kelman (2015) have questioned why people would want to “bounce back” to a situation that is anywhere near the original circumstances that produced their vulnerability. Despite these critiques, resilience persists as a core goal in international disaster management.

The Resilience Goals of Disaster Management

Resilience approaches aim to reduce the likelihood of disruption, damage, and death caused by disasters and enable communities to absorb change and recover quickly while exhausting the least amount of resources (United Nations International Strategy for Disaster Risk Reduction [UNISDR], 2009; Intergovernmental Panel on Climate Change [IPCC], 2012). But little consensus or formal clarity has been provided on the specific elements required for resilience or how to achieve them (Arbon, 2014; Neely, 2014; Goode et al., 2015). It is not our intention to review and translate all interpretations of resilience goals, nor to provide an alternative unifying consensus. Instead, we have identified common elements and goals of resilience found throughout previous interpretations, focusing on social resilience, institutional resilience, and community capital (Cutter et al., 2010), and elements most relevant to communities and individuals (as opposed to the natural environment, economics, or infrastructure). Factors deemed important for community disaster resilience were determined through examination and review of relevant resilience policy documents (e.g., United Nations International Strategy for Disaster Risk Reduction [UNISDR], 2005, 2015; Council of Australian Governments [COAG], 2011; Barnes et al., 2014), recent academic literature relating to community disaster resilience (e.g., Paton, 2006; Cutter et al., 2010; Renschler et al., 2010; Dufty, 2012; McLennan and Handmer, 2012, 2013; Taylor et al., 2012; Arbon, 2014; Neely, 2014; Goode et al., 2015), and through research experience of the authors gained via collaboration with DRR and community development units within formal emergency management organizations. For the purpose of this article, the contributions of VGI and digital volunteerism to community disaster resilience are summarized and discussed through the elements depicted in Figure 1.

FIGURE 1

FIGURE 1. Factors important for community disaster resilience as identified by the authors’ review of relevant policy documents, scholarly literature, and research experience. Graphics have been modified from artwork by Minduka, SimpleIcons, grin, GDJ, Iaobc, yves_guillou, russel, Fred the Oyster, and Netalloy in the Open Clip Art Library and are licensed under a Creative Commons Attribution 4.0 International license.

The Good: Potential for VGI to Enhance Community Resilience

Resilience thinking recognizes that multiple environmental and social stressors influence a community’s adaptive capacity, and different context-dependent factors will determine whether a community is adversely impacted by a hazard (Murphy, 2015). Numerous published articles in recent years describe the benefits of VGI in disaster management, particularly in crisis response, often implying (and sometimes boasting) that VGI aids in building community resilience. What follows in this section is a brief discussion of how VGI may contribute to enhancing each of the elements of community disaster resilience presented in Figure 1.

Effective Communication

Resilience is intimately associated with good communication, whereby two-way dialog delivers both resources to communities and intelligence regarding community needs to relevant agencies (Nicholls, 2012). The most effective disaster communication is locally relevant, so those at risk can access and act upon specific information about their household and risk reduction (Boon, 2014). Information transfer and knowledge networks, both formal and informal, required for resilience building, are dependent on effective communication. In a study of emergency management professionals in Australia, increased levels of communication were identified as the most significant opportunity presented by VGI (Haworth, 2016). Social media platforms, in particular, increase the speed and reach of communication between community members and emergency organizations, as demonstrated during the 2011 Cyclone Yasi and Queensland flood disaster (Taylor et al., 2012). Social media and other mechanisms for timely communication help spread important information, such as emergency warnings (Dufty, 2012), and are vital components of resilient communities (Nicholls, 2012). Further, online VGI platforms can reduce potential barriers to resilience building, including the inaccessibility of traditional communication forums, such as community meetings due to other commitments.

Understanding Risks and Associated Uncertainty

Increased awareness of local risk, hazards, and vulnerability shared through VGI enables individuals to make informed assessments about their individual and community preparation, planning, and likelihood of danger or damage. VGI through participatory mapping was useful for increasing risk awareness among youth in the Philippines (Gaillard and Pangilinan, 2010). Rather than simply trying to educate on risk through one-way communication, participatory mapping increases risk awareness and engagement in DRR by involving marginalized groups, valorizing their inputs, and materializing the hazard and risk as something personally relevant to the participants (Gaillard and Pangilinan, 2010).

Flanagin and Metzger (2008) report on the manifold increase in the number of information sources provided through social media and VGI. Research indicates that community perceptions and actions are influenced by exposure to risk and preparedness information, with Basolo et al. (2009) arguing individuals exposed to multiple sources of information may feel more knowledgeable about disaster preparedness. They found that receiving preparedness information from multiple sources was correlated with an individual taking protective actions.

Local Knowledge and Resource Sharing

The building and transmission of local knowledge² has been associated with increased adaptive capacity in social–ecological systems (Folke et al., 1998). Inclusion of local knowledge is crucial for increasing community disaster resilience (Council of Australian Governments [COAG], 2011; Giordano et al., 2013), and resilience-building strategies with an emphasis on local knowledge have positive impacts (Manyena, 2006). Yet, the integration of local and scientific knowledge to support disaster monitoring is not standard practice (Giordano et al., 2013). VGI has potential to facilitate increased collection, exchange, and use of local knowledge and resources in disaster management and resilience building. The ability of VGI to capture local observations and interpretations provides contextual data at fine spatio-temporal scales of relevance to communities that is otherwise unavailable in aggregate data. The merging of local knowledge and authoritative information sources can demonstrate differences in opinions and perspectives on risk or vulnerability, providing new insight for improving disaster planning. Participatory mapping can facilitate co-learning and encourage communication between stakeholders (Lynam et al., 2007). Giordano et al. (2013) demonstrated a participatory methodology for improving the usability of local knowledge from different stakeholders in analyzing drought impacts at Lake Trasimeno in Italy. Synthesis of knowledge from different sources allowed for identification of the impacts most relevant to local communities. Haworth et al. (2016) demonstrated both individual and community level benefits of community-based participatory mapping for bushfire disaster resilience, with participants reporting the personal relevance of shared knowledge and improved understanding of broader community bushfire risk and preparation levels as valuable to both community members and authorities.

Social Connectedness

Involvement in activities that engender a sense of community (feelings of belonging and attachment for people and places), efficacy, and problem solving, strengthen peoples’ disaster-resilience (Paton and Johnston, 2001). Sense of community fosters involvement in community disaster response and increases access to social networks (Paton and Johnston, 2001). Well-connected communities are able to draw on internal resources and competencies that will allow them to manage the challenges of future hazards (Frandsen, 2012) and the role of VGI in fostering social connectedness is well documented (e.g., Taylor et al., 2012). The social aspect of VGI, with people collectively sharing geographic information, has been shown to be even more valuable for community DRR and resilience building than local knowledge (Haworth et al., 2016). Removing constraints of time or geography, social media have made it simpler to interact with community members (Dufty, 2012). Analysis of usage trends during response to cyclones and floods found social media directly contributed to increased disaster resilience through promoting connectedness, with individuals feeling supported and encouraged by others, and was identified as a source of psychological first aid for those directly and indirectly affected by the disaster (Taylor et al., 2012). Social media can increase and improve social networks, leadership and support systems, and provide support to people during and after a disaster (Dufty, 2012). Social capital in the form of trust and social networks is a recognized source of resilience, enabling a social–ecological system’s capacity to adapt and shape change (Folke, 2006). Social media can preserve and strengthen existing ties and also facilitate the creation of new social relations (Dufty, 2012).

Empowerment

Volunteered geographic information promotes decentralization of top-down power held by disaster authorities and disrupts control over the production, handling, and dissemination of information (Haworth, 2016). McLennan and Handmer (2013) argue that for shared responsibility to be effective in developing community disaster resilience, control over decisions must also be shared. Through these shifts in control and power, citizens may become more empowered in their own disaster management decision-making, choices, and capacities.

Elwood (2002) framework for assessing three forms of empowerment associated with community-based participatory geographic information systems (GISs) can help to understand how VGI and social media can contribute to empowerment. The first element, distributive empowerment, relates to material changes and outcomes allied with greater access to goods and services and increased participation opportunities (Elwood, 2002). VGI has been shown to increase opportunities for public involvement in disaster management through tasks such as volunteer-mapping to assist with humanitarian aid efforts, including for individuals located outside the disaster location (e.g., Meier, 2012). Social media resources like Twitter have also empowered individuals with the desire to help in response to crisis events by enabling remote assistance provision (Starbird and Palen, 2011).

Procedural change involves shifts in processes resulting in communities’ contributions and knowledge being granted greater legitimacy in decision-making (Elwood, 2002). Responding to the 2010 Haiti earthquake crisis, VGI and the global citizen–volunteer mapping effort harnessing OSM, the Ushahidi Crowdmap platform, social media data, and information collected via SMS from impacted individuals (see Meier, 2012; Crawford and Finn, 2015) influenced decision-making in an unprecedented way. In this event, and inspired cases since, VGI technologies and practices facilitated collection, use, and legitimation of citizen contributions in ways previously unseen, demonstrating capacity for VGI to support procedural dimensions of empowerment.

And finally, capacity building improves the ability of communities to take action on their own behalf through skill acquisition, community-based knowledge production, or new understandings of community conditions (Elwood, 2002). VGI empowers citizens to georegister and transmit their own observations through the Internet (Goodchild and Glennon, 2010) and provides mechanisms for empowering people to help themselves and each other, thus enhancing community autonomy and capacity for independence from emergency authorities for various tasks (e.g., Paton and Irons, 2016). Community-based knowledge production and new understandings of community conditions have been associated with VGI through participatory mapping in community bushfire preparation (Haworth et al., 2016). Similarly, Taylor et al. (2012) described the empowerment of individuals and communities to help themselves during cyclones and floods where social media delivered a new mechanism for connecting with others, which for many constituted skill acquisition.

Preparation Engagement

Individual and community disaster preparation to enable effective emergency response broadly involves both physical preparedness activities (e.g., making changes to the home, storing emergency supplies, identifying safe areas) and psychological processes (e.g., formulating household plans, engaging with neighbors, considering how one might feel, or be affected emotionally during and after a crisis) (Frandsen, 2012). Frandsen (2012) demonstrated that community engagement is an effective, sustainable, and economical approach to increasing bushfire preparedness and disaster resilience. However, merely providing relevant information and community education doesn’t translate to meaningful engagement in disaster preparation (Lindell and Perry, 2000; McGee, 2005; Frandsen et al., 2011; McFarlane et al., 2011). Engaged preparation is dependent on inclusive and participatory approaches to DRR (Frandsen et al., 2011).

Volunteered geographic information provides diverse mechanisms for individuals to engage in DRR at multiple spatial scales. For example, global mapping efforts like the Humanitarian OpenStreetMap Team (HOT) utilize volunteers to co-create, curate, and disseminate free and up-to-date spatial information for disaster management³. These projects involving map creation for disaster preparedness (e.g., Malawi Flood Preparedness) harness the potential of the crowd, allowing large numbers of people to engage, improving broader disaster awareness, and increasing feelings of self-worth for contributors. The activities also provide geospatial products to assist local people and authorities to effectively engage in DRR. Although collaborative online mapping provides greater flexibility for individual involvement by removing geographical and temporal constraints (Haworth et al., 2016), participatory mapping through less technologically sophisticated platforms, such as paper sketch maps and three-dimensional maps using pushpins, have also been demonstrated to contribute to engagement in DRR (Gaillard and Pangilinan, 2010).

Collaboration Between Stakeholders

Collaborative disaster management is recognized as an important contributor to resilience but requires coordination and pre-existing trust between multiple government agencies, NGOs, private sector, and the community (Kapucu, 2008). Through VGI technologies and practices, opportunities are created for collaborative disaster management between individuals, communities, and authorities, where increased value placed on citizen-information and bottom-up activities can lead to co-operation.

The 2010 Haiti earthquake disaster, which occurred with a dearth of official high-quality spatial information, saw the emergence of crowdsourced and volunteered mapping to meet the needs of the humanitarian response (see Meier, 2012). Volunteers used OSM and the Ushahidi platform, Crowdmap, to trace satellite imagery, collate information from other online sources, and capture reports from people directly in the disaster area via SMS from mobile phones (Meier, 2012). Here, VGI created a new disaster response paradigm in which citizens (in this case largely citizens from other countries) take the lead (Levental, 2012), or at least complement the activities of authorities (Heinzelman and Waters, 2010), in emergency operations with consequences for disaster management globally.

Developing Flexibility

A resilient system must be flexible to avoid undesirable transformation (Allison and Hobbs, 2004), with inflexibility eroding adaptive capacity and leading to what have been termed “rigidity traps” (Carpenter and Brock, 2008). In a “rigidity trap,” strong self-reinforcing controls prevent the flexibility required for systems to adapt to change, thereby increasing the risk of system breakdown (Carpenter and Brock, 2008) and preventing movement to a more desirable regime if the current state becomes untenable. A potential rigidity trap lies in the top-down, “command and control” style of authoritative emergency management. The hierarchical, bureaucratic approach of emergency management, centered on agency control of information, risks limiting resilience through inflexibility and an inability to adapt to change. Web 2.0 and VGI assist in deconstructing this rigid system by redistributing power over information creation and sharing among other stakeholders, including private individuals (Haworth, 2016). Through VGI, people are able to contribute alternative representations of and responses to risk, increasing flexibility in the system with diversity of information.

For instance, social media provided flexibility in crisis communication for earthquakes in Japan and Haiti, bombings in India, and severe storms in North America and Australia, particularly when power and telecommunications were disrupted. Sites like Facebook and Twitter became more reliable with robust capacity to handle much larger activity volumes than other services (Yates and Paquette, 2011; Bird et al., 2012). Through exposing people to new ways of doing things, VGI and social media can add to future flexibility and resilience by improving citizens’ ability to be agile in other contexts.

Inclusion of alternate viewpoints enabled by VGI through participatory mapping can lead to greater flexibility incorporated into decision-making over disaster planning and response strategies. For example, local knowledge shared on a community preparation map showing potential evacuation sites may reveal alternative and improved site options to those designated by authorities. Participatory mapping provides an enabling environment for assessing risks, identifying solutions, and the integration of various strategy options into action plans (Cadag and Gaillard, 2012), and thus aids in building flexibility into community and authoritative disaster management.

Capacity for Self-Organization

The Internet and mobile devices are empowering individuals to organize themselves in ways previously unimaginable, as evidenced by activities such as Wikileaks, Twitter, and citizen journalism, or the formation of spontaneous volunteering groups like the Student Volunteer Army in response to the 2011 Christchurch earthquake (Neely, 2014). Social media platforms provided a space for volunteers to mutually self-organize with others in sharing information and resources in response to the 2010 Haiti earthquake (Starbird and Palen, 2011) and bushfires in Australia in 2013 (Irons et al., 2015), where VGI enabled a form of self-sufficiency and self-responsibility in sharing and coordinating people and resources previously not possible through more traditional disaster response approaches or communication media. A capacity for self-organization is particularly important in disaster management, where citizen-led initiatives are often more timely, responsive to local needs, and effective over the longer term than external responses (Scanlon et al., 2014; Whittaker et al., 2015).

The Bad: Potential for VGI to Undermine Community Resilience

Next we shift focus to the challenges presented by VGI for resilience-building in disaster management, which have potential to lead to undesirable action or erode a community’s capacity to adapt to the inherent uncertainty associated with living in high risk areas.

Compounding Risk

It is important to consider the compounding processes in which VGI practices may exacerbate disaster risk and the consequences for disaster response planning and resourcing. VGI initiatives with recognized benefits can also provide mechanisms for precipitating other events with potentially negative consequences. For example, in January 2013, a request for help posted on the Tassie Fires We Can Help (TFWCH) Facebook page (established by a community member to coordinate the sharing of information and resources related to a bushfire emergency in Tasmania, Australia) identified several people who had become isolated and needed supplies delivered (Australian Broadcasting Corporation [ABC], 2013). The only access route was via boat across open water. In response to the Facebook post, hundreds of people gathered together geographically with donated goods, and 35 boats were involved in delivering supplies (Australian Broadcasting Corporation [ABC], 2013). But when water conditions became dangerous, several boats required rescuing, thus diverting emergency response efforts and resources. In this instance, VGI practices triggered unsafe actions and increased the risk of harm to VGI users and others.

During the Parliament Hill shooting crisis in Ottawa in 2014 social media posts unintentionally endangered others (Zoltick, 2014). It was reported that individuals geotagged at the scene were tweeting live about the events as they unfolded to indicate they were safe and to keep people informed. This practice publicized their locations as well as the location of police response activities for potential exploitation by the then still-active gunman. These practices also have liability implications for emergency response agencies potentially acting on misinformation (see the section “Responsibility for Community-Led Initiatives”).

Data Quality and Trust

Data quality is a recognized challenge associated with VGI, including elements such as positional accuracy and content inaccuracies (see Ostermann and Spinsanti, 2011; Bird et al., 2012; Goodchild and Li, 2012; Haworth, 2016; Senaratne et al., 2017). During the Haiti earthquake response, locational uncertainty constrained the publishing of reports on Crowdmap to just 3,854 of the 15,000–60,000 reports collected (Morrow et al., 2011) and only 202 of those published reports were marked as “verified” (Heinzelman and Waters, 2010). In the 2011 Queensland floods Crowdmap, 75% of reports were verified by the map conveners, but many of these had been submitted by identifiable organizations, and, significantly, anonymous individuals were responsible for almost all of the unverified messages (Bittner et al., 2016). Pond (2016) argues that treating verification of crowdsourced data as a binary variable (verified or unverified) on platforms such as Ushahidi may limit the quantity of information that can be used for situational awareness. There is a need to consider the cognitive dimension of uncertainty and how perceptions of reported accuracy translate in decision-making processes (Bruce, 2004). Alternative approaches for communicating data uncertainty extending beyond standard authentication criteria for determining “truthfulness” and categories of data verification would allow the context-specific value of the information to be judged by end users. Methods for qualifying VGI-derived data, which are informed by the communities relying on those data, would not restrict the publishing of crowdsourced reports to those considered by “experts” as verified and would engage people in critical assessment of data sources as part of the disaster preparation process. Although this will assume a level of data literacy that may not always be present in end users such as responders, journalists, or the general public.

There is also potential for people to intentionally share information that is false or misleading, particularly through social media platforms, with demonstrated implications for disaster management (Mendoza et al., 2010), responses to terror incidents (Starbird et al., 2014), and financial markets (Rapoza, 2017). Vosoughi et al. (2018) studied news stories distributed on Twitter from 2006 to 2017 and found that false information diffused farther, faster, deeper, and more broadly than true information, although effects were stronger for false political news than for news related to terrorism or disasters.⁴ Gupta et al. (2013) studied the role of Twitter in spreading fake images during Hurricane Sandy, such as images of sharks swimming in flooded motorways. They found that most (86%) of the tweets spreading fake images were retweets (as opposed to original tweets) by a very small proportion (.3%) of Twitter users. While some research exists on classifying, measuring, and filtering false information, further work is needed to elucidate why people are motivated to share misinformation and to comprehend the wider implications of this behavior for community disaster resilience.

For resilience, uncertainties in data quality and source trustworthiness have important implications. Trust is important for social capital and is necessary for individuals to engage with collective activities, such as community or neighborhood groups, either online or offline (Valenzuela et al., 2009). Much of the uncertainty around VGI is due to a lack of known credibility of information sources and, therefore, trust (Flanagin and Metzger, 2008; Hung et al., 2016; Haworth, 2018). Low levels of trust in information provided by the general public were a rationale for Tasmanian community members’ limited willingness to engage on social media for bushfire communication (Haworth et al., 2015). Public perceptions of community-supplied information on Facebook during flooding events revealed VGI was more up-to-date than government information, but was considered less accurate and less trustworthy (Bird et al., 2012).

If trust in VGI is nurtured, for example through past positive experiences, future complications may arise if an initiative is not sustainable, platforms are not maintained, past key individuals are absent, or a community becomes reliant on a single system. If people expect a particular previous source of VGI (e.g., a Crowdmap or social media page) to be present this may cause tensions in communities, disappointment, or anxiety, all of which negatively impact elements of community resilience, such as social capital, empowerment, and individual and community confidence. Further, precedence does not necessarily equate to perceived reliability or future proof VGI initiatives.

Exposure to VGI and social media could be embedded in disaster preparedness programs, involving techniques for using, interpreting, evaluating, and contributing content to these platforms, and management of issues of data quality and trust. Encouraging learning and flexibility to engage effectively in these new technologies when confronted with a disaster event would strengthen community and individual resilience.

Under-Representing the “Crowd”

Due to the phenomenon of participant inequality (Haklay, 2016) we argue that VGI does not provide adequate opportunity for inclusive community participation. VGI inherently discriminates based on technology access and usage patterns associated with demographics, socioeconomic circumstances, and technical skills, knowledge, and interest (see the “digital divide”; Chinn and Fairlie, 2007; Sui et al., 2013). It is also important to remember that the platforms and tools used for VGI data collection and analysis have traditionally been created by an elite technology industry dominated by mainly Western men. As such, just as critical GIS has highlighted, “the uneven spatial distribution of technologies across the globe suggests an uneven representativeness of participation in the determination of how those technologies will be developed” (Burns, 2014, p. 3). Thus, the marginalized are under-represented in VGI and the tools and modes of data collection and analysis. Information gained through social media or other VGI sources will therefore only present partial and skewed representations (Crawford and Finn, 2015; Klonner et al., 2016; Haworth, 2018). Exclusion of some groups and individuals (Zook et al., 2010; Burns, 2015) means VGI practices may operate at the detriment of resilience building for some community members.

We must consider who can and does contribute VGI (and who creates the platforms they rely on). Bittner et al. (2016) critique the idea that crisis maps are the product of “ordinary citizens” pooling their expertise. VGI often represents the elite over the ordinary, as only those people with time, access to necessary social and technological systems, and required skills are able to contribute (Bittner et al., 2016). Less privileged individuals may be unable to participate, thus being excluded from the resilience-building benefits of VGI. Marginalized groups and individuals are often not represented on maps, and, particularly troubling for resilience, those people already marginalized are often the most vulnerable to disasters (Hewitt, 1997), especially since preparedness information is often not directed at vulnerable groups (Verrucci et al., 2016). Echoing critiques in the mid-1990s of the uneven community access to GIS technologies (e.g., Sawicki and Craig, 1996) as well as critique of participatory GIS in the 2000s (Elwood, 2006), Haworth (2018) recognizes the importance of understanding barriers to participation in community applications of VGI, and Burns (2018) urges us to recognize the social and political imbalances of big data, understanding that representations of the “real world” in data are rooted in spatial inequalities.

Volunteered geographic information may in fact enhance existing inequalities and vulnerabilities by further isolating those already economically, socially, or technologically disadvantaged, as has also been described for other technologies such as mobile phones (Read et al., 2016). Crawford and Finn (2015) showed how the crowdmapping efforts of the Haiti earthquake response exacerbated power differentials between the rich and the Haitian poor as VGI submitted in the local Kreyòl language was mapped in English only, excluding non-English speakers from benefiting from their own contributions. Thus, VGI curation practices and who is responsible may impact what is included in “ordinary voices” (Bittner et al., 2016). Furthermore, the geography of technology access has been linked with the geography of risk, whereby those in urban areas with greater Internet access also experience lower disaster risk in contrast to those in rural or isolated areas with poor Internet and mobile coverage but potentially greater disaster risk (Haworth, 2016). Even those with stable Internet access may have their contributions limited through forms of online censorship, language barriers, or page ranking algorithms, and hence in disaster scenarios VGI may represent only a privileged minority (Bittner et al., 2016).

The nature and characteristics of the “offline community” may also influence participation in an “online community,” particularly for social media, potentially limiting resilience building. For example, differences between urban and rural environments, whereby sense of community has been shown to be greater in rural communities, positively associated with social joining and negatively related to antisocial actions (Roussi et al., 2006), may impact online participation in more dispersed city communities, reducing the number of people who view, let alone contribute specific VGI together, undermining potential benefits such as increased social connectedness. This coupled with geographic disparities in Internet access further distorts the picture of who VGI actually represents and in what capacity (i.e., as passive viewers, information recipients, contributors). Further, it has been shown that strong offline community groups do not necessarily translate online, being unable to attract large numbers of friends and followers, with key stakeholders remaining passive in social media networks like Twitter (Williamson and Ruming, 2015). It has also been argued that VGI activities may be most useful for DRR at local community scales (Haworth, 2018).

Critiques of public participation GIS (PPGIS), where complexity of the technologies can contribute to marginalization of individuals contrary to the promise of citizen-empowerment (Corbett and Keller, 2005), also apply to VGI. Use of technologies required for participation in VGI practices, such as computers, the Internet, smartphones, social media platforms, location enabled mobile devices, satellite imagery, and online map-making software may result in a technological learning curve effect that precludes “non-experts,” potentially negatively impacting community disaster resilience with some people disproportionally benefiting while others are left off the maps.

Compromised Privacy and Security

The focus on community empowerment and democratic participation in much of the geoweb and VGI narrative has tended to equate power with public visibility and neglected to consider the potential for exploitation of VGI-derived data by external groups (Young and Gilmore, 2014). VGI is often publically available once contributed, potentially increasing vulnerability and risk exposure. Contributors’ information may be exposed to unintended uses, either by governments, companies, other individuals, or those with malicious intent. Greater openness exposes users to increased online security threats, such as malware, inappropriate content, and breaches of confidential information (Shanley et al., 2013).

Many people are not confident in their understanding of how and by whom their data can be accessed (Crawford and Finn, 2015). Moreover, in high-stress situations like disasters privacy may be less of a priority for individuals than in “normal” settings, and thus individuals may be increasingly vulnerable (Crawford and Finn, 2015). Lack of awareness of VGI features, such as image geotags captured from GPS-enabled smartphones, has had demonstrable implications for privacy, personal safety, and the protection of assets (Kruszelnicki, 2012). As data remain on the Internet after their initial use and can be repurposed in other ways, privacy and ethical issues persist into the future (Shanley et al., 2013). While some digital humanitarian initiatives have been designed to protect identities (e.g., the Libya Crisis Map; Meier, 2012), further critical research on ethical and technical dimensions is needed to establish methods for acquisitioning the value of geoweb-enabled platforms for disaster-prone communities while maintaining their privacy and security (Young and Gilmore, 2014).

Increased Community Tensions

Tension within communities generated or exacerbated by mapping activities or outcomes can undermine the resilience of disaster-prone communities. The TFWCH example in which well-intended vessel owners responded to a VGI-enabled request for assistance highlights the potential for serious negative outcomes and resultant tensions between those involved. Although facilitating capacity for self-organization, the outcomes of poorly orchestrated logistical response actions such as this may lead to feelings of blame and resentment; community ties could be broken, thus reducing community disaster resilience. Carroll et al. (2006) examined sources of social conflict following wildfires in the American West, finding that conflict can occur when social relations are disturbed by non-local entities, leading to a perceived loss of local agency. VGI can facilitate greater participation of “outsiders” in community responses to disaster, and therefore creates potential for increased community tension or conflict.

Using VGI and participatory mapping for disaster preparation can involve the identification of particular residents, properties, or areas as unprepared or problematic in the wider community context. This process may motivate residents to improve their level of preparedness, but may also provoke feelings of shame, guilt, or resentment toward those involved in the mapping. A sense of inadequacy in meeting the standards of others more actively engaged in DRR may weaken community connectedness and reduce an individual’s confidence in their ability to respond to disaster. Furthermore, VGI contributions may disclose personal or sensitive information unrelated to disaster potentially leading to tension.

Capacity to participate varies depending on status and position within the networks producing maps (Bittner et al., 2016). Therefore, VGI in disaster management is linked to the wealth, class status, community connections, reputation, politics, and power of particular community members. Disparities between individuals may be highlighted or aggravated by who can and cannot contribute. Berkes and Ross (2013) argue that a community resilience approach that integrates socio–ecological systems research and developmental psychology would give greater recognition to the importance of the ability to cope with divisions within community (Kulig, 2000 as cited in Berkes and Ross, 2013) and key resilience dimensions of community resources and collective action.

Responsibility for Community-Led Initiatives

Community-led VGI initiatives promote resilience benefits, including a sense of ownership and power, and foster innovation and projects that work for specific communities. But with ownership also comes responsibility and expectations, including project establishment, site/platform maintenance, and data management. Community-led social media pages, community maps, or other VGI initiatives require some knowledge and expertise in use of these technologies, and individuals with some level of autonomy and community trust who can champion these efforts. Not all communities will have access to this social capital – a further limitation to the broader social inclusiveness potential of VGI. In addition to knowledge and technological skills, responsibility for VGI project and data maintenance also requires considerable time and resources, which may become burdensome leading to volunteer fatigue (Deutsch and Ruiz-Córdova, 2015), thus reducing community connectedness and disaster resilience.

Failure or abandonment of VGI initiatives when contributions cease, or if the responsible personnel change or leave, can have consequences. It is important to anticipate potential turnovers which may lead to the abandoning of map updates or shifts in objectives or data use (Gaillard and Maceda, 2009). The sustainability of a small-scale bushfire risk mapping project across Australia, Bushfire Connect, was dependent on not only the sustainability of the volunteers, but the relationship of these volunteers with supportive emergency management professionals and how the participating/viewing public regarded the project and outputs, which ultimately affected its long-term viability (Bittner et al., 2016).

In line with shared responsibility, organizations and communities could work together on VGI initiatives in fostering community disaster resilience. However, consideration needs to be given to well-documented obstacles associated with agency involvement in participatory style mapping projects. These include undervaluing of community input, regularity barriers, unpredictability associated with public activities, lack of effective administrative structures, and lack of genuine community participation (Corbett and Keller, 2005; Brown, 2012). Further research is needed on the social learning value of community-led VGI initiatives, and community preparedness efforts will benefit from dissemination of positive outcome stories, a strategy known to aid in reducing barriers to participatory approaches such as volunteer dropout (Deutsch and Ruiz-Córdova, 2015).

Disruptions to Authoritative Emergency Management

The potential for VGI practices to disrupt official disaster management activities can reduce resilience in impacted or at-risk communities. Public trust gained for social media pages or VGI contributions for a disaster event presents the risk that the public will give precedence to these information sources over official disaster information. Disaster messaging is designed to be clear, concise, and consistent to provide the public with accurate and understandable information for given scenarios. Deviations from this messaging can be detrimental to community safety. Further, the accelerated rate of information exchange enabled by social media and VGI during an event may become incompatible with the logistics involved in coordinating response activities (Pond, 2016). Public preference for multiple disaster information sources emphasizes that VGI should not aim to replace more traditional forms of disaster communication (Taylor et al., 2012; Haworth et al., 2015). A balance between bottom-up and top-down practices needs to be maintained where appropriate value is procured from both systems for effective resilience development.

The Limits of Data as Knowledge

As has been noted, “data is not knowledge, nor is it capacity to analyse it” (Read et al., 2016, p. 1315). In order for VGI to contribute to building individual and community disaster resilience, it is important to recognize the blind spots and limitations of dominant ways of thinking about data. One of the key issues is practical; that the capacity to increase the amount of information collected about disasters through VGI and other crowdsourcing techniques has grown faster than the concurrent increase in the capacity to analyze that information, in many cases. This can lead to an expectations gap, where people believe sharing information in disasters will lead directly to receiving aid, when capacity to respond may not match demand. It also invites a focus on the new and novel in regards to data, often driven by advances in technology not response to need (Read et al., 2016), which can exacerbate the problems identified in the section “Responsibility for Community-Led Initiatives” around maintenance of platforms and ownership of data.

Additionally, increased and increasingly diverse data, which VGI offers, invite more technological solutions to the problem of analyzing those data. The computational techniques which define these new data forms (Boyd and Crawford, 2012), far from automatically empowering affected communities, can make possible the governance of disaster at a distance, through a reliance on “a cybernetic rationality” that “eschews causality or a need to acknowledge the motives and beliefs that shape actual behaviour” (Duffield, 2016, p. 147). The resilience that these developments offer is not progressive but a “case of survivalism through encouraging improvisation, making do, and inventive bricolage with existing communications infrastructure, architecture and social capital” (Duffield, 2016, p. 148). From this perspective, the optimism around both VGI and resilience is bound up with neoliberalism which “rework[s] disasters as a positive development opportunity” which requires communities experiencing disasters to take responsibility for their recovery (Duffield, 2016, p. 153).

Instead of leading to a growth in the inclusion of local knowledge, there is the potential for a loss of “ground truth” (Duffield, 2014) and a reliance on technological forms which, drawing on financial modeling and intelligence analysis are “concerned with rendering perceptible and actionable that which would otherwise be beyond the threshold of human observation” (Amoore, 2018, p. 1). This has contributed to what Burns (2014) describes as “moments of closure” through a fixing of the politics of knowledge in which these new technologies privilege and encode as more legitimate certain data forms and a “cybernetic rationality” (Halpern, 2015). This sediments a system in which data that can be analyzed computationally are privileged, and so too are those who have the skills to analyze it, undermining the valuing of local knowledge and knowledge exchange which, as earlier sections noted, have been cited as key to building community disaster resilience.

The Uncertain: Areas for Further Research

The application and relevance of emergent VGI practices to building community resilience require continuing critical debate involving disaster-prone communities, researchers, practitioners, and policy makers. Drawing on recent literature and policy documents, we considered key elements of resilience to examine the community disaster resilience-building potential of VGI practices, and obstacles that may undermine resilience-building processes. In this concluding section, we focus on areas of uncertainty where we perceive further work is needed. In doing so, we discuss various points of relevance to both practice-based disaster management and the disciplines of geography and disaster management.

Although beyond the scope of this paper, indicators and frameworks have been offered for measuring community resilience (Cutter et al., 2010; Renschler et al., 2010). Future work to examine the impacts of VGI practices on resilience through alignment with these indices may clarify, for example, whether the increased risk to personal safety and issues associated with the digital divide outweigh the potential benefits of VGI, such as community connectedness and risk awareness.

The evolving and relatively embryonic nature of VGI in disaster management means practical adoption of VGI into current emergency management approaches has been limited to date and harnessing of VGI has often been ad hoc or opportunistic. Despite a growing body of literature (Goodchild and Glennon, 2010; Ostermann and Spinsanti, 2011; Bird et al., 2012; Dufty, 2012; Meier, 2012; Taylor et al., 2012; Shanley et al., 2013; Paton and Irons, 2016; Pond, 2016; Haworth, 2018) and identified opportunities associated with VGI, on-going cultural change involving adaption by authoritative emergency management to less information control and increased citizen participation may be required before VGI in disaster management becomes a more established field with measurable outcomes for community disaster resilience.

With much research on the data and technological components (Granell and Ostermann, 2016), the social and behavioral elements of VGI are lacking critical analyses; directing greater attention to issues such as underrepresentation of the “crowd” and the nuances of power relations in online and offline social networks will build confidence in VGI as a resource in developing community resilience. Further research on the ethical and technical dimensions of VGI practices is required to inform the development of novel methods for maintaining the privacy and security of disaster-prone communities. Incorporating awareness and training on collaborative geoweb technologies, including data ethics, effective VGI practice, and potential digital footprint, into community preparedness programs will equip individuals to make informed judgments on VGI content during a disaster event and reduce unintended consequences of social media initiatives.

The positioning of VGI within critical GIS has been contested (McCall et al., 2015), but drawing on established approaches to community mapping from within the discipline of geography, such as PPGIS or participatory mapping (Brown and Fagerholm, 2015), may offer promise for understanding the social and behavioral elements of VGI and confining the use of VGI technologies to spatial scales most conducive to achieving the aim of increased community disaster resilience. By establishing which elements of VGI should be emphasized and which require careful management, a form of “controlled” or “facilitated” VGI at community scales with pre-established data systems, protocols, intended outcomes, and appropriate links to authoritative emergency management may be implemented. This can enable VGI in disaster management to be less opportunistic, data-centric, disruptive to authoritative activities and exclusionary, and allow for more reliable, community-centric, complementary, and socially inclusive practices. Recognition of the consequences of these divergent outcomes is critical for ensuring effective resilience strategies that encourage flexibility to cope with disaster, self-organizing responses, and inclusive participation.

Author Contributions

BH was responsible for defining the research aims, collating and reviewing the literature and previous research for the discussion, conceptualizing the article structure, writing the manuscript draft, and figure conception and execution. EB and JW jointly agreed upon the research objectives; provided supportive analyses and interpretations; and contributed, reviewed, and edited text. RR contributed and edited text, and critically reviewed the article aims, analyses, and discussion.

Funding

The Bushfire and Natural Hazards Cooperative Research Centre (Australia) supported parts of this research.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank the journal editors and reviewers for their constructive feedback and support.

Footnotes

^While citizen science and VGI both involve members of the general public participating in what have been traditionally “expert” or professional activities, such as science or map-making, an important distinction is that citizen science involves participation in defined scientific projects and VGI occurs in a wider range of contexts, with a variety of participant motivations, and is not necessarily attached to a specific project. Further, citizen science data and activities do not always involve a geographic component, as is always the case for VGI.
^The authors adopt a broad definition of “local knowledge” to mean any individual or collective knowledge possessed by local people for a particular location or community group, which may include, but is not limited to, knowledge of past events, indigenous knowledge, environmental and resource knowledge, and/or local social, economic, and political knowledge. Broadly, local knowledge is distinct from other forms of knowledge (i.e., scientific or professional) in that it typically derives from personal and lived experiences, and is often informal, tacit, and associated with shared values and histories (see Raymond et al., 2010; Giordano et al., 2013).
^https://hotosm.org
^The authors adopt a broad definition of “news,” which refers to any story or claim with an assertion in it. News was classified as true or false using information from six independent fact-checking organizations that exhibited 95–98% agreement on classifications (see Vosoughi et al., 2018).

References

Allison, H. E., and Hobbs, R. J. (2004). Resilience, adaptive capacity, and the “lock-in trap” of the Western Australian agricultural region. Ecol. Soc. 9:3. doi: 10.5751/ES-00641-090103