Ocean Integration: The Needs and Challenges of Effective Coordination Within the Ocean Observing System

Understanding and sustainably managing complex environments such as marine ecosystems benefits from an integrated approach to ensure that information about all relevant components and their interactions at multiple and nested spatiotemporal scales are considered. This information is based on a wide range of ocean observations using different systems and approaches. An integrated approach thus requires effective collaboration between areas of expertise in order to improve coordination at each step of the ocean observing value chain, from the design and deployment of multi-platform observations to their analysis and the delivery of products, sometimes through data assimilation in numerical models. Despite significant advances over the last two decades in more cooperation across the ocean observing activities, this integrated approach has not yet been fully realized. The ocean observing system still suffers from organizational silos due to independent and often disconnected initiatives, the strong and sometimes destructive competition across disciplines and among scientists, and the absence of a well-established overall governance framework. Here, we address the need for enhanced organizational integration among all the actors of ocean observing, focusing on the occidental systems. We advocate for a major evolution in the way we collaborate, calling for transformative scientific, cultural, behavioral, and management changes. This is timely because we now have the scientific and technical capabilities as well as urgent societal and political drivers. The ambition of the United Nations Decade of Ocean Science for Sustainable Development (2021–2030) and the various efforts to grow a sustainable ocean economy and effective ocean protection efforts all require a more integrated approach to ocean observing. After analyzing the barriers that currently prevent this full integration within the occidental systems, we suggest nine approaches for breaking down the silos and promoting better coordination and sharing. These recommendations are related to the organizational framework, the ocean science culture, the system of recognition and rewards, the data management system, the ocean governance structure, and the ocean observing drivers and funding. These reflections are intended to provide food for thought for further dialogue between all parties involved and trigger concrete actions to foster a real transformational change in ocean observing.

Background

The ocean is a complex dynamic environment with a strong interplay of physical, geochemical and biological phenomena at multiple and nested spatiotemporal scales. An integrated approach is thus essential to ensure that information about all relevant components and their interactions are considered. To adequately observe and predict the ocean state and variability across various disciplines and scales, ocean observations are best made from well-coordinated multiple in situ and remote platforms. An integrated approach therefore requires effective collaboration among the different domains of expertise, across disciplines and technologies, to optimally combine such multi-platform observations in a consistent ensemble of data. The goal is for these data to constitute a coherent dataset that can be used in the value-added chain and deliver useful products that provide information on ocean phenomena, sometimes through data assimilation in numerical models. However, except for some specific process studies or regional programs, this integrated approach has not yet been fully achieved, and this is an important challenge that needs to be addressed by the ocean observing community.

The value chain is a concept adopted from economics which allows to organize a system (in this case “ocean observing”) into subsystems, each adding value with inputs, transformation procedures, and outputs, in a continual and iterative process (Bahurel et al., 2010; Garçon et al., 2019; Pinardi et al., 2019). The term “ocean observing” thus refers to the whole value chain, from the initial stakeholder/societal/scientific engagement that identifies the requirements for observations and establishes the design of multi-platform observations to the process of collecting/calibrating/validating data to the creation and delivery of products and services, often through data assimilation in numerical models. Ocean observing therefore includes the modeling and ocean prediction communities.

Ocean integration is a multi-dimensional effort. It is used here to designate the process of efficiently collaborating among all the actors of ocean observing in order to optimally combine, across scales and disciplines, in situ and remote observations and numerical models, exploiting the complementarities among different types of data, to produce an observing system that is more relevant than the sum of its individual contributions. Ocean integration thus encompasses not only the integration of each element across the value chain, but also involves strong and efficient coordination between each of these elements in order to work toward a common goal and reach a system-level focus rather than a sum of network approach (JCOMMOPS Review, 2018). In situ and remote ocean observations and data delivery are at the basis of this value chain, and should therefore be designed and coordinated with all the stakeholder communities that exploit these observations to develop fit-for-purpose quality data, ocean prediction outputs and products and services, all of which benefit both science and society. The stakeholders include the engineering, technology, satellite, modeling, operational, weather, climate, and service delivery communities. Ocean integration therefore involves strong and efficient coordination between observing networks, between in situ and remote sensing, between modelers and observers, between disciplines and domains of expertise, and between researchers, institutions and nations.

Integration has been a general concern in ocean science for at least 20 years and particularly since OceanObs’09 (Fischer et al., 2010), since it is essential for the creation of value-added products combining multiple data streams. Moreover, it is crucial for the ocean observing system to fulfill the observational requirements of a wide range of users (e.g., Lindstrom et al., 2012; Moltmann et al., 2019). It is also now widely recognized that enhanced integration is needed to deliver more complete, consistent and sustained observations globally and better address the new and emerging scientific Grand Challenges [e.g., National Research Council, 2011; International Oceanographic Commission (IOC)-UNESCO, 2017; European Marine Board, 2019; OceanObs’19, 2019; Tanhua et al., 2019a]. Such coordination is also essential to reduce the duplication of efforts and optimize the limited resources available (European Marine Board, 2013, 2021; OceanOPS Strategic Plan, 2020).

An important advance has been the development of the “Framework for Ocean Observing” (FOO, Lindstrom et al., 2012), which has provided a useful framework to GOOS for guiding its implementation and strategy (IOC, 2019). The overall FOO strategy follows the value chain approach, and has provided the basis for designing basin-wide or regional ocean observing systems, such as AtlantOS in the Atlantic (deYoung et al., 2019), TPOS in the Tropical Pacific (Smith et al., 2019), or IndOOS-2 in the Indian Ocean (Beal et al., 2019), among others. An important concept of the FOO is the focus on Essential Ocean Variables (EOVs) that have been advocated by the ocean observing community based on feasibility and impact. The EOVs are independent of the observational platform, and have thus helped guiding the ocean observing system toward better integration across observing networks.

During the last decades, there has been considerable progress by the GOOS Observations Coordination Group in the capabilities and in the integration across the value chain of the individual observing networks (Argo, OceanSITES, SVP drifters, OceanGliders, HFRadar, etc.). Significant coordination has been established in areas such as metrics, standards and best practices, with the elaboration of the FAIR (Findable, Accessible, Interoperable, Re-usable) Data Principles (Wilkinson et al., 2016; Tanhua et al., 2019b) and the establishment of the Ocean Best Practices System (OBPS, Pearlman et al., 2019). Considerable progress has also been made by GOOS toward enhanced collaboration among national systems, regional alliances, global networks, and in situ observing and remote sensing (Moltmann et al., 2019). All this has contributed in making substantial advancement in the process of integrating multi-platform and multi-disciplinary data and modeling for the creation of value-added products. We could cite as examples of this progress the basin-wide and regional ocean observing systems such as the above-mentioned TPOS, AtlantOS, IndOOS, as well as IMOS in Australia, IOOS (and its Regional Coastal Ocean Observing Systems) in the U.S., and EuroGOOS, MonGOOS, and the Copernicus Marine Service (CMEMS, Le Traon et al., 2019) in Europe, among others.

Despite this progress, there are still many areas where more coordination is needed. As detailed in deYoung et al. (2019), issues remain that are hindering these systems from becoming fully integrated. In this paper, we focus on the barriers that prevent a full integration within the European/North American/Australian systems, where the authors have expertise. We do not represent all the ocean communities, and our views are not necessarily representative of all regional associations. There are, of course, routine ocean observations in countries outside this geographical area, but the integration issues in these countries and regions may vary according to their specific cultures and policies, and an evaluation of how best to achieve full international coordination is outside the scope of this paper.

In occident, the major obstacles to ocean integration are the insufficient collaboration between the observing networks and systems,² the too-narrow focus of the existing observational networks, and the lack of reliable long-term funding. The ocean observing systems still suffer from organizational silos due to the large number of disconnected initiatives, the strong and sometimes ‘‘destructive’’³ competition between scientists (Fülöp and Orosz, 2015) and institutes, and the lack of a clear governance structure for coordinating the end-to-end value chain (e.g., Davidson et al., 2019). In many cases, each network, team or nation establishes their own priorities and direction without substantial interaction with others (European Ocean Observing System [EOOS] Conference 2018 report and Call to Action, Larkin et al., 2019 Ocean Observation European Commission Consultation Inception Impact Assessment¹). As a result, the existing networks tend to be focused on specific scientific and/or societal needs and are not well integrated with other observing systems, even in the same geographical area (Tanhua et al., 2019a).

This lack of integration provides an unnecessary impediment to closing gaps in ocean observing coverage. Some important processes are insufficiently measured, limiting our ability to precisely quantify some relevant mechanisms and interacting scales related to global and societal challenges, such as climate change (IPCC, 2019 Special Report on the Ocean and Cryosphere in a Changing Climate) and a sustainable management of the ocean health, as well as operational services. Moreover, the sparsity of ocean observations and the lack of internationally agreed data standards makes implementation of data assimilation, data science and model verification frameworks difficult (National Academies of Sciences, Engineering, and Medicine, 2017; Davidson et al., 2019), which prevents the science of ocean prediction from advancing at a faster rate. This lack of coordination also restricts data sharing and the implementation of the FAIR principles (Tanhua et al., 2019b), rendering a lot of observations not findable, accessible and interoperable, and therefore not re-usable. This leads to a non-optimum use of resources (European Marine Board, 2013) and limits investment (Smith, 2021 internal report).

What We Need to Achieve

We have reached the point where it is vital to break down the silos and promote better integration among all the components of the ocean observing system. According to the European Commission public consultation on Ocean Observations in November 2020, it is clear that there is a consensus on the need for a better organization of the ocean observing system.¹ This is timely because we now have the scientific and technical capabilities as well as urgent societal and political drivers. The unprecedented scientific and environmental challenges facing our society today call for “big science” based on “big data,” and requires leadership and a mission-oriented approach, i.e., driven by a clear top-down direction setting, while progressing through bottom-up research and innovation (Mazzucato, 2018). We therefore need to foster a more unified and collaborative environment that allows for better coordination and sharing, enabling each team to fit their work into the bigger picture. This is crucial for delivering the ocean science required to boost and assist better ecosystem-based management of the ocean (Visbeck, 2018; Lubchenco and Gaines, 2019; OceanObs’19, 2019, Pendleton et al., 2020). Moreover, it is also essential for facilitating global communication and mutual learning across research and stakeholder communities, which is a key objective of the United Nations Decade of Ocean Science for Sustainable Development 2021–2030, hereafter referred to as the UN Ocean Decade.

To reach this goal, all members of the ocean observing system need to establish a shared vision and commit to common priorities to form a unified whole much bigger and relevant than the sum of its components. This is what is called “organizational integration,” which is defined as “the extent to which distinct and interdependent organizational components constitute a unified whole, rapidly and adequately responding and adapting to each other while pursuing common organizational goals” (Barki and Pinsonneault, 2005; Ricciardi et al., 2018). This organizational integration calls for a major evolution in the way we work with each other, and implies cultural, behavioral and management changes. We need to achieve an organizational innovation in ocean science, and move beyond a “business as usual” approach to a new innovative functional organization. Private sector companies are accustomed to adapting their organizational procedures to changes in external conditions such as the economy or the market. They have well-proven strategies for helping their human capital adjust to these changes. Unfortunately, such adjustment does not happen as frequently in the public sector, especially in science and education which remain very much discipline-oriented, often stuck in silos (Tress et al., 2007; OECD, 2020).

Today we are living in a time of change, with an international trend toward more open science, more open data infrastructures, and a more concerted effort to conduct mission-oriented research (Mazzucato, 2018), and to translate science into services for society (e.g., World Meteorological Organization (WMO) reform of 2019⁴ ; UN Ocean Decade). We have therefore reached a point where the ocean observing community needs to trigger internal organizational changes to ensure an optimal and efficient response to the priorities of international science and society needs. As called for by Lubchenco and Rapley (2020), “It is time for strategic, collective action to change the culture of academia and create the enabling conditions for science to serve society better.” And this can only be achieved if the scientists themselves, including institutional leaders in academia and ocean governance, shape this ambition and embrace it.

Organizational integration aims at directly benefiting all parties, including the scientists themselves, and is fully compatible with their individual intellectual and financial autonomy. This work began within the framework of the EU-funded EuroSea project⁵ (Task 3.9), by independent researchers whose primary focus is ocean science. However, it has rapidly evolved toward a more international and multi-sectoral dimension as described in this paper. We are well aware that there are gaps in our understanding of ocean processes that prevent our capacity to resolve major critical human needs. We therefore acknowledge the need and relevance of basic process studies to understand the relevant interacting scales, and we are convinced that creativity and diversity are essential factors for the advancement of knowledge. However, a collective and coordinated vision can add significant value to individual knowledge and help in advancing toward a transdisciplinary and complete understanding of the picture.⁶ Moreover, individual researchers are the first to suffer from a non-collaborative and hypercompetitive culture, and a lack of good management and leadership is linked to many poor research practices that have an adverse impact on their well-being (e.g., Van Noorden, 2018; Bleasdale, 2019). Developing a shared vision fosters a culture of exchange and constructive competition,⁷ enabling the community to constructively collaborate on challenging issues in a more positive and pleasant working environment. It is interesting that in a sector such as the economy of the sea, experts are also calling for an integrated approach to guarantee a proper balance between all stakeholders, taking into account the differing and sometimes conflicting needs of each of them (Marques, 2020). Such an integrated approach would lead to significant benefits regarding sustainability and innovation (Marques, 2021).

The Challenges Ahead

Organizational integration must build upon well-defined and common goals, so that all partners work together collaboratively toward a shared purpose and vision. In this section we analyze some examples. In oceanography, an example of organizational integration has been the Global Ocean Data Assimilation Experiment (GODAE) whose aim was to demonstrate the feasibility and utility of real-time global ocean forecasting (Smith and Lefebvre, 1997; Bell et al., 2009). In order to complement satellite observations, the international Argo program was developed as a joint venture between GODAE and CLIVAR (Le Traon, 2013), and the integrated approach merging in situ Argo observations with satellite altimetry and numerical models allowed to establish the state and variability of the large-scale open ocean circulation, at the heart of the development of operational oceanography and of the present European Copernicus Marine Service (CMEMS, Le Traon et al., 2017). Another example of organizational integration is the worldwide satellite altimetry community, which brings together a wide range of expertise and combines multiple observations and data processing methods to obtain and publicly disseminate the final gridded sea level product (International Altimetry Team, 2021). Outside the field of oceanography, meteorology also offers a good example of an integrated system focused on ensuring that the delivery of meteorological services responds to societal needs (Pinardi et al., 2019; World Meteorological Organisation, 2019).

There are, however, fundamental differences between these examples and the ocean observing system. The major difference is that these examples are mission-oriented, driven by a well-defined primary operational purpose, and operated on a top-down approach. Most of the present ocean observing system, on the other hand, is multi-faceted and led by a bottom-up science-based approach, and therefore driven by scientific objectives such as discovery, understanding and excellence. The majority of national and regional agencies responsible for funding and running in situ ocean observing systems are research-based, rather than operational. In many cases, at local to regional scales, in situ observations are led by individual scientists working to establish themselves as principal investigators as part of their career ladder. The framework of these projects is driven by research agendas, and thus suffers in areas such as integration, sustainability and sharing, beyond the immediate interest. Local and regional environmental consultants also collect and accumulate quite a large amount of data for their reports, in many cases responding to requests from public entities. However, such data may not be at all findable or quality-checked or trusted. To implement integration, we rely on the goodwill of the various actors in creating harmonious and effective coordination and fostering open science. At a global level, the GOOS Observations Coordination Group has been successful in enhancing the capabilities of the individual observing networks (Argo, OceanSITES, SVP drifters, OceanGliders, HFRadar, etc.), and significant coordination has been established in areas such as metrics, standards and best practices, new technologies, and data (Moltmann et al., 2019). However, as mentioned above, these networks have not yet reached the same level of maturity and do not yet fully exploit their complementarities.

In addition, the examples given above (e.g., satellite altimetry, meteorology, etc.) benefit from a strong leadership and a well-defined governance system. In the case of satellite altimetry, a clear governance structure with a few key players was established, mostly led initially by research agencies, NASA, CNES, and ESA, and now extended to the Committee on Earth Observations Satellites (CEOS) and operational agencies (NOAA, Eumetsat). In the case of meteorology, WMO is a strong centralized organization where nations contribute based on a treaty, with clear mandates and legal obligations. In the much wider field of ocean observing, the governance structure is unclear and fragmented (International Oceanographic Commission (IOC)-UNESCO, 2017; Tanhua et al., 2019a; European Marine Board, 2021; Smith, 2021). A large variety of institutions and initiatives deal with ocean management and governance at local, regional, national and international levels (Valdés et al., 2017; Muñiz Piniella and Heymans, 2020). However, they often overlap geographically and/or in their mandates or subject agendas, with only marginal coordination between them. At national level, responsibilities and financing for ocean observing activities are often distributed across multiple ministries and sectors, and unfortunately, formalized national coordination structures such as national focal points or national ocean committees are rare or dysfunctional (deYoung et al., 2019; Lara-Lopez et al., 2021; Smith, 2021). Internationally, ocean affairs are spread throughout a number of UN organizations (Valdés, 2017), with some leadership for ocean observing coming through GOOS and IOC-UNESCO. To integrate national systems into regional ones, thirteen GOOS Regional Alliances (GRAs) have been created, covering most regions of the globe. Yet there is significant heterogeneity in the governance, funding and data policies and indeed the performance of GRAs (Moltmann et al., 2019; Tanhua et al., 2019a). The present form of arrangement lacks authority, clarity and transparency, which leads to confusion around roles, responsibilities, accountability, leadership, and cross-support system engagement and coordination (Smith, 2021).

Another difference between meteorology and the ocean observing system is that public and political interest in weather forecasts has led to sustained funding, including private companies lobbying to maintain observing infrastructures (e.g., Spiegler, 2007; Weller et al., 2019; National Academies of Sciences, Engineering, and Medicine, 2020). In comparison, the global and regional-scale ocean observing systems mostly lack reliable sustained funding. Most in situ observing networks are financed through research projects that are subject to 3--5-year funding cycles, with no funds or time-budget allocated to cover the costs and the time needed for the coordination with other research projects. Integration is very time-demanding and requires stability and long-term planning, which is difficult without appropriate resources and support structures. Sustained core funding associated with a centralized infrastructure greatly enhances integration, as seen for example in the U.S. Integrated Ocean Observing System (IOOS) and the Australian Integrated Marine Observing System (IMOS). In Europe, for more than 10 years, there have been discussions on how to improve ocean governance and create new innovative funding models to support the ocean observing system (and more specifically the in situ system) as a public-good infrastructure⁸ (e.g., European Marine Board, 2013; Expert group on marine research infrastructures, European Commission, 2013). One possible solution currently envisioned is a European hub-and-spoke model, similar to IOOS and IMOS, with a centralized European backbone entity with subscription based or binding Nationally Determined Contributions, combined with national infrastructures (European Marine Board, 2021).

To support ocean observations, there has been a debate in the US on how to expand funding opportunities (National Academies of Sciences, Engineering, and Medicine, 2017, 2020; Weller et al., 2019). It has been argued that, while the ocean observing system should be primarily a federal responsibility, a new, flexible and nimble organization engaging with non-profits organizations, philanthropic organizations, academia, government agencies, and the private sector could address some of the challenges in maintaining sustained observations. This organization should be based on the principles of a collective impact organization (Kania and Kramer, 2011), which gives a framework for organizing a decentralized landscape of stakeholders interested in addressing a common complex issue. This governance model is similar to, and founded in, the polycentric multi-level governance promoted by Ostrom for managing the world’s common resources (Ostrom, 2010), and could be a model that could be drawn upon to create an efficient ocean governance that would provide a sustained foundation for ocean integration. A collective impact organization involves establishing a common agenda, continuous communication, shared metrics, a backbone infrastructure with a dedicated staff, and mutually reinforcing activities among all participants. This framework is similar to that of the Integrated Coastal Zone Management (ICZM, e.g., Cicin-Sain and Knecht, 1998; Diedrich et al., 2011) and much could be learned from some of the solid advances in the implementation of ICZM guidelines worldwide (e.g., The Pegaso Project 2014⁹).

Finally, the lack of collaboration among networks, institutions and scientists stems from insufficient incentives to participate in the coordination process in our science culture. Indeed, integration is a matter of long-term collective multi-dimensional efforts, which can be out of line with traditional career-advancement metrics. Research is currently assessed essentially through bibliometric indicators, making publication the primary objective of research, at the expense of the other aspects of the research mission such as delivering solutions to societal problems that are regarded as “unproductive” activities by scientists since these activities do not tangibly further their careers (e.g., Hicks et al., 2015; Wilsdon et al., 2015; Delgado-López-Cózar et al., 2021). Moreover, the evaluation process predominantly targets individual researchers or individual networks rather than teams, departments, or observing systems. This reinforces the silos and can have a distorting impact on integration, interdisciplinarity, information sharing and well-being more generally (e.g., Coriat, 2019; Moher et al., 2020). Given the societal expectations of ocean science in the context of the UN Ocean Decade, there should be concern about the misalignment between these expectations and the way scientists are evaluated for their work. Many of the skills required for effective collaboration, such as communication skills, community relationship building, and the open-mindedness to constructively debate, are undervalued in academia (e.g., Lubchenco and Rapley, 2020; Hernández-Aguilera et al., 2021). This acts as a disincentive for scientists contemplating participation in coordination activities, and threatens the continuity of the workforce to sustain ocean observing in the future (National Academies of Sciences, Engineering, and Medicine, 2017; Weller et al., 2019). Excellence and rigor are still one of the pillars in an integrated approach, but there is a need to also recognize and reward engagement with the community, communication, coordination and sharing, as well as the essential need to document data and observation/analysis methods (Pearlman et al., 2019). As Lubchenco said, “not all scientists will want to (or should!) engage, but all should value and support those who do” (Lubchenco, 2017).

This is in line with the general international trend toward the redefinition of scientific excellence (e.g., Benedictus and Miedema, 2016; Nature Editorial, 2018), which considers that research should be assessed in terms of multiple aspects rather than just on journal-based publication metrics (Raff, 2012; Leiden Manifesto, Hicks et al., 2015). The development of new indicators together with a more careful and reflective human review to ensure an appropriate translation and interpretation of indicators are regarded as strongly necessary. Change is progressively taking place: several European funding agencies are now adopting the use of narrative curricula vitae formats, and various institutional initiatives have proposed new evaluation models (e.g., University of Cambridge¹⁰ and Exeter,¹¹ UK; Macquarie University,¹² Australia; Utrecht University, Netherlands¹³ ; see the blog, resources page, and case studies of the DORA website¹⁴ for more examples). Dutch research institutes and funders have come up with a most notable initiative. They have announced the development of a new system of recognition and rewards, with equal emphasis on five key areas of education, research, impact on society, leadership and (for university medical centers) patient care, as detailed in the position paper “Room for everyone’s talent: toward a new balance in the recognition and rewards of academics” (VSNU et al., 2019). Considering that “it is unrealistic as well as unnecessary for each academic to excel in each of the key areas,” they advocate for a greater diversity in competences and talents among academics. Among other things, bibliometric publication indicators will no longer be used and the inclusion of research output on curricula vitae and application forms will take on a more narrative form. At Utrecht university for example, the impact factor is formally abandoned and by early 2022 every department will judge its scholars by other standards, including their commitment to teamwork and their efforts to promote open science (Woolston, 2021).

Moving Forward

This ground-breaking advancement in the way Dutch academics are evaluated and rewarded is fully aligned with the spirit of the UN Ocean Decade, which aspires for real transformational change and encourages the scientific community to think beyond business as usual. To advance toward a more integrated approach in ocean science, we may likewise need to change the status quo and rethink some parts of our ocean science system. In the following subsections, we suggest nine recommendations (Figure 1) focusing on those aspects that we consider as essential for initiating cultural, behavioral, organizational and management changes within the European, North American and the Australian scientific community in order to reach a better organizational integration in ocean observing. We consider that all the proposed approaches are complementary and necessary, and the order of the items does not reflect priority ranking. However, the first four points (i.e., redefining scientific excellence, agreeing on a common agenda, redesigning ocean governance and elaborating sustainable funding mechanisms) are certainly among the most important since they would lay the foundation upon which this ocean integration could be built and from which the other points could naturally derive. They should therefore be the main priority.

FIGURE 1

Figure 1. Overview of the challenges to be faced in order to reach efficient integration within the ocean observing system. The most critical ones are highlighted in gray.

Redefining Scientific Excellence

Dealing with the multi-faceted challenges of integration requires a long-term vision and the interplay of a rich diversity of talents and skills. The incentive system of ocean science thus urgently needs to change in order to meet current needs and allow for greater diversity in possible career paths and profiles. Research assessment should focus more on long-term outcomes and on the overall ocean observing system benefits than on short-term results. The contribution to ocean observing systems and their implementation (including governance) should be highly valued, as well as the participation in coordination work. Sharing data and the methods for collecting or creating data is key to this integrated approach (more details in section “Coordinating Data Management and Delivery”). Gathering data and providing FAIR and open data in a timely manner, together with the methods and best practices used to collect or create the data (Pearlman et al., 2021), should therefore be properly recognized and rewarded (Hodson, 2018; Tanhua et al., 2019b). This would help to foster a cultural change toward more information sharing, encouraging scientists to make their data and methods FAIR and publicly available even before publication. Moreover, the long-term implication of data gathering needs to be considered. Finally, communication activities, training, education, and mentorship should also be duly recognized, as well as the compliance with the Responsible Research and Innovation (RRI) principles.¹⁵

In light of the growing worldwide trend toward reforming the research evaluation system aforementioned, and following the initiative of the Dutch research institutes and funders, a new system of recognition should be developed, with balanced emphasis on publications, coordination, contribution to the overall ocean observing system, impact on society, data gathering, data management and sharing, best practices development, communication, education, mentorship, training, leadership and team building, without constraining scientists to excel in all these areas. This would encourage the scientific community to be more deeply involved in coordination and integration tasks, and would ensure a greater recognition of the technical, engineering, communicative and collaborative aspects of the work that are indispensable for the achievement of ocean integration. It would also contribute to a more diverse, equal and inclusive ocean observing community and make the ocean science system a more coherent whole. A diverse set of metrics should be developed to quantify these accomplishments, and inappropriate indicators that create the wrong kind of incentives such as journal impact factors and citation counts should no longer be used, in favor of more comprehensive metrics and a more narrative-based assessment.

Agreeing on a Common Agenda and Principles

Building on the mission-oriented approach proposed by Mazzucato (2018) to address the global challenges of our time, on the principles of collective impact organizations described by Kania and Kramer (2011) and put forward by Weller et al. (2019), and on the principles of ICZM (e.g., EU ICZM Recommendation 2002/413/EC¹⁶), we suggest applying the same approach to establish a more mission-oriented and mutually supportive scientific community. To work collaboratively toward a shared vision, a common agenda that will define the missions on which to focus should be agreed. As explained by Mazzucato (2018), missions should be broad enough to engage the whole community, but focused enough in order to join forces toward clear goals. Missions do not specify how to achieve success, but stimulate the development of a wide range of different solutions to achieve the common goal. It is thus a powerful tool for establishing a clear direction while fostering bottom-up research and innovation solutions.¹⁷ The ocean observing community has therefore everything to gain from addressing the Grand Challenges related to the ocean as missions. This will result in an innovation spillover that may have many unforeseen positive benefits on ocean science (Mazzucato, 2018).

To achieve this agenda, the ocean observing community should agree on fundamental principles that will serve as governing rules for reconciling divergent interests or perspectives and setting up a joint approach, similar to the above-mentioned principles on which ICZM is based. Those principles should define the primary objectives of the integrated approach, the various dimensions that need to be integrated (platforms, networks, regions, disciplines, modelers and observers etc.), the common agenda to which all members commit, the responsibility in which they engage, their respective central roles, and the ways progress will be measured and reported. These principles should determine the essential elements with standards of data and measurements, and the requirements for an observing element to be considered as respecting the principles of integration. Regional components should be agreed in order to align global priorities with regional ones and create benefits at local, national and regional scales, but all regions should have a common baseline. Those principles should serve as guidelines for all members to establish their own specific strategy and activities. Each organization is free to chart its own course as long as it fits into the overarching plan and is coordinated with the actions of others. The success of organizational integration will come from the back-and-forth discussions between all members and from the coordination of mutually reinforcing activities.

Redesigning Ocean Governance at Various Levels

Integration does not happen automatically and should therefore be seen as a necessary part of the work that needs to be tackled from the beginning (Tress et al., 2006). Coordination takes time, so providing adequate financial and human resources and agreement on the responsibility to coordinate the integration process is essential. Such work requires a separate infrastructure with a dedicated staff with a very specific set of skills to serve as the backbone, just as in collective impact organizations. As indicated by Weller et al. (2019), this backbone infrastructure is not designed to impose things or to set the common agenda, but should be responsible for providing strategic high level intentions and organizational capabilities to coordinate the integration process and to plan, manage, carry out and support that agenda, based on the fundamental principles previously agreed by all members. As most ocean observations are funded at a national level, permanent national organizations such as ocean agencies, equivalent to the meteorological agencies, involving both scientists and funders and supported by a sound governance structure, should be set up to improve the communication between the various national observing systems, with significant savings in maintenance and operating costs. It would also enable countries to act more coherently at the international level, and improve the dialogue with policy makers and funding agencies. However, in some cases, regional organizations might be more efficient than national ones for harmonizing and coordinating the ocean observing value chain at the regional scale. In Europe, the European Ocean Observing System (EOOS) could be designed as the backbone entity for harmonizing and coordinating these national and regional organizations together with the European ones, with a coordination body funded with subscription-based or binding Nationally Determined Contributions, as proposed by the European Marine Board (2021).

At a global level, GOOS should act as the international backbone infrastructure providing authoritative guidance and supporting the decentralized landscape of ocean-related organizations worldwide, coordinating local, regional, national and basin-wide organizations to span all spatial scales and disciplines. To reach this goal, the GOOS infrastructure should be rejuvenated in order to develop the organization’s ability needed to meet these objectives. We therefore fully support the current work that is now being carried out within GOOS to reorganize its governance.¹⁸ GOOS should have a dedicated staff, separate from the other ocean observing system organizations, who can plan, manage and support the integration process through ongoing facilitation, technology and communications support. There should be clarity and transparency about the respective responsibilities of GOOS, the regional organizations and the national ones, with clear connections between them. A GOOS funding model similar to the one proposed for EOOS, with binding contributions from national and regional entities, might perhaps reinforce their engagement with GOOS. Finally, as also recommended by Moltmann et al. (2019), the benefits of being part of GOOS need to be much more apparent to countries, institutions, programs and ocean observers. In addition to incentivizing researchers and facilities for their contribution to the overall ocean observing system (see section “Redefining Scientific Excellence”), there is also a strong need for GOOS to work on building leadership in order to become more attractive. Providing authoritative guidance and assistance for establishing the design and implementation of observing systems (see section “Establishing Clear Design and Implementation Plans”), for helping the transition of observing systems from research to operational (see section “Facilitating the Transition From Research to Integrated Sustained Operations”), and for leading the interdisciplinarity process (see section “Connecting the Diverse Communities”) would certainly be elements contributing toward achieving this objective.

Elaborating Sustainable Funding Mechanisms

The ocean observing system should be considered as a public-good, and sustainable funding should support its long-term collective nature. Given the economic importance of ocean observing (see section “Efficiently Communicating the Value of Ocean Observing”), public, private, and academic sectors could cooperate with mutual benefits, as is the case in meteorology with the Weather Enterprise (e.g., Spiegler, 2007). The ocean observing community could create an equivalent level of visibility, lobbying, and advocacy for funding, for example through initiatives such as the Benefits of Ocean Observations Catalog¹⁹ drawn up by IOOS. However, to be convincing, the ocean observing community needs to be better organized to reach out to the public in a more coordinated and impactful manner. As Smith said, “support will begin to flow once the “house” is in order and is investible” (Smith, 2021). Moreover, a common agenda should be agreed so that the ocean observing community all align behind clear priorities for missions (see section “Agreeing on a Common Agenda and Principles”). This would help to attract sustainable funding, for example from Member States subscriptions, as is the case for WMO. In addition, the financial needs should be quantified in order to provide concrete numbers on what is needed to support the system. There is currently no robust estimate of the cost and human capital of the ocean observing infrastructure needed, as this cost depends on the purpose and final design (Kite-Powell, 2009). This makes the discussion with policy makers and funders difficult (e.g., National Academies of Sciences, Engineering, and Medicine, 2020; European Marine Board, 2021). Designing how a fully integrated ocean observing system might look (see section “Establishing Clear Design and Implementation Plans”) is therefore the first step for estimating the resources and time-budget required.

In addition, overheads need to be clearly estimated and funded. Although integration undoubtedly has a strong impact on the cost-effectiveness of the system, it requires substantial financial base that is generally not supported by research-funded projects. New types of funders such as mission-based entities are therefore needed to help create and sustain the long-term collective process, recognizing that ocean integration will not come from a single breakthrough, but through the gradual alignment of all parts of the system. At a national level, there are already initiatives to fund more mission-oriented projects (i.e., Ocean-Shots in the US). Global and regional programs should therefore be in phase with such national initiatives. Finally, the role of the private sector needs also to be recognized, both as a provider of technologies and as a key intermediate user for the creation of services tailored to specific end-user needs. This is what is expected for the development of the “New Blue Economy” (Hotaling and Spinrad, 2021), with technology providers, public agencies and public and private intermediaries delivering value-added information products and services to a wide range of end-users. The “Ocean Enterprise” commercial activity is already significant in scale and scope (NOAA, 2017, 2021; Rayner et al., 2019a), and a greater connectivity with the commercial players could strengthen the organizational and financial case for the ocean observing system.

Connecting the Diverse Communities

Observational data are only useful for a given application if they meet the specific requirements needed for this application. The planning and design of ocean observations therefore need to be carefully thought out and discussed across all ocean-related communities in order to improve the return on investment and produce data that are fit for multiple purposes. Far too often observational data, or the associated meta-data, do not meet key requirements and cannot be used to their full extent. For example, of the large volume of in situ data collected, only a limited portion is actually directly suitable for satellite calibration/validation activities, either because the measurements are not directly comparable, or because the observation is not properly located or provided with enough accuracy, or because the data are not accessible (Sterckx et al., 2020). Similarly, a large number of in situ observations are not assimilated into numerical models due to insufficient spatiotemporal coverage (National Academies of Sciences, Engineering, and Medicine, 2017), or because the data are not provided with enough details about their quality, accuracy and precision.

Engagement with stakeholders to understand their needs and requirements is thus essential for an efficient design of the ocean observing value chain. The planning and design of an integrated ocean observing system will therefore need to be defined jointly with other scientific communities, such as the weather, the climate, the satellite, the modeling, the operational and the service delivery communities. For some dimensions (e.g., coastal ocean, ecosystem state, fisheries), the perspectives of local and indigenous communities as well as other coastal-ocean related disciplines should also be considered. This requires overcoming the silos between disciplines and going beyond conventional practices by including more diverse perspectives within the ocean observing community. For example, an increased intergenerational exchange by mixing early, mid- and late-career professionals could result in broader perspectives and more integrative approaches. At a higher level, closer collaboration is needed between the GOOS OCG, the GOOS discipline-based expert panels, the Expert Team on Operational Ocean Forecasting System (ETOOFS), the Joint WMO-IOC Collaborative Board, the CEOS working groups, as well as with other programs such as CLIVAR,²⁰ OceanPredict,²¹ and CoastPredict.²² For example, an internationally agreed organization could help to coordinate communication between the different scientific disciplines at project conception.

Establishing Clear Design and Implementation Plans

Agreeing on a common agenda entails agreeing on the objective we want to achieve. This means that we need to prioritize certain observations and collectively design how a fully integrated ocean observing system should look (as pursued by the AtlantOS (deYoung et al., 2019) and TPOS2020 (Smith et al., 2019) projects, among others), in accordance with the agreed agenda, and develop a long-term (5–10 years) implementation plan to put this system in place. This involves challenging ourselves to provide answers to practical questions, such as the exact number, location and the type of instruments and platforms that are the most appropriate to sufficiently resolve the specific spatiotemporal scales of a given ocean phenomenon. The GOOS driven effort to agree on Essential Ocean Variables is an example of this and shift the discussion from the need to have X number of platforms Y, to the need to observe an EOV at a determined accuracy and spatiotemporal coverage, possibly using many observing networks. The next step would therefore be to agree on Essential Ocean Phenomena and their corresponding observing configurations, and combine them to obtain the fully integrated observing design. Various spatial scales (global, basin, regional, coastal and local) need to be considered and integrated among themselves. For ocean prediction, this means identifying the observations that lead to improved forecasts (Davidson et al., 2019). Providing concrete numbers would not only ensure better communication with policy makers and funders, but would also provide useful guidance that will enable a more strategic approach to this investment. Moreover, once this implementation plan will be approved and funded, the competition for funds between the networks would be hindered.

This collective work requires the observing networks to be objective and look beyond their own interests, taking care to avoid network-specific approaches, as recommended by Moltmann et al. (2019). The multidisciplinary potential of each platform should be fully exploited in order to enhance the overall cost efficiency. This calls for trust and flexibility by all parties involved, and a willingness to reach the common goal. It is also key that the community be open to vigorous and regular re-examination of the ocean observing system elements to insure that the agreed objectives are being met. A kind of peer-review open process through a coordinating body is needed in order to assess the quality and overall efficiency of the system, which entails discussions with stakeholders. This requires a dynamic, agile process that could also retire old networks that are not needed or not sufficiently cost-efficient. It could be the responsibility of GOOS, in collaboration with other experts, to establish this implementation plan and the evaluation process.

The involvement of the modeling community is key for several aspects, since it can identify which observations are needed to improve the models and reduce the biases and provide guidance on the optimal observing design using data assimilation tools (Davidson et al., 2019). In fact, Observing System Evaluation (OS-Eval) methods have been used for decades by space agencies and others to test different designs of new satellite systems prior to their launch, and help justify investments in new observing systems (Zeng et al., 2020). Due to the large systematic errors in ocean models, increased sophistication in OS-Eval methodologies is needed, as well as multi-system evaluations to improve the robustness of the results and reduce system-dependency (Fujii et al., 2019). This entails strong international coordination and enhanced communication between the modeling and observational communities in order to increase the reliability of those experiments and take full advantage of these techniques, as recommended by Fujii et al. (2019).

Facilitating the Transition From Research to Integrated Sustained Operations

The overall ocean observing system should be enhanced by upgrading and integrating existing research-based elements into a global integrated sustained operational framework. Observing systems must have certain characteristics in order to meet the criteria for operationality, sustainability, and integration. First, they must meet a user’s needs (ideally multi-purposed), and be coordinated and complementary to other observing systems. Moreover, they must be sustainable and must provide data with the necessary metadata in a timely, cost-effective, and efficient method. These operational observing systems undoubtedly provide critical information for basic research but there is often a compromise between flexibility for the research question and meeting the criteria for integration, sustainability, and application to a broad range of users. There is therefore a strong need for facilitating these twin aspects and assisting the research teams in the transition from research to integrated sustained operations.

First, we need to set standards regarding the requirements for a research-based observing system to adapt and be accepted as part of the integrated sustained observing system. It should be the responsibility of a rejuvenated GOOS or the regional organizations to take up this responsibility and assist the transition. In developing countries, where this transition would be most difficult due to limited sources of public funding, GOOS could coordinate pools of donor funds to assist such transitions where it is needed the most, for example through the WMO Systematic Observations Financing Facility.²³ Second, research teams should be funded and incentivized for this work (see section “Redefining Scientific Excellence”). Third, new policies at regional and global level could be key elements to foster this transition. Finally, we need a shift in our ocean science culture. In the same way operational weather prediction centers contribute to research and also benefit from it, in oceanography the linkage between research and operational teams is mutually beneficial, and a close link between these communities is essential. Once scientists are incentivized for operational work, oceanographers could decide whether they prefer to work on research projects or be involved in sustained operational capabilities, in a similar way as weather measurements are sustained by operational entities, rather than aggregates of researchers.

Coordinating Data Management and Delivery

Observational data are only usable if they are delivered with the appropriate accuracy and precision, and with enough details on their collection practices and provenance for the users to decide which data to use for a specific purpose. Data should therefore be FAIR, but also timely distributed for feeding the operational ocean prediction systems (Davidson et al., 2019) and made freely available. The methods used to collect or create the data should also be open and available (Pearlman et al., 2021). Despite the great advances toward these objectives, partly through the activities of the IOC/UNESCO’s International Oceanographic Data and Information Exchange (IODE), much progress still needs to be made. While the key elements for a good data management plan have been established (see e.g., Tanhua et al., 2019b) and for some teams and even regional organizations incorporated into their work routines [e.g., the Argo (Tanhua et al., 2019b) and the IMOS (Lara-Lopez et al., 2016) data management systems], there are still barriers to sharing, inherent to the culture and the organization of our science system. We therefore need to find ways to put all these elements in place and foster a systematic application of these principles among all the ocean observing community.

First, to avoid duplication and heterogeneity, a common data delivery approach must be adopted by all actors and all organizations involved in data acquisition and management. This requires strong coordination, so we need an international collaborative framework to implement this work at the different levels: national/regional research infrastructures, networks and clusters as well as at the international level. This framework should consolidate the work developed by research teams into best practices and should be agile enough to make the best practices evolve to follow the progress of research and to handle new platforms/sensors/variables implemented by researchers. The data integrators (such as SeaDataNet,²⁴ CMEMS,²⁵ or EMODnet²⁶ in Europe) should also be able to adapt to new observational data flows. This framework should include organizations dedicated to provide guidance and training for assisting the data management work of the data providers, since this work is time-demanding and can be beyond the technical reach of many science groups.

Second, following the FAIR and open access principles, developing best practices, and providing training to assist their implementation, should be rewarded and acknowledged to be of equal value to publishing scientific papers (see section “Redefining Scientific Excellence”). Many options exist in order to give credit to datasets through data citation tools, such as DOIs or Persistent Identifiers for Data (PIDs) and/or products (see Tanhua et al., 2019b for more details). Traceability of use tools using DOIs and PIDs should be implemented to provide feedback to the originators when the data are shared or used through a downstream product. Similarly, DOIs should be assigned to methods used in the value chain to provide recognition of scientists that create and document the practices used in their research. Moreover, a standardized approach is needed for the attribution and acknowledgment of funders. Incentivizing researchers and facilities to make their data FAIR and publicly available would help ensure that the data management work is contemplated and funded in all research projects, whether the transition to accepted operational status is envisioned or not. The TPOS 2nd report²⁷ and Tanhua et al. (2019b) suggest 10% of the funding of science projects should be devoted to data management, and this is what IMOS dedicates to this activity from its core funds (Lara-Lopez et al., 2016). Finally, binding international regulations could also help guarantee that this data delivery approach is followed. The current review of WMO data policy²⁸ is a good example of a top-down measure that will encourage more data sharing. Considerable progress has been made in Europe with the INSPIRE directive,²⁹ but implementation is still insufficient.

Efficiently Communicating the Value of Ocean Observing

In the same way meteorology and space observations are considered essential for all who live on earth, the general public and decision-makers should understand that ocean observing is also of primary importance for all the world’s people, and especially for achieving the Sustainable Development Goals (SDGs) of the 2030 Agenda.³⁰ Ocean heat distribution controls the weather and climate and is crucial for forecasting natural or climate change-induced hazards. Ocean carbon absorption controls atmospheric CO2 accumulation and is a key element for establishing the carbon budget. Ocean currents are key for operational services, and ocean biodiversity and productivity impact fisheries and ocean health (e.g., National Academies of Sciences, Engineering, and Medicine, 2017). Consequently, many of the SDGs are related to the ocean. Moreover, although the benefit-cost ratio of ocean observations is difficult to estimate since it strongly depends on the cost of the observing system and on the economic importance of the user sectors (Kite-Powell, 2009), it is becoming clear that ocean observations support a wide range of societal and economic benefits (e.g., Rayner et al., 2019b). For IMOS for example, this benefit-cost ratio has been estimated to be in the range of 7.6–12 to 1 for the Australian government (Lateral Economics, 2021).

However, far too often the importance of ocean observations in the daily life of citizens and their nations is not communicated well, in part because we, as scientists, do not know how to efficiently communicate and provide answers to practical questions (National Academies of Sciences, Engineering, and Medicine, 2020). More effective and coordinated outreach efforts to communicate the value of ocean observing to broad audiences are therefore needed. The pressing issue of climate change and the increasing demand for a sustainable management of the ocean health and operational services is a suitable conduit for these efforts. Incentivizing researchers and facilities to communicate on the value of ocean observing (see section “Redefining Scientific Excellence”) will strongly contribute to enhancing the sharing and communication of science, and would encourage scientists to improve their communication skills and actively team with communication experts. Moreover, social scientists as well as professional communicators could also be engaged.

Closing Remarks

Achieving a truly integrated ocean observing system requires a profound change in our research and operational cultures and in the organization of our ocean observing community. This change is of fundamental importance for both ocean science and society. This is timely since we now have the knowledge, the technical capacities, as well as urgent societal and political drivers to deliver “the ocean science we need for the ocean we want” (UN Ocean Decade, 2021–2030). The scientific community needs therefore to act in order to remove the barriers between ocean research activities and take full advantage of the scientific and technical advances made in the last decades. This evolution in the organization of how we have been working so far in oceanography will not be easy, and will only be possible if scientists, institutions and funders embrace this change and collectively reflect on how to implement it. This paper aims at being a first step opening the way toward more reflection. Our aim is to provide food for thought for further dialogue between all the parties involved on the concrete actions to undertake.

Notably, this study is limited by the author team’s regional perspective, and only presents the view from the occidental science culture, since the barriers to integration might be different in other regions where the culture is different. In view of a worldwide integration, international dialogue would be needed to include more diverse perspectives and avoid “colonial science,” also referred as “parachute science” (e.g., Stefanoudis et al., 2021). Also, our expertise is mainly drawn from physical and biogeochemical oceanography, given that it is the area where real time observations are more advanced and modeling has reached good predictive capabilities, both allowing enhanced data assimilation and good initiatives to respond to societal and stakeholders’ needs, along the value chain. However, we believe our approach could be largely applied to the other fields of ocean science, since the lack of coordination between teams and disciplines is a problem that is common to many basic and applied endeavors (i.e., Tress et al., 2006; OECD, 2020), and already reported in marine biological observations (e.g., Guidi et al., 2020).

We proposed here nine lines of approach that we believe could lay the foundation of and stimulate a real transformational change in the internal organization and the culture of ocean science. They would promote a working environment that is more conducive to innovation and the sharing of experience and expertise. We call for wider discussions between all the actors of ocean observing in order for the proposed recommendations to be followed by the development of a set of coordinated initiatives, for example within a project under the UN Ocean Decade, in order to achieve a truly integrated ocean observing system by 2030.

Author Contributions

AR, JT, and JV conceived the study and coordinated the author contributions. AR and JT were the primary authors with all co-authors substantially contributing to the manuscript ideas and text. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the European Union’s Horizon 2020 Research and Innovation Programme under grant agreement No. 862626, project EuroSea (Improving and Integrating European Ocean Observing and Forecasting Systems for Sustainable Use of the Oceans).

Conflict of Interest

TP was employed by company Octopus Ocean Consulting LLC. MM was employed by company Blue Info By Skipper & Wool. BW was employed by company Fugro.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^ https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/12539-Ocean-observation-sharing-responsibility_en
2. ^ Networks are defined as an ensemble of platforms (such as Argo for profiling floats, OceanSites for moored time series, GO-SHIP for large scale hydrography, among others), while a system is an ensemble of networks.
3. ^ “Competition can really fall into one of two categories: constructive competition or destructive competition.” https://thriveglobal.com/stories/constructive-competition-vs-destructive-competition/
4. ^ https://public.wmo.int/en/governance-reform
5. ^ https://eurosea.eu
6. ^ ESA Knowledge Management Initiative https://ideas.esa.int/servlet/hype/IMT? documentTableId=45087625531035594&userAction=Browse&templateName=&documentId=1355b822559eaa009275bf70f75ff984 Our individual knowledge is like a piece of a jigsaw puzzle: it is fundamental, but it only works if joined with other pieces to fit together and have a complete understanding of the picture.
7. ^ “To ensure that competition is constructive, you only need to pay attention to one thing: a shared vision.” https://thriveglobal.com/stories/constructive-competition-vs.-destructive-competition/
8. ^ https://emodnet.ec.europa.eu/en/emodnet-vision-statement-marine-board-eurogoos-perspective
9. ^ http://www.coastalwiki.org/w/images/4/43/Pegaso_D2_3_annex3.pdf
10. ^ https://www.cam.ac.uk/research/news/cambridge-university-signs-san-francisco-declaration-on-research-assessment
11. ^ http://www.exeter.ac.uk/staff/exeteracademic/yourdevelopment/
12. ^ https://staff.mq.edu.au/work/development/academic-promotion/new-scheme
13. ^ https://www.uu.nl/en/research/open-science/tracks/recognition-and-rewards
14. ^ https://sfdora.org
15. ^ https://www.rri-practice.eu/about-rri-practice/what-is-rri/
16. ^ 2002/413/EC, Recommendation of the European Parliament and of the Council of 30 May 2002 concerning the implementation of integrated coastal zone management, OJ L148 of 6.6.2002. http://data.europa.eu/eli/reco/2002/413/oj
17. ^ The Ocean-Shots concept from the US UN Decade initiative is an ambitious, transformational research concept that draws inspiration and expertise from multiple disciplines and fundamentally advances ocean science for sustainable development. https://www.nationalacademies.org/our-work/us-national-committee-on-ocean-science-for-sustainable-development-2021-2030
18. ^ GOOS 10th Steering Committee https://www.goosocean.org/index.php?option=com_oe&task=viewDocumentRecord&docID=28265
19. ^ https://ioos.noaa.gov/ioos-in-action/benefits-of-ocean-observing-catalog/
20. ^ https://www.clivar.org
21. ^ https://oceanpredict.org
22. ^ https://www.coastpredict.org
23. ^ https://public.wmo.int/en/our-mandate/how-we-do-it/development-partnerships/Innovating-finance
24. ^ https://www.seadatanet.org
25. ^ https://marine.copernicus.eu
26. ^ https://emodnet.eu/en
27. ^ https://tpos2020.org/project-reports/second-report/
28. ^ https://meetings.wmo.int/SERCOM-1-II/_layouts/15/WopiFrame.aspx?sourcedoc=/SERCOM-1-II/Presentations/SERCOM-1(II)-SG-DIP-SBarrell.pptx&action=default
29. ^ https://inspire.ec.europa.eu/inspire-directive/2
30. ^ https://sdgs.un.org/publications/transforming-our-world-2030-agenda-sustainable-development-17981

References

Bahurel, P., Adragna, F., Bell, M., Jacq, F., Johannessen, J. A., Le Traon, P. Y., et al. (2010). “Ocean monitoring and forecasting core services, the european myocean example,” in Proceedings of OceanObs’09: Sustained Ocean Observations and Information for Society, Vol. 1, eds J. Hall, D. E. Harrison, and D. Stammer (Venice: ESA Publication), 21–25. doi: 10.5270/OceanObs09.pp.02

CrossRef Full Text | Google Scholar

Barki, H., and Pinsonneault, A. (2005). A model of organizational integration, implementation effort, and performance. Organ. Sci. 16, 165–179. doi: 10.1287/orsc.1050.0118

PubMed Abstract | CrossRef Full Text | Google Scholar

Beal, L. M., Vialard, J., Roxy, M. K., Ravichandran, M., McPhaden, M. J., Feng, M., et al. (2019). Executive Summary. IndOOS-2: A Roadmap to Sustained Observations of the Indian Ocean For 2020-2030. Qingdao: CLIVAR-4/2019, GOOS-237, 204. doi: 10.36071/clivar.rp.4.2019

CrossRef Full Text | Google Scholar

Bell, M. J., Lefèbvre, M., Le Traon, P.-Y., Smith, N., and Wilmer-Becker, K. (2009). GODAE: the global ocean data assimilation experiment. Oceanography 22, 14–21.

Google Scholar

Benedictus, R., and Miedema, F. (2016). Fewer numbers, better science. Nature 538, 453–454. doi: 10.1038/538453a

PubMed Abstract | CrossRef Full Text | Google Scholar

Bleasdale, B. (2019). Researchers pay the cost of research. Nat. Mater. 18:772. doi: 10.1038/s41563-019-0443-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Cicin-Sain, B., and Knecht, R. W. (1998). Integrated Coastal and Ocean Management: Concepts and Practices. Washington, DC: Island Press, 37–64.

Google Scholar

Coriat, A. M. (2019). PhD merit needs to be defined by more than just publications. Nat. Hum. Behav. 3:1007. doi: 10.1038/s41562-019-0727-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Davidson, F., Azcarate, A. A., Barth, A., Brassington, G. B., Chassignet, E. P., Clementi, E., et al. (2019). Synergies in operational oceanography: the intrinsic need for sustained ocean observations. Front. Mar. Sci 6:450. doi: 10.3389/fmars.2019.00450

CrossRef Full Text | Google Scholar

Delgado-López-Cózar, E., Ràfols, I., and Abadal, E. (2021). Letter: a call for a radical change in research evaluation in Spain. Prof. Inf. 30:e300309. doi: 10.3145/epi.2021.may.09

CrossRef Full Text | Google Scholar

deYoung, B., Visbeck, M., de Araujo Filho, M. C., Baringer, M. O., Black, C. A., Buch, E., et al. (2019). An integrated all-atlantic ocean observing system in 2030. Front. Mar. Sci. 6:428. doi: 10.3389/fmars.2019.00428

CrossRef Full Text | Google Scholar

Diedrich, A., Balaguer, P., and Tintoreì, J. (2011). “Concepts, methods, and tools to sup- port science-based decision-making in inte- grated coastal and ocean management: examples from the balearic islands,” in ICZM as an Evolution of Territorial Planning and Governance, ed. J. Farinoìs (Valencia: University of Valencia), 89–110.

Google Scholar

European Commission (2013). Towards European Integrated Ocean Observation: Expert Group On Marine Research Infrastructures: Directorate-General for Research and Innovation. Luxembourg: European Commission, doi: 10.2777/29343

CrossRef Full Text | Google Scholar

European Marine Board (2019). “Navigating the future V: marine science for a sustainable future,” in Proceedings of the Position Paper 24 of the European Marine Board, Ostend.

Google Scholar

European Marine Board (2013). “Navigating the future IV,” in Proceedings of the osition Paper 20 of the European Marine Board, Ostend.

Google Scholar

European Marine Board (2021). Sustaining In Situ Ocean Observations in the Age of the Digital Ocean: EMB Policy Brief No. 9, June 2021. Oostende: European Marine Board, doi: 10.5281/zenodo.4836060

CrossRef Full Text | Google Scholar

Fischer, A. S., Hall, J., Harrison, D. E., Stammer, D., and Benveniste, J. (2010). “Conference summary-ocean information for society: sustaining the benefits, realizing the potential,” in Proceedings of OceanObs’09: Sustained Ocean Observations and Information for Society WPP-306, 21-25 September 2009, Vol. 1, eds J. Hall, D. E. Harrison, and D. Stammer (Venice: ESA Publication), doi: 10.5270/OceanObs09.Summary

CrossRef Full Text | Google Scholar

Fujii, Y., Rémy, E., Zuo, H., Oke, P., Halliwell, G., Gasparin, F., et al. (2019). Observing system evaluation based on ocean data assimilation and prediction systems: on-going challenges and a future vision for designing and supporting ocean observational networks. Front. Mar. Sci. 6:417. doi: 10.3389/fmars.2019.00417

CrossRef Full Text | Google Scholar

Fülöp, M., and Orosz, G. (2015). “State of the art in competition research,” in Emerging Trends in the Social and Behavioral Sciences: An Interdisciplinary, Searchable, and Linkable Resource, eds R. A. Scott, M. C. Buchmann, and S. M. Kosslyn (Hoboken, NJ: John Wiley and Sons), 1–16. doi: 10.1002/9781118900772.etrds0317

CrossRef Full Text | Google Scholar

Garçon, V., Karstensen, J., Palacz, A., Telszewski, M., Aparco, Lara T, Breitburg, D., et al. (2019). Multidisciplinary observing in the world ocean’s oxygen minimum zone regions: from climate to fish — the voice initiative. Front. Mar. Sci. 6:722. doi: 10.3389/fmars.2019.00722

CrossRef Full Text | Google Scholar

Guidi, L., Fernandez Guerra, A., Canchaya, C., Curry, E., Foglini, F., Irisson, J.-O., et al. (2020). “Big data in marine science,” in Proceedings of the Future Science Brief 6 of the European Marine Board, eds B. Alexander, J. J. Heymans, A. MunPiz Piniella, P. Kellett, and J. Coopman (Ostend: European Marine Board), doi: 10.5281/zenodo.3755793

CrossRef Full Text | Google Scholar

Hernández-Aguilera, J. N., Anderson, W., Bridges, A. L., Fernandez, M. P., Hansen, W. D., Maurer, M. L., et al. (2021). Supporting interdisciplinary careers for sustainability. Nat. Sustain. 4, 374–375. doi: 10.1038/s41893-020-00679-y

CrossRef Full Text | Google Scholar

Hicks, D., Wouters, P., Waltman, L., Rijcke, S. D., and Rafols, I. (2015). The leiden manifesto for research metrics. Nat. News 520, 429–431. doi: 10.1038/520429a

PubMed Abstract | CrossRef Full Text | Google Scholar

Hodson, J. (2018). FAIR Data Action Plan: Interim Recommendations and Actions from the European Commission Expert Group on FAIR Data. Zenodo. doi: 10.5281/zenodo.1285290

CrossRef Full Text | Google Scholar

Hotaling, L., and Spinrad, R. W. (2021). Preparing a Workforce for the New Blue Economy: People, Products and Policies. Amsterdam: Elsevier Science Publishing Co Inc.

Google Scholar

International Altimetry Team (2021). Altimetry for the future: Building on 25 years of progress. Adv. Space Res 68, 319–363. doi: 10.1016/j.asr.2021.01.022

CrossRef Full Text | Google Scholar

International Oceanographic Commission (IOC)-UNESCO (2017). Global Ocean Science Report – The Current Status Of Ocean Science Around The World, eds L. Valdeìs et al. (Paris: UNESCO Publishing).

Google Scholar

IOC (2019). The Global Ocean Observing System 2030 Strategy. IOC Brochure 2019-5 (IOC/BRO/2019/5 rev.2), GOOS Report No.239. Paris: IOC.

Google Scholar

IPCC (2019). IPCC Special Report on the Ocean and Cryosphere in a Changing Climate, eds H.-O. Pörtner, D. C. Roberts, V. Masson-Delmotte, P. Zhai, M. Tignor, E. Poloczanska, et al. (Geneva: IPCC).

Google Scholar

JCOMMOPS Review (2018). 2018 Review of the JCOMM in situ Observations Programme Support Centre. Available online at: https://www.jcomm.info/index.php?option=com_oe&task=viewDocumentRecord&docID=23372 (Accessed December 12, 2018).

Google Scholar

Kania, J., and Kramer, M. (2011). Collective impact. Stanford Soc. Innov. Rev. 9, 36–41.

Google Scholar

Kite-Powell, H. L. (2009). Economic considerations in the design of ocean observing systems. Oceanography 22, 44–49. doi: 10.5670/oceanog.2009.37

CrossRef Full Text | Google Scholar

Lara-Lopez, A., Heslop, E., and Lips, I. (2021). European GOOS National Focal Points Survey: Funding and Coordination across Ocean Observing and Marine monitoring in Europe: A European Ocean Observing System (EOOS) Report.

Google Scholar

Lara-Lopez, A., Moltmann, T., and Proctor, R. (2016). Australia’s Integrated marine observing system (imos): data impacts and lessons learned. Mar. Technol. Soc. J. 50, 22–33. doi: 10.4031/MTSJ.50.3.1

CrossRef Full Text | Google Scholar

Larkin, K., Marsan, A.-A., Tonné, N., and Calewaert, J.-B. (eds) (2019). EOOS Conference 2018 Report and Call to Action. Connecting European Ocean Observing Communities for End-to-End Solutions 21-23 November 2018. Brussels: The Egg.

Google Scholar

Lateral Economics (2021). Integrated Marine Observing System (IMOS) Estimated return on investment. June 2021 Final Report.

Google Scholar

Le Traon, P. Y. (2013). From satellite altimetry to Argo and operational oceanography: three revolutions in oceanography. Ocean Sci. 9, 901–915. doi: 10.5194/os-9-901-2013

CrossRef Full Text | Google Scholar

Le Traon, P. Y., Ali, A., Alvarez Fanjul, E., Aouf, L., Axell, L., Aznar, R., et al. (2017). The Copernicus marine environmental monitoring service: main scientific achievements and future prospects. Spec. Issue Mercator Océan J. 56:101. doi: 10.25575/56

CrossRef Full Text | Google Scholar

Le Traon, P. Y., Reppucci, A., Fanjul, E. A., Aouf, L., Behrens, A., Belmonte, M., et al. (2019). From observation to information and users: the copernicus marine service perspective. Front. Mar. Sci 6:234.

Google Scholar

Lindstrom, E., Gunn, J., Fischer, A., McCurdy, A., Glover, L., Alverson, K., et al. (2012). A Framework for Ocean Observing: By the Task Team for an Integrated Framework for Sustained Ocean Observing, UNESCO 2012, IOC/INF-1284. Paris: UNESCO, doi: 10.5270/OceanObs09-FOO

CrossRef Full Text | Google Scholar

Lubchenco, J. (2017). Environmental science in a post-truth world. Front. Ecol. Environ. 15:3. doi: 10.1002/fee.1454

CrossRef Full Text | Google Scholar

Lubchenco, J., and Gaines, S. D. (2019). A new narrative for the ocean. Science 364:911. doi: 10.1126/science.aay2241

PubMed Abstract | CrossRef Full Text | Google Scholar

Lubchenco, J., and Rapley, C. (2020). Our moment of truth: the social contract realized? Environ. Res. Lett. 15:110201. doi: 10.1088/1748-9326/abba9c

CrossRef Full Text | Google Scholar

Marques, M. (2020). PWC HELM Circumnavigation: An Integrated Approach to the Economy of the Sea. PwC Economy of the Sea Barometer (World), 5 Edn. Lisbon: PwC.

Google Scholar

Marques, M. (2021). Blue Info by Skipper & Wool lda, Burgee – Oceans, Seas and Rivers Socioeconomic Report. Available online at: https://blueinfo.pt/blue-info-summit/ (Accessed June 2021).

Google Scholar

Mazzucato, M. (2018). Mission-Oriented Research & Innovation in the European Union: A Problem-Solving Approach to Fuel Innovation-Led Growth. Brussels: European Commission.

Google Scholar

Moher, D., Bouter, L., Kleinert, S., Glasziou, P., Sham, M. H., Barbour, V., et al. (2020). The hong kong principles for assessing researchers: fostering research integrity. PLoS Biol. 18:e3000737. doi: 10.1371/journal.pbio.3000737

PubMed Abstract | CrossRef Full Text | Google Scholar

Moltmann, T., Turton, J., Zhang, H.-M., Nolan, G., Gouldman, C., Griesbauer, L., et al. (2019). A global ocean observing system (GOOS), delivered through enhanced collaboration across regions, communities, and new technologies. Front. Mar. Sci. 6:291. doi: 10.3389/fmars.2019.00291

CrossRef Full Text | Google Scholar

Muñiz Piniella, A. ì, and Heymans, J. J. (2020). Report On Initiatives, Strategies And Roadmaps That Contribute To Foresight In Ocean Observation: EU EuroSea Project. doi: 10.5281/zenodo.3956082

CrossRef Full Text | Google Scholar

National Academies of Sciences, Engineering, and Medicine (2017). Sustaining Ocean Observations to Understand Future Changes in Earth’s Climate. Washington, DC: The National Academies Press, doi: 10.17226/24919

CrossRef Full Text | Google Scholar

National Academies of Sciences, Engineering, and Medicine (2020). Sustaining Ocean Observations: Proceedings of a Workshop in Brief. Washington, DC: The National Academies Press, doi: 10.17226/25997

CrossRef Full Text | Google Scholar

National Research Council (2011). Critical Infrastructure for Ocean Research and Societal Needs in 2030. Washington, DC: The National Academies Press.

Google Scholar

Nature Editorial (2018). Science needs to redefine excellence. Nature 554, 403–404. doi: 10.1038/d41586-018-02183-y

CrossRef Full Text | Google Scholar

NOAA (2017). The Ocean Enterprise: A study of US Business Activity in Ocean Measurement, Observation and Forecasting. Washington, DC: NOAA.

Google Scholar

NOAA (2021). The Ocean Enterprise 2015 – 2020: A Study of U.S. New Blue Economy Business Activity. Washington, DC: NOAA.

Google Scholar

OceanObs’19 (2019). OceanObs’19 Conference Statement. Available online at: http://www.oceanobs19.net/wp-content/uploads/2019/09/OO19-Conference-Statement_online.pdf

Google Scholar

OceanOPS Strategic Plan (2020). A 5-Year Strategic Plan for OceanOPS 2021-2025. Available online at: https://www.ocean-ops.org/strategy/

Google Scholar

OECD (2020). “Addressing societal challenges using transdisciplinary research,” in OECD Science, Technology and Industry Policy Papers, No. 88 (Paris: OECD Publishing). doi: 10.1787/0ca0ca45-en

CrossRef Full Text | Google Scholar

Ostrom, E. (2010). Polycentric systems for coping with collective action and global environmental change. Glob. Environ. Change 20, 550–557. doi: 10.1016/j.gloenvcha.2010.07.004

CrossRef Full Text | Google Scholar

Pearlman, J., Bushnell, M., Coppola, L., Karstensen, J., Buttigieg, P. L., Pearlman, F., et al. (2019). Evolving and sustaining ocean best practices and standards for the next decade. Front. Mar. Sci. 6:277. doi: 10.3389/fmars.2019.00277

CrossRef Full Text | Google Scholar

Pearlman, J., Buttigieg, P. L., Bushnell, M., Delgado, C., Hermes, J., Heslop, E., et al. (2021). Evolving and sustaining ocean best practices to enable interoperability in the UN decade of ocean science for sustainable development. Front. Mar. Sci. 8:619685. doi: 10.3389/fmars.2021.619685

CrossRef Full Text | Google Scholar

Pendleton, L., Evans, K., and Visbeck, M. (2020). We need a global movement to transform ocean science for a better world. Proc. Natl. Acad. Sci. U.S.A. 117, 9652–9655. doi: 10.1073/pnas.2005485117

PubMed Abstract | CrossRef Full Text | Google Scholar

Pinardi, N., Stander, J., Legler, D. M., O’Brien, K., Boyer, T., Cuff, T., et al. (2019). The joint IOC (of UNESCO) and WMO collaborative effort for met-ocean services. Front. Mar. Sci. 6:410. doi: 10.3389/fmars.2019.00410

CrossRef Full Text | Google Scholar

Raff, J. W. (2012). San francisco declaration on research assessment (DORA). Biol. Open 2, 533–534. doi: 10.1242/bio.20135330

PubMed Abstract | CrossRef Full Text | Google Scholar

Rayner, R., Gouldman, C., and Willis, Z. (2019a). The Ocean Enterprise – understanding and quantifying business activity in support of observing, measuring and forecasting the ocean. J. Oper. Oceanogr. 12(Suppl. 2), S97–S110. doi: 10.1080/1755876X.2018.1543982

CrossRef Full Text | Google Scholar

Rayner, R., Jolly, C., and Gouldman, C. (2019b). Ocean observing and the blue economy. Front. Mar. Sci. 6:330. doi: 10.3389/fmars.2019.00330

CrossRef Full Text | Google Scholar

Ricciardi, F., Zardini, A., and Rossignoli, C. (2018). Organizational integration of the IT function: a key enabler of firm capabilities and performance. J. Innov. Knowledge 3, 93–107. doi: 10.1016/j.jik.2017.02.003

CrossRef Full Text | Google Scholar

Smith, N. (2021). Report of study on Support Provided to Global and Regional Ocean Observing Systems. Internal Report. Available online at: https://drive.google.com/file/d/13SLTbUgikcOYhQc1XSJU9JpKWSpEkHEz/view

Google Scholar

Smith, N., and Lefebvre, M. (1997). “The global ocean data assimilation experiment (GODAE),” in Paper Presented At Monitoring The Oceans in the 2000s: An Integrated Approach, Biarritz.

Google Scholar

Smith, N., Kessler, W. S., Cravatte, S., Sprintall, J., Wijffels, S., Cronin, M. F., et al. (2019). Tropical pacific observing system. Front. Mar. Sci. 6:31. doi: 10.3389/fmars.2019.00031

CrossRef Full Text | Google Scholar

Spiegler, D. B. (2007). Community: the private sector in meteorology – an update. Bull. Am. Meteorol. Soc. 88, 1272–1275.

Google Scholar

Stefanoudis, P. V., Licuanan, W. Y., Morrison, T. H., Talma, S., Veitayaki, J., and Woodall, L. C. (2021). Turning the tide of parachute science. Curr. Biol. 31, 184–185. doi: 10.1016/j.cub.2021.01.029

PubMed Abstract | CrossRef Full Text | Google Scholar

Sterckx, S., Brown, I., Kääb, A., Krol, M., Morrow, R., Veefkind, P., et al. (2020). Towards a european Cal/Val service for earth observation. Int. J. Remote Sens. 41, 4496–4511. doi: 10.1080/01431161.2020.1718240

CrossRef Full Text | Google Scholar

Tanhua, T., McCurdy, A., Fischer, A., Appeltans, W., Bax, N., Currie, K., et al. (2019a). What we have learned from the framework for ocean observing: evolution of the global ocean observing system. Front. Mar. Sci. 6:471. doi: 10.3389/fmars.2019.00471

CrossRef Full Text | Google Scholar

Tanhua, T., Pouliquen, S., Hausman, J., O’Brien, K., Bricher, P., de Bruin, T., et al. (2019b). Ocean FAIR data services. Front. Mar. Sci. 6:440. doi: 10.3389/fmars.2019.00440

CrossRef Full Text | Google Scholar

Tress, B., Tress, G., and Fry, G. (2006). “Chapter 17: ten steps to success in integrative research projects,” in From Landscape Research to Landscape Planning: Aspects of Integration, Education and Application, Vol. 12, eds B. Tress, G. Tress, G. Fry, and P. Opdam (Wageningen: Wageningen University Frontis Series).

Google Scholar

Tress, G., Tress, B., and Gary, F. (2007). Analysis of the barriers to integration in landscape research projects. Land Use Policy 24, 374–385. doi: 10.1016/j.landusepol.2006.05.001

CrossRef Full Text | Google Scholar

Valdés, L. (2017). “The UN architecture for ocean science knowledge and governance,” in Handbook on the Economics and Management of Sustainable Oceans, eds A. Markandya, L. E. Svensson, and P. A. L. D. Nunes (Cheltenham: Edward Elgar Publishing), 381–395.

Google Scholar

Valdés, L., Mees, J., and Enevoldsen, H. (2017). “International organizations supporting ocean science,” in Global Ocean Science Report – The Current Status of Ocean Science Around the World, eds L. Valdés, J. Mees, and H. Enevoldsen (Paris: UNESCO Publishing).

Google Scholar

Van Noorden, R. (2018). Some hard numbers on science’s leadership problems. Nature 557, 294–296. doi: 10.1038/d41586-018-05143-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Visbeck, M. (2018). Ocean science research is key for a sustainable future. Nat. Commun. 9:690. doi: 10.1038/s41467-018-03158-3

PubMed Abstract | CrossRef Full Text | Google Scholar

VSNU, NFU, KNAW, NWO, and ZONMW (2019). Room for Everyone’s Talent: Towards a New Balance in the Recognition and Rewards of Academics. The Hague.

Google Scholar

Weller, R. A., Baker, D. J., Glackin, M. M., Roberts, S. J., Schmitt, R. W., Twigg, E. S., et al. (2019). The challenge of sustaining ocean observations. Front. Mar. Sci. 6:105. doi: 10.3389/fmars.2019.00105

CrossRef Full Text | Google Scholar

Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., et al. (2016). The FAIR guiding principles for scientific data management and stewardship. Sci. Data 3:160018. doi: 10.1038/sdata.2016.18

PubMed Abstract | CrossRef Full Text | Google Scholar

Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S. A., et al. (2015). The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management. London: Higher Education Funding Council for England. doi: 10.13140/RG.2.1.4929.1363

CrossRef Full Text | Google Scholar

Woolston, C. (2021). Impact factor abandoned by Dutch university in hiring and promotion decisions. Nat. Carrer News 595:462. doi: 10.1038/d41586-021-01759-5

PubMed Abstract | CrossRef Full Text | Google Scholar

World Meteorological Organisation (2019). Vision For The WMO Integrated Global Observing System (WIGOS) in 2040. Geneva: WMO.

Google Scholar

Zeng, X., Atlas, R., Birk, R. J., Carr, F. H., Carrier, M. J., Cucurull, L., et al. (2020). Use of observing system simulation experiments in the United States. Bull. Am. Meteorol. Soc. 101, E1427–E1438. doi: 10.1175/BAMS-D-19-0155.1

CrossRef Full Text | Google Scholar