Attribution of multi-annual to decadal changes in the climate system: The Large Ensemble Single Forcing Model Intercomparison Project (LESFMIP)

Smith, Doug M.; Gillett, Nathan P.; Simpson, Isla R.; Athanasiadis, Panos J.; Baehr, Johanna; Bethke, Ingo; Bilge, Tarkan A.; Bonnet, Rémy; Boucher, Olivier; Findell, Kirsten L.; Gastineau, Guillaume; Gualdi, Silvio; Hermanson, Leon; Leung, L. Ruby; Mignot, Juliette; Müller, Wolfgang A.; Osprey, Scott; Otterå, Odd Helge; Persad, Geeta G.; Scaife, Adam A.; Schmidt, Gavin A.; Shiogama, Hideo; Sutton, Rowan T.; Swingedouw, Didier; Yang, Shuting; Zhou, Tianjun; Ziehn, Tilo

doi:10.3389/fclim.2022.955414

PERSPECTIVE article

Front. Clim., 16 September 2022

Sec. Predictions and Projections

Volume 4 - 2022 | https://doi.org/10.3389/fclim.2022.955414

This article is part of the Research Topic Insights in Predictions and Projections: 2022 View all 7 articles

Attribution of multi-annual to decadal changes in the climate system: The Large Ensemble Single Forcing Model Intercomparison Project (LESFMIP)

$\nDoug M. Smith$ Doug M. Smith¹^*

Nathan P. Gillett²

Isla R. Simpson³

Panos J. Athanasiadis⁴

Johanna Baehr⁵

Ingo Bethke⁶

Tarkan A. Bilge⁶

Rémy Bonnet⁷

Olivier Boucher⁷

Kirsten L. Findell⁸

Guillaume Gastineau⁷

Silvio Gualdi⁴

Leon Hermanson¹

L. Ruby Leung⁹

Juliette Mignot⁷

Wolfgang A. Müller¹⁰

Scott Osprey^11,12

Odd Helge Otterå⁶

Geeta G. Persad¹³

Adam A. Scaife^1,14

Gavin A. Schmidt¹⁵

Hideo Shiogama¹⁶

Rowan T. Sutton^17,18

Didier Swingedouw¹⁹

Shuting Yang²⁰

Tianjun Zhou^21,22

Tilo Ziehn²³

¹Met Office Hadley Centre, Exeter, United Kingdom
²Canadian Centre for Climate Modelling and Analysis, Environment and Climate Change Canada, Victoria, BC, Canada
³Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, United States
⁴Centro Euro-Mediterraneo sui Cambiamenti Climatici, Bologna, Italy
⁵Institute of Oceanography, Center for Earth System Research and Sustainability (CEN), Universität Hamburg, Hamburg, Germany
⁶Geophysical Institute, University of Bergen and Bjerknes Centre for Climate Research, Bergen, Norway
⁷LOCEAN Laboratory, Institut Pierre Simon Laplace (IPSL), Sorbonne Universités, Paris, France
⁸Geophysical Fluid Dynamics Laboratory/NOAA, Princeton, NJ, United States
⁹Pacific Northwest National Laboratory, Atmospheric Sciences and Global Change Division, Richland, WA, United States
¹⁰Max-Planck-Institut für Meteorologie, Hamburg, Germany
¹¹National Centre for Atmospheric Science, Oxford, United Kingdom
¹²Department of Physics, University of Oxford, Oxford, United Kingdom
¹³Department of Geological Science, The University of Texas at Austin, Austin, TX, United States
¹⁴College of Engineering, Mathematics and Physical Sciences, Exeter University, Exeter, United Kingdom
¹⁵NASA Goddard Institute for Space Studies, New York, NY, United States
¹⁶Earth System Division, National Institute for Environmental Studies (NIES), Tsukuba, Japan
¹⁷National Centre for Atmospheric Science, Leeds, United Kingdom
¹⁸Department of Meteorology, University of Reading, Reading, United Kingdom
¹⁹CNRS-EPOC, Université de Bordeaux, Pessac, France
²⁰Danish Meteorological Institute, Copenhagen, Denmark
²¹State Key Laboratory of Numerical Modeling for Atmospheric Sciences and Geophysical Fluid Dynamics, Institute of Atmospheric Physics, Chinese Academy of Sciences, Beijing, China
²²University of Chinese Academy of Sciences, Beijing, China
²³CSIRO Oceans and Atmosphere, Aspendale Gardens, VIC, Australia

Multi-annual to decadal changes in climate are accompanied by changes in extreme events that cause major impacts on society and severe challenges for adaptation. Early warnings of such changes are now potentially possible through operational decadal predictions. However, improved understanding of the causes of regional changes in climate on these timescales is needed both to attribute recent events and to gain further confidence in forecasts. Here we document the Large Ensemble Single Forcing Model Intercomparison Project that will address this need through coordinated model experiments enabling the impacts of different external drivers to be isolated. We highlight the need to account for model errors and propose an attribution approach that exploits differences between models to diagnose the real-world situation and overcomes potential errors in atmospheric circulation changes. The experiments and analysis proposed here will provide substantial improvements to our ability to understand near-term changes in climate and will support the World Climate Research Program Lighthouse Activity on Explaining and Predicting Earth System Change.

Introduction

Climate variability and change cause major impacts on human society and natural ecosystems. Of particular concern are large-scale changes in atmospheric circulation, since they can alter rainfall patterns, threatening food and water security, and increase the chances of extreme events. Prominent examples on decadal timescales include the United States “Dust Bowl” during the 1930s (Schubert et al., 2004), the Sahel droughts during the 1970s (Kandji et al., 2006), the mid-1990s increase in Atlantic hurricane activity (Goldenberg et al., 2001), the early twentieth century warming (Hegerl et al., 2018) and the early twenty-first century slowdown in global surface warming (Medhaug et al., 2017). Recent multi-decadal trends in atmospheric circulation also likely contributed to several extreme seasonal events including heatwaves in Europe and Russia, floods in Pakistan, and wildfires across the west coast of the United States (Teng et al., 2022) and Australia (Canadell et al., 2021), and even daily extremes such as rainfall and storms are strongly modulated by multidecadal fluctuations in large scale patterns such as the North Atlantic Oscillation (NAO; Scaife et al., 2008; Dawkins et al., 2016). Some recent extremes are also far outside the range of previously observed variability, causing severe impacts and creating serious challenges for climate adaptation (Fischer et al., 2021). Hence there is an urgent need to understand the causes of regional multi-annual to decadal changes in climate.

Decadal climate predictions are now issued operationally by the World Meteorological Organisation (WMO) and offer the potential to provide early warnings of changes in climate (Kushnir et al., 2019). By comparing retrospective forecasts (hindcasts) with observations, it has been established that decadal predictions can potentially forecast many aspects of climate over the coming years, including Atlantic Multidecadal Variability (AMV, Doblas-Reyes et al., 2013; Hermanson et al., 2014; Yeager and Robson, 2017; Borchert et al., 2021), Atlantic hurricane activity (Smith et al., 2010; Caron et al., 2018), Sahel rainfall (Sheen et al., 2017; Yeager et al., 2018), droughts and wildfires in southwestern United States (Chikamoto et al., 2017), carbon fluxes (Li et al., 2019; Lovenduski et al., 2019a,b), European precipitation and temperature in summer (Müller et al., 2012; Yeager et al., 2018) and winter (Simpson et al., 2019), summer temperatures in north-east Asia (Monerie et al., 2018), the occurrence of warm summer temperature extremes (Borchert et al., 2019), Tibetan Plateau summer rainfall (Hu and Zhou, 2021), and winter atmospheric circulation in the North Atlantic (Athanasiadis et al., 2020; Smith et al., 2020). Each year the WMO issues a Global Annual to Decadal Climate Update (GADCU, Hermanson et al., 2022) that synthesizes the forecasts from several international centers. These forecasts show clear signals for atmospheric circulation and precipitation, as well as for near surface temperature (Hermanson et al., 2022), but greater confidence is needed for them to become “actionable”. In particular, skill in hindcasts does not guarantee skill in forecasts since the drivers may be different. Similarly, absence of skill in hindcasts does not necessarily mean absence of skill in forecasts since the drivers that produce predictable signals may only just be emerging. Understanding the drivers of multi-annual changes in climate is therefore essential for gaining further confidence in forecasts.

Despite its importance, our ability to understand and attribute multi-annual to decadal changes in climate is immature, as evidenced by the recent debate over the slowdown in global surface warming (Medhaug et al., 2017). To address this, the World Climate Research Programme (WCRP) has set up a Lighthouse Activity on Explaining and Predicting Earth System Change (LHA-EPESC) with three major themes (Findell et al., 2022): (i) monitoring and modeling Earth system change; (ii) integrated attribution, prediction and projection; and (iii) assessment of current and future hazards. A key component of this especially for theme (ii) will be the Large Ensemble Single Forcing Model Intercomparison Project (LESFMIP) experiments proposed here.

Potential drivers of multi-annual to decadal changes in climate

On multi-annual to decadal timescales climate is potentially influenced by several factors: changes in greenhouse gas concentrations, anthropogenic aerosols, volcanic aerosols, solar irradiance, ozone, land use, biomass burning, dust, and internal variability especially involving the ocean (some of which may be predictable). In most regions a long-term warming trend from greenhouse gases has been partially offset by the cooling effects of anthropogenic aerosols. However, aerosol effects are much more heterogeneous both in time and space, due to strong regional patterns of emission changes and indirect effects through interactions with clouds. Many studies have shown potential anthropogenic aerosol impacts on decadal climate including AMV (Booth et al., 2012; Bellucci et al., 2017; Murphy et al., 2017; Bellomo et al., 2018; Watanabe and Tatebe, 2019), Atlantic Meridional Overturning Circulation (AMOC, Menary et al., 2020; Hassan et al., 2021), the Aleutian Low and Pacific Decadal Variability (PDV, Allen et al., 2014; Smith et al., 2016; Takahashi and Watanabe, 2016; Oudar et al., 2018; Wilcox et al., 2019; Dittus et al., 2021; Dow et al., 2021), mid-latitude atmospheric jets (Wang Y. et al., 2020; Dong et al., 2022), southern hemisphere atmospheric circulation (Gillett et al., 2013; Rotstayn et al., 2013; Wang H. et al., 2020), Atlantic hurricanes (Mann and Emanuel, 2006; Dunstone et al., 2013), Sahel rainfall (Ackerley et al., 2011; Marvel et al., 2020; Hirasawa et al., 2022), and monsoon rainfall (Bollasina et al., 2011; Polson et al., 2014; Ma et al., 2017; Zhou et al., 2020), though aerosol indirect effects are particularly uncertain and these links are still debated (Oudar et al., 2018; Zhang R. et al., 2019; Baek et al., 2022).

Major volcanic eruptions inject aerosols into the stratosphere, warming the equatorial stratosphere and cooling global mean surface temperature for several years. Warming of the equatorial stratosphere strengthens the polar vortex in both hemispheres with subsequent surface impacts (Robock, 2000; Shindell et al., 2004), though the size of the response (Azoulay et al., 2021; DallaSanta and Polvani, 2022) and whether it is underestimated by climate models (Stenchikov et al., 2006; Driscoll et al., 2012; Swingedouw et al., 2017; Hermanson et al., 2020) is debated. Although not predictable in advance, volcanic eruptions are important for understanding climate variability and provide predictability for several years after they occur (Timmreck et al., 2016; Ménégoz et al., 2018). Multi-annual to decadal impacts of volcanic eruptions include AMV (Otterå et al., 2010; Knudsen et al., 2014; Swingedouw et al., 2017; Wang J. et al., 2017; Birkel et al., 2018; Mann et al., 2021), AMOC (Stenchikov et al., 2009; Mignot et al., 2011; Iwi et al., 2012; Zanchettin et al., 2013; Swingedouw et al., 2015; Hermanson et al., 2020), El Niño (Maher et al., 2015; Khodri et al., 2017; Zuo et al., 2018; Hermanson et al., 2020), PDV (Wang T. et al., 2012; Gregory et al., 2019), tropical cyclones (Evan, 2012; Pausata and Camargo, 2019), and global precipitation patterns (Iles and Hegerl, 2014; Tejedor et al., 2021) including Sahel rainfall (Haywood et al., 2013), monsoons (Fasullo et al., 2019; Zuo et al., 2019a) and rainfall in global arid regions (Zuo et al., 2019b).

Aerosols can also change through biomass burning and natural variations in dust. Biomass burning has been implicated in decadal changes in temperature and the hydrological cycle (Fasullo et al., 2022; Heyblom et al., 2022), and dust variations have been linked to AMV, Sahel rainfall and tropical cyclones (Evan et al., 2011; Wang C. et al., 2012; Strong et al., 2018).

Although changes in solar irradiance are small, they can have significant impacts on climate (Gray et al., 2010). “Top-down” influences occur through increases in ultra-violet radiation and associated ozone feedbacks which warm the tropical stratosphere, strengthening the polar vortex (Kodera and Kuroda, 2002) and promoting a positive NAO. This is seen both in the response to the 11-year solar cycle (Ineson et al., 2011; Gray et al., 2013; Andrews et al., 2015; Thiéblemont et al., 2015; Ma et al., 2018; Kuroda et al., 2021) and on multi-decadal timescales (Shindell et al., 2001; Ineson et al., 2015; Maycock et al., 2015; Chiodo et al., 2016; Spiegl and Langematz, 2020), potentially influencing the Atlantic Ocean and the AMOC (Scaife et al., 2013; Menary and Scaife, 2014). “Bottom-up” influences occur mainly through changes in visible radiation which particularly affect the tropical Pacific, though there is uncertainty whether increased solar irradiance causes a La Niña response (Meehl et al., 2009) or a reduction in the Walker Circulation more typical of El Niño (Misios et al., 2019), and whether the response depends on the timescale of the forcing (Meehl et al., 2013). Furthermore, changes in the tropical Pacific may generate Rossby waves that could impact the NAO (Swingedouw et al., 2011; Chiodo et al., 2016). Solar influences on tropical tropospheric temperatures have also been suggested to influence multi-decadal variations of the Southern Annular Mode (Wright et al., 2022).

Since stratospheric ozone warms the stratosphere by absorbing solar radiation, depletion of ozone over the Antarctic caused local cooling that increased the meridional temperature gradient and promoted a positive SAM in austral summer (Thompson and Solomon, 2002; Son et al., 2010; McLandress et al., 2011; Polvani et al., 2011; Thompson et al., 2011). Conversely, ozone recovery tends to shift the SAM toward the negative phase, opposing the positive SAM signal driven by greenhouse gases (Perlwitz et al., 2008; Banerjee et al., 2020). Stratospheric ozone-driven changes in the SAM are accompanied by changes in the Southern Hemisphere Hadley cell and associated sub-tropical jet (Jebri et al., 2020) and precipitation (Kang et al., 2011; Delworth and Zeng, 2014). Tropospheric ozone may also affect tropical expansion (Allen et al., 2012) and Southern Ocean warming (Liu et al., 2022).

Land use changes affect surface albedo, evapotranspiration and surface roughness, and hence the exchange of heat, moisture, and momentum between the land surface and the atmosphere (Findell et al., 2007; Lawrence et al., 2016; Pongratz et al., 2021). This can result in local changes in temperature, rainfall, and associated extremes (Mueller et al., 2016; Findell et al., 2017), and remote impacts are possible through atmospheric teleconnections (Snyder, 2010; Teng et al., 2019, 2022; Wang et al., 2019).

Multi-annual to decadal changes in climate may also occur in the absence of external drivers through natural internal variability (Cassou et al., 2018), especially in the Atlantic (Knight et al., 2005; Delworth et al., 2007), Pacific (Power et al., 2021), and Southern oceans (Zhang L. et al., 2019), but also the atmosphere (Dimdore-Miles et al., 2022). Moreover, many multi-annual to decadal changes in climate may occur through a combination of external forcing and internal variability (e.g., Huang et al., 2020; Bonnet et al., 2021; Wu et al., 2021) complicating the separation of forcing's from the natural variability in observed changes.

LESFMIP experiments and analysis

Observations alone are insufficient for robust attribution because it is difficult to separate multiple external drivers and internal variability, causality cannot be assessed unequivocally, and they are not yet available to assess forecast signals. Hence, we propose numerical model experiments that are designed to isolate the impacts of individual external drivers (Table 1). To the extent that the representation of the relevant physical processes in a model can be trusted, these model experiments can indicate the causal influence of external forcing factors on historical trends. However, analysis of these experiments must consider potential model errors (see below), non-additivity of drivers, potential errors in forcings (Wang et al., 2021; Fasullo et al., 2022) and potential impacts of missing processes including glacial melt (Rye et al., 2020; Devilliers et al., 2021). Any changes that cannot be explained by external factors after accounting for all of these factors would be assumed to be internal variability.

TABLE 1

Table 1. LESFMIP coordinated model experiments.

Experiment set 1 will enable the modeled response to individual forcings to be determined over the historical period. Experiments 1.1–1.4 are identical to those proposed by the Detection and Attribution Model Intercomparison Project (DAMIP, Gillett et al., 2016) but with increased ensemble size. Experiment 1.5 (hist-totalO3) has the same ozone prescribed as that used in the historical and SSP2-4.5 simulations. In models with interactive ozone chemistry, the simulated ozone from the historical and SSP2-4.5 simulations should be prescribed, though we encourage groups that are able to assess sensitivities to this (Ivanciu et al., 2021). These simulations are complementary to the hist-stratO3 experiments in DAMIP and are included here to complete the set of individual forcings and allow their additivity to be assessed by comparing with All-forcing simulations (experiment 3.1). We encourage groups to perform hist-stratO3 with at least 10 ensemble members if they are able to allow the effects of tropospheric and stratospheric ozone changes to be separately identified. Experiment 1.6 is an important addition to assess the impact of land use and land cover changes in a consistent manner, providing additional information to the experiments proposed in the Land Use Model Intercomparison Project (Lawrence et al., 2016).

Experiment set 2 extends set 1 by using updated estimates and projections of forcings. This will be especially important in the event of the next future major volcanic eruption, but will also allow deviations in aerosol and GHG emissions and other forcings from the scenario used in experiment set 1 to be assessed, potentially allowing the effects of mitigation measures to be simulated, and possibly detected in observations (e.g., Tebaldi and Friedlingstein, 2013). A focus of LHA-EPESC will be to create updated forcing datasets for experiment set 2 in a timely manner, ensuring their compatibility with the CMIP6 forcings used in experiment set 1. Experiment set 2 extends up to 10 years into the future to provide attribution of forecast signals as well as improved attribution of recent changes.

Experiment set 3 enables the additivity of multiple forcings to be assessed (Shiogama et al., 2013) by comparing with the sum of individual forcings from experiment set 1. Note that solar and volcanic influences on ozone that were applied in the single forcing DAMIP protocol will need to be accounted for.

Experiment sets 4 and 5 are similar to 1 and 2 except that each individual forcing is held constant with all others specified according to the All-forcing simulations (Deser et al., 2020). Experiments 3.1 and 3.3 are therefore also required to make use of experiments 4 and 5. By comparing with experiment sets 1 and 2 experiment sets 4 and 5 will allow the influence of changes in the background climate on attribution to be assessed (Meehl et al., 2004; Ming and Ramaswamy, 2009; Deng et al., 2020), and the assumption of linear additivity of the response to forcings to be tested (e.g., Bindoff et al., 2013).

Large ensembles (ideally 50 members, but a minimum of 10) are requested in order to quantify forced signals on regional scales in the presence of internal variability. This requirement exists regardless of model errors, but is amplified by specific challenges in simulating changes in atmospheric circulation which are discussed below. Although fully coupled climate models are needed to simulate the full responses, we also encourage the use of a hierarchy of models, including slab oceans, to assess the roles of ocean-atmosphere coupling and dynamic ocean variability (e.g., Chemke et al., 2022).

A key challenge in analyzing these experiments will be accounting for model errors (Bellprat et al., 2019). For example, although some models simulate AMV in response to external forcings that closely matches observations (Booth et al., 2012; Murphy et al., 2017; Bellomo et al., 2018) whether they do so for the right reasons is debated (Zhang R. et al., 2019). In particular, AMV is strongly related to the NAO integrated over previous decades in observations, but this relationship is much weaker in models (Peings et al., 2016; O'Reilly et al., 2019; Lai et al., 2022). Furthermore, models underestimate multi-decadal variability, including for temperatures (Cheung et al., 2017; Kravtsov, 2017; Qasmi et al., 2017), especially for the North Atlantic (Wang X. et al., 2017; Kim et al., 2018), the North Atlantic atmospheric jet (Bracegirdle et al., 2018; Simpson et al., 2018), and trends in the NAO (Eade et al., 2022).

A potentially very important model error is the underestimation of predictable or forced atmospheric circulation signals. This has been termed the “signal-to-noise paradox” (SNP, Scaife and Smith, 2018) since a model can unexpectedly predict the real world better than itself despite being a perfect representation of itself. First diagnosed in seasonal forecasts of the NAO (Eade et al., 2014; Scaife et al., 2014) the SNP has now been found in sub-seasonal (Domeisen et al., 2020), seasonal (Baker et al., 2018; Dunstone et al., 2018), interannual (Dunstone et al., 2016, 2020), multi-annual (Sheen et al., 2017; Yeager et al., 2018; Hu and Zhou, 2021) and decadal (Smith et al., 2019a, 2020; Athanasiadis et al., 2020) forecasts. On seasonal timescales the SNP occurs mainly in the extratropics, especially the North Atlantic where the NAO signal is underestimated by a factor of 2 to 3 (Eade et al., 2014; Scaife et al., 2014; Dunstone et al., 2016; Baker et al., 2018). However, on decadal timescales the SNP appears to be stronger, with the NAO underestimated by an order of magnitude (Smith et al., 2020), and more widespread, affecting the tropics as well as the extratropics (Eade et al., 2014; Smith et al., 2019a). Although the SNP may vary over time or be model dependent, it appears to be present whenever the NAO skill is high (Weisheimer et al., 2019). Where atmospheric signals are underestimated, taking models at face value (Deser et al., 2017) will give misleading conclusions about the role of irreducible internal variability. Importantly, the SNP is also found in uninitialized historical simulations (Sévellec and Drijfhout, 2019; Zhang and Kirtman, 2019; Klavans et al., 2021; Zhang et al., 2021). It is therefore not simply an artifact of initialization, and models underestimate the response to at least some external forcings. This is consistent with other evidence that modeled responses to volcanoes (Stenchikov et al., 2006; Driscoll et al., 2012; Swingedouw et al., 2017; Hermanson et al., 2020) and solar variability (Stott et al., 2003; Matthes et al., 2004; Gray et al., 2013; Scaife et al., 2013) may be too weak, and may explain the lack of signals noted in some studies (Schurer et al., 2014; Chiodo et al., 2019; Azoulay et al., 2021; DallaSanta and Polvani, 2022).

The cause of the SNP is currently unknown, though underestimated eddy feedback (Scaife et al., 2019; Hardiman et al., 2022) and errors in ocean-atmosphere interactions (Ossó et al., 2020; Zhang et al., 2021) could be important. For atmospheric circulation patterns such as the NAO, the error can potentially be overcome by taking the mean of a very large ensemble to diagnose the modeled signal and then inflating its variance to match the observed predictable component (Eade et al., 2014; Smith et al., 2020). The ensemble size required scales as the square of the signal deficit: an order of magnitude underestimation of the decadal NAO signal requires 100 times more ensemble members than would a perfect model (Smith et al., 2020). Hence, LESFMIP requests large ensembles (Table 1). However, scaling will not overcome SNP-related errors in variables such as temperature and rainfall, which are impacted by atmospheric circulation errors (Smith et al., 2020). To illustrate why, we partition a climate signal (T) into dynamically (T_dyn) and thermodynamically (T_therm) driven components:

\begin{array}{l} T = T_{d y n} + T_{t h e r m} + ε & (1) \end{array}

where ε is the residual. In some regions, T_dyn≫T_therm in reality, but T_dyn≪T_therm in the model ensemble mean if the atmospheric circulation response to forcings is severely underestimated. Scaling the ensemble mean will retain the incorrect ratio $\frac{T_{d y n}}{T_{t h e r m}}$ and fail to attribute dynamical signals.

To overcome this error new approaches are needed to analyse model output. We illustrate this for the extreme positive NAO period (1986–1997) for which observations show a clear pattern of warm anomalies over northern Europe and south-east USA and cold anomalies over northern Africa and eastern Canada (Figure 1A). The raw ensemble mean of decadal predictions shows very little signal (Figure 1B) and scaling to match the observed variance shows little improvement (Figure 1C) because the dynamical signals that produced the observed extremes are underestimated. A potential solution is to select ensemble members that, by chance, have the correct magnitude of dynamical signals. The correct magnitude is diagnosed by scaling the ensemble mean NAO to match the observed predictable component, and can be obtained in forecasts and projections since it only requires the scaling factor to be diagnosed from past cases. Note that if the predictable or forced signals are underestimated then no single ensemble member would be expected to reproduce the observed timeseries, and the selected ensemble members will likely be different at each time. This approach, referred to as “NAO-matching” (Smith et al., 2020), reveals much more realistic temperature patterns (Figure 1D) and similar improvements for northern European rainfall (Figures 1E,F). It also improves AMV (Figures 1G,H) suggesting that NAO variations cannot be solely attributed to AMV and revealing an important dynamical influence on AMV that would affect attribution analysis. Note that other approaches such as using observed regression, analogs from models or observations (Deser et al., 2016; Sippel et al., 2019), or ensemble sub-selection based on drivers (Dobrynin et al., 2018), are also possible and warrant further investigation.

FIGURE 1

Figure 1. The need for new approaches to assess model output. Near surface temperature anomalies for the extreme NAO period (1986–1997) in (A) observations, and decadal predictions shown as (B) raw ensemble mean, (C) scaled ensemble mean to match the observed variance at each grid point, (D) NAO-matched ensemble mean. Middle and right columns show observed 8-year mean time-series (black) and decadal predictions (years 2–9, ensemble mean in red with 5–95% confidence interval shaded) as raw ensemble means (middle) and NAO-matched ensemble means (right) for (E,F) northern European rainfall (10°W−25°E, 55–70°N) and (G,H) AMV index (Trenberth and Shea, 2006). Decadal predictions start each year from 1960 to 2005 and consist of a total of 169 ensemble members from 13 different models. (A–D) Show standardized values, obtained by dividing by the standard deviation of rolling 8-year means at each grid point. (A,B) show signals standardized by the observed variability, (C,D) show signals standardized by the ensemble mean variability. NAO-matching is achieved by selecting the 20 ensemble members at each time that are closest to the ensemble mean after scaling this to account for its underestimation of the magnitude of the predictable signal. All panels show boreal winter (December-March) anomalies relative the average over all year 2–9 predictions. Adapted from Smith et al. (2020), where further details are available.

An important aspect of the analysis of LESFMIP experiments will be to diagnose the real-world response to individual forcings over the historical period. We propose to develop “emergent constraints” (Hall et al., 2019) that exploit differences between models and relate them to observed quantities. For example, the atmospheric response to future Arctic sea ice loss depends on atmospheric eddy feedback which is underestimated by models, constraining the real world toward the upper end of the model simulations (Smith et al., 2022). Multi-model LESFMIP simulations are therefore essential, as is the need to develop constraints directly tied to the underlying physical processes (Smith et al., 2022).

The optimum analysis procedure will require further research and will be developed as the experimental results become available. Based on the discussion above, the key ingredients must allow for potentially underestimated atmospheric circulation signals and should therefore not treat model ensemble members as alternative realizations of the observations. As a start we suggest the following:

1. For a given event or forecast signal, identify the relevant patterns of atmospheric circulation.

2. For each model ensemble mean, perform “detection and attribution” analysis (multiple linear regression) on the atmospheric circulation patterns to obtain scaling factors for each driver.

3. Develop emergent constraints to exploit differences in scaling factors between models to diagnose the influence of each driver in the real-world.

4. Sub-select ensemble members, or obtain model or observed analogs, which match the real-world influence of each driver diagnosed in step 3.

5. For the variable of interest compute the contribution of each driver using the sub-selected or analogs ensembles from step 4.

Step 3 in particular is non-trivial and will require further research to understand the physical reasons for model differences. Even if constraints cannot be found, the experiments will still be extremely valuable for assessing model uncertainties in the potential roles of individual drivers and internal variability, and for highlighting specific areas where model improvements are needed. Step 2 assumes linear additivity of forcings which should be assessed using experiment set 3 or other methods. The LESFMIP experiments will also allow the investigation of pattern recognition to single out the influence of external forcing in ensemble simulations (Wills et al., 2020). Step 4 requires further research to extend NAO-matching or similar techniques to other modes of atmospheric variability. Further developments will also be required to account for uncertainties in forcing and observations, and to assess potential non-linearities and dependencies on the background state. The analysis plan proposed here could potentially be applied to extreme events as well as multi-year to decadal signals, enabling model errors and differences to be taken into account in extreme event attribution studies (e.g., van Oldenborgh et al., 2022), though further testing will be required to assess this application.

Data request

The requested diagnostics are the same for all LESFMIP experiments. Given the large ensemble sizes the data request is substantially reduced and is the same as for the Polar Amplification Model Intercomparison Project (PAMIP, Smith et al., 2019b): Appendix D in Boer et al., 2016 with the addition of wave activity diagnostics [Table 3 in Smith et al. (2019b), see Gerber and Manzini (2016), for details on how to compute these variables]. In addition, we request monthly mean ambient aerosol optical thickness at 550 nm (od550aer) to assess aerosol forcing. We stress that the data request is not intended to exclude other variables and participants are encouraged to retain variables requested by other MIPs if possible. The model output from LESFMIP will be distributed through the Earth System Grid Federation (ESGF).

Summary

There is an urgent need to better understand the drivers of multi-annual to decadal changes in climate both to attribute recent events, including extremes, and to gain further confidence in forecasts that are issued each year by the WMO. Multi-annual to decadal changes in climate are influenced by multiple factors including greenhouse gases, aerosols, ozone, land use, volcanic eruptions, solar variations, and internal variability. The Large Ensemble Single Forcing Model Intercomparison Project (LESFMIP) documented here proposes a set of coordinated model experiments to isolate and assess the impacts of these different factors. Additional experiments are included to take advantage of updated estimates of forcings, and to assess potential non-linearities and dependencies on the background state. A key part of the analysis will be to account for model errors. For this, multi-model simulations are needed to diagnose the real-world situation by exploiting model differences with constraints based on the key physical processes, though research will be required to identify these. Large ensembles are also needed to extract the forced signals which may be too weak in models, especially for changes in atmospheric circulation. In this case it will also be necessary to adopt new approaches to analyse the model output, possibly by selecting those ensemble members with the required magnitude of atmospheric circulation signals. The simulations proposed here could lead to a step change in our ability to understand regional climate variability, change and predictability, and will be an important contribution to the WCRP Lighthouse Activity on Explaining and Predicting Earth System Change goal to develop an operational capability to attribute multi-annual to decadal changes in climate.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

Author contributions

DS led the design and writing. NG, IS, PA, JB, IB, TB, RB, OB, KF, GG, SG, LH, LL, JM, WM, SO, OO, GP, AS, GS, HS, RS, DS, SY, TZh, and TZi commented on the design and manuscript. All authors contributed to the article and approved the submitted version.

Funding

DS, LH, and AS were supported by the Met Office Hadley Centre Climate Programme funded by BEIS and Defra. DS, RB, and OB were also supported by the European Commission Horizon 2020 CONSTRAIN project (GA 820829). Climate modeling at GISS is supported by the NASA Modeling, Analysis and Prediction program. LL was supported by U.S. Department of Energy Office of Science Biological and Environmental Research through the Global and Regional Model Analysis program area. PNNL is operated for DOE by the Battelle Memorial Institute under contract DE-AC05-76RL01830. TZi received funding from the Australian Government under the National Environmental Science Program. RS was supported by the UK National Centre for Atmospheric Science. HS was supported by the Advanced Studies of Climate Change Projection (SENTAN, JPMXD0722680395) of the Ministry of Education, Culture, Sports, Science and Technology of Japan. SY was supported by the Danish National Centre for Climate Research (NCKF). IS was supported by the National Center for Atmospheric Research, which is a major facility sponsored by the National Science Foundation under the Cooperative Agreement 1852977. JM and GG were also supported by the JPI climate/JPI ocean ROADMAP ANR-19-JPOC-003 and ANR-MOPGA ARCHANGE projects. TZh was supported by the National Natural Science Foundation of China under Grant No. 41988101.

Acknowledgments

Thanks to Hiro Murakami for providing valuable comments on a draft of the manuscript and to Dave Plummer for input on ozone datasets.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Ackerley, D., Booth, B. B., Knight, S. H., Highwood, E. J., Frame, D. J., Allen, M. R., et al. (2011). Sensitivity of 20th-century Sahel rainfall to sulfate aerosol and CO₂ forcing. J. Clim. 24 4999–5014 doi: 10.1175/JCLI-D-11-00019.1