Engagement opportunities in OECD NEA benchmark development

A myriad of opportunities is available to collaborate via international benchmark exercises and experimental data preservation activities. Many such opportunities abound under the auspices of the Nuclear Science Committee of the Organisation for Economic Co-operation and Development Nuclear Energy Agency (NEA). Key projects and activities of relevance to the development of advanced reactors design include the International Criticality Safety Benchmark Evaluation Project (ICSBEP), the International Reactor Physics Experiment Evaluation Project (IRPhEP), the International Assay Data of Spent Nuclear Fuel Database (SFCOMPO), the Shielding Integral Benchmark and Archive Database (SINBAD), and The International Experimental Thermal HYdraulicS Database (TIETHYS), and various cooperative benchmark exercises. Interested participants are encouraged to contact the leadership and secretariat of the various Technical Working Groups and Working Parties to become more engaged. This paper provides a summary of the current benchmark exercises and experimental databases available for international participation.

1 Introduction

The Organisation for Economic Co-operation and Development (OECD) Nuclear Energy Agency (NEA) is an intergovernmental agency that fosters and facilitates international collaboration to advance nuclear technology infrastructures enabling excellence in nuclear safety, technology, science, environment, and law. The objective of the NEA is to assist its member countries in maintaining and developing the scientific, technological, and legal foundation requisite for safe, environmentally sound, and economical use of nuclear energy for peaceful purposes. It also provides authoritative assessments and forges common understandings on key issues as input to government decisions on nuclear energy policy and to broader OECD analyses in areas such as energy and the sustainable development of low-carbon economies. However, the NEA is not a policy-making agency; policy decisions related to the use of information provided by NEA activities are made on a national or, as applicable, a regional level. This nuclear foundation is relevant to today’s nuclear facilities as well as future advanced reactor systems.

The NEA oversees a broad range of activities organized within standing technical committees, including the Committee on Nuclear Regulatory Activities (CNRA), Committee on the Safety of Nuclear Installations (CSNI), the Radioactive Waste Management Committee (RWMC) and the Nuclear Science Committee (NSC) Some of the standing committees have established up discipline-oriented Working Parties which report directly to them, or expert groups in charge of specific tasks, reporting either to the working parties or directly to the standing technical committees. The Agency’s nuclear science program is led by the NSC, while the CSNI is responsible for maintaining and advancing the scientific and technical knowledge base of the safety of nuclear installations. Both are comprised of high-level international scientific experts, and within each, various Working Parties and Technical Working Groups exist and operate to promote the co-ordination of work in different member countries that serve to maintain and enhance competence in nuclear safety matters, including the establishment of joint undertakings and assist in the distribution of the work to participating organisations. We will give a very abbreviated overview of CSNI and NSC activities to provide and understanding of activities that are relevant for the study of advanced reactor systems.

Within the CSNI, the Working Group on Fuel Safety (WGFS) is tasked with advancing the understanding of nuclear fuel safety issues by assessing the technical basis for current safety criteria and their applicability to high burnup and to new fuel designs and materials. WGFS aims to facilitate international convergence in this area, including as regards experimental approaches and interpretation and the use of experimental data relevant for fuel safety. One of the key areas in fuel safety is the analysis of fuel behavior under reactivity-initiated accident (RIA) conditions for which the WGFS has led major fuel performance codes benchmarking activities over the last decade. CSNI also oversees a number of International Standard Problems (ISPs) exercises, which are comparative exercises in which predictions or recalculations of a given physical problem with different best-estimate computer code that are compared with each other and above all with the results of a carefully specified experimental study.

Within the NSC, the WPFC Expert Group on Innovative Fuel Elements (EGIFE) is coordinating a benchmark related to fuel performance of fast reactor fuel. Among its constituent Expert Groups, the Expert Group on Innovative Fuels (EGIF) was created with the objective of conducting joint and comparative studies to support the development of innovative fuels such as homogeneous and heterogeneous fuels, accelerator driven system (ADS) fuels, and oxide, metal, nitride and carbide fuels, all of which can be implemented in advanced nuclear fuel cycles with fast reactors. The Working Party on Nuclear Criticality Safety (WPNCS) deals with technical and scientific issues relevant to criticality safety. Specific areas of interest include (but are not limited to) investigations of static and transient configurations encountered in the nuclear fuel cycle. These include fuel fabrication, transport, and storage. The Working Party on Scientific Issues and Uncertainty Analysis of Reactor Systems (WPRS) studies the reactor physics, fuel performance, and radiation transport and shielding in present and future nuclear power systems. The Working Party also studies the uncertainties associated with the modelling of these phenomena, particularly the modelling of reactor transient events. The current structure of the NEA committees and subsidiary bodies is shown in Figure 1, and can also be found online at the following weblink: https://www.oecd-nea.org/jcms/pl_36698/structure-of-nuclear-energy-agency-committees-and-subsidiary-bodies.

FIGURE 1

FIGURE 1. Structure of the Nuclear Energy Agency Committees and Subsidiary Bodies.

The purpose of this paper is to present a synopsis of a subset of the myriad of opportunities available via the NEA that serve to preserve and evaluate experimental data, to establish reliable benchmark cases, and to challenge state-of-the-art tools in cooperative benchmark exercises of relevance to the development of advanced reactors. In the context of this paper, advanced reactors, refers to essentially any reactor outside the water reactor arena, including, but not limited to, molten salt reactors, high-temperature gas reactors (HTGRs) using graphite as a moderator and helium as a coolant, sodium-, lead-, and gas-cooled fast as well as small modular reactors (SMRs) and micro-reactors that are designed using non-water coolant. We make this distinction as data on water cooled reactors is more readily available. The activities described herein originate within and report on various Working Parties and Expert Groups established by the NSC. Interested participants are encouraged to become engaged in these groups by contacting their respective leadership and NEA secretariats which are summarized in Section 3.

2 International collaborative efforts within the NEA nuclear science committee

2.1 International criticality safety benchmark evaluation project (ICSBEP)

The International Criticality Safety Benchmark Evaluation Project (ICSBEP) (Briggs, Scott, and Nouri, 2003), a sanctioned program under the auspices of the OECD NEA since 1995, is considered the gold standard for experimental benchmarking within the nuclear community. Its main purpose is to preserve and evaluate new and legacy integral experiment data and publish it in a standardized handbook format to provide quality benchmarks for modern and future criticality safety and nuclear data validation. Many neutronics codes around the world use ICSBEP benchmarks in their test suites and nuclear data libraries use the benchmark predictions as a main indicator of library fidelity. The ICSBEP and the International Reactor Physics Experiment Evaluation Project (see Section 2.3) have been invaluable sources of benchmark data from many countries, allowing access to experimental benchmarks that are unique, otherwise unavailable, and would be cost-prohibitive to replicate.

The 2022 edition of the ICSBEP Handbook (NEA, 2022a) includes 592 evaluations containing acceptable benchmark specifications for 5,144 critical, subcritical, or near-critical configurations, representing contributions from 26 countries. An additional 838 configurations deemed unacceptable to support criticality safety requirements yet are valuable to the community are also preserved within the handbook. Additionally, there are 10 criticality alarm placement/shielding evaluations containing a total of 46 benchmark configurations, and 11 fundamental physics evaluations containing a total of 238 measurements relevant to criticality safety applications.

Many advanced reactor design concepts fall well outside of the established validation coverage used for traditional light water reactors, involving novel materials (molten salts, liquid metals, new fuel alloys, increased enrichments) and harder neutron spectra. Integral benchmarks to test neutronics codes and their underlying nuclear data are vital to assessing the fidelity of code predictions, and the ICSBEP benchmarks represent a significant breadth of experiments with fissile and non-fissile materials with various energy spectra. Validation gaps exist for advanced reactors and their fuel cycles, but a thorough review of the ICSBEP Handbook should identify existing applicable, evaluated benchmarks to reduce the number of new experiments that must be performed and evaluated in support of the deployment of these novel reactor systems.

2.2 International reactor physics experiment evaluation project (IRPhEP)

The OECD-NEA International Reactor Physics Experiment Evaluation Project (IRPhEP) (Briggs and Gulliford, 2014) seeks to preserve and evaluate integral reactor physics experiment data to support nuclear energy and technology needs. International contributions are collated within the IRPhEP Handbook (NEA, 2022b). Measurements found in the Handbook include criticality, buckling and extrapolation length, spectral characteristics, reactivity effects, reactivity coefficients, kinetics, reaction rate distributions, power distributions, isotopic compositions, and/or other miscellaneous types of measurements for various types of reactor systems. Distributed with the IRPhEP Handbook and available online is the IRPhEP Database and Analysis Tool (IDAT) (Hill et al., 2014), allowing users to search and interrogate the data.

A total of 26 countries have contributed to the past and continued success of these projects as benchmark evaluations, technical reviews, and experimental data using their own time and resources. Contributing countries include Argentina, Austria, Belarus, Belgium, Brazil, Canada, People’s Republic of China, Czech Republic, Denmark, France, Germany, Hungary, India, Italy, Japan, Kazakhstan, Republic of Korea, Russian Federation, Serbia, Slovenia, South Africa, Spain, Sweden, Switzerland, United Kingdom, and the United States of America.

The IRPhE project is patterned after and closely coordinated with the International Criticality Safety Benchmark Evaluation Project (ICSBEP) (Briggs et al., 2003; NEA, 2020b) in order to avoid duplication of effort and publication of conflicting information. Some benchmark data are applicable to both nuclear criticality safety and reactor physics technology. Some have already been evaluated and published by the ICSBEP. However, the ICSBEP has focused primarily on critical and subcritical configurations and radiation transport measurements that are relevant to determining the need for and placement of criticality alarm systems.

The value of the IRPhEP is demonstrated by use of the benchmarks themselves. In (Palmiotti, et al., 2014) it is stated that the benchmark projects “have and will continue to make, vast amounts of valuable legacy and new data available to current and future nuclear energy–related programs.” This includes advanced reactor designs, including systems for which traditional physics experiments cannot be easily performed. For example, the Versatile Test Reactor (VTR) project has used two relevant benchmarks for validation of their calculational methods:

• Evaluation of the Initial Isothermal Physics Measurements at the Fast Flux Test Facility, a Prototypic Liquid Metal Fast Breeder Reactor, FFTF-LMFR-RESR-001, and

• ZPR-3 Assembly 56B: A Cylindrical Assembly of Mixed (Pu,U) Oxide and Sodium with a Nickel-Sodium Reflector, ZPR-LMFR-EXP-004.

The DOE/NNSA Material Management and Minimization (M3) program has made extensive use of the IRPhEP benchmark Advanced Test Reactor: Serpentine Arrangement of Highly Enriched Water-Moderated Uranium-Aluminide Fuel Plates Reflected by Beryllium, HEU-MET-THERM-022.

A number of benchmarks will be of value to the DOE Advanced Reactor Technologies and the Department of Defense TRISO-Based Microreactor Design programs:

• Evaluation of the Start-Up Core Physics Tests at Japan’s High Temperature Engineering Test Reactor (Fully-Loaded Core), HTTR-GCFR-RESR-001,

• HTR-PROTEUS Pebble Bed Experimental Program Cores 9 10: Columnar Hexagonal Point-On-Point Packing with a 1:1 Moderator-To-Fuel Pebble Ratio, PROTEUS-GCR-EXP-004, and

• Temperature Effect on Reactivity in VHTRC-1 Core, VHTRC-GCR-EXP-001.

Under the US Department of Energy’s (DOE) Nuclear Engineering University Programs (NEUP), a number of benchmarks for advanced reactors are being developed, including:

• Development and Evaluation of Neutron Thermalization Integral Benchmarks for Advanced Reactor Applications, PI: Ayman I. Hawari, North Carolina State University (2019),

• Regenerating Missing Experimental Parameters with Data-Assimilation Methods for MSRE Transient Benchmark Development and Evaluation, PI: Zeyun Wu, Virginia Commonwealth University (2021), and

• Separate and Mulitphysics Effects IRPhEP Benchmark Evaluation using SNAP Experiments, PI: Dan Kotlyar, Georgia Institute of Technology (2021).

In general, individuals and organizations solicit funding from their companies or other avenues of research and development support to develop a benchmark. And as mentioned earlier, development of IRPhEP benchmarks is often supported under DOE NEUP. Benchmarks evaluations are prepared based upon historic or recently performed experimental data using guides and example evaluations provided in the IRPhEP Handbook. One or more evaluators provide the primary assessment of a benchmark experiment, which is followed by an in-house verification of the analyses, including adherence to the handbook guidance and procedures performed by internal reviewer(s). Independent review is coordinated via the IRPhEP and NEA to verify the analysis; independent reviewers typically are external, often foreign, participants. In the event that insufficient personnel are available to support internal review for a given evaluation, then at least two independent reviewers are assigned to ensure sufficient peer-review prior to submission to the IRPhE TRG. Reviewers often serve on a voluntary basis or are supported by their own research programs. Individuals interested in participating as TRG reviewers should contact IRPhE leadership. Regardless of experience, many would benefit from the experience gained when reviewing these benchmarks (DeHart, et al., 2022).

2.3 International assay data of spent nuclear fuel database (SFCOMPO)

SFCOMPO is the largest international database of open experimental assay data for spent nuclear fuel, publicly available online at https://www.oecd-nea.org/jcms/pl_21515/sfcompo-2-0-spent-fuel-isotopic-composition. The database is hosted by the NEA and managed by the SFCOMPO Technical Review Group (TRG) under the NEA Nuclear Science Committee - Working Party on Nuclear Criticality Safety (NEA/NSC/WPNCS). Assay data in the SFCOMPO database consists of datasets of measured nuclide concentrations of well characterized irradiated nuclear fuel samples, with description of samples’ characteristics and operation histories being provided with adequate detail for potential use in benchmark models. Assay data are provided for 750 samples selected from fuel irradiated in 44 reactors, with 8 reactor types and over 24,000 measurement entries, and cover measurement data for 91 nuclides.

SFCOMPO originates from the database compiled in the 1990s by the Japan Atomic Energy Research Institute (JAERI), which consisted of a series of webpages with tables of measured data for fuel samples from 7 pressurized water reactors (PWRs) and 7 boiling water reactors (BWRs). This database was transferred from JAERI to NEA in 2001, has been hosted by NEA since and has been expanded significantly through the international community’s concerted effort led by the NEA Expert Group of Assay Data of Spent Nuclear Fuel (EGADSNF) during 2001–2013. Development of a new, modern functionality database with a standardized format and improved accessibility was initiated in 2013 and culminated with the release in 2016 of the SFCOMPO 2.0 new graphical user interface (Michel-Sendis et, al. 2017). This interface enables user-friendly content browsing and data visualization, and easy access to the primary references.

The SFCOMPO TRG has been mandated by the NEA/NSC/WPNCS to maintain and further coordinate the development of the SFCOMPO database. The TRG’s members and contributors are science and engineering experts from academia, industry, and research institutions worldwide. The SFCOMPO TRG’s mission is complementary to the efforts of IRPhEP and ICSBEP TRGs, to extend the applications beyond nuclear criticality safety or steady-state in-core analyses and support a broad range of fuel cycle needs, including radiological safety, source terms, shielding, and repository analyses. The primary missions of the SFCOMPO TRG are: 1) preservation of data (preserve and capture legacy data as well as new data as they become available); 2) accessibility of data (ad hoc formatting of the data, and continuous adaptation of the SFCOMPO database and its interface tools to address current and future needs); 3) evaluation of data (perform international peer-reviewed assessments to produce qualified benchmarks suitable for code validation); and 4) knowledge transfer (develop guidance for qualification of data evaluations and train new generations of experts through the evaluation/review process).

Evaluation of data for developing benchmarks is the current focus (Ilas, et al., 2020) of the TRG. Since 2019, two evaluations have been approved for release and are pending publication (“Evaluation of Three Mile Island Unit 1 Fuel Samples—Assemblies NJ05YU and NJ070G (Type 15 × 15)” by Georgeta Radulescu, and “Evaluation of Fukushima-Daini-1 Samples—Assemblies 2F1ZN2 and 2F1ZN3 (Type 9×9–9)” by Ugur Mertyurek). A further set of three draft evaluations are pending finalization of the independent review and approval by the TRG. To address community’s great interest in decay heat data addition as a new key spent fuel metric to the SFCOMPO database, a task force was established in January 2022 to prepare, review, and curate full-assembly decay heat experimental data for addition as new entry datasets in the SFCOMPO database. The requirements that are being developed for decay heat addition will be consolidated with previous enhancements of the database and its interface.

SFCOMPO is an invaluable asset serving the needs of the international community. Validity of safety assessments for handling irradiated nuclear fuel including transportation, storage, processing and recycling, and repository applications is largely based on capabilities to accurately predict the evolution of nuclides during and after irradiation in fuel and structural materials. Experimental assay data are essential for evaluating bias and uncertainties in spent nuclear fuel safety analyses and provide one means for determining uncertainties in integral quantities important to safety, such as decay heat or spent fuel reactivity, and to validate nuclear data. The importance of experimental assay data for code and associated nuclear data validation goes well beyond the back end of the fuel cycle applications, to impact any area where accurate estimation of nuclide inventories is impactful. Most of the experimental data in SFCOMPO applies to light-water reactor fuel. However, data are available from previous MAGNOX and advanced gas-cooled reactor (AGR) fuel experiments and the database can be easily expanded to include any advanced reactor fuel assay data once they would become available. The current database and the evaluations being developed provide great support in assessing the abilities of the underlying methods and nuclear data for advanced reactors of accurately predicting the nuclear transmutation and decay physics.

2.4 Shielding integral benchmark and archive database (SINBAD)

An international shielding benchmark was first proposed in 1988 at the 7^th International Conference on Radiation Shielding (ICRS-7), which resulted in the Shielding Integral Benchmark Archive Database (SINBAD) being established in 1996 as a joint effort between the Nuclear Energy Agency (NEA) and the Radiation Safety Information Computational Center (RSICC) (https://www.oecd-nea.org/jcms/pl_32139/shielding-integral-benchmark-archive-and-database-sinbad) (Kodeli, Sartori, and Kirk, 2006; Kodeli, et al., 2014). The goal of this type of database is to provide the community a way to validate their shielding or fixed source simulations and evaluated nuclear data. Today SINBAD continues under the auspices of the Expert Group on Physics of Reactor Systems (EGPRS), which is a subgroup of the NEA’s Working Party on Scientific Issues and Uncertainty Analysis of Reactor Systems (WPRS). The current release of SINBAD, which is available from the NEA Data Bank and RSICC, contains evaluations of 102 benchmark experiments. These experiments are broken into three broad categories, fission systems, fusion systems, and accelerator systems, which have 31, 48, and 23 benchmarks, respectively.

In February 2021 the EGPRS established the SINBAD Task Force (TF) to oversee the future development of SINBAD, which is consistent with the strategy of the NEA to continuously improve data available from their data bank. The TF will operate for 3 years, and then the EGPRS will evaluate the performance and determine how SINBAD development should continue in the future. The proposed aim of the SINBAD TF is to maintain and begin modernizing SINBAD. There are two major factors that led to the EGPRS establishing the TF. First, is the reduction in new benchmarks being added to the database. The second motivating factor is to modernize the database while building upon previous work. This previous work includes all entries currently in SINBAD, but also the quality reviews (Kodeli and Sartori, 2021). The goals prescribed by the EGPRS for the SINBAD TF are to provide new database entries and to improve the quality of the existing database entries. Providing new database entries is self-explanatory. The goal of improving the quality of the existing database entries is a very broad goal. Discussions with the EGPRS and TF participants have led to specific deliverables that achieve this goal, which are:

• when updating current evaluations or producing new evaluations, have a single summary document following the SINBAD evaluation guide approved in 2019 (NEA, 2022c),

• perform sensitivity and uncertainty quantification,

• provide accurate models of the geometry (CAD or some code agnostic format), materials, sources, and detector response parameters, and

• provide supplemental resources like sample code inputs and outputs, variance reduction parameters, tools to convert data to code input, and tools to post process code output.

The TF participants have also defined the following additional goals:

• do not lose or remove any existing information from the database,

• involve the nuclear data community, and

• capture the output of the Working Party on International Nuclear Data Evaluation Cooperation (WPEC) Subgroup 47, which was focused on the use of SINBAD for nuclear data validation (Kodeli, et al., 2022).

From the perspective of advance reactor design, like with criticality, thermal hydraulics, and other topics mention herein, many concepts are outside the domain of current benchmark experiments used for validation. Different fuel types, moderators, and coolants have been proposed that do not have benchmark quality experimental data. This becomes a shielding concern for biological dose around advanced reactors and their spent fuel, but also for radiation damage of reactor and fuel components. There is certainly a need for additional SINBAD benchmarks to help address these gaps in validation for advanced reactor designs.

2.5 The international experimental thermal Hydraulics database (TIETHYS)

Nuclear reactor design and accident analyses are generally multi-physics simulation problems. Validation of relevant multi-physics codes requires that the individual physics models are also validated in order to avoid compensating effects. One important component of multi-physics simulation is thermal-hydraulic (TH) physics and requires relevant TH data. Some of the TH codes are at a system level and others such as CFD (computational fluid dynamics) address more details of local thermal hydraulic phenomenon such as at sub-channel level. These system TH analytical tools have balance equations and many hundreds of correlations to provide information or model parameters about interactions between different phases at the interfaces. The shape and size of the interfaces determines the phasic transfer terms. In order to assess that capability of the codes and fidelity of the predictions, these codes must be validated with tests that represent phenomena in the plant for the specific scenario. The Element 2 in the US Nuclear Regulatory Commission (USNRC) suggested approach in Regulatory guide 1.203 (USNRC, 2005) for transient and accident analysis indicates that any analyses for licensing will require a validation matrix consisting of separate effects and integral effect tests. All advanced reactors will need design certification from USNRC. Similarly, other relevant guides exist for the UK (ONR, 2019), France (ASN, 2017), and Japan (AESJ, 2015).

The TH data is scattered in different locations and in different formats. Some of the data is in danger of being lost. The Nuclear Science Committee (NSC) of the NEA has developed a user-friendly GUI (graphical user interface) and a relational database, The International Experimental Thermal Hydraulic Systems database (TIETHYS) (Rohatgi, Dyrda, and Soppera, 2018), to organize and preserve the international TH test data for various reactor concepts and different scenarios. The database has an expandable platform with place holders for molten salt reactors, high temperature gas cooled reactors, CANDU, and liquid metal reactors. TIETHYS provides better access to and preserves this valuable information. In addition, the database will also expand as more information becomes available for given tests such as application, instrumentation uncertainty, and user guidelines. For TH code validation, it is important to provide accurate descriptions of geometry and initial conditions.

The validation of a code for a given application requires tests that cover the conditions expected in that application. The TIETHYS development has two parallel paths. One path includes a database for tests with searchable attributes. The second part is linking of these tests to applications through possible phenomena identification table, part of PIRT (Phenomena Identification and Ranking Table) process. The current database platform includes scenarios for PWRs, BWRs, vodo-vodyanoi enyergeticheskiy reactors (VVERs), and corresponding SETs (separate effects tests) and IETs (integral effects tests) along with specific benchmarks for CFD modelling. Currently, the database includes 45 integral facilities and 223 separate effects tests (which have been mostly derived from previous efforts carried-out by the OECD-NEA in the framework of the Computer Code Validation Matrix (CCVM) (NEA, 1994; NEA, 1996; NEA, 1997; NEA, 2001; NEA, 2017), and nine CFD-relevant tests (NEA, 2015). Numerous others have been identified for later inclusion for LWRs. This will meet some of the needs for the multi-physics development programs around the world. These tests are currently for LWRs, spanning PWR, BWR, and VVER designs.

It is important to note that it is difficult to define benchmarks for TH codes as they do not undergo rigorous nodalization studies as prediction can change with nodalization due to flow regime map definition, and related interfacial transfer of heat, mass, and momentum. The goal of this database is to provide descriptions of tests, geometry, initial conditions and transient test data, and examples of application. The code user can apply their own code guidelines to create nodalization.

The initial version of relational database TIETHYS and GUI software are freely open to public and available for testing via the NEA website (https://www.oecd-nea.org/tiethysweb/). Going forward the database will be extended to include additional links and data as they become available. The organizations conducting TH tests are encouraged to submit their test data to NEA for inclusion in database for wider exposure and for preservation.

Another initiative which is linked to TIETHYS and is worth mentioning is the OECD/NEA THEMPo (Harmonization of Methodologies for System Thermal-Hydraulics Experimental Meta-Data Preservation, Collection and Qualification) which will commence in 2023. The objective is to develop and/or to improve and then harmonize existing methodologies (Petruzzi and D’Auria, 2016) for collection, preservation, qualification, organization and then use of an exhaustive “Set of Experimental Information” (SEI). The derived methodology will provide the guide for the creation of a relational database of experimental meta-data with a standardization of the procedures by which the SEI are collected, qualified, and organized into a consolidated database which finally allows the code analysists for a faster and more accurate development of computational simulation models of experimental tests to be exploited for code validation purposes.

2.6 Challenging state-of-the-art tools in cooperative benchmark exercises

Benchmark exercises at OECD NEA challenge our current state-of-the-art tools and provide international best practice guidance on simulation methods and tools. Benchmarks are typically either based on existing experimental benchmark cases in the NEA databases or lead to the development of new benchmark cases which then become part of the databases.

Key activities with relevance to the development of advanced reactors design include the multi-physics benchmark exercises in the Expert Group on Reactor Systems Multi-Physics (EGMUP), which is here presented as a showcase. EGMUP originates from the Expert Group on Multi-Physics Experimental Data, Benchmarks and Validation (EGMPEBV) which was established in 2014 by the NEA to establish processes for certification of experimental data and development of benchmark models for validation of multi-physics computational methods (Finck, et al., 2016; Avramova, et al., 2017). This Expert Group was reorganized as the Expert Group on Reactor Systems Multi-Physics (EGMUP) and placed under the direction of the NEA Working Party on Scientific Issues and Uncertainty Analysis of Reactor Systems (WPRS) (Valentine, et al., 2021). The EGMUP seeks to advance the state-of-the-art in establishing processes and procedures for certifying experimental data and benchmarking multi-physics multi-scale modelling and simulation (M&S). The term multi-physics implies computationally coupled interaction of two or more of the following physical phenomena (physics) that include, but are not limited to, reactor physics, thermal-hydraulics, fuel performance, structural mechanics, materials chemistry, and heat transfer.

While single-physics benchmarks have significant value, in real-world systems all physics are coupled to one degree or another. In recognition of this truth, multi-physics methods have evolved internationally over the last 2 decades with varying degrees of fidelity (DeHart, et al., 2017). It has also been recognized that few measurements have been performed in which simultaneous measurements of coupled phenomena have been performed. Hence, this Expert Group is focused on the coupled physics aspects of both steady-state and transient simulations for both existing and advanced nuclear systems, along with uncertainty quantification and propagation through different scales (multi-scale M&S) and different physics phenomena (multi-physics M&S).

The expert group provides recommendations to the WPRS and the nuclear community on scientific development needs, e.g., data and methods, validation experiments, scenario studies, etc., for multi-physics and multi-scale M&S, including sensitivity and uncertainty methodologies for analysis of different reactor systems and scenarios. The Expert Group aims to develop guidance and recommendations for verification of experimental data for multi-physics multi-scale M&S and to apply this data to the benchmarking of models. To this end, the Expert Group will work to provide:

1. Standardized benchmark models with detailed uncertainty evaluations and uncertainty methodology guidelines;

2. Guidance on best practices to combine high fidelity and low fidelity simulation tools;

3. A framework and consensus recommendations for validating multi-physics simulations;

4. Sensitivity and uncertainty methods to facilitate quantification and ranking of coupled physics;

5. Evaluation methods for uncertainty quantification of the following parameters on multi-physics simulations:

a. Data (e.g., geometry, physical properties),

b. Numerical methods, and

c. Physical models.

6. Training opportunities to demonstrate validation principles and practices; and

7. Demonstrations of the validation recommendations for specific applications.

To support its activities, the group will collect and evaluate multi-physics data from available integral facilities, test and research reactor and nuclear power plant experimental data; analytical and numerical benchmarking will also be used to fulfil the objectives.

All nuclear reactor systems operate in a multi-physics domain. Simulations of many advanced reactor systems have demonstrated tightly coupled behavior between different physical aspects of the core, including micro-reactors, molten salt and fast reactor systems and nuclear thermal propulsion concepts. With these generally reactors still in early design phases and prototypes being developed for testing, multi-physics data from such systems may become available in the next few years. However, multi-physics data to support such design is in short supply. This need drives the importance of this Expert Group to strive to fill this gap.

Other similar benchmark exercise activities exist for all domains within the scientific portfolio covered by the OECD NEA Nuclear Science Committee. For more information, interested parties should reach out to the Secretariats of the OECD-NEA Nuclear Science Committee (see Section 3).

3 Active engagements within the NEA benchmark activities

The various activities rely on international participation, typically coordinated through in-person meetings once or twice a year, typically but not always held the NEA headquarters in Paris. In-person meeting were replaced by virtual meetings during COVID travel restrictions between 2020 and 2022, and future meetings will likely include virtual participation. However, time zone differences limit virtual meetings to short time windows. Nevertheless, each of the benchmark activities will move forward beyond 2023 with continued international coordination meetings.

Both the ICSBEP and IRPhEP Technical Review Groups (TRG) have traditionally been very active since their inceptions, meeting together once or twice a year to review new experimental benchmarks for inclusion into new respective handbook editions. Experiments are independently evaluated by scientists and engineers at institutions around the world, undergoing an extensive internal and external review process. The final step in this process is a final review by the TRG for potential inclusion into the handbook during the in-person meeting.

A pilot SFCOMPO TRG meeting was hosted by NEA in March 2019. Since then, more participants have joined the effort and are actively contributing. In 2021, a task force was formed under the TRG to review the existing evaluation guidance (NEA, 2016) and provide feedback on how to further improve it and address the uncertainty evaluation challenge, based on recent lessons learned and drawing from similar IRPhEP and ICSBEP experience.

Before WPRS’s EGPRS established the SINBAD Task Force in 2021, SINBAD meetings were often held in tandem with IRPhEP/ICSBEP meetings. At the time of this writing, the schedule for future meetings of the SINBAD Task Force has not been determined, but the Task Force may continue to hold meetings in tandem with IRPhEP/ICSBEP meetings.

The first workshop on preservation of Thermal-Hydraulics experimental data workshop (TH-1) was conducted under the auspices of new Expert Group on Reactor Core Thermal Hydraulics (EGRCTH) within the WPRS. The meeting was hosted by Gesellschaft für Anlagen-und Reaktorsicherheit (GRS) gGmbH, in Garching, Germany in June, 2019. This workshop elected to endorse the effort to provide for preservation of experimental data for model development and validation in the form of the existing TIETHYS database. Since this time updates to TIETHYS are reported to EGRCTH, but workshops have held independently of WPRS meetings.

As was noted earlier, the EGMUP originated from the earlier EGMPEBV, which met twice a year, usually at NEA headquarters, beginning in 2014. After reformed as the EGMUP, this Expert Group has met in tandem with the WPRS since 2021 at NEA Headquarters.

Each of these benchmark projects continues to expand and improve through the concerted contributions of the international community, and each is actively and continuously soliciting new participants from NEA member countries; the nature of the participation (e.g., evaluators, reviewers, observers, etc.) varies by benchmark activity. International expertise and enthusiasm are needed to build the future of each project. There is ongoing work that practically all experts in various aspects of the nuclear energy community will find interesting and applicable to their daily work. Some of the projects currently do not have enough available volunteers to wholly fulfill their charters.

4 Conclusion

The OECD NEA enables international collaboration in advanced nuclear technology in support of nuclear safety, technology, science, environment, and law to ensure safe, economical, and environmentally sound use of peaceful nuclear energy. There are numerous ongoing activities relevant in support of modern nuclear facilities and future advanced reactor systems. International participants serving in the various efforts such as ICSBEP, IRPhEP, SFCOMPO, SINBAD, TIETHYS, and other cooperative benchmark exercises contribute their time and expertise to ensure continued success. The continued success of these projects has concatenated priceless information into utile resources supporting various aspects of modeling, design, simulation, and validation. This paper provides a summary of current ongoing work in these activities, and interested participants are encouraged to become engaged in these groups by contacting their respective leadership and NEA secretariat (Table 1).

TABLE 1

TABLE 1. Contact and leadership information for OECD NEA international benchmark exercises.

To get more information on other benchmark exercises, please contact the following Secretariats within the NEA Division of Nuclear Science and Education:

• Working Party on Scientific Issues of Advanced Fuel Cycles (WPFC): d3BmY0BvZWNkLW5lYS5vcmc=

• Working Party on Nuclear Criticality Safety (WPNCS): d3BuY3NAb2VjZC1uZWEub3Jn

• Working Party on International Nuclear Data Evaluation Co-operation (WPEC): d3BlY0BvZWNkLW5lYS5vcmc=

• Working Party on Scientific Issues and Uncertainty Analysis of Reactor Systems (WPRS): d3Byc0BvZWNkLW5lYS5vcmc=

Data availability statement

The data analyzed in this study is subject to the following licenses/restrictions: The data from these working groups are available to OECD NEA member countries and participating scientists/organizations. Requests to access these datasets should be directed to the NEA Secretariats.

Author contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Funding

Contributions, financial and otherwise, towards the success of the OECD NEA international benchmark projects have come from a myriad of international sources.

Acknowledgments

The OECD NEA activities are collaborative efforts that involve numerous scientists, engineers, and administrative support personnel across the world. The authors would like to acknowledge the efforts of all these dedicated individuals without whom these efforts would not be possible.

Conflict of interest

JB is employed by JFoster & Associates. AP is employed by Nuclear and INdustrial Engineering.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer PR declared a shared affiliation with the author EI to the handling editor at the time of review.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

AESJ (2015). AESJ guide for the assessment of nuclear simulation credibility: 2015. AESJ-SC-A008E: 2015. Tokyo, Japan: Atomic Energy Society of Japan. Available at: https://www.aesj.net/publish-1722.

Google Scholar

ASN (2017). Qualification of scientific computing tools used in the nuclear safety case – 1st barrier. guide No. 28, autorité de Sûreté Nucléaire, montrouge, France. Available at: https://www.french-nuclear-safety.fr/asn-regulates/asn-guides/asn-guide-no.-28.

Google Scholar

Avramova, M., Ivanov, K., DeHart, M., Dyrda, J., Hudelot, J-P., Petruzzi, A., et al. (2017). OECD/NEA EGMPEBV activities in multi-physics verification and validation. Trans. Am. Nucl. Soc. 116 (1), 1297–1300. Available at: https://epubs.ans.org/?p=trans.

Google Scholar

Briggs, J. B., and Gulliford, J. (2014). An overview of the international reactor physics experiment evaluation project. Nucl. Sci. Eng. 178 (3), 269–279. doi:10.13182/NSE14-27

CrossRef Full Text | Google Scholar

Briggs, J. B., Scott, L., and Nouri, A. (2003). The international criticality safety benchmark evaluation project. Nucl. Sci. Eng. 145 (1), 1–10. doi:10.13182/NSE03-14

CrossRef Full Text | Google Scholar

DeHart, M. D., Bess, J. D., Ivanova, T., and Hill, I. (2022). A Review of the Status of the Evaluated Reactor Physics Benchmark Experiments Project, 127. Transactions of the American Nuclear Society. Available from: https://www.ans.org/pubs/transactions/.

Google Scholar

DeHart, M. D., Le Pallec, J.-C., and Hudelot, J.-P. (2017). Multinational efforts for development of novel multi-physics modelling and simulation tools. Paris, France: OECD Nuclear Energy Agency. NEA/NSC/DOC(2017)12.

Google Scholar

Finck, P., Valentine, T., Dyrda, J., Gulliford, J., Hudelot, J.-P., and Rohatgi, U. (2016). “OECD-NEA expert group on multi-physics experimental data, benchmarks and validation,” in Proceedings of PHYSOR 2016: Unifying Theory and Experiments in the 21st Century, Sun Valley, Idaho, May 1–5. Available from: https://www.ans.org/store/item-700403/.

Google Scholar

Hill, I., Soppera, N., and Bossant, M. (2014). Idat: The international handbook of evaluated reactor physics benchmark experiments database and analysis tool. Nucl. Sci. Eng. 178 (3), 280–294. doi:10.13182/NSE14-37

CrossRef Full Text | Google Scholar

Ilas, G., Gauld, I., Ortego, P., and Tsuda, S. (2020). “SFCOMPO database of spent nuclear fuel assay data - the next frontier,” in Proceedings of PHYSOR 2020 – International Conference on Physics of Reactors: Transition to a Scalable Nuclear Future, Cambridge, UK. doi:10.1051/epjconf/202124710019

CrossRef Full Text | Google Scholar

Kodeli, I., Fleming, M., Cabellos, O., Leal, L., Çelik, Y., Ding, Y., et al. (2022). “Outcomes of WPEC SG47 on use of shielding integral benchmark archive and database for nuclear data validation,” in Proceedings 15th International Conference on Nuclear Data for Science and Technology (ND2022), Virtual Meeting, July 25-29.

Google Scholar

Kodeli, I., Milocco, A., Ortego, P., and Sartori, E. (2014). 20 Years of SINBAD (shielding integral benchmark archive and database). Prog. Nucl. Sci. Technol., 4, pp. 308–311. doi:10.15669/pnst.4.308

CrossRef Full Text | Google Scholar

Kodeli, I., Sartori, E., and Kirk, B. (2006). “SINBAD shielding benchmark experiments status and planned activities,” in Proceedings of the 14th Topical Meeting of the Radiation Protection and Shielding Division (RPSD 2006), Carlsbad, New Mexico, April 3-6.

Google Scholar

Kodeli, I., and Sartori, E. (2021). Sinbad – radiation shielding benchmark experiments. Ann. Nucl. Energy 159, 108254. doi:10.1016/j.anucene.2021.108254

CrossRef Full Text | Google Scholar

Michel-Sendis, F, Gauld, I., Martinez, J., Alejano, C., Bossant, M., Boulanger, D., et al. (2017). SFCOMPO-2.0: An OECD NEA database of spent nuclear fuel isotopic assays, reactor design specifications, and operating data. Ann. Nucl. Energy, 110, 779–788. doi:10.1016/j.anucene.2017.07.022

CrossRef Full Text | Google Scholar

NEA (2017). A state-of-the-art report on scaling in system thermal-hydraulics applications to nuclear reactor safety and design. NEA/CSNI/R(2016)14. Paris, France: OECD Nuclear Energy Agency. Available from: https://www.oecd-nea.org/jcms/pl_19744/scaling-in-system-thermal-hydraulics-applications-to-nuclear-reactor-safety-and-design-a-state-of-the-art-report.

Google Scholar

NEA (2015). Assessment of CFD codes for nuclear reactor safety problems – revision 2. NEA/CSNI/R(2014)12. Paris, France: OECD Nuclear Energy Agency. Available from: https://www.oecd-nea.org/jcms/pl_19550/assessment-of-cfd-codes-for-nuclear-reactor-safety-problems-revision-2.

Google Scholar

NEA (1996). CSNI integral test facility validation matrix for the assessment of thermal-hydraulic codes for LWR LOCA and transients. NEA/CSNI/R(1996)17. Paris, France: OECD Nuclear Energy Agency. Available from: https://www.oecd-nea.org/jcms/pl_16134/csni-integral-test-facility-validation-matrix-for-the-assessment-of-thermal-hydraulic-codes-for-lwr-loca-and-transients.

Google Scholar

NEA (2016). Evaluation guide for the evaluated spent nuclear fuel assay database (SFCOMPO). NEA/NSC/R(2015)8. France: OECD Nuclear Energy Agency. Available from: https://www.oecd-nea.org/jcms/pl_19678/evaluation-guide-for-the-evaluated-spent-nuclear-fuel-assay-database-sfcompo.

Google Scholar

NEA (1997). Evaluation of the separate effects tests (SET) validation matrix. NEA/CSNI/R(1996)16. Paris, France: OECD Nuclear Energy Agency. Available from: https://www.oecd-nea.org/jcms/pl_16132/evaluation-of-the-separate-effects-tests-set-validation-matrix-1997.

Google Scholar

NEA (2022a). International handbook of evaluated criticality safety benchmark experiments. Paris, France: OECD Nuclear Energy Agency. Available from: https://www.oecd-nea.org/jcms/pl_20291/icsbep-handbook.

Google Scholar

NEA (2020b). International handbook of evaluated reactor physics benchmark experiments. Paris, France: OECD Nuclear Energy Agency. Available from: https://www.oecd-nea.org/jcms/pl_20279/international-handbook-of-evaluated-reactor-physics-benchmark-experiments-irphe.

Google Scholar

NEA (1994). Separate effects test matrix for thermal-hydraulic code validation, vol. 1 and 2. NEA/CSNI/R(1993)4. Paris, France: OECD Nuclear Energy Agency. Available from: https://www.oecd-nea.org/jcms/pl_15968/separate-effects-test-matrix-for-thermal-hydraulic-code-validation-vol-1-and-2-1994.

Google Scholar

NEA (2022c). SINBAD project format/content guide. NEA/NSC/WPRS/DOC(2022)5. Paris, France: OECD Nuclear Energy Agency.

Google Scholar

NEA (2001). Validation matrix for the assessment of thermal-hydraulic codes for VVER LOCA and transients. NEA/CSNI/R(2001)4. Paris, France: OECD Nuclear Energy Agency. Available from: https://www.oecd-nea.org/jcms/pl_17492/validation-matrix-for-the-assessment-of-thermal-hydraulic-codes-for-vver-loca-and-transients.

Google Scholar

ONR (2019). Validation of computer Codes and calculation methods. NS-TAST-GD-042 rev. 5, office for nuclear regulation. Bootle, UK. Available from: www.onr.org.uk/operational/tech_asst_guides/ns-tast-gd-042.pdf.

Google Scholar

Palmiotti, G., Briggs, J. B., Kugo, T., Trumble, E. F., Kahler, A. C. S., and Lancaster, D. (2014). Applications of integral benchmark data. Nucl. Sci. Eng. 178 (3), 295–310. doi:10.13182/nse14-33

CrossRef Full Text | Google Scholar

Petruzzi, A., and D’Auria, F. (2016). Standardized consolidated calculated and reference experimental database (sccred): A supporting tool for V&V and uncertainty evaluation of best-estimate system codes for licensing applications. Nucl. Sci. Eng. 182 (1), 13–53. doi:10.13182/NSE15-79

CrossRef Full Text | Google Scholar

Rohatgi, U., Dyrda, J., and Soppera, N. (2018). “The international experimental thermal hydraulic systems database (TIETHYS): A new NEA validation tool,” in Proceedings of the 26th International Conference on Nuclear Engineering, July 22-26 (London, England. doi:10.1115/ICONE26-82631

CrossRef Full Text | Google Scholar

USNRC (2005). Regulatory guide 1.203: Transient and accident analysis methods. Washington, D.C: US Nuclear Regulatory Commission. Available from: https://www.nrc.gov/docs/ML0535/ML053500170.pdf.

Google Scholar

Valentine, T., Avramova, M., Fleming, M., Hursin, M., Ivanov, K., Petruzzi, A., et al. (2021). Overview of the OECD-NEA expert group on multi-physics experimental data, benchmarks and validation. EPJ Web Conf. 247, 06048. doi:10.1051/epjconf/202124706048

CrossRef Full Text | Google Scholar