Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Modeling software”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,243 records · Page 69

Systematic and objective evaluation of Earth system models: PCMDI Metrics Package (PMP) version 3

Systematic, routine, and comprehensive evaluation of Earth system models (ESMs) facilitates benchmarking improvement across model generations and identifying the strengths and weaknesses of different model configurations. By gauging the consistency between models and observations, this endeavor is becoming increasingly necessary to objectively synthesize the thousands of simulations contributed to the Coupled Model Intercomparison Project (CMIP) to date. The Program for Climate Model Diagnosis and Intercomparison (PCMDI) Metrics Package (PMP) is an open-source Python software package that provides quick-look objective comparisons of ESMs with one another and with observations. The comparisons include metrics of large- to global-scale climatologies, tropical inter-annual and intra-seasonal variability modes such as the El Niño–Southern Oscillation (ENSO) and Madden–Julian Oscillation (MJO), extratropical modes of variability, regional monsoons, cloud radiative feedbacks, and high-frequency characteristics of simulated precipitation, including its extremes. The PMP comparison results are produced using all model simulations contributed to CMIP6 and earlier CMIP phases. An important objective of the PMP is to document the performance of ESMs participating in the recent phases of CMIP, together with providing version-controlled information for all datasets, software packages, and analysis codes being used in the evaluation process. Among other purposes, this also enables modeling groups to assess performance changes during the ESM development cycle in the context of the error distribution of the multi-model ensemble. Quantitative model evaluation provided by the PMP can assist modelers in their development priorities. In this paper, we provide an overview of the PMP, including its latest capabilities, and discuss its future direction.

54 ENVIRONMENTAL SCIENCES↗

Evaluation of distributed process-based hydrologic model performance using only a priori information to define model inputs

Fully distributed, integrated surface–subsurface hydrological models (ISSHMs) have seen renewed interest due to availability of better software, high performance computing facilities, and high-resolution, spatially extensive data products. ISSHMs are valuable as tools for advancing system understanding as they can resolve multiple processes defined on the plot scale including three-dimensional interaction of surface water and groundwater. Here, we evaluated the performance of an ISSHM, the Advanced Terrestrial Simulator (ATS), on seven diverse catchments across the continental US using widely available data products to define model inputs without calibration. We compare the ATS-simulated streamflow and evapotranspiration with gauge observations and MODIS-derived evapotranspiration, respectively. Using the Kling-Gupta Efficiency (KGE) as metric, ATS with default data products performed reasonably well at 6 of 7 catchments for streamflow. However, in one of those 6 catchments ATS had poor performance on baseflow and ATS’s overall performance was thus judged to be inadequate despite the acceptable KGE. ATS performance for evapotranspiration was good in all 7 catchments using default data products. In the two catchments where ATS streamflow performance using default data products was not acceptable, the performance was significantly improved by using local information on subsurface properties below the soil. We also compare the model-simulated streamflow and evapotranspiration with the Sacramento soil moisture accounting (SAC-SMA) model, a semi-distributed model that was calibrated on a catchment-by-catchment basis. Uncalibrated ATS performance is comparable to the calibrated SAC-SMA model in terms of streamflow while ATS performance is similar to or better (much better in certain catchments) in reproducing MODIS-derived evapotranspiration. Reasonably good performance of ATS without catchment-specific calibration provides new confidence in the ISSHM class of models and community data products as tools for advancing understanding of watershed function in a changing environment.

54 ENVIRONMENTAL SCIENCES↗

Thermodynamic modeling of the Nb-Ni system with uncertainty quantification using PyCalphad and ESPEI

Here, the Nb–Ni system is remodeled with uncertainty quantification (UQ) using software tools of PyCalphad and ESPEI (the Extensible, Self-optimizing Phase Equilibria Infrastructure) with the presently implemented capability of modeling site fraction based on Wyckoff positions. The five- and three-sublattice models are used to model the topologically close pack (TCP) μ-Nb 7 Ni 6 and δ-NbNi 3 phases according to their Wyckoff positions. The inputs for CALPHAD-based thermodynamic modeling include the thermochemical data as a function of temperature predicted by first-principles and phonon calculations based on density functional theory (DFT), ab initio molecular dynamics (AIMD) simulations, together with phase equilibrium and site fraction data in the literature. In addition to phase diagram and thermodynamic properties, the CALPHAD-based predictions of site fractions of Nb in μ-Nb 7 Ni 6 agree well with experimental data. Furthermore, the UQ estimation using the Markov Chain Monte Carlo (MCMC) method as implemented in ESPEI is applied to study the uncertainty of site fraction in μ-Nb 7 Ni 6 and enthalpy of mixing (ΔH mix ) in liquid.

36 MATERIALS SCIENCE↗

EI_MS_ML

The unambiguous identification of compounds from their electron ionization mass (EI-MS) spectra remains a significant unsolved problem in the field of metabolomics and analytical chemistry as a whole. Typically EI-MS spectra are compared using various mathematical operations that convert the spectral similarity or differences into a distance-like metric that roughly approximates the similarity of any two spectra. A commonly used metric for this is the cosine similarity metric which has values close to one for very similar spectra and a value of zero for very dissimilar spectra; however, no metric is perfect. Due to the prevalence of structurally-similar compounds such as isomers and the prevalence of certain fragmentation patterns across structurally-dissimilar compounds, the unambiguous assignment of EI-MS spectra compounds remains difficult. Frequently, querying an observed EI-MS spectrum against a large database such as the NIST17 library yields multiple possible assignments requiring the end user to distinguish between multiple high scoring hits, or multiple low scoring hits while keeping in mind that the correct hit may not be in the database at all. Although techniques such as orthogonal information from techniques such as chromatography can greatly aid in unambiguous assignment, this also requires more complicated experimental designs and access to more complicated analytical instrumentation. Substructures can be trivially detected and represented as strings using a previously published technique called node coloring from a known chemical structure. However, for experimentally-derived EI-MS spectra this information must be derived from the spectra itself (i.e., because we do not know what compound it represents). To achieve this, the software uses techniques from the field of machine learning and a large training dataset of EI-MS spectra corresponding to known structures annotated with substructure strings, to build models that can predict the presence of a given chemical substructure from an EI-MS spectrum directly.If these predictions are of high-quality (i.e., are unlikely to be false positives), the presence of one or more predicted substructures can be used to constrain the number of possible hits for a query spectrum. Mathematically, this restriction could be expressed in many forms, but the most straight-forward implementation is to weight the cosine similarity of a query spectrum and a plausible database match with a Tanimoto-like coefficient based on the ratio of the number of substructures predicted to the number of substructures present in the potential database hit. Determining which combination of models best reduces assignment ambiguity will be achieved using a combination of manual curation and optimization techniques such as genetic algorithms. This software will perform all the steps necessary to construct said models from a training dataset and evaluate them using a holdout dataset. Various statistical analyses can be performed to determine if this approach does decrease assignment ambiguity. For example, if this approach works, on average, the rank-order of the correct assignment for the holdout set of EI-MS spectra should decrease and the weighted cosine similarities for most of the possible matches in the database should be better than the unweighted cosine similarities. Furthermore, this same pipeline can be used on real experimental data to generate less ambiguous assignments.

Mitchell, Joshua↗

Using the ATLAS experiment software on heterogeneous resources

With the large dataset expected from 2030 onwards by the HL-LHC at CERN, the ATLAS experiment is reaching the limits of the current data processing model in terms of traditional CPU resources based on x86_64 architectures and an extensive program for software upgrades towards the HL-LHC has been set up. The ARM CPU architecture is becoming a competitive and energy efficient alternative. Accelerators like GPUs are available in any recent HPC. In the past years ATLAS has successfully ported its full data processing and simulation software framework Athena to ARM and has invested significant effort in porting parts of the reconstruction and simulation algorithms to GPUs. We report on the successful usage of the ATLAS experiment offline and online software framework Athena on ARM and GPUs through the PanDA workflow management system at various WLCG sites. Furthermore we report on performance optimizations of the builds for ARM CPUs and the GPU integration efforts. We will discuss performance comparisons of different ARM and x86_64 architectures on WLCG resources and Cloud compute providers like GCP and AWS using ATLAS productions workflows as used in the Hep-Score23 benchmark suite.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Using PyBioNetFit to leverage qualitative and quantitative data in biological model parameterization and uncertainty quantification

Data generated in studies of cellular regulatory systems are often qualitative. For example, measurements of signaling readouts in the presence and absence of mutations may reveal a rank ordering of responses across conditions but not the precise extents of mutation-induced differences. Qualitative data are often ignored by mathematical modelers or are considered in an ad hoc manner, as in the study of Kocieniewski and Lipniacki (2013) [Phys Biol 10: 035006], which was focused on the roles of MEK isoforms in ERK activation. In this earlier study, model parameter values were tuned manually to obtain consistency with a combination of qualitative and quantitative data. This approach is not reproducible, nor does it provide insights into parametric or prediction uncertainties. Here, starting from the same data and the same ordinary differential equation (ODE) model structure, we generate formalized statements of qualitative observations, making these observations more reusable, and we improve the model parameterization procedure by applying a systematic and automated approach enabled by the software package PyBioNetFit. We also demonstrate uncertainty quantification (UQ), which was absent in the original study. Our results show that PyBioNetFit enables qualitative data to be leveraged, together with quantitative data, in parameterization of systems biology models and facilitates UQ. These capabilities are important for reliable estimation of model parameters and model analyses in studies of cellular regulatory systems and reproducibility.

59 BASIC BIOLOGICAL SCIENCES↗

Overview of Advanced Reactor Simulation Capabilities to Support Demonstrations

The Virtual Test Bed (VTB) was established to support the deployment of advanced reactors by facilitating the use of state-of-the-art modeling and simulation capabilities. These tools are able to solve previously prohibitive challenge problems in greater levels of fidelity and in a multi-physics framework. A VTB repository was recently set up to host models for a wide variety of reactor types and using a range of NEAMS-developed software. Table I summarizes the different capabilities already available in the repository and up-and-coming near/long term ones. The repository currently hosts models for a wide variety of reactor types, including High-Temperature Gas Cooled Reactors (HTGR), Sodium Fast Reactors (SFR), Fluoride salt cooled High-temperature Reactors (FHR), Molten Salt Reactors (MSR), and Heat-Pipe Microreactors (HP-MR). It should be noted that work is ongoing for all of the models and each example problem is at a different state of maturity.

99 GENERAL AND MISCELLANEOUS↗

Simulation of Isotopic Concentrations and Gamma Spectra from Dynamic Fission Sources

A tool was developed to rapidly generate synthetic gamma-ray spectra to evaluate safeguards material control and accounting methods for liquid-fueled molten salt reactors. Molten salt reactor operations pose unique challenges to nuclear safeguards methods and protocols compared to deployed reactors designs (e.g., light water reactors). This research evaluates the use of gamma-ray spectroscopy to monitor fission product isotopic flow through a reactor model to understand expected operations and investigate changes to the spectra with material diversion scenarios. The large design space of molten salt reactors (e.g., liquid-fueled, liquid-cooled, online separations) could potentially lead to many measurement points within the reactor system. The developed analytical tool generates and evaluates synthetic gamma-ray spectra from dynamic reactor simulations by extracting isotopic inventory to generate source terms. An implementation in the Gamma Detector Response and Analysis Software (GADRAS) Application Program Interface (API) uses the source terms, a model of the reactor component, a detector response function, and measurement plan to quickly generate and analyze spectra. Prospective measurements are then evaluated in the more accurate but slower Geant4 simulations. Utilization of the developed modeling tool and analysis of the subsequent spectra enables optimization of collimation, shielding requirements, and expected count rates that are used to determine key measurement points in the modeled reactor design.

O'Brien, Sean↗

Scalable Data-Intensive Geocomputation: A Design for Real-Time Continental Flood Inundation Mapping

The convergence of data-intensive and extreme-scale computing enables an integrated software and data ecosystem for scientific discovery. Developments in this realm will fuel transformative research in data-driven interdisciplinary domains. Geocomputation provides computing paradigms in Geographic Information Systems (GIS) for interactive computing of geographic data, processes, models, and maps. Because GIS is data-driven, the computational scalability of a geocomputation workflow is directly related to the scale of the GIS data layers, their resolution and extent, as well as the velocity of the geo-located data streams to be processed. Unique in high user interactivity and low end-to-end latency requirements, geocomputation applications will dramatically benefit from the convergence of high-end data analytics (HDA) and high-performance computing (HPC). The application level challenge, however, is to identify and eliminate computational bottlenecks that arise along a geocomputation workflow. Indeed, poor scalability at any of the workflow components is detrimental to the entire end-to-end pipeline. Here, we study a large geocomputation use case in flood inundation mapping that handles multiple national-scale geospatial datasets and targets low end-to-end latency. We discuss benefits and challenges for harnessing both HDA and HPC for data-intensive geospatial data processing and intensive numerical modeling of geographic processes. We propose an HDA+HPC geocomputation architecture design that couples HDA (e.g., Spark)-based spatial data handling and HPC-based parallel data modeling. Key techniques for coupling HDA and HPC to bridge the two different software stacks are reviewed and discussed.

Liu, Yan↗

Using a Large Language Model as a Building Block to Generate Usable Validation and Verification Suite for OpenMP

In the HPC area, both hardware and software move quickly. Often new hardware is developed and deployed, the corresponding software stack, including compilers and other tools, are under active development while leading edge software developers are working to port and tune their applications, all at the same time. While the software ecosystem is in flux, one of the key challenges for users is obtaining insight into the state of implementation of key features in the programming languages and models their applications are using – whether they have been implemented, and whether the implementation conforms to the specification, especially for newly implemented features (less tested by widespread use). OpenMP is one of the most prominent shared memory programming models used for on-node programming in HPC. With the shift towards accelerators (such as GPUs and FPGAs) and heterogeneous programming OpenMP features are getting more complex. It is natural to ask whether generative AI approaches, and large language models (LLMs) in particular, can help in producing validation and verification test suites to allow users better and faster insights into the availability and correctness of OpenMP features of interest. In this work, we explore the use of ChatGPT-4 to generate a suite of tests for OpenMP features. We have chosen a set of directives and clauses, a total of 78 combinations, which first appeared in OpenMP 3.0 (released in May 2008) but are also relevant for accelerators. We prompted ChatGPT to generate tests in the C and Fortran languages, for both host (CPU) and device (accelerator). On the Summit super-computer using the GNU implementation, we found that, of the 78 generated tests 67 C tests and 43 Fortran tests compiled successfully and fewer than those executed to completion. On further analysis we show that not all generated tests are valid. We document the process, results, and provide detailed analysis regarding the quality of tests generated. With the aim of providing input to a production quality validation and verification suite, we manually implement the corrections required to make the tests valid according to the current OpenMP specification. We quantify this effort as small, medium, or large, and record the lines of code changed to correct the invalid tests. With the corrected tests we validate recent implementations from HPE, AMD, and GNU on the Frontier supercomputer. Our experiment and subsequent analysis show that although LLMs are capable of producing HPC specific codes, they are limited by their understanding of the deeper semantics and restrictions of programming models such as OpenMP. Unsurprisingly more commonly used features have better support, while some OpenMP 3.0 directives such as sections and tasking are not universally supported on accelerators. We demonstrate that successful compilation and execution to completion are inadequate metrics for evaluating generated code and that, at this time, commodity LLMs require expert intervention for code verification. This points to gaps in the training data that is currently available for HPC. We demonstrate that with "small" effort 37% of generated invalid C tests and 63% of generated invalid Fortran tests could be corrected. This improves productivity of test generation as we circumvent writing from scratch and the common programming errors associated with it.

Pophale, Swaroop [ORNL] (ORCID:0000000185446367)↗

Wind Energy Forecasting with the Weather Research and Forecasting Model

This was a collaborative effort between Lawrence Livermore National Security, LLC as manager and operator of Lawrence Livermore National Laboratory (LLNL) and Siemens Energy, Inc. (Siemens) to develop a wind resource forecasting tool. LLNL was to develop an independent high-resolution mesoscale modeling capability forecasting tool that could be implemented in conjunction with existing wind farm control and monitoring software to provide forecasting of wind resources using local observations of winds and temperature. Research with LLNL’s state-of-the-art large-eddy simulation meteorological prediction model, based on the community WRF model and innovative turbulence parameterizations, would improve that model’s applicability to large wind farms offshore and in complex terrain. The modeling capability would include uncertainty quantification. Finally, the application of the modeling tool and existing global climate change predictions would enable the delineation of the likely effects of climate change on wind resources. Siemens was to provide high time resolution hub-height wind speed and other meteorological data streams, including temperature profiles from wind farms, for LLNL to incorporate into the modeling system, to validate and tune this forecasting model for their locations of interest. These data streams would also be used for longer-term studies of correlations of wind resources to climate oscillations to indicate how long-term climate change trends may affect the available wind resource. Siemens would also provide information and observations of turbine wakes for incorporation into the modeling tool. By implementing state-of-the-art turbulence parameterizations into a simulation model and/or ensembles of simulation models, and by integrating real-time hub height wind speed and other meteorological datastreams from wind farms into that model or ensemble of models, LLNL would develop a forecasting tool that could be implemented by Siemens as an add-on to existing wind farm control and monitoring software to provide owners with useful resource forecasting. The desired outcome was that the accuracy level of the output would be sufficient to substantiate power output commitments. The final deliverable for this work would consist of a document outlining the algorithms and software tools that could be integrated into Siemens Wind Park Supervisor.

17 WIND ENERGY↗

Elastic Bayesian Model Calibration

Functional data are ubiquitous in scientific modeling. For instance, quantities of interest are modeled as functions of time, space, energy, density, etc. Uncertainty quantification methods for computer models with functional response have resulted in tools for emulation, sensitivity analysis, and calibration that are widely used. However, many of these tools do not perform well when the computer model’s parameters control both the amplitude variation of the functional output and its alignment (or phase variation). This paper introduces a framework for Bayesian model calibration when the model responses are misaligned functional data. The approach generates two types of data out of the misaligned functional responses: (1) aligned functions so that the amplitude variation is isolated and (2) warping functions that isolate the phase variation. These two types of data are created for the computer simulation data (both of which may be emulated) and the experimental data. The calibration approach uses both types so that it seeks to match both the amplitude and phase of the experimental data. The framework is careful to respect constraints that arise, especially when modeling phase variation, and is framed in a way that it can be done with readily available calibration software. In conclusion, we demonstrate the techniques on two simulated data examples and on two dynamic material science problems: a strength model calibration using flyer plate experiments and an equation of state model calibration using experiments performed on the Sandia National Laboratories’ Z-machine.

97 MATHEMATICS AND COMPUTING↗

CFD-trained ANN Model for Approximating Near-occupant Condition in Real-time Simulations

The main drawback of Computational Fluid Dynamics (CFD) simulations has been the time and resource consuming nature which is not suitable for real-time applications. In this work, we first generated numerous CFD models of a given indoor space to obtain airspeed, temperature, and mean radiant temperature near an occupant as training data. Several artificial neural networks (ANN) models were trained using this CFD simulated data to approximate near real-time environmental conditions for a given occupant. This trained ANN model approach is a part of a real-time simulation of building operations using a combination of software and real hardware (HVAC equipment) approaches. The preliminary results suggest that the CFD- generated training data and the trained ANN model can accurately approximate such conditions in a real-time application, a method that has great potential in building simulation and building digital twin areas of research.

Zhang, Yun↗

Hybrid-RL-MPC4CLR (Hybird-Reinforcement-Learning-Model-Predictive-Control-for-Reserve-Policy-Assisted-Critical-Load-Restoration-in-Distribution-Grids)

Hybrid-RL-MPC4CLR was developed as a hybrid controller for active distribution grid critical load restoration, combining deep reinforcement learning (RL) and model predictive control (MPC) aiming at maximizing total restored load following an extreme event. The RL determines a policy for quantifying operating reserve requirements, thereby hedging against uncertainty, while the MPC models grid operations incorporating the RL policy actions (i.e., reserve requirements), renewable (wind and solar) power predictions, and load demand forecasts. The developers formulated the reserve requirement determination problem as a sequential decision-making problem based on the Markov Decision Process (MDP) and design an RL learning environment based on the OpenAI Gym framework and MPC simulation. The RL agent reward and MPC objective function aim to maximize and monotonically increase total restored load and minimize load shedding and renewable power curtailment. The software is developed using various software packages in Python. The MPC's optimal power flow (OPF) model is implemented using the Pyomo package, the RL simulation environment is implemented using the MPC simulation with various scenarios of renewable energy and load demand profiles and power outage beginning times, based on the OpenAI Gym framework. The RL agent training is performed using the RLlib Ray package. The RL algorithm is trained offline using historical forecasts of renewable generation and load demand profiles. Simulation analysis and performance tests are conducted using a modified IEEE 13-bus distribution test feeder containing wind turbine, photovoltaic, microturbine, and battery.

Eseye, Abinet Tesfaye↗

Improving Frequency Stability and Minimizing Load Shedding Events by Adopting Grid-Scale Energy Storage with Grid Forming Inverters

The upward adoption trend of renewable generation not only means cleaner energy integrated into modern power grids, but also that most new generation sources are based on front-end inverter bridges, used as interfaces to most wind generation and all the solar PV. It is well known that due to their power electronics-based construction rather than rotational shafts, these sources do not provide inertia inherently, nor substantial amounts of short-circuit currents. However, stable energy such as what can be stored in energy storage systems, although interfaced via inverters, can be controlled to respond to system disturbances in a manner that emulates inertial behavior. This paper focuses on the application of such energy storage systems to augment inertia in the island of Puerto Rico. To do so, a user defined inverter model that contains grid forming capabilities and fast frequency response is modeled and integrated into the real transmission system in power flow and dynamics software. Energy storage is then connected to two selected areas so that it not only provides frequency regulation to avoid widespread load shedding events, but also other tangible benefits. The simulated cases suggest that even relatively small energy storage systems can avert load shedding events if adequately placed in the transmission network.

Grid-forming inverters, IBR, Inertia↗

Analysis of ZPPR-15 experimental data for VTR software validation

As part of the Versatile Test Reactor (VTR) program, extensive validation work is being performed to demonstrate the fidelity of the Argonne Reactor Code (ARC) software package presently used for the core design studies. It is anticipated that this software will be also used for initial operations although additional software may be introduced at a later time to supplement or replace some components. The validation work was focused on accumulating reactor models that are consistent with the VTR and thus can be used to validate the accuracy of predictions made by the software used to design and operate the VTR. The U.S. does not have any active experimental facilities that can neutronically mockup the VTR core design. However, in the past, the U.S. had extensive experimental facilities for fast reactor neutronics. In that regard, the ZPPR-15 experiments carried out at ANL during the 1980s work best because the fuel form is closest to the VTR design, there is a considerable amount of processed data available for use, and existing staff are already familiar with the experimental machine and measurements. Because of previous collaborative work, many of the ZPPR-15 loadings of interest have already been processed into MCNP and preliminary ARC models, and a full uncertainty quantification has already been completed for many of the ZPPR-15 measurements. This manuscript covers the recent analysis work on the ZPPR-15 experiments and compares it with the experimental results. With respect to the ARC software components, this manuscript focuses on validation of the MC 2 , DIF3D and GAMSOR using the ZPPR-15 experimental data.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

UPDATES FROM THE INVOLUTE WORKING GROUP

The HFIR, RHF, and FRM II reactors represent a particular class of Research and Test Reactors that provide some of the most intense and continuous neutron fluxes for science, industry, and medical applications. These high-performance reactors have achieved compact cores by operating with Highly Enriched Uranium fuel (HEU, 235U/U ≥ 20 wt. %) and utilizing fuel plates curved as an involute. Due to the proliferation risks, the international community aims to reduce or eliminate, when possible, the use of HEU fuel in civilian facilities by converting them to a Low-Enriched Uranium fuel (LEU, 235U/U < 20 wt. %). Conversion of these reactors without significantly compromising their performance or safety is a challenging endeavor that can tremendously benefit from advanced computational tools and thus, eliminate unnecessary conservatism to ensure sufficient thermal margins. Therefore, models are being developed using modern Computational Fluid Dynamics (CFD) and Computational Structural Mechanics (CSM) software to evaluate the steady-state safety margins of various LEU designs instead of being reliant on the more traditional, conservative methods. To gain the confidence and acceptance of high-fidelity modeling by the nuclear regulators, Argonne National Laboratory (ANL) and the involute reactors have formed an informal scientific group, the Involute Working Group (IWG). The IWG facilitates inter-organizational collaboration on experimental benchmarking, code-to-code comparisons, and Verification and Validation (V&V). This paper describes some of the recent IWG efforts in validating software against the existing experimental data, as well as code-to-code comparisons of different software used by the IWG members.

Bergeron, Aurelien↗