Detection of Random Test Failures and their Impact on CI Testing Pipelines
Explore the source record for details and available documents.
Bartlett, Roscoe [Sandia National Laboratories (SN↗
SEARCH · Engineering Papers
Results for “CI testing”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Software Quality Assurance for the MOOSE-Based Open-Source Multiphysics Code Cardinal - An Expanded CI Testing Suite
Cardinal is a wrapping of the GPU-oriented spectral element Computational Fluid Dynamics (CFD) code NekRS and the Monte Carlo particle transport code OpenMC within the Multiphysics Object-Oriented Simulation Environment (MOOSE). Cardinal provides high-resolution thermal-hydraulics and/or radiation transport feedback to MOOSE multiphysics simulations. Multiphysics feedback is implemented in a geometry-agnostic manner which eliminates the need for rigid one-to-one mappings. A generic data transfer implementation also allows NekRS and OpenMC to couple to any MOOSE application, enabling a broad set of multiphysics capabilities. Cardinal simulations can also leverage combinations of MPI, OpenMP, and GPU resources. Cardinal continuous development and improvement efforts have led to the software being considered as a high-fidelity design and licensing tool for key areas of nuclear reactor relevant physics, including neutron transport, fluid flow, heat transfer, and mechanical processes. The fast development and expansion of the software from a pure R&D framework towards its application in the nuclear industry and regulation require a focus on developing, enhancing and, maintaining Cardinal’s software quality through strict adherence to a Software Quality Assurance (SQA) framework and SQA program. To facilitate compliance with SQA standards, the Cardinal SQA Program has been initiated during Fiscal Year 2023 (FY23). During the development of the Cardinal SQA Program, multiple gaps have been identified. These gaps are primarily related to model verification and code pedigree as they relate to the use of Cardinal as a safety analysis tool. These gaps have been captured in a report published in 2023. A second report highlighted the progress made during Fiscal Year 2024 (FY24) and described Argonne’s effort to document and integrate software verification within Cardinal’s software development process. This report documents a snapshot of the verification test cases currently available for Cardinal and NekRS in their assimilation into a Continuous Integration (CI) platform. Following the CI practice permits the integrating of source code changes frequently and ensuring that the integrated codebase clears the verification testing for the software. It should be noted that the SQA program itself, including the program plans, procedures, configuration management, and testing strategies, need to be developed in a future step of this task.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
Cholla-MHD: An Exascale-capable Magnetohydrodynamic Extension to the Cholla Astrophysical Simulation Code
Abstract We present an extension of the massively parallel, GPU native, astrophysical hydrodynamics code Cholla to magnetohydrodynamics (MHD). Cholla solves the ideal MHD equations in their Eulerian form on a static Cartesian mesh utilizing the Van Leer + constrained transport integrator, the HLLD Riemann solver, and reconstruction methods at second and third order. Cholla’s MHD module can perform ≈260 million cell updates per GPU-second on an NVIDIA A100 while using the HLLD Riemann solver and second order reconstruction. The inherently parallel nature of GPUs combined with increased memory in new hardware allows Cholla’s MHD module to perform simulations with resolutions ∼500 3 cells on a single high-end GPU (e.g., an NVIDIA A100 with 80 GB of memory). We employ GPU direct Message Passing Interface to attain excellent weak scaling on the exascale supercomputer Frontier, while using 74,088 GPUs and simulating a total grid size of over 7.2 trillion cells. A suite of test problems highlights the accuracy of Cholla’s MHD module and demonstrates that zero magnetic divergence in solutions is maintained to round off error. We also present new testing and CI tools using GoogleTest, GitHub Actions, and Jenkins that have made development more robust and accurate and ensure reliability in the future.
Astronomy & Astrophysics↗
SHOCK TUBE IGNITION STUDIES OF RENEWABLE DIESEL FUELS FOR MEDIUM AND HEAVY-DUTY TRANSPORTATION
Currently extensive research on alternative fuels is being conducted due to their increasing demand to reduce greenhouse emissions. One renewable fuel studied in this work is dimethyl ether (DME) blended with propane(C3H8) as a potential mixture for heavy-duty engines used in semi-trucks. The blend has the potential to drastically reduce particulate and greenhouse gas emissions compared to a conventional diesel engine operating under similar conditions. To develop the use of mixture, one must conduct detailed conceptual and simulation studies before progressing to detail studies in CFD, engine modifications, and live testing. For simulations, accurate high-fidelity chemical kinetic models are necessary. However, the validity of the chemical kinetic mechanism for operating conditions of a heavy-duty mixing-controlled compression (MCCI) engine was widely unknown until recent work presented here and published. In this work, we studied the ignition of DME and propane blends in a shock tube under MCCI engine conditions. Ignition delay time (IDT) gathered behind the reflected shock for DME-propane mixtures for heavy-duty compression ignition (CI) engine parameters. Testing was conducted for undiluted varieties spanning from temperatures of 700 to 1100 K at pressures ranging from 55 to 84 bar for various blends (100% CH3OCH3, 100% C3H8, 60% CH3OCH3/ 40% C3H8) of DME and propane were combusted in synthetic air (21% O2/ 79% N2). Several experiments were conducted at higher pressures (90-120 bar) to improve the model performance and accuracy. The ignition delay times (IDTs) were compared to recent mechanisms, including Aramco3.0, NUIG, and Dames et al. A common trend among the mechanisms was overpredicted experimental IDTs. Further studies were conducted by a sensitivity analysis using the Dames et al. model, and critical reactions sensitive to IDTs of DME-propane mixture near 60 bar are outlined. Chemical analysis was conducted on the NTC region to explain chemical kinetics which is critical for developing MCCI heavy duty engines.
Mohammed, Zuhayr Pasha↗
Progress Towards NQA-1 for Cardinal in FY25
Cardinal is a wrapping of the GPU-oriented spectral element Computational Fluid Dynamics (CFD) code NekRS and the Monte Carlo particle transport code OpenMC within the Multiphysics Object-Oriented Simulation Environment (MOOSE). Cardinal provides high-resolution thermal-hydraulics and/or radiation transport feedback to MOOSE multiphysics simulations. Multiphysics feedback is implemented in a geometry-agnostic manner which eliminates the need for rigid one-to-one mappings. A generic data transfer implementation also allows NekRS and OpenMC to couple to any MOOSE application, enabling a broad set of multiphysics capabilities. Cardinal simulations can also leverage combinations of MPI, OpenMP, and GPU resources. Cardinal continuous development and improvement efforts have led to the software being considered as a high-fidelity design and licensing tool for key areas of nuclear reactor relevant physics, including neutron transport, fluid flow, heat transfer, and mechanical processes. The fast development and expansion of the software from a pure R&D framework towards its application in the nuclear industry and regulation require a focus on developing, enhancing,and maintaining Cardinal’s software quality through strict adherence to a Software Quality Assurance (SQA) framework and SQA program. To facilitate compliance with SQA standards, the Cardinal SQA Program was initiated during Fiscal Year 2023 (FY23). During the development of the Cardinal SQA Program, multiple gaps have been identified. These gaps are primarily related to model verification and code pedigree as they relate to the use of Cardinal as an analysis tool. These gaps were captured in a report published in 2023. A second report highlighted the progress made during Fiscal Year 2024 (FY24) and described Argonne’s effort to document and integrate software verification within Cardinal’s software development process. This report documents the progress made towards NQA-1 for Cardinal in the Fiscal Year 2025 (FY25). All cases in the expanded Continuous Integration (CI) suite of NekRS are included in this report which test the solvers and modules available in NekRS exhaustively. The NekRS tests are integrated with the Cardinal CI suite and made available in publicly accessible Github documentation. Following the CI practice permits integrating of source code changes frequently and ensuring that the integrated codebase clears the verification testing for the software. Also in this report is a brief overview of the development of the Cardinal Software Quality Assurance Plan (SQAP) that was done in FY25, though it should be noted that the rest of the documentation for the SQA program needs to be developed in a future step of this task.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
Thoroughly testing and integrating hundreds of Pull Requests per month: ROOT’s new Cost-efficient and Feature Rich GitHub-based CI
ROOT is an open source framework, freely available on GitHub, at the heart of data acquisition, processing and analysis of HE(N)P experiments, and beyond. It is developed collaboratively: contributions are not authored only by ROOT team members, but also by the user community at large: developers and scientists from universities, labs as well as the private sector. More than 1500 GitHub Pull Requests are merged on average per year. It is in this context that code integration acquires a primary role. The review of code contributions isn’t enough: not only they need to be thoroughly reviewed, they also need to be thoroughly tested through a powerful CI infrastructure on several different platforms to comply with the high code quality standards of the project. Since the end of 2023, ROOT moved its continuous integration system from Jenkins to GitHub Actions. In this contribution, we characterise the transition to the GitHub CI, focussing on our strategy, its implementation and the lessons learned, as well as the advantages the new system offers with respect to the previous one. Particular emphasis will be given to the evaluation of the cost-benefit ratio for Jenkins and GitHub Actions for the ROOT project. We also describe how we manage to run in less than one hour thousands of unit, integration, functional and end-to-end tests on different flavours of Windows, four versions of macOS, as well as about ten of the most used Linux distributions, taking advantage of the CERN computing infrastructure.
Piparo, Danilo [CERN]↗
dCache CI/CD migration to Kubernetes
For over two decades, the dCache project has provided open-source to satisfy ever-more demanding storage requirements. More than 80 sites worldwide rely on dCache to provide services for LHC experiments, Belle-II, Eu- XFEL, and others. This can be achieved only with a well-established process from a whiteboard, where ideas are created through development, packaging, and testing. The project’s build and test infrastructure is based on Jenkins CI and a set of virtual machines. This infrastructure is maintained by dCache developers. With the introduction of the DESY-central Gitlab server, the developers have started migrating from VM-based testing to container-based deployments in the onsite Kubernetes cluster. As a result, we have packaged dCache containers and Helm charts that can be used by other sites to reproduce our test and build steps quickly or to evaluate new releases on their pre-production systems and, eventually, become a standard model of dCache deployment at the sites. This paper describes the challenges we have faced, the techniques we used to solve them, and the issues that still need to be addressed.
Mkrtchyan, Tigran [DESY]↗
Evaluating Possible Formation Mechanisms of Criegee Intermediates during the Heterogeneous Autoxidation of Squalene
Organic molecules in the environment oxidatively degrade by a variety of free radical, microbial, and biogeochemical pathways. A significant pathway is heterogeneous autoxidation, in which degradation occurs via a network of carbon and oxygen centered free radicals. Recently, we found evidence for a new heterogeneous autoxidation mechanism of squalene that is initiated by hydroxyl (OH) radical addition to a carbon–carbon double bond and apparently propagated through pathways involving Criegee Intermediates (CI) produced from β-hydroxy peroxy radicals (β-OH-RO 2 •). It remains unclear, however, exactly how CI are formed from β-OH-RO 2 •, which could occur by a unimolecular or bimolecular pathway. Combining kinetic models and multiphase OH oxidation measurements of squalene, we evaluate the kinetic viability of three mechanistic scenarios. Scenario 1 assumes that CI are formed by the unimolecular bond scission of β-OH-RO 2 •, whereas Scenarios 2 and 3 test bimolecular pathways of β-OH-RO 2 • to yield CI. Scenario 1 best replicates the entire experimental data set, which includes effective uptake coefficients vs [OH] as well as the formation kinetics of the major products (i.e., aldehydes and secondary ozonides). Finally, although the unimolecular pathway appears to be kinetically viable, future high-level theory is needed to fully explain the mechanistic relationship between CI and β-OH-RO 2 • in the condensed phase.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Combining Machine Learning and Comparative Effectiveness Methodology to Study Primary Care Pharmacotherapy Pathways for Veterans With Depression
Our objective is to demonstrate an innovative method combining machine learning with comparative effectiveness research techniques and to investigate a hitherto unstudied question about the effectiveness of common prescribing patterns. For Operation Enduring Freedom/Operation Iraqi Freedom veterans with major depressive disorder, we generate pharmacotherapy pathways (of antidepressants) using process mining and machine learning. We select the medication episodes that were started at subtherapeutic doses by the first assigned primary care physician and observe the paths that those medication episodes follow. Using 2-stage least squares, we test the effectiveness of starting at a low dose and staying low for longer versus ramping up fast while balancing observable and unobservable characteristics of patients and providers through instrumental variables. We leverage predetermined provider practice patterns as instruments. We collected outpatient pharmacy data for selective serotonin reuptake inhibitors and selective norepinephrine reuptake inhibitors, patient and provider characteristics (as control variables), and the instruments for our cohort. All data were extracted for the period between 2006 and 2020. There is a statistically significant positive effect (0.68, 95% CI 0.11–1.25) of “ramping up fast” on engagement in care. When we examine the effect of “ramping up slow”, we see an insignificant negative impact on engagement in care (−0.82, 95% CI −1.89 to 0.25). As expected, the probability of drop-out also seems to have a negative effect on engagement in care (−0.39, 95% CI −0.94 to 0.17). We further validate these results by testing with medication possession ratios calculated periodically as an alternative engagement in care metric. Our findings contradict the “Start low, go slow” adage, indicating that ramping up the dose of an antidepressant faster has a significantly positive effect on engagement in care for our population.
60 APPLIED LIFE SCIENCES↗
Software stewardship and advancement of a high-performance computing scientific application: QMCPACK
Here, we provide an overview of the software engineering efforts and their impact in QMCPACK, a production-level ab-initio Quantum Monte Carlo open-source code targeting high-performance computing (HPC) systems. Aspects included are: (i) strategic expansion of continuous integration (CI) targeting CPUs, using GitHub Actions own runners, and NVIDIA and AMD GPUs used in pre-exascale systems, (ii) incremental reduction of memory leaks using sanitizers, (iii) incorporation of Docker containers for CI and reproducibility, and (iv) refactoring efforts to improve maintainability, testing coverage, and memory lifetime management. We quantify the value of these improvements by providing metrics to illustrate the shift towards a predictive, rather than reactive, maintenance approach. Our goal, in documenting the impact of these efforts on QMCPACK, is to contribute to the body of knowledge on the importance of research software engineering (RSE) for the stewardship and advancement of community HPC codes to enable scientific discovery at scale.
97 MATHEMATICS AND COMPUTING↗
Understanding the Competition between Alcohol Formation and Dimerization during Electrochemical Reduction of Aromatic Carbonyl Compounds
The electrochemical reductive dimerization of small aromatic carbonyl compounds derived from lignocellulosic biomass is a crucial C−C coupling reaction for upgrading small molecules to longchain hydrocarbons, particularly in the synthesis of drop-in sustainable aviation fuels. Although other electrochemical reduction reactions of these reactants (i.e., hydrogenation and hydrogenolysis) have undergone extensive mechanistic investigation, the understanding of dimerization remains relatively underdeveloped. Most importantly, there is a lack of understanding of the selectivity-determining step between dimerization and monomer reduction and critical factors that can affect this step. In this study, we provide a comprehensive mechanistic model to explain the competition between dimerization and monomer reduction of benzaldehyde under various conditions. Our model proposes that the selectivity between dimerization and monomer reduction depends on the competition between desorption of a ketyl radical from the electrode, necessary for dimerization, and further reduction of the ketyl radical to an alcohol on the electrode by proton-coupled electron transfer (PCET). Computationally comparing the adsorption/desorption energy and PCET activation barrier energy is challenging because conventional DFT calculations substantially underestimate the PCET kinetic barriers. In this study, we employed constrained DFTbased configuration interaction (CDFT-CI) to obtain a reliable comparison of these energies. Our mechanistic model was tested and supported by experimental results obtained with four electrodes (Cu, Pb, Bi, graphite), three pH conditions (acidic, neutral, basic), and three potentials. Our study offers a coherent mechanistic foundation that can explain how each of these conditions impacts the desorption and PCET processes and the selectivities for dimerization and alcohol production.
09 BIOMASS FUELS↗
Reducing stomatal density by expression of a synthetic epidermal patterning factor increases leaf intrinsic water use efficiency and reduces plant water use in a C4 crop
Abstract Enhancing crop water use efficiency (WUE) is a key target trait for climatic resilience and expanding cultivation on marginal lands. Engineering lower stomatal density to reduce stomatal conductance (gs) has improved WUE in multiple C3 crop species. However, reducing gs in C3 species often reduces photosynthetic carbon gain. A different response is expected in C4 plants because they possess specialized anatomy and biochemistry which concentrates CO2 at the site of fixation. This modifies the relationship of photosynthesis (AN) with intracellular CO2 concentration (ci), such that photosynthesis is CO2 saturated and reductions in gs are unlikely to limit AN. To test this hypothesis, genetic strategies were investigated to reduce stomatal density in the C4 crop sorghum. Constitutive expression of a synthetic epidermal patterning factor (EPF) transgenic allele in sorghum led to reduced stomatal densities, reduced gs, reduced plant water use, and avoidance of stress during a period of water deprivation. In addition, moderate reduction in stomatal density did not increase stomatal limitation to AN. However, these positive outcomes were associated with negative pleiotropic effects on reproductive development and photosynthetic capacity. Avoiding pleiotropy by targeting expression of the transgene to specific tissues could provide a pathway to improved agronomic outcomes.
59 BASIC BIOLOGICAL SCIENCES↗
pnnl-predictive-phenomics/csc052-gem
Genome-Scale Metabolic Model Continuous Validation with Memote for CarbStor Community Member Bacillus These repositories contain the continuous validation environment for an organism-specific genome-scale metabolic model (GEM) using Memote. Memote is a software tool that provides a suite of tests to ensure the quality and consistency of metabolic models. By integrating Memote into a continuous integration (CI) workflow, we can automatically validate updates to the GEM, ensuring that model modifications improve or maintain the model's integrity.
Torres, Victor E.↗
pnnl-predictive-phenomics/csc031-gem
Genome-Scale Metabolic Model of CarbStor Community member Microbacterium (csc031) Continuous Validation with Memote These repositories contain the continuous validation environment for an organism-specific genome-scale metabolic model (GEM) using Memote. Memote is a software tool that provides a suite of tests to ensure the quality and consistency of metabolic models. By integrating Memote into a continuous integration (CI) workflow, we can automatically validate updates to the GEM, ensuring that model modifications improve or maintain the model's integrity
McNaughton, Andrew [@PNNL]↗
pnnl-predictive-phenomics/csc009-gem
Genome-Scale Metabolic Model of CarbStore Community member Curtobacterium (csc009) Continuous Validation with Memote These repositories contain the continuous validation environment for an organism-specific genome-scale metabolic model (GEM) using Memote. Memote is a software tool that provides a suite of tests to ensure the quality and consistency of metabolic models. By integrating Memote into a continuous integration (CI) workflow, we can automatically validate updates to the GEM, ensuring that model modifications improve or maintain the model's integrity
Lin, Tesia↗
pnnl-predictive-phenomics/csc040-gem
Genome-Scale Metabolic Model Continuous Validation with Memote for CarbStor Community Member Rhodococcus These repositories contain the continuous validation environment for an organism-specific genome-scale metabolic model (GEM) using Memote. Memote is a software tool that provides a suite of tests to ensure the quality and consistency of metabolic models. By integrating Memote into a continuous integration (CI) workflow, we can automatically validate updates to the GEM, ensuring that model modifications improve or maintain the model's integrity.
McNaughton, Andrew [@PNNL]↗
pnnl-predictive-phenomics/csc043-gem
Genome-Scale Metabolic Model Continuous Validation with Memote for CarbStor Community Member Paenibacillus These repositories contain the continuous validation environment for an organism-specific genome-scale metabolic model (GEM) using Memote. Memote is a software tool that provides a suite of tests to ensure the quality and consistency of metabolic models. By integrating Memote into a continuous integration (CI) workflow, we can automatically validate updates to the GEM, ensuring that model modifications improve or maintain the model's integrity.
Zucker, Jeremy [Pacific Northwest National Laborat↗
Machine-learning-based estimates of global natural vegetated wetland methane emissions (2000–2025)
Wetlands are the largest natural source of atmospheric methane (CH 4 ), yet comprehensive global budgets are typically delayed by years, preventing a timely understanding of CH 4 sources, sinks, and trends. To reduce this delay, we present a model emulator-driven framework and accompanying workflow that enable timely, continuous emission updates using a machine-learning emulator to reconstruct spatially explicit monthly emission fields at 1° × 1° resolution. We apply this framework to a global dataset of natural vegetated wetland CH 4 emissions to extend the most recent Global Methane Budget (GMB; Saunois et al., 2025) record that covers the 2000–2020 emissions through 2025. In the test data (∼ 30 % of the total dataset), the emulator achieved a global R 2 of 0.65 ± 0.003 (mean ± 95 % CI, hereafter) and an RMSE of 5.49 ± 0.12×10 -3 Tg CH 4 yr −1 . The emulator is trained on 35 GMB model estimates, including 22 process-based models and 13 atmospheric inversions, paired with 10 ensemble realizations of 11 gridded climate predictor variables from atmospheric reanalyses. Our results show that the global mean predicted wetland CH 4 emissions for 2021–2025 (157.8 ± 2.4 Tg CH 4 yr −1 ) are not significantly higher (∼ 0.05 Tg CH 4 yr −1 ) than the 2000–2020 baseline. However, this stability masks a significant hemispheric redistribution of emissions. We detect an increase in Northern Hemisphere (NH) emissions in 2021–2025, with mid- and high-latitudes increasing by 0.76 ± 0.07 and 0.35 ± 0.03 Tg CH 4 yr −1 , respectively, while the tropics and Southern Hemisphere (SH) extratropics show offsetting negative trends (−0.95 ± 0.19 and -0.11 ± 0.02 Tg CH 4 yr −1 , respectively). The predicted emissions are able to capture the low emissions in 2023 in South America linked to El Niño-related drought, as reported by recent studies (Ciais et al., 2026; Quinn et al., 2025). Furthermore, we identify a distinct seasonal amplification of global emission trends that peaks in late boreal summer. This new modeled dataset and operational framework bridge the gap between the latest updated budgets and low-latency monitoring, providing a scalable capacity to frequently update global emission estimates and critical early warnings of regional wetland feedback loops. The data are publicly available at https://doi.org/10.5281/zenodo.18870108 (Li et al., 2026).
Li, Mengze [National University of Singapore (Sing↗