SEARCH · Engineering Papers

Results for “Software testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

All sources NASA NTRS DOE OSTI

DOE OSTI2021

QXF Cold Mass Friction Test

The purpose of this test is to determine the longitudinal force on the stainless steel shell that is required to break the friction between the stainless steel shell and the aluminum shell underneath. This is achieved by suspending the short magnet between two backing plates on the eight grade 8 threaded rods which replace the 8 stainless steel toke tie rods in the magnet assembly. Eight hydraulic cylinders are mounted to each backing plate. Six, 22" sections of the stainless steel shell are welded in place on the magnet, each of which has a slightly different developed length, starting from the baseline design, and becoming progressively looser. A pushing flange is mounted to the stainless steel shells via small tack welds. Four load cells are then attached to the flange at designated points, while additional tooling is attached to each of the cylinders to facilitate the transfer of force from the cylinders to the stainless steel shell During the test, the cylinders will be slowly energized to no more than 7000 psi, with a resultant, longitudinal force on the flange of no more than 125,400 lbf. For each set of shells, the test ends when the load cell monitoring software detects a sharp decrease in force. This will indicate that the friction has broken. The cullenders will then be de-energized and the cycle will repeat for the next test. Welding operations outlined in the procedure (US-HiLumi-doc-4200) are outside the scope of the HA as the work is occurring under a previously-issued burn permit by qualified welders.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2026

Best practices in software development for robust and reproducible geoscientific models based on insights from the Global Carbon Budget's dynamic vegetation models

Computational models play an increasingly vital role in scientific research by enabling the numerical simulation of complex processes. Such models are also fundamental in geosciences. For instance, they offer critical insights into the impacts of global change on the Earth system today and in the future. Beyond their value as research tools, models are also software products and should therefore adhere to certain established software engineering standards. However, scientists are rarely trained as software developers, which can lead to potential deficiencies in software quality like unreadable, inefficient, or erroneous code. The complexity of models, coupled with their integration into broader workflows, also often makes it challenging to reproduce results, evaluate processes, and build upon them. In this paper, we review the state and current practices of the development processes of the state-of-the-art land surface models used by the Global Carbon Budget. We combine the experience of modelers from the respective research groups with the expertise of software engineers from tech companies to outline key principles and tools for improving software quality in research. We explore four main areas: (1) model testing and validation, (2) scientific, technical, and user documentation, (3) version control, continuous integration, and code review, and (4) the portability and reproducibility of workflows. Our review reveals that while modeling communities are incorporating many best practices, significant room for improvement remains in areas such as automated testing, automated documentation, and reproducibility. Therefore, we here identify and promote essential software engineering practices, including numerous examples of practices from within the community that can serve as guidelines for other models and could help streamline processes across the entire community. We conclude with an open-source example implementation of these principles, demonstrating portable and reproducible data flows, a continuous integration setup, and web-based visualizations. This example may serve as a practical resource for model developers, users, and all scientists engaged in scientific programming.

Gregor, Konstantin [Technical Univ. of Munich (Ger↗

DOE OSTI2020

Substation Secondary Asset Health Monitoring and Management System (SSHM)

Electric Power Group, LLC (EPG) was awarded DOE OE0000850 to design, develop, and demonstrate a real-time software application for substation secondary equipment health monitoring and management at host utility American Electric Power (AEP), a cost share partner. This project addresses the need for monitoring substation equipment health and providing operators with tools to identify and take pre-emptive action to avoid catastrophic equipment failure. By monitoring synchrophasor data in real time, data anomalies that indicate potential asset failure can be detected and alerts can be sent to operators in time to take corrective actions. EPG developed two data driven algorithms to identify abnormal equipment signature patterns as well as a method using substation linear state estimation (SLSE). The software has been deployed on hardened PC’s and tested and validated for cost share partner AEP’s substations - 138kV and 765 kV. The SSHM software has been accepted by AEP and final demonstration of DOE - OE0000850 for AEP and DOE was successfully completed on March 17th, 2020. EPG developed the Substation Secondary Asset Health Monitoring (SSHM) Platform to analyze equipment failure signatures in PMU data and alert substation personnel when pre-emptive inspection, repairs and other actions are warranted to prevent potential catastrophic failures. This is the first system that automatically monitors the health of substation secondary assets and alerts users to emerging failures using synchrophasor measurements. SSHM uses high resolution PMU data from PTs, CTs, and CCVTs to analyze equipment signatures and identify anomalies that are precursor indicators of potential equipment failure. When an anomaly is detected, there is a likelihood of potential failure of monitored equipment, the system alerts the user with visual alarms including alarm trend charts and indicator lights on a oneline diagram of the substation.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

Practical Insights on Applying Simulation-Based Control Methods in Experimental Studies

Advanced nuclear reactors are crucial to the future of energy both in the United States and around the globe. In contrast to the current operating fleet, they are characterized as being deployable in remote locations and able to operate in semi-autonomous or autonomous fashion. This leap forward necessitates a new reactor control paradigm. Because advanced nuclear reactors are still under development in the United States, the creation of new control methods to achieve autonomous operations has been based on systems modeling and simulation. However, an important factor in successfully deploying these new control methods is the ability to seamlessly transition from simulation environments to real-world settings. Control methods tested in both simulation and experimental settings need to be investigated in the context of advanced reactor applications. This work developed a series of simple controllers for Idaho National Laboratory (INL)’s Microreactor Applications Research Validation and Evaluation (MARVEL) microreactor operating in load-following scenarios. These controllers were tested in both simulation and experimental settings, and a comparative performance analysis was performed. The simulation tests leveraged the Control and Optimization Modular Modeling Application for Nuclear Deployment (COMMAND) software developed in a previous stage of the current effort, along with the MARVEL Reactor Excursion and Leak Analysis Program (RELAP5-3D) and Monte Carlo N-Particle (MCNP) models. The experimental tests leveraged the COMMAND software, MARVEL models, and the U.S. Department of Energy Microreactor Program’s Microreactor Automated Control System (MACS). MACS was developed to serve as a control method testbed. It was customized to mirror the MARVEL microreactor, and COMMAND enabled MACS to emulate the physics of MARVEL. The load-following controller was developed using the simulation platform, with efforts to emulate real systems by introducing actuator saturation and noise. These factors were incrementally accounted for in the controller design. After finalizing the controller design, it was implemented with the experimental setup. The experimental conditions tested included an initial test under conditions similar to the final simulation test, and two additional scenarios. The first scenario introduced additional actuator saturation to account for equipment aging over time, which was unknown to the controller. The second scenario introduced sensor delay, a phenomenon anticipated with the use of remote operations or wireless communication in advanced reactors. These tests revealed several notable differences. While the controller performed well in simulation, it exhibited several limitations when transitioning to hardware. The main challenges involved maintaining the steady-state target power, as evidenced by larger error values between the true reactor power and setpoint power, as well as persistent oscillations in controlled reactor power. These issues could lead to unacceptable transient conditions in real reactor testing. Introducing actuator aging and stochastic delays in the experimental setup significantly impacted controller performance, resulting in increased overshoot and undershoot, and exacerbated error and oscillations previously mentioned. These findings underscore the importance of experimental testbeds for testing and validating control methods, as controllers developed using only theory and/or simulation may perform unexpectedly when applied to actual hardware. This research emphasizes the need for an experimental testbed for achieving such validation.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗

DOE OSTI2025

Testing convolutional neural network based deep learning systems: a statistical metamorphic approach

Machine learning technology spans many areas and today plays a significant role in addressing a wide range of problems in critical domains,i.e., healthcare, autonomous driving, finance, manufacturing, cybersecurity,etc. Metamorphic testing (MT) is considered a simple but very powerful approach in testing such computationally complex systems for which either an oracle is not available or is available but difficult to apply. Conventional metamorphic testing techniques have certain limitations in verifying deep learning-based models (i.e., convolutional neural networks (CNNs)) that have a stochastic nature (because of randomly initializing the network weights) in their training. In this article, we attempt to address this problem by using a statistical metamorphic testing (SMT) technique that does not require software testers to worry about fixing the random seeds (to get deterministic results) to verify the metamorphic relations (MRs). We propose seven MRs combined with different statistical methods to statistically verify whether the program under test adheres to the relation(s) specified in the MR(s). We further use mutation testing techniques to show the usefulness of the proposed approach in the healthcare space and test two CNN-based deep learning models (used for pneumonia detection among patients). The empirical results show that our proposed approach uncovers 85.71% of the implementation faults in the classifiers under test (CUT). Furthermore, we also propose an MRs minimization algorithm for the CUT, thus saving computational costs and organizational testing resources.

Computer Science↗

DOE OSTI2022

Requirements for Cataloging Hanford Geophysical Datasets

Environmental management activities at the Hanford Site produce extensive data about site conditions, contaminants, cleanup, and more. Managing and archiving that data requires a high degree of collaboration among site contractors and a high level of awareness by project managers and staff. Part of that effort is developing a Hanford Environmental Information and Data Index (HEIDI) to organize the data and maximize its value by making it findable and available for reuse. The objective is to catalog the disparate data sets collected to address the evolving needs of planning, executing, and documenting cleanup over several decades up to the present day, including links to active data sources when available. A properly implemented data catalog makes finding environmental datasets related to an area or theme a routine, reliable process, without requiring the searcher to have special knowledge that a data set exists and where it may be stored. In this project, a working group, including the U.S. Department of Energy, the Hanford Site contractors, and Pacific Northwest National Laboratory staff, identified needs and requirements for handling complex site data. Geophysical data was chosen as a test case because it can be large and complex and often involves multiple processing steps to extract the information incorporated into deliverables. The ability to document those steps was one of the requirements identified for the catalog. In addition to developing requirements, other activities included selecting a metadata schema and initial testing with the objective of determining whether the workflow and capabilities of selected data catalog software platforms were sufficient to implement and impose the identified requirements. This initial testing involved running the default catalog instance using the software platform of interest and altering the configuration to achieve each requirement, if possible. Where configuration alone was insufficient, the possibility of modifying the software by changing the code was examined, but not implemented. A follow-on task is planned to reprogram the code as necessary to implement requirements in a prototype catalog.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2022

Noisy intermediate-scale quantum algorithms

A universal fault-tolerant quantum computer that can efficiently solve problems such as integer factorization and unstructured database search requires millions of qubits with low error rates and long coherence times. While the experimental advancement toward realizing such devices will potentially take decades of research, noisy intermediate-scale quantum (NISQ) computers already exist. These computers are composed of hundreds of noisy qubits, i.e., qubits that are not error corrected, and therefore perform imperfect operations within a limited coherence time. In the search for achieving quantum advantage with these devices, algorithms have been proposed for applications in various disciplines spanning physics, machine learning, quantum chemistry, and combinatorial optimization. The overarching goal of such algorithms is to leverage the limited available resources to perform classically challenging tasks. In this review, a thorough summary of NISQ computational paradigms and algorithms is provided. The key structure of these algorithms and their limitations and advantages are discussed. Finally, a comprehensive overview of various benchmarking and software tools useful for programming and testing NISQ devices is additionally provided.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2024

Efficient end-to-end simulation of time-dependent coherent X-ray scattering experiments

Physical optics simulations for beamlines and experiments allow users to test experiment feasibility and optimize beamline settings ahead of beam time in order to optimize valuable beam time at synchrotron light sources like NSLS-II. Further, such simulations also help to develop and test experimental data processing methods and software in advance. The Synchrotron Radiation Workshop ( SRW ) software package supports such complex simulations. We demonstrate how recent developments in SRW significantly improve the efficiency of physical optics simulations, such as end-to-end simulations of time-dependent X-ray photon correlation spectroscopy experiments with partially coherent undulator radiation (UR). The molecular dynamics simulation code LAMMPS was chosen to model the sample: a solution of silica nanoparticles in water at room temperature. Real-space distributions of nanoparticles produced by LAMMPS were imported into SRW and used to simulate scattering patterns of partially coherent hard X-ray UR from such a sample at the detector. The partially coherent UR illuminating the sample can be represented by a set of orthogonal coherent modes obtained by simulation of emission and propagation of this radiation through the coherent hard X-ray (CHX) scattering beamline followed by a coherent-mode decomposition. GPU acceleration is added for several key functions of SRW used in propagation from sample to detector, further improving the speed of the calculations. The accuracy of this simulation is benchmarked by comparison with experimental data.

36 MATERIALS SCIENCE↗

DOE OSTI2024

A general Bayesian algorithm for the autonomous alignment of beamlines

Autonomous methods to align beamlines can decrease the amount of time spent on diagnostics, and also uncover better global optima leading to better beam quality. The alignment of these beamlines is a high-dimensional expensive-to-sample optimization problem involving the simultaneous treatment of many optical elements with correlated and nonlinear dynamics. Bayesian optimization is a strategy of efficient global optimization that has proved successful in similar regimes in a wide variety of beamline alignment applications, though it has typically been implemented for particular beamlines and optimization tasks. In this paper, we present a basic formulation of Bayesian inference and Gaussian process models as they relate to multi-objective Bayesian optimization, as well as the practical challenges presented by beamline alignment. We show that the same general implementation of Bayesian optimization with special consideration for beamline alignment can quickly learn the dynamics of particular beamlines in an online fashion through hyperparameter fitting with no prior information. We present the implementation of a concise software framework for beamline alignment and test it on four different optimization problems for experiments on X-ray beamlines at the National Synchrotron Light Source II and the Advanced Light Source, and an electron beam at the Accelerator Test Facility, along with benchmarking on a simulated digital twin. We discuss new applications of the framework, and the potential for a unified approach to beamline alignment at synchrotron facilities.

47 OTHER INSTRUMENTATION↗

DOE OSTI2022

Challenges of and Opportunities for a Large Diverse Software Team

A large software team consisting of members with different expertise, skillsets, personalities, ethnicities, and involving collaboration on a large and complex software product presents many technical and cultural challenges, but also provides unique opportunities. In this article, we discuss the essential issues we faced when successfully transforming a collection of various independently developed software libraries into one large integrated product: the eXtreme-scale scientific Software Development Kit (xSDK). Furthermore, we argue it is just as important to pay attention to cultural challenges, such as establishment of reliable communication channels that considers, among others, differences in personalities and backgrounds as well as overcoming geographical separation and time-zone distribution when collaborating, as technical challenges. Finally, we discuss opportunities stemming from participating in a large diverse software team, such as increased internal expertise, variety of skillsets, broadened connections to external experts, and access to a larger pool of ideas or solutions.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Ring Pull Strain Analysis

Ring Pull Strain Analysis (RPSA) is a framework for analysis of Digital Image Correlation (DIC) data of gaugeless ring pull testing. It is meant for analysis of rings cut from tube cross sections and pulled in a displacement-controlled tensile test. This code takes load frame data, images take during testing, and csv files from the DIC analysis of those images and allows the user to analyze and plot data from the test. Some important features of the software include getting the strain values as a function of radial distance and angle, plotting of the strain around the ring, and stress-strain analysis of the load frame data.

Beck, Peter↗

DOE OSTI2024

Ring Pull Strain Analysis

Beck, Peter↗

DOE OSTI2022

Evaluating an Eco-Cooperative Automated Control System

The paper evaluates an Eco-Cooperative Automated Control (Eco-CAC) system on a large-scale network considering a combination of internal combustion engine vehicles (ICEVs), hybrid electric vehicles (HEVs), and battery-only electric vehicles (BEVs) in a microscopic traffic simulation environment. We used a novel integrated control system that: (1) routes ICEVs, HEVs, and BEVs in a fuel/energy-efficient manner; (2) selects vehicle speeds based on anticipated traffic network evolution; (3) minimizes vehicle fuel/energy consumption near signalized intersections; and (4) intelligently modulates the longitudinal motion of vehicles along freeways within a cooperative platoon to minimize fuel/energy consumption. The study tested the system using the INTEGRATION software on the Los Angeles (LA), U.S., downtown network for three different demand levels: no congestion, mild congestion, and heavy congestion. The results demonstrated that the Eco-CAC system effectively reduces vehicle fuel and energy consumption, travel time, total delay, and stopped delay in heavily congested conditions. However, different vehicle compositions produced different results. In particular, the maximum energy consumption savings for BEVs (36.9%) for a current vehicle composition occurred at a 10% market penetration rate (MPR) of connected automated vehicles (CAVs) in mild congestion, while the maximum savings for a future vehicle composition (35.5%) occurred at a 50% CAV MPR in no congestion. The system reduced fuel consumption for ICEVs and HEVs by up to 5.4% and 6.3% at a 25% CAV MPR in heavy congestion for current and future vehicle compositions, respectively. However, the system increased total fuel consumption by up to 4.6% at a 50% CAV MPR in no congestion for a current vehicle composition. The study demonstrates that the effectiveness of the Eco-CAC system depends on traffic conditions, including congestion level, network configuration, CAV MPR, and vehicle composition.

Engineering↗

DOE OSTI2024

Multilevel Monte Carlo Estimators For Derivative-Free Optimization Under Uncertainty

Optimization is a key tool for scientific and engineering applications; however, in the presence of models affected by uncertainty, the optimization formulation needs to be extended to consider statistics of the quantity of interest. Optimization under uncertainty (OUU) deals with this endeavor and requires uncertainty quantification analyses at several design locations; i.e., its overall computational cost is proportional to the cost of performing a forward uncertainty analysis at each design location. An OUU workflow has two main components: an inner loop strategy for the computation of statistics of the quantity of interest, and an outer loop optimization strategy tasked with finding the optimal design, given a merit function based on the inner loop statistics. Here, in this work, we propose to alleviate the cost of the inner loop uncertainty analysis by leveraging the so-called multilevel Monte Carlo (MLMC) method, which is able to allocate resources over multiple models with varying accuracy and cost. The resource allocation problem in MLMC is formulated by minimizing the computational cost given a target variance for the estimator. We consider MLMC estimators for statistics usually employed in OUU workflows and solve the corresponding allocation problem. For the outer loop, we consider a derivative-free optimization strategy implemented in the SNOWPAC library; our novel strategy is implemented and released in the Dakota software toolkit. We discuss several numerical test cases to showcase the features and performance of our approach with respect to its Monte Carlo single fidelity counterpart.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Battery Performance, Thermal, and Life Modeling for Southern California Edison (Cooperative Research and Development Final Report)

The objective of this project is to develop battery performance models, thermal models and life models for SCE to use in simulation and analysis of grid services involving energy storage. NREL will fit the models to three separate chemistries using test data provided by SCE. The dataset for each chemistry will include electrical, thermal and aging response to different temperatures and cycling conditions, measured under a variety of cell-level and module-level experiments described in SCE test protocol documents. Once tuned to a test dataset, the battery system model software will provide predictions of battery energy and power loss, and thus cycle and calendar life, for any energy storage grid service power profile of interest to SCE. The automated battery life modeling and simulation tool can also be applied to any other grid battery systems as long as necessary input data is available to accelerate battery model development and battery lifetime analysis.

25 ENERGY STORAGE↗

DOE OSTI2021

CFD Analysis of RLUOB Zone 1 HEPA Filter Plenum and Testing Manifolds

The purpose of this computational fluid dynamics (CFD) analysis is to ensure that Camfil Farr’s upstream and downstream injection and sampling manifolds can meet or exceed the requirements outlined in the ASME AG-1 1997 a(2000) Code on Nuclear Air and Gas Treatment for the testing of HEPA and adsorbent filters. This paper will present a numerical simulation of airflow in the Radiological Laboratory Utility Office Building (RLUOB) zone 1 HEPA filter plenum and testing manifolds using the commercial CFD software ANSYS FLUENT 2020R1. The CFD analysis focuses on the investigation of the air flow distribution and air-aerosol mixing uniformity. The evaluation was done for all steps of the modeling process: grid generation, physics setup, simulation, and post-processing. The mass flow rate in each section of the zone 1 injection and sampling manifolds is also reported.

42 ENGINEERING↗

DOE OSTI2020

FY20 Proxy App Suite Release: Report for ECP Proxy App Project Milestone ADCD-504-10

Version 4.0 of the ECP Proxy App Suite is practically unchanged from the previous release. The current set of proxies has proven useful for many aspects of benchmarking and co-design and we see little reason to alter the suite. Although there have been few changes to the ECP suite, the team has been hard at work in other areas. In the area of Machine Learning (ML) we have now created a separate proxy suite dedicated to this scientific applications of ML. The suite includes: miniGAN (Generative Adversarial Networks), miniRL (Reinforcement Learning), CRADL (inline inference), Cosmoflow-Benchmark (Convolutional Neural Network), and MLPerf-DeepCam (Climate Segmentation Benchmark). Section 3 contains more information about these proxies as well as the principles that are guiding the development of the suite. We have surveyed available proxies for several application domains including Computational Fluid Dynamics, Quantum Chemistry, Quantum Computing Simulation, Molecular Dynamics, Monte Carlo Transport, and Density Functional Theory to identify gaps in proxy coverage. Several new proxy apps are either already available or will be released soon to fill these gaps. Section 4 provides full details. Finally, section 5 reports on our continued collaboration with the ECP Continuous Integration (CI) effort to use proxy apps to help identify problems and roadblocks to cross-lab CI. We have also assisted the El Capitan Center of Excellence (COE) to stand up a CI system that will be used to test releases of HPE and AMD software stacks. We hope that the COE effort can serve as a model for ECP by showing how the Proxy App Team can work with AD and ST teams identify critical features, kernels, patterns, etc. and incorporate them into a CI system that will help ensure that Frontier and Aurora will provide those needed capabilities.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Advanced Monitoring and Control in the ANL METL Facility Using an Engineering Digital Twin

The potential benefits of using an engineering digital twin to achieve greater autonomy for monitoring and control functions in advanced reactors was investigated for the Mechanisms Engineering Test Loop (METL) liquid sodium facility at Argonne National Laboratory. The METL sodium purification system served as a representative system as it requires significant human in-the-loop interaction to accomplish its design function. The objective was to demonstrate how real-time operation could be automated while preserving oversight of the operator for ensuring that the system design functions are met. A digital twin model of the purification system was developed for both the cold trap purification loop and plugging meter diagnostic loop using information from the METL piping and instrumentation diagram (P&ID). Automated monitoring and diagnosis of component degradation in the METL facility was demonstrated in tests using the PRO-AID health monitoring software with the digital twin model incorporated in the library of components. Component failures were introduced and were successfully diagnosed in real time. These tests serve to demonstrate an advanced monitoring capability able to differentiate sensor degradation from component degradation, to generate a rank ordering of probabilities of different failure mechanisms that serves to circumvent the false alarm problem with current anomaly detection methods, and how facility monitoring can be transformed from anomaly detection to identification of a specific fault. Automated control of the purification system was demonstrated through simulations that exercised a model predictive controller designed using the digital twin model. Results of these simulations compared favorably with experimental data showing very good reference tracking response with negligible overshoot. In conclusion, these pilot tests and simulations successfully demonstrated the use of a digital twin for improved automation of monitoring and control. It was shown how the digital twin enables switching between control modes from cold trap operation where impurities are removed to plugging meter operation where impurity concentrations are measured.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗