Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “automated testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Dyno-in-the-Loop: An Innovative Hardware-in-the-Loop Development and Testing Platform for Emerging Mobility Technologies

Today’s transportation is quickly transforming with the nascent advent of connectivity, automation, shared-mobility, and electrification. These technologies will not only affect our safety and mobility, but also our energy consumption, and environment. As a result, it is of unprecedented importance to understand the overall system impacts due to the introduction of these emerging technologies and concepts. Existing modeling tools are not able to effectively capture the implications of these technologies, not to mention accurately and reliably evaluating their effectiveness with a reasonable scope. To address these gaps, a dynamometer-in-the-loop (DiL) development and testing approach is proposed which integrates test vehicle(s), chassis dynamometer, and high fidelity traffic simulation tools, in order to achieve a balance between the model accuracy and scalability of environmental analysis for the next generation of transportation systems. With this DiL platform, a connected eco-operation system for the plug-in hybrid electric bus (PHEB) has been developed and tested, which can optimize the vehicle dynamics (and potentially powertrain control via smart energy management) to reduce the operational energy consumption as well as tailpipe emissions of the target PHEB. The system performance has been evaluated on the DiL platform with respect to a variety of traffic congestion levels. The results have shown that the developed system can save fuel by more than 13% while reducing the electricity consumption by 2% in the test scenarios.

Wu, Guoyuan↗
DOE OSTI2023

Artificial intelligence models, photos, and data associated with the manuscript “Quantifying Streambed Grain Size, Uncertainty, and Hydrobiogeochemical Parameters Using Machine Learning Model YOLO” (v2)

This data package is associated with the manuscript “Quantifying Streambed Grain Size, Uncertainty, and Hydrobiogeochemical Parameters Using Machine Learning Model YOLO” published in Water Resources Research (Chen et al., 2024). This data package includes the training, validation, testing, and prediction data used by the artificial intelligence (AI) model for automated grain size and hydro-biogeochemistry quantification using streambed photos. The grain size data are extracted for each photo using You Look Only Once (YOLO), a pre-trained object detection model. This data package was originally published in October 2023. It was updated August 2025 (v2; new and modified files). File and folder names were not revised to indicate changes. See the change history section in the readme for more details. Please see flmd.csv for a list of all files contained in this data package and descriptions for each. Please see dd.csv for a data dictionary that defines the column headers of .csv files in the data package. This dataset is comprised of one data folder containing (1) file-level metadata; (2) data dictionary; (3) readme; and (4) six subfolders. Subfolders 1 to 4 include the training, validation, testing, and prediction data. Subfolder 5_Summary includes the summary results of different combinations of training, validation, testing, and prediction data. Subfolder 6_SupplementalData includes additional data downloaded from public sources (Kaufman et al., 2023a; Kaufman et al., 2023b; Garefalakis et al., 2023; Mair et al., 2024; https://github.com/river-corridors-sfa/Geospatial_variables). In total, the data package includes 110 folders and 44,283 files. These files include 9,047 .jpg photos, 1 .png photo, 3 .tif photos; 26,639 photo labels and individual grain sizes and probability from AI (.txt); 8,447 grain size distribution data (.dat); and 126 CSV files for results summary, and 14 required metadata files (.xlsx). The summary CSV files contain 68 columns and approximately 2,200 rows that represent photo names, site locations, recording time, GPS coordinates, grains sizes (D10, D50, D60, and D84), number of grains, and additional hydro-biogeochemical data such as water depth, flow velocity, Manning’s coefficient, friction factor, hydraulic conductivity, permeability, streambed interstitial velocity magnitude, mass transfer rate, and nitrate uptake velocity. The photos were obtained from 75 sites in the Yakima River Basin and the Columbia River shorelines, and other associated data from samples and sensors obtained when the photos were taken are publicly available (Fulton et al. 2022; Grieger et al. 2023). All files are .csv, .txt, .dat, .jpg, or .pdf. We acknowledge the Yakama Nation as owners and caretakers of the lands where we collected some of these data. We thank the Confederated Tribes and Bands of the Yakama Nation Tribal Council and Yakama Nation Fisheries for working with us to facilitate sample collection and optimization of data usage according to their values and worldview.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

LLM Benchmarking with LLaMA2: Evaluating Code Development Performance Across Multiple Programming Languages

The rapid evolution of large language models (LLMs) has opened new possibilities for automating various tasks in software development. This paper evaluates the capabilities of the LLaMA 2-70B model in automating these tasks for scientific applications written in commonly used programming languages. Using representative test problems, we assess the model's capacity to generate code, documentation, and unit tests, as well as its ability to translate existing code between commonly used programming languages. Our comprehensive analysis evaluates the compilation, runtime behavior, and correctness of the generated and translated code. Additionally, we assess the quality of automatically generated code, documentation, and unit tests. Here, our results indicate that while LLaMA 2-70B frequently generates syntactically correct and functional code for simpler numerical tasks, it encounters substantial difficulties with more complex, parallelized, or distributed computations, requiring considerable manual corrections. We identify key limitations and suggest areas for future improvements to better leverage AI-driven automation in scientific computing workflows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Fuel Performance Analysis of Fast Flux Test Facility MFF-3 and -5 Fuel Pins Using BISON with Post Irradiation Examination Data

Using the BISON fuel-performance code, simulations were conducted of an automated process to read initial and operating conditions from the Pacific Northwest National Laboratory (PNNL) database and reports, which contain metallic-fuel data from the Fast Flux Test Facility (FFTF) MFF Experiments. This work builds on previous modeling efforts involving 1977 EBR-II metallic fuel pins from experiments. Coupling the FFTF PNNL reports to BISON allowed for all 338 pins from MFF-3 and MFF-5 campaigns to be simulated. Each BISON simulation contains unique power and flux histories, axial power and flux profiles, and coolant-channel flow rates. Fission-gas release (FGR), fuel axial swelling, cladding profilometry, and burnup were all simulated in BISON and compared to available post-irradiation examination (PIE) data. Cladding profilometry, FGR, and fuel axial swelling simulation results for full-length MFF metallic pins were found to be in agreement with PIE measurements using FFTF physics and models used previously for EBR-II simulations. The main two peaks observed within the cladding profilometry were able to be simulated, with fuel-cladding mechanical interaction (FCMI), fuel-cladding chemical interaction (FCCI), and thermal and irradiation-induced creep being the cause. A U-Pu-Zr hot-pressing model was included in this work to allow pore collapse within the fuel matrix. This allowed better agreement between BISON-simulated cladding profilometry and PIE measurements for the peak caused by FCMI. This work shows that metallic fuel models used to accurately represent fuel performance for smaller EBR-II pins may be used for full-length metallic fuel, such as FFTF MFF assemblies and the Versatile Test Reactor (VTR). As new material models and PIE measurements become available, FFTF MFF assessment cases will be reassessed to further BISON model development.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Understanding and modeling current and future coastal wetland methane dynamics (Final Report)

The coastal terrestrial-aquatic interface (TAI) is a highly dynamic component of the Earth system that plays a critical role in biogeochemical cycling. Due to its dynamic nature, the processes that regulate decomposition and methane (CH 4 ) emissions are of greater significance at the TAI than in upland systems. Despite this, we have limited mechanistic understanding of how climate stressors interact to regulate the electron acceptors and donors that determine decomposition pathways within TAIs, including the generation of hot spots and hot moments. Accurately modeling these processes is critical for incorporating the coastal TAI into Earth systems models, such as DOE’s E3SM. With previous DOE support, we adapted an aerobic terrestrial representation of decomposition using PFLOTRAN, a reactive flow and transport model, and added anerobic decomposition pathways, salinity, and oxygen (O 2 ) that fluctuates independently of water table level. However, because this model (PFLOTRAN TAI ) is based on decomposition rates and organic matter carbon to nitrogen ratios from terrestrial systems, its performance in TAI systems is currently limited by the lack of empirical data to properly parameterize variables. In addition, while PFLOTRAN TAI can simulate movement of O 2 into sediments, it is not currently capable of tracking the movement of CH 4 gas through plant tissues due to both current model structure and lack of available data. Our overall objective of the project was the increase our mechanistic understanding of CH 4 dynamics in response to environmental change, such that we can improve the representation of these dynamics in PFLOTRAN. We installed automated flux chambers in a new field-scale active soil warming experiment and set up marsh organs (mesocosms) to test effects of warming, flooding, and salinity. This resulted in a new dataset consisting of chamber-level CH 4 flux measurements across multiple sites, ecological conditions, and timeframes, as well as corresponding measurements on porewater chemistry, soil carbon quality, plant biomass, and redox reaction rates. Using these data, we improved PFLOTRAN TAI to more accurately model CH 4 dynamics and successfully tested our hypotheses. This grant contributed to the professional development of 3 postdocs, 4 undergraduate interns, 4 teacher interns, 17 technicians, and 15 participatory scientists. The automated chamber technology designed for this grant has also been shared with multiple new projects.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Automated and Distributed Monte Carlo Generation for GlueX

MCwrapper is a set of systems that manages the entire Monte Carlo production workflow for GlueX and provides standards for how that Monte Carlo is produced. MCwrapper was designed to be able to utilize a variety of batch systems in a way that is relatively transparent to the user, thus enabling users to quickly and easily produce valid simulated data at home institutions worldwide. Additionally, MCwrapper supports an autonomous system that takes user’s project submissions via a custom web application. The system then atomizes the project into individual jobs, matches these jobs to resources, and monitors the jobs status. The entire system is managed by a database which tracks almost all facets of the systems from user submissions to the individual jobs themselves. Users can interact with their submitted projects online via a dashboard or, in the case of testing failure, can modify their project requests from a link contained in an automated email. Beginning in 2018 the GlueX Collaboration began to utilize the Open Science Grid (OSG) to handle a bulk of simulation tasks; these tasks are currently being performed on the OSG automatically via MCwrapper. This talk will outline the entire system of MCwrapper, its use cases, and the unique challenges facing the system.

Britton, Thomas↗
DOE OSTI2025

Testing Capabilities for Superconducting Magnets at Fermilab

This presentation will focus on the current testing capabilities for superconducting magnets at Fermilab, highlighting existing systems, recent upgrades, and future plans. The Vertical Magnet Test Facility (VMTF) remains a cornerstone for testing superconducting magnets in R&D applications, maintaining its established capabilities. Stand 4 continues to support the Hi-Lumi AUP production line of magnets in cryostats and has achieved significant improvements through a redesign of lead connections, resolving issues with liquid helium levels and resistance, culminating in a successful endurance test without interruption. Similarly, Stand 7 underwent commissioning and a lead redesign to reduce heat load, while its core testing capabilities remain unchanged. Looking forward, Stand 3 is undergoing a redesign, with plans to restart and begin commissioning by the end of 2025, introducing new features such as background magnetic field testing and upgraded power supplies. Fermilab is also advancing the High Field Vertical Magnet Test Facility (HFVMTF), designed to support the fusion and magnet R&D program, with commissioning scheduled for 2025 and critical components, including the cryostat and power supplies, being delivered. Enhancing all operations, the newly commissioned IB1 cryoplant now offers automated overnight helium production and efficient transfer to a 10,000-liter dewar, streamlining support across all test stands. This talk will provide a comprehensive overview of testing capabilities, and showcasing how Fermilab is advancing its capabilities to meet the evolving demands of superconducting magnet testing.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Investigation of Nanoparticle Degradation in Hydrogen Fuel Cell Systems through Automated Electron Microscopy

Proton exchange membrane fuel cells (PEMFC) are promising devices for the deployment of hydrogen-powered heavy-duty vehicles, providing a higher efficiency for similar driving range and fueling time than the existing ones. However, PEMFCs still encounter durability challenges mainly due to catalyst degradation in the cathode. Mitigating these performance losses requires a better understanding of the degradation mechanisms under heavy-duty accelerated stress tests (ASTs) [1]. Scanning transmission electron microscopy (STEM) combined with energy dispersive X-ray spectroscopy (EDS) are key tools for the analysis of Pt and PtCo nanoparticle size, spatial distribution and composition [2]. Electron tomography is also used to determine the rate and type of degradation of catalyst nanoparticles as a function of their position on the carbon support. In this work, automated data acquisition software, paired with a custom Python code, have been used to study the effect of different accelerated stress tests (ASTs) on nanoparticle coarsening [2]. Figure 1 shows high-angle annular dark-field (HAADF)-STEM images and EDS maps comparing the cathodes of membrane electrode assemblies (MEAs) following an electrocatalyst AST performed under H2/N2 with that of the heavy-duty AST performed under H2/air. We will discuss how AST conditions affect considerably the spatial distribution of the nanoparticles across the electrode between the membrane and microporous layer. Although the median particle size increased more in the MEA aged under the heavy-duty AST, as determined using a high-throughput image analysis, the quantitative EDS measurements demonstrate that the electrocatalyst AST resulted in more Pt and Co dissolution from the cathode, which is another important indicator of electrocatalyst degradation. We will further present the impact of the relative humidity (% RH) on the degradation mechanisms demonstrated using the same approach. Electron tomography has been used to distinguish the Pt nanoparticles residing on the carbon support surface (exterior) from those within the pore structure (interior) in order to determine the relative stability of interior and exterior nanoparticles. As shown in Figure 2, we will compare the Pt catalyst particle size at the beginning of test (BOT) and end of test (EOT), and discuss the importance of automating the electron tomography workflow, i.e. acquisition, reconstruction, and visualization, to increase sampling and determine the standard deviation of these measurement. The outlook for utilizing low-dose cryo-tomography for limiting damage to the catalyst, support, and especially proton-conducting ionomer will also be discussed [3].

Amichi, Lynda↗
DOE OSTI2024

A nuclear power plant digital twin for developing robot navigation and interaction

As robot technologies are rapidly improving, an increasing number of new ideas on utilizing robots for automated operation and maintenance tasks in nuclear power plants (NPPs) are being studied. However, due to safety concerns, researchers hardly found opportunities to test their new robot solutions on physical NPPs. In that sense, an efficient and realistic simulation environment plays a vital role in the development of automation systems for an NPP. In this paper, we propose the design of a 3D digital twin system capable of simulating NPP in real-time. This system obtains the data from a full-scope NPP simulator to reproduce the operating conditions of the plant. In addition, a scenario of a team of robots performing inspection tasks like temperature and pressure measurements will highlight its usability. This system enables development of intelligent robot swarms to deploy for inspection and maintenance purposes. It will positively impact autonomous control and operations for many types of reactors by reducing uncertainty in autonomous control and providing the tools necessary for remote intervention.

Energy & Fuels↗
DOE OSTI2025

Exploring Large Language Model Agents in Cybersecurity: A Literature Review with Experiments

The accelerated development and integration of large language model (LLM) agents have led researchers and developers to explore their effectiveness in cybersecurity, specifically with penetration testing (pentesting). Recent research efforts have attempted to use LLM agents to automate the process of pentesting because of the cost and time requirements that are required to perform a manual review. However, not all of the tools perform as expected. This paper reviews some of the newest and most popular autonomous pentesting frameworks, highlighting the capabilities and limitations of each one with the goal of providing the components needed to successfully and effectively build an autonomous pentesting agent in the future.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

An Automated Disruption-Tolerant Key Management Framework for Critical Systems

Key management is critical to secure operation. Distributed control systems, such as Supervisory Control and Data Acquisition (SCADA) systems, have unique operational requirements that make conventional key management solutions less effective and burdensome. This paper presents a novel Kerberos-based framework for automated, disruption-tolerant key management for control system environments. Here, experimental tests and their results are presented to quantify the expected performance overhead of this approach. Additionally, Zeek sensor analytics are presented to aid in monitoring the health and security of the key management framework operation.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2021

Cyberbiosecurity and Public Health in the Age of COVID-19

Cyberbiosecurity, the aspect of biosecurity involving the digital representation of biological data, had already been emerging as a matter of public concern even prior to the onset of the COVID-19 pandemic. Key issues of concern include, among others, the privacy of patient data, the security of public health databases, the integrity of diagnostic test data, the integrity of public biological databases, the security implications of automated laboratory systems and the security of proprietary biological engineering advances. With the onset of the COVID-19 pandemic, and the importance of digital resources in combatting it, concern about the potential for cyber attacks by state-based or non-state actors has been elevated. To illuminate the challenges, we focus on the cyber vulnerabilities that need to be addressed in public health activities such as disease surveillance and outbreak management. In particular, we examine cyber issues raised by the accelerated pace of development for COVID mitigations, treatments, and vaccines.

cybersecurity, biosecurity↗
DOE OSTI2023

Semi-Automatic Geographic Information System Framework for Creating Photo-Realistic Digital Twin Cities to Support Autonomous Driving Research

Digital twin cities are frequently used in vehicle and traffic simulations to render realistic on-road driving scenarios under various traffic and environmental conditions. These digital twins provide a high-fidelity replica of the physical world (e.g., buildings, roads, infrastructures, traffic) to create three-dimensional (3D) virtual-physical environments to support various emerging vehicle and transportation technologies such as connected and automated vehicles. These virtual environments provide a cost-effective digital proving ground to evaluate, validate, and test emerging technologies that include control algorithms, localization, perception, and sensors. Replicating a real-world traffic scenario in a digital twin using a traditional 3D modeling approach is a time-consuming and labor-intensive effort. Here this paper presents a semi-automated spatial framework to construct realistic 3D digital twin cities to support autonomous driving research using readily available geographic information system (GIS) data and 3D prefabricated (prefab) models. We start with a comprehensive review of geospatial data sources of essential digital entities required in a 3D digital twin city and present an integrated GIS-3D modeling pipeline using customized QGIS/GDAL and Blender scripting in Python. The pipeline outputs are realistic 3D digital twin cities compatible with common vehicle simulation software, such as CARLA and IPG CarMaker. The paper closes with a showcase to demonstrate the quality and usability of a digital twin city created to replicate the Shallowford Road corridor in Chattanooga in both Unity and Unreal engine-based virtual environment. The generated digital twin city can be applied to a hardware-in-the-loop simulation environment with an actual testing vehicle to facilitate autonomous driving research.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2025

If We Build Them, They Will Run: Automated HPC Apps Deployment and Profiling with eBPF in Cloud

The high performance computing (HPC) community is in a period of transition. The rise of AI/ML coupled with a changing landscape of resources deems portability a new metric of performance, and methods to move between on-premises and cloud environments and assess compatibility are paramount. Here we design and test a strategy for bridging the gap between traditional HPC and Kubernetes environments – first containerizing applications, providing automated orchestration to run studies, and packaging the setup with automated means to assess performance using low overhead eXtended Berkeley Packet Filter (eBPF) programs. We first assess different designs for eBPF collection, demonstrating a tradeoff between number of programs deployed on a node and overhead added. We develop 5 low overhead eBPF programs that combine with streaming ML models to assess CPU, futex, TCP, shared memory, and file access across four different builds of an HPC application for CPU and GPU. We use eBPF data to generate insights into the possible underlying etiology of scaling issues. We then assess compatibility of a well-known benchmark, HPCG, across matrices of micro-architectures and optimization levels (217 containers across 24 instance types and over 7500 runs). We provide to the community 30 applications to deploy in our automated setup and perform a scaling study from 4 to a maximum of 256 nodes for both CPU and GPU applications. Finally, we use our gained knowledge about performance to generate compatibility artifacts that are used by a newly developed Kubernetes controller to intelligently select instance type based on optimizing a figure of merit. Along with insights to scaling in this environment with a collection of applications and templates to work from, we provide an overall strategy for approaching HPC application deployment and image selection based on compatibility in cloud.

Computer science↗
DOE OSTI2023

Automated Air Sealing Demonstration: Denver Federal Center Building 40

This project aimed to demonstrate building airtightness achieved by automated air sealing in a commercial building setting. The automated air sealing demonstrated in this project uses a modified blower door to pressurize and distribute the sealing aerosol to achieve the required building airtightness. To demonstrate this technology, Building 40 at the Denver Federal Center, a federally owned campus under the jurisdiction, custody and control of the U.S. General Services Administration (GSA), was selected for testing (Figure 2). This building is currently undergoing several retrofit projects, including increasing insulation, installing high-performance windows, and adding advanced equipment, control systems, and automated air sealing. This demonstration project involved installing automated air sealing and measuring the improvements in the building's airtightness. The automated air sealing was installed over two days by AeroBarrier, the vendor. An external blower door test contractor measured the airtightness of the demonstration space before and after air sealing. The new airtightness value and the percentage increase in airtightness were used to evaluate the energy savings potential of automated air sealing. These airtightness values were used to compute the energy savings and CO 2 emissions reduction for different climate zones, building types, and initial airtightness conditions. In addition, the heating, ventilation and air conditioning (HVAC) load reduction attributable to the reduced building air leakage was studied. This study included research to determine the cost and time reduction of automated air sealing. Finally, the automated air sealing performance was evaluated qualitatively using a focus group discussion that included GSA and Bristol, the general contractor. The installation has shown the demonstration site, with a floor size of 4,462 ft 2 , air leakage has reduced by more than 50% in less than 7 hours, including preparation, site sealing, and cleanup. The performance objectives were classified as quantitative or qualitative based on the evaluation metrics used to assess the project’s success. The key performance objectives for this project were the level of airtightness achieved, the time and cost required to perform the sealing, and the HVAC load reduction attributable to air sealing. Table 1 shows the quantitative performance objectives.

42 ENGINEERING↗
DOE OSTI2021

A Computed Tomography Automation Architecture Developed from a Comprehensive Literature Review

This paper presents a literature review on techniques related to the computed tomography procedure that incorporate automation elements in their research investigations or industrial applications. Computed tomography (CT) is a non-destructive testing (NDT) technique in that the imaging and inspection are performed without damaging the sample, allowing for additional or repeated analysis if necessary. The reviewed literature is organized based on the steps associated with a general NDT task in order to define an end-to-end computed tomography automation architecture. The process steps include activities prior to image collection, during the scan, and after the data are collected. It further reviews efforts related to repeating this process based on a previous scan result. By analyzing the multiple existing but disparate efforts found in the literature, we present a framework for fully automating NDT procedures and discuss the remaining technical gaps in the developed framework.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Improvements to CTF Code Verification and Unit Testing (FY2020)

In 2010, the U.S. Department of Energy created its first Energy Innovation Hub, which focuses on improving Light Water Reactors (LWRs) through Modeling and Simulation. This hub, named the Consortium for the Advanced Simulation of LWRs (CASL), attempts to characterize and understand LWR behavior under normal operating conditions and use any gained insights to improve their efficiency. In collaboration with North Carolina State University (NCSU), CASL has worked extensively on the thermal-hydraulic subchannel code Coolant Boiling in Rod Arrays—Three Field (COBRA-TF). The NCSU/CASL version of COBRA-TF has been rebranded as CTF. This document focuses on code verification test problems that ensure CTF converges to the correct answer for the intended application. The suite of code verification tests are mapped to the underlying conservation equations of CTF, and significant gaps are addressed. Convergence behavior and numerical errors are quantified for each of the tests. Tests that converge at the correct rate to the corresponding analytic solution are incorporated into the CTF automated regression suite. A new verification utility is created for this purpose, which enables code verification by generalizing the process. For problems that do not behave correctly, the results are reported but the problem is not included in the regression suite. In addition to verification studies, this document also quantifies the existing tests of constitutive models. A few existing gaps are addressed by adding new unit tests.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗