Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “automated testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Automated Testing System

The Automated Testing System (ATS) is an open-source, Python-based tool for automating the running of tests of an application across a broad range of high performance computers.

Dempsey, StephanieM.↗
DOE OSTI2023

Challenges and Strategies for Testing Automation Practices at Sandia National Laboratories

Sandia National Laboratories is a premier United States national security laboratory which develops science-based technologies in areas such as nuclear deterrence, energy production, and climate change. Computing plays a key role in its diverse missions, and within that environment, Research Software Engineers (RSEs) and other scientific software developers utilize testing automation to ensure quality and maintainability of their work. We conducted a Participatory Action Research study to explore the challenges and strategies for testing automation through the lens of academic literature. Through the experiences collected and comparison with open literature, we identify these challenges in testing automation and then present strategies for mitigation grounded in evidence-based practice and experience reports that other, similar institutions can assess for their automation needs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

ATLAS-MAP: An Automated Test Station for Gated Electronic Transport Measurements

The diversification of electronic materials in devices provides a strong incentive for methods to rapidly correlate device performance with fabrication decisions. In this work, we present a low-cost automated test station for gated electronic transport measurements of field-effect transistors. Utilizing open-source PyMeasure libraries for transparent instrument control, the “ATLAS-MAP” system serves as a customizable interface between sourcemeters and samples under test and is programmed to conduct transfer curve and van der Pauw methods with static and sweeping gate voltages. Zinc oxide transistors of variable thickness (5, 10, and 20 nm) and channel size (50 μm to 3 mm, of equal length and width) were fabricated to validate the design. Standardization of testing procedures and raw data formatting enabled automated data analysis. A detailed list of parts and code files for the system are provided.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Maximizing Efficiency and Quality: Leveraging Automated Testing for Laboratory Commissioning

The traditional commissioning process uses sampling to select equipment for functional acceptance testing when large quantities of equipment are present. Although this approach is generally effective in identifying wide-spread issues, it has several shortcomings: it fails to evaluate equipment not included in the sample, provides only a one-time validation of equipment operation, and the standard documentation is a simple checklist of pass/fail questions. During the construction and commissioning process of the new Research and Innovation Laboratory (RAIL) in Golden, CO, the National Renewable Energy Laboratory team engaged Group14 Engineering to implement a Connected Commissioning process using fault detection and diagnostic software for automated functional acceptance testing. This presentation highlights the advantages offered by automated functional testing in this critical laboratory setting: (1) sampling 100% of BAS-connected equipment during functional testing, (2) testing results backed by data beyond the traditional pass/fail checklist, and (3) an automated test process that can be regularly executed by the building management team for ongoing commissioning throughout the life of the building. The presentation will also cover technical challenges associated with Connected Commissioning and the important conversations with key stakeholders that need to occur well before functional acceptance testing in order to successfully implement the automated testing processes.

automated testing↗
DOE OSTI2023

Maximizing Efficiency and Quality: Leveraging Automated Testing for Laboratory Commissioning

The traditional commissioning process for a building often implements sampling rates to select equipment for functional acceptance testing when large quantities of equipment are present. While this approach is generally effective in identifying wide-spread issues, it has several key shortfalls. (1) It provides a one-time validation of equipment operation in which the standard documentation is a simple checklist of pass/fail questions. (2) It fails to evaluate equipment not included in the sample rate selection. During the construction and commissioning process of the new Research and Innovation Laboratory (RAIL) in Golden, CO, the National Renewable Energy Laboratory team engaged Group14 Engineering to implement a Connected Commissioning process using fault detection and diagnostic software for automated functional acceptance testing. The goal of this presentation will be to highlight the advantages offered by automated functional testing in this critical laboratory setting. These are primarily (1) sampling 100% of BAS-connected equipment during functional testing, (2) testing results backed by data beyond the traditional pass/fail checklist, and (3) an automated test process that can be regularly executed by the building management team for ongoing commissioning throughout the life of the building. The presentation will also cover technical challenges associated with connected commissioning and the important conversations with the key stakeholders that need to occur well before functional acceptance testing in order to successfully implement the automated testing processes.

automated testing↗
DOE OSTI2024

KLOKKOS: Automated test generation for performance-portable programs

SAND2024-13830O The KLOKKOS project provides a suite of test generation and validation tools to increase maintainability and portability of Kokkos programs. KLOKKOS, a portmanteau of Klee and Kokkos, is developing an automated test generation for performance-portable programs using Clang/low-level virtual machine (LLVM) and formal methods. KLOKKOS is based on several topics in computer science: the development of symbolic and concolic analysis of LLVM and C++ codes, formal semantics and type theory, C++ templates and language features, LLVM compiler infrastructure, and data parallel primitives. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525

SciDAC↗
DOE OSTI2025

Hardware-in-Loop Modules for Testing Automated Ventilator Controllers

Automated ventilator controllers have the potential to simplify oxygen and carbon dioxide management for trauma. In the pre-hospital or military medicine environment, trauma care can be required for prolonged periods by personnel with limited ventilator management training. As such, there is a need for closed-loop control systems that can adapt ventilator management to a complex, ever-changing medical environment. Here, we present a novel hardware-in-loop test platform for the independent troubleshooting and evaluation of oxygen and carbon dioxide automated ventilator management capabilities. The oxygen management system provides an analogue blood oxygen signal that is responsive to the fraction of inspired oxygen and the peak inspiratory pressure ventilator settings. A tested oxygenation controller successfully reached the target oxygen saturation within 5 min. The carbon dioxide removal system integrates with commercial ventilator technology and mimics carbon dioxide generation, lung compliance, and airway resistance while providing an end-tidal carbon dioxide level that is responsive to changes in the tidal volume and respiratory rate settings. A test mechanical ventilator controller was able to regulate EtCO2 regardless of the starting value within 10 min. This highlights the system’s functionality and provides proof-of-concept demonstrations for how the hardware-in-loop test platforms can be used for evaluating closed-loop controller technologies.

Berard, David (ORCID:0000000322863846)↗
DOE OSTI2025

Automating Testing of DUNE Electronics via a Finite State Machine

The Deep Underground Neutrino Experiment (DUNE) is a flagship international collaboration designed to study neutrinos tiny, nearly massless particles that may hold answers to fundamental questions about the Universe. Fermilab s Robotic Test Stand (RTS) plays a critical role in ensuring the quality of approximately 50,000 Application-Specific Integrated Circuit (ASIC) chips that will be used in DUNE s massive liquid argon detectors. These electronics will be inside the cryostat; therefore, they will need to have a high yield of working chips and low noise. To improve the automation and reliability of the RTS, this project focused on designing and implementing a Python-based finite state machine (FSM) to manage chip handling workflows. The FSM was developed as a modular software framework to coordinate robotic arm movements, manage chip tray positions, and monitor system states during testing. Key features include robust error handling routines, a pause/resume system for safe mid-cycle interruptions, and a simulation mode for iterative testing without hardware dependencies. The system was designed to prepare for seamless integration with RTS hardware components such as the robotic arm and vision system. This integration will streamline collaboration and enable efficient deployment of updates across the six total institutions performing testing. The outcomes of this internship contribute to Fermilab s mission to advance high-energy physics and support the DOE s national goals by directly improving the testing of equipment to be used in DUNE. The project also provided valuable experience in software design and contributing to the success of DUNE.

Kang, Caleb [William Rainey Harper Coll.]↗
DOE OSTI2025

Automating Testing of DUNE Electronics via a Finite State Machine

The Deep Underground Neutrino Experiment (DUNE) is a flagship international collaboration designed to study neutrinos—tiny, nearly massless particles that may hold answers to fundamental questions about the Universe. Fermilab’s Robotic Test Stand (RTS) plays a critical role in ensuring the quality of approximately 50,000 Application-Specific Integrated Circuit (ASIC) chips that will be used in DUNE’s massive liquid argon detectors. These electronics will be inside the cryostat; therefore, they will need to have a high yield of working chips and low noise. To improve the automation and reliability of the RTS, this project focused on designing and implementing a Python-based finite state machine (FSM) to manage chip handling workflows. The FSM was developed as a modular software framework to coordinate robotic arm movements, manage chip tray positions, and monitor system states during testing. Key features include robust error handling routines, a pause/resume system for safe mid-cycle interruptions, and a simulation mode for iterative testing without hardware dependencies. The system was designed to prepare for seamless integration with RTS hardware components such as the robotic arm and vision system. This integration will streamline collaboration and enable efficient deployment of updates across the six institutions performing testing. The outcomes of this internship contribute to Fermilab’s mission to advance high-energy physics and support the DOE’s national goals by directly improving the testing of equipment to be used in DUNE. The project also provided valuable experience in software design and contributing to the success of DUNE.

Kang, Caleb [Fermilab]↗
DOE OSTI2026

A Scalable Test-Bed for Direct Current Current Transformer Automated Functional Testing and Characterization

This article describes a scalable test-bed for automation of measurements of the direct current current transformers (DCCT). Automation of measurement involves functional measurements, fault event based measurements and data validation. The test-bed is designed to measure up to 20 DCCTs at a time with software based measurement repetition checks and validation of the measurements.

43 PARTICLE ACCELERATORS↗
DOE OSTI2021

Production and tests of the ADDC for the Micromegas detector of the ATLAS New Small Wheel

The Large Hadron Collider (LHC) at CERN will perform a series of upgrades to allow luminosity increases during the next physics runs expected from 2022 on. These will also significantly increase the trigger rates of all the detectors. As part of the ATLAS Phase-I upgrade, the current Small Wheel muon detectors will be replaced with state-of-the-art New Small Wheel (NSW) detectors to cope with the increased luminosity of the LHC. The Address in Real Time Data Driver Card (ADDC) is designed to transmit the trigger data from the resistive Micro-Mesh Gaseous Structure (Micromegas) detectors of the NSW. The ART ASIC on the ADDC is a customdesigned chip to receive theAddress in Real Time (ART) signals from theVMMfront-end ASIC and perform hit-selection processing. The processed trigger data is then sent to the Trigger Processor (TP) through optical links. A total of 512 ADDCs will be installed on the detector close to the front-end boards. Therefore, those cards must be able to work properly in a high radiation and magnetic field environment. Furthermore, after four rounds of prototyping, the ADDC production was launched in January of 2019. 600 ADDC boards have been produced and tested with an automated test stand. This test setup can simulate the front-end signals and provide "Level-1 Data Driver Card" (L1DDC) functions as well as part of the TP functions. Thus, the ADDC functionality and stability can be verified without the remaining NSW electronics. This paper describes the ADDC hardware prototype development, radiation and integration testing, and the ADDC automated production test procedure.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

ON THE EFFECTIVENESS OF LLMS IN UNIT TEST GENERATION FOR STRUCTURED TEXT PROGRAMS

The reliability of industrial automation systems heavily depends on the correctness of Programmable Logic Controller (PLC) programs, which are often written in Structured Text (ST). While Large Language Models (LLMs) have shown promise in automating test generation for mainstream programming languages, their effectiveness for the syntactically strict ST language remains underexplored. This thesis presents a systematic empirical evaluation of three state-of-the-art LLMs—GPT-4o, Gemini 2.5 Pro, and Claude Sonnet 4.5—for generating ST unit tests. We examine three prompting strategies: Natural Language (NL), Code Language (CL), and Chain-of-Thought (CoT), across a curated set of 11 ST function blocks. The quality of the generated tests is assessed using Compilation Success Rate (CSR), Statement Coverage (SC), and Branch Coverage (BC). In the zero-shot setting, Claude Sonnet 4.5 achieves the highest CSR, while Gemini 2.5 Pro consistently delivers the best statement and branch coverage, particularly under CL prompts. By incorporating a one-shot CL prompt, all models exhibit substantial improvements—most notably GPT-4o, whose CSR increases from 45.45% to 90.91%, with substantial gains in both SC and BC. To further contextualize these findings, we compare GPT-4o’s one-shot results with PLCAutoTester, a state-ofthe- art ST unit test generation tool, on an additional benchmark dataset. While LLMgenerated tests approach competitive coverage levels, PLCAutoTester maintains significantly higher and more stable coverage across programs. This study provides the first comprehensive benchmark of modern LLMs for ST unit testing, highlighting their strengths, limitations, and improvements through one-shot prompting, and positioning their performance relative to specialized automated testing tools in industrial automation.

42 ENGINEERING↗
DOE OSTI2022

kynema-fmb [SWR-23-07]

Kynema-FMB (FKA: Kynema) is an open-source performance portable flexible multibody (FMB) dynamics solver designed for time-domain simulations. While originally tailored for wind turbine structural dynamics, the formulation and implementation are those of a general flexible-multidbody dynamics solver that can readily be applied to a wide range of systems. Kynema was designed with a narrow focus, namely to provide a lightweight, fast, accurate FMD solver for coupling to computational-fluid-dynamics (CFD) codes, especially the CFD codes in the Kynema suite, for fluid-structure-interaction (FSI) simulations. Kynema-FMB is equipped to model systems that can be represented as a collection of beams and rigid bodies that are connected through constraints. Degrees of freedom are defined in the inertial/global frame of reference and include displacements and rotations (formally as rotation matrices, but stored as quaternions). The underlying formulation is built on a Lie-group time integrator designed for index-3 differential-algebraic equations, which is second-order accurate in time (Bruls et al., 2012). Beam models are based on geometrically exact beam theory and are discretized as high-order spectral finite elements similar to those in BeamDyn (Wang et al., 2017). The governing equations for a FMD system like a wind turbine constitute a highly nonlinear system of constrained partial-differential equations. Kynema-FMB uses analytical Jacobians in the nonlinear-system solves in each time step. Linear systems use sparse storage and several third-party sparse-linear-system solvers are enabled. Ill conditioning of linear systems is mitigated with preconditioning described in Bottasso et al, 2008. Kynema-FMB is integrated with a simple open-source controller (ROSCO). There is an application programming interface (API) for coupling to geometry-resolved CFD (like that in Sharma et al., 2023) and actuator-force CFD (like that in Kuhn et al., 2025). In the latter, for actuator-line models, Kynema-FMB includes an internal blade-element solver that depends on user-provided lookup tables for coefficients of lift and drag, i.e., aerodynamic polars. Kynema-FMB is written in C++ and leverages Kokkos and Kokkos-Kernels (KokkosEcosystem) as its performance portability layer enabling simulations on both CPU and GPU systems. The repository is equipped with extensive automated testing at the unit and regression/system levels. The following describes the high-level development objectives conceived for Kynema: *Kynema will follow modern software development best practices, including test-driven development (TDD), version control, hierarchical automated testing, and continuous integration (CI) for a robust development environment. *The core data structures are memory efficient and enable vectorization and parallelization at multiple levels. *Data structures are data-oriented to exploit methods for accelerated computing including high utilization of chip resources (e.g., single instruction multiple data (SIMD) instruction sets) and parallelization using GP-GPUs. *The computational algorithms incorporate robust open-source libraries for mathematical operations, resource allocation, and data management. *The API design considers multiple stakeholder needs and ensure integration with existing and future ecosystems for data science, machine learning, and AI. *Kynema-FMB is written in modern C++ and leverages Kokkos as its performance-portability library with inspiration from the kynema stack.

Sprague, MichaelA.↗