SEARCH · Engineering Papers

Results for “Software Test Tool”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

All sources NASA NTRS DOE OSTI

DOE OSTI2025

ON THE EFFECTIVENESS OF LLMS IN UNIT TEST GENERATION FOR STRUCTURED TEXT PROGRAMS

The reliability of industrial automation systems heavily depends on the correctness of Programmable Logic Controller (PLC) programs, which are often written in Structured Text (ST). While Large Language Models (LLMs) have shown promise in automating test generation for mainstream programming languages, their effectiveness for the syntactically strict ST language remains underexplored. This thesis presents a systematic empirical evaluation of three state-of-the-art LLMs—GPT-4o, Gemini 2.5 Pro, and Claude Sonnet 4.5—for generating ST unit tests. We examine three prompting strategies: Natural Language (NL), Code Language (CL), and Chain-of-Thought (CoT), across a curated set of 11 ST function blocks. The quality of the generated tests is assessed using Compilation Success Rate (CSR), Statement Coverage (SC), and Branch Coverage (BC). In the zero-shot setting, Claude Sonnet 4.5 achieves the highest CSR, while Gemini 2.5 Pro consistently delivers the best statement and branch coverage, particularly under CL prompts. By incorporating a one-shot CL prompt, all models exhibit substantial improvements—most notably GPT-4o, whose CSR increases from 45.45% to 90.91%, with substantial gains in both SC and BC. To further contextualize these findings, we compare GPT-4o’s one-shot results with PLCAutoTester, a state-ofthe- art ST unit test generation tool, on an additional benchmark dataset. While LLMgenerated tests approach competitive coverage levels, PLCAutoTester maintains significantly higher and more stable coverage across programs. This study provides the first comprehensive benchmark of modern LLMs for ST unit testing, highlighting their strengths, limitations, and improvements through one-shot prompting, and positioning their performance relative to specialized automated testing tools in industrial automation.

42 ENGINEERING↗

NASA NTRS2012

Analytical and Experimental Evaluation of Digital Control Systems for the Semi-Span Super-Sonic Transport (S4T) Wind Tunnel Model

An important objective of the Semi-Span Super-Sonic Transport (S4T) wind tunnel model program was the demonstration of Flutter Suppression (FS), Gust Load Alleviation (GLA), and Ride Quality Enhancement (RQE). It was critical to evaluate the stability and robustness of these control laws analytically before testing them and experimentally while testing them to ensure safety of the model and the wind tunnel. MATLAB based software was applied to evaluate the performance of closed-loop systems in terms of stability and robustness. Existing software tools were extended to use analytical representations of the S4T and the control laws to analyze and evaluate the control laws prior to testing. Lessons were learned about the complex windtunnel model and experimental testing. The open-loop flutter boundary was determined from the closed-loop systems. A MATLAB/Simulink Simulation developed under the program is available for future work to improve the CPE process. This paper is one of a series of that comprise a special session, which summarizes the S4T wind-tunnel program.

Wieseman, Carol D.↗

NASA NTRS1992

Software-Engineering Process Simulation (SEPS) model

The Software Engineering Process Simulation (SEPS) model is described which was developed at JPL. SEPS is a dynamic simulation model of the software project development process. It uses the feedback principles of system dynamics to simulate the dynamic interactions among various software life cycle development activities and management decision making processes. The model is designed to be a planning tool to examine tradeoffs of cost, schedule, and functionality, and to test the implications of different managerial policies on a project's outcome. Furthermore, SEPS will enable software managers to gain a better understanding of the dynamics of software project development and perform postmodern assessments.

Lin, C. Y.↗

DOE OSTI2016

Methodology to Assess No Touch Audit Software Using Field Data

The research presented in this report builds upon these previous efforts and proposes a set of tests to assess no touch audit tools using real utility bill and on-site data. The proposed assessment methodology explicitly investigates the behaviors of the monthly energy end uses with respect to outdoor temperature, i.e., the building energy signature, to help understand the Tool's disaggregation accuracy. The project team collaborated with Field Diagnosis Services, Inc. (FDSI) to identify appropriate test sites for the evaluation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NASA NTRSDate not supplied

Supporting Exploration Missions by Enabling Exploration Mission System Software

Future exploration missions will consist of a multitude of data sources, systems, and operators collaborating to complete mission objectives. Presently, NASA is instantiating the contractual mechanisms, such as the xEVAS and HLS contracts, to produce these mission assets. Architectural planning is also underway to establish the networking protocols and infrastructure to digitally create and connect mission elements, such as LunaNET. However, without new horizontally integrated data systems, these advancements will be limited in their ability to get mission data appropriately integrated into the plan, train, fly, explore workflow of the operations workforce. Here we describe several mission system software development efforts underway that are designed to support human spaceflight missions. We describe the current iterations of a suite of tools to support EVA procedure authoring and execution, for both ISS and Artemis missions, as well as a software solution to establish and interact with mission context and data products. These tools have been developed iteratively and continue to be tested in several NASA facilities such as the Neutral Buoyancy Lab (NBL), Artemis field testing, and in present-day International Space Station (ISS) operations on orbit. Our solutions demonstrate how software development can be aligned with ongoing operations development activities to discover the features that best support future human spaceflight missions.

EVA Mission System Software↗

NASA NTRS2005

Hardware in-the-Loop Demonstration of Real-Time Orbit Determination in High Earth Orbits

This paper presents results from a study conducted at Goddard Space Flight Center (GSFC) to assess the real-time orbit determination accuracy of GPS-based navigation in a number of different high Earth orbital regimes. Measurements collected from a GPS receiver (connected to a GPS radio frequency (RF) signal simulator) were processed in a navigation filter in real-time, and resulting errors in the estimated states were assessed. For the most challenging orbit simulated, a 12 hour Molniya orbit with an apogee of approximately 39,000 km, mean total position and velocity errors were approximately 7 meters and 3 mm/s respectively. The study also makes direct comparisons between the results from the above hardware in-the-loop tests and results obtained by processing GPS measurements generated from software simulations. Care was taken to use the same models and assumptions in the generation of both the real-time and software simulated measurements, in order that the real-time data could be used to help validate the assumptions and models used in the software simulations. The study makes use of the unique capabilities of the Formation Flying Test Bed at GSFC, which provides a capability to interface with different GPS receivers and to produce real-time, filtered orbit solutions even when less than four satellites are visible. The result is a powerful tool for assessing onboard navigation performance in a wide range of orbital regimes, and a test-bed for developing software and procedures for use in real spacecraft applications.

Moreau, Michael↗

NASA NTRS2023

CST: A Tool for Optimizing the Efficiency and Effectiveness of Static-Code Analysis Tools

Static Code Analysis (SCA) is a vital component of NASA IV&V’s mission assurance for safety-critical software as it reduces the likelihood of software-induced hazards impacting mission success. Static Code Analysis achieves this by identifying hazards that may not have been otherwise detectable by typical code reviews or other testing. Using SCA tools, however, can be intimidating due to steep learning curves, especially considering tool performance and defect coverage varies greatly. Because of this variation amongst SCA tools, understanding which tools support certain defects and which do not, as well as understanding how to run an analysis based on steps that are unique to each tool, can be difficult to both new and experienced analysts alike. To mitigate this, the SCAWG or the IV&V Static Code Analysis Working Group, created the SCA Checker Taxonomy and Starting Point Profiles. The Checker Selection Tool (CST) incorporates these two SCAWG products into an interactive tool which allows the user to: select organized categories of defects they would like the SCA tools to discover, select default checkers depending on their mission type (e.g. flight), and configure multiple SCA tools at once. C/C++, Java, and Python defect checkers from four common SCA tools were utilized in this iteration of the CST. This iteration also includes the addition of training, SCA tool specific help, and taxonomy guide links, into its design to help users new to Static Code Analysis learn how to perform SCA more efficiently. The CST has been subject to beta testing by experienced static code analysts from the SCAWG to ensure a usable and accurate final product. The implications of the CST in the mission assurance of NASA safety-critical software are profound, as the CST can help identify and reduce false positives and false negatives, fundamentally improving overall SCA efficiency and accuracy.

static code analysis↗

NASA NTRS2011

Mars Science Laboratory Flight Software Internal Testing

The Mars Science Laboratory (MSL) team is sending the rover, Curiosity, to Mars, and therefore is physically and technically complex. During my stay, I have assisted the MSL Flight Software (FSW) team in implementing functional test scripts to ensure that the FSW performs to the best of its abilities. There are a large number of FSW requirements that have been written up for implementation; however I have only been assigned a few sections of these requirements. There are many stages within testing; one of the early stages is FSW Internal Testing (FIT). The FIT team can accomplish this with simulation software and the MSL Test Automation Kit (MTAK). MTAK has the ability to integrate with the Software Simulation Equipment (SSE) and the Mission Processing and Control System (MPCS) software which makes it a powerful tool within the MSL FSW development process. The MSL team must ensure that the rover accomplishes all stages of the mission successfully. Due to the natural complexity of this project there is a strong emphasis on testing, as failure is not an option. The entire mission could be jeopardized if something is overlooked.

entry, descent, and landing (EDL)↗

NASA NTRS1976

Man-rated flight software for the F-8 DFBW program

The design, implementation, and verification of the flight control software used in the F-8 DFBW program are discussed. Since the DFBW utilizes an Apollo computer and hardware, the procedures, controls, and basic management techniques employed are based on those developed for the Apollo software system. Program assembly control, simulator configuration control, erasable-memory load generation, change procedures and anomaly reporting are discussed. The primary verification tools are described, as well as the program test plans and their implementation on the various simulators. Failure effects analysis and the creation of special failure generating software for testing purposes are described.

Bairnsfather, R. R.↗

DOE OSTI2013

Deep in Data: Empirical Data Based Software Accuracy Testing Using the Building America Field Data Repository: Preprint

An opportunity is available for using home energy consumption and building description data to develop a standardized accuracy test for residential energy analysis tools. That is, to test the ability of uncalibrated simulations to match real utility bills. Empirical data collected from around the United States have been translated into a uniform Home Performance Extensible Markup Language format that may enable software developers to create translators to their input schemes for efficient access to the data. This may facilitate the possibility of modeling many homes expediently, and thus implementing software accuracy test cases by applying the translated data. This paper describes progress toward, and issues related to, developing a usable, standardized, empirical data-based software accuracy test suite.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2015

Building energy simulation in real time through an open standard interface

Building energy models (BEMs) are typically used for design and code compliance for new buildings and in the renovation of existing buildings to predict energy use. We present the increasing adoption of BEM as standard practice in the building industry presents an opportunity to extend the use of BEMs into construction, commissioning and operation. In 2009, the authors developed a real-time simulation framework to execute an EnergyPlus model in real time to improve building operation. This paper reports an enhancement of that real-time energy simulation framework. The previous version only works with software tools that implement the custom co-simulation interface of the Building Controls Virtual Test Bed (BCVTB), such as EnergyPlus, Dymola and TRNSYS. The new version uses an open standard interface, the Functional Mockup Interface (FMI), to provide a generic interface to any application that supports the FMI protocol. In addition, the new version utilizes the Simple Measurement and Actuation Profile (sMAP) tool as the data acquisition system to acquire, store and present data. Lastly, this paper introduces the updated architecture of the real-time simulation framework using FMI and presents proof-of-concept demonstration results which validate the new framework.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2012

n-dimensional Statistical Inverse Graphical Hydraulic Test Simulator

nSIGHTS (n-dimensional Statistical Inverse Graphical Hydraulic Test Simulator) is a comprehensive well test analysis software package. It provides a user-interface, a well test analysis model and many tools to analyze both field and simulated data. The well test analysis model simulates a single-phase, one-dimensional, radial/non-radial flow regime, with a borehole at the center of the modeled flow system. nSIGHTS solves the radially symmetric n-dimensional forward flow problem using a solver based on a graph-theoretic approach. The results of the forward simulation are pressure, and flow rate, given all the input parameters. The parameter estimation portion of nSIGHTS uses a perturbation-based approach to interpret the best-fit well and reservoir parameters, given an observed dataset of pressure and flow rate.

Kuhlman, KristopherL↗

DOE OSTI2015

MOOSE IPL Extensions (Control Logic)

In FY-2015, the development of MOOSE was driven by the needs of the NEAMS MOOSE-based applications, BISON, MARMOT, and RELAP-7. An emphasis was placed on the continued upkeep and improvement MOOSE in support of the product line integration goals. New unified documentation tools have been developed, several improvements to regression testing have been enforced and overall better software quality practices have been implemented. In addition the Multiapps and Transfers systems have seen significant refactoring and robustness improvements, as has the “Restart and Recover” system in support of Multiapp simulations. Finally, a completely new “Control Logic” system has been engineered to replace the prototype system currently in use in the RELAP-7 code. The development of this system continues and is expected to handle existing needs as well as support future enhancements.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2017

MOOSE IPL Extensions (Documentation)

In FY-2015, The development of MOOSE was driven by the needs of the NEAMS MOOSE-based applications, BISON, MARMOT, and RELAP-7. An emphasis was placed on the continued upkeep and improvement MOOSE in support of the product line integration goals. New unified documentation tools have been developed, several improvements to regression testing have been enforced and overall better software quality practices have been implemented. In addition the Multiapps and Transfers systems have seen significant refactoring and robustness improvements, as has the “Restart and Recover” system in support of Multiapp simulations. Finally, a completely new “Control Logic” system has been engineered to replace the prototype system currently in use in the RELAP-7 code. The development of this system continues and is expected to handle existing needs as well as support future enhancements.

99 GENERAL AND MISCELLANEOUS↗

NASA NTRS1998

Workstation-Based Simulation for Rapid Prototyping and Piloted Evaluation of Control System Designs

The development and optimization of flight control systems for modem fixed- and rotary-. wing aircraft consume a significant portion of the overall time and cost of aircraft development. Substantial savings can be achieved if the time required to develop and flight test the control system, and the cost, is reduced. To bring about such reductions, software tools such as Matlab/Simulink are being used to readily implement block diagrams and rapidly evaluate the expected responses of the completed system. Moreover, tools such as CONDUIT (CONtrol Designer's Unified InTerface) have been developed that enable the controls engineers to optimize their control laws and ensure that all the relevant quantitative criteria are satisfied, all within a fully interactive, user friendly, unified software environment.

Mansur, M. Hossein↗

NASA NTRS2019

Application of Machine Learning Techniques to Delay Tolerant Network Routing

This dissertation discusses several machine learning techniques to improve routing in delay tolerant networks (DTNs). These are networks in which there may be long one-way trip times, asymmetric links, high error rates, and deterministic as well as non-deterministic loss of contact between network nodes, such as interplanetary satellite networks, mobile ad hoc networks and wireless sensor networks. This work uses historical network statistics to train a multi-label classifier to predict reliable paths through the network. In addition, a clustering technique is used to predict future mobile node locations. Both of these techniques are used to reduce the consumption of resources such as network bandwidth, memory and data storage that is required by replication routing methods often used in opportunistic DTN environments. Thesis contributions include: an emulation tool chain developed to create a DTN test bed for machine learning, the network and software architecture for a machine learning based routing method, the development and implementation of classification and clustering techniques and performance evaluation in terms of machine learning and routing metrics.

Dudukovich, Rachel M.↗

DOE OSTI2012

n-dimensional Statistical Inverse Graphical Hydraulic Test Simulator

Source record↗

DOE OSTI2020

Machine Learning for Automated Extraction of Building Geometry

As data science comes to buildings, the promise of using machine learning and novel sources of data has received much attention. Advances in machine learning and computer vision algorithms, combined with increased access to unstructured data (e.g., images and text), have created an opportunity for automated extraction of building characteristics – cost-effectively, and at scale. Acquisition of features such as footprint are time consuming and costly to acquire with today’s manual methods, but can be streamlined through intelligent software-based solutions applied to satellite images. When combined with aerial RGB and thermal images, full 3D geometries and thermal maps can be constructed to determine additional characteristics such as window to wall ratio, height, number of stories and envelope thermal characteristics. In this paper we present three contributions to accelerate these high potential opportunities: (1) a methodical analysis of how these features can be integrated into today’s simulation and data driven software tools to enhance efficiency measure identification and owner/operator decision making; (2) development and accuracy testing of open source deep neural network methods to extract building footprints from satellite imagery, including the curation and application of openly available GIS datasets for training and continued development by others; and (3) an open framework for drone-based image capture and creation of 3D building geometries. This work represents an important bridge between high-level studies that span diverse application areas and those that detail point solutions yet cannot be easily replicated or extended.

Touzani, Samir↗