Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Software Test Tool”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

Longitudinal Aerodynamic Modeling of the Adaptive Compliant Trailing Edge Flaps on a GIII Airplane and Comparisons to Flight Data

A pair of compliant trailing edge flaps was flown on a modified GIII airplane. Prior to flight test, multiple analysis tools of various levels of complexity were used to predict the aerodynamic effects of the flaps. Vortex lattice, full potential flow, and full Navier-Stokes aerodynamic analysis software programs were used for prediction, in addition to another program that used empirical data. After the flight-test series, lift and pitching moment coefficient increments due to the flaps were estimated from flight data and compared to the results of the predictive tools. The predicted lift increments matched flight data well for all predictive tools for small flap deflections. All tools over-predicted lift increments for large flap deflections. The potential flow and Navier-Stokes programs predicted pitching moment coefficient increments better than the other tools.

aerodynamics↗

Assessment of Sensor Data Accuracy within Gazebo/ROS for High-Precision Autonomous In-Space Robotic Operations

Modeling high-precision in-space servicing, assembly, and manufacturing operations in a simulated environment is a critical step in the development of robotic systems that will be used to autonomously assemble large-scale structures in space. Limited facility size and high costs for manufacturing prototypes make it challenging to conduct full-scale operational testing under appropriate environmental conditions; therefore, testing in a modular, high-fidelity simulation environment is necessary for verification and validation of technology and architecture designs prior to launch. Several modeling and simulation environments exist both within NASA and industry that can be used to test robotic system design and operations, including the widely used commercial tool Gazebo integrated with Robotic Operating System software. Because the performance of autonomous robotic systems relies heavily on the quality of sensor input data, this paper focuses on assessing the accuracy of pose data from an optical sensor model in the Gazebo environment against the behavior of real hardware. The results of the tests will help developers using Gazebo for large-scale, high-precision simulation to account for modeling inaccuracies within their robotic control system algorithms.

simulation↗

A Lyapunov Function Based Remedial Action Screening Tool Using Real-Time Data

This report summarizes the outcome of a research project that comprised the development of a Lyapunov function based remedial action screening tool using real-time data (L-RAS). The L-RAS is an advanced computational tool that is intended to assist system operators in making real-time redispatch decisions to preserve power grid stability. The tool relies on screening contingencies using a homotopy method based on Lyapunov functions to avoid, to the extent possible, the use of time domain simulations. This enables transient stability evaluation at real-time speed without the use of massively parallel computational resources. The project combined the following components. 1. Development of a methodology for contingency screening using a homotopy method based on Lyapunov functions and real-time data. 2. Development of a methodology for recommending remedial actions based on the screening results. 3. Development of a visualization and operator interaction interface. 4. Testing of screening tool, validation of control actions, and demonstration of project outcomes on a representative real system simulated on a Real-Time Digital Simulator (RTDS) cluster. The project was led by Michigan State University (MSU), where the theoretical models including homotopy-based screening, trajectory correction using real-time data, and remedial action were developed and implemented in the form of research-grade software. Los Alamos National Laboratory (LANL) contributed to the development of energy margin sensitivity dynamics, which constituted a part of the remedial action portfolio. Florida State University (FSU) and Southern California Edison (SCE) developed a model of the SCE system that was implemented on FSU's RTDS cluster to simulate real-time data that was streamed over the internet to MSU where the L-RAS tool was executed and remedial actions were communicated back to FSU to execute stabilizing controls on the simulated system. LCG Consulting developed the visualization and operator interaction interface, based on specifications provided by MSU. The project was performed from October 2012 to December 2016, at the end of which the L-RAS tool, as described above, was completed and demonstrated. The project resulted in the following innovations and contributions: (a) the L-RAS software prototype, tested on a simulated system, vetted by utility personnel, and potentially ready for wider testing and commercialization; (b) an RTDS-based test bed that can be used for future research in the field; (c) a suite of breakthrough theoretical contributions to the field of power system stability and control; and (d) a new tool for visualization of power system stability margins. While detailed descriptions of the development and implementation of the various project components have been provided in the quarterly reports, this final report provides an overview of the complete project, and is demonstrated using public domain test systems commonly used in the literature. The SCE system, and demonstrations thereon, are not included in this report due to Critical Energy Infrastructure Information (CEII) restrictions.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Field Tested Service Oriented Robotic Architecture: Case Study

This paper presents the lessons learned from six years of experiments with planetary rover prototypes running the Service Oriented Robotic Architecture (SORA) developed by the Intelligent Robotics Group (IRG) at NASA Ames Research Center. SORA relies on proven software methods and technologies applied to the robotic world. Based on a Service Oriented Architecture and robust middleware, SORA extends its reach beyond the on-board robot controller and supports the full suite of software tools used during mission scenarios from ground control to remote robotic sites. SORA has been field tested in numerous scenarios of robotic lunar and planetary exploration. The results of these high fidelity experiments are illustrated through concrete examples that have shown the benefits of using SORA as well as its limitations.

Flueckiger, Lorenzo↗

CisLunar Habitat Internal Architecture Design Criteria

BACKGROUND: In preparation for human exploration to Mars, there is a need to define the development and test program that will validate deep space operations and systems. In that context, a Proving Grounds CisLunar habitat spacecraft is being defined as the next step towards this goal. This spacecraft will operate differently from the ISS or other spacecraft in human history. The performance envelope of this spacecraft (mass, volume, power, specifications, etc.) is being defined by the Future Capabilities Study Team. This team has recognized the need for a human-centered approach for the internal architecture of this spacecraft and has commissioned a CisLunar Phase-1 Habitat Internal Architecture Study Team to develop a NASA reference configuration, providing the Agency with a "smart buyer" approach for future acquisition. THE CISLUNAR HABITAT INTERNAL ARCHITECTURE STUDY: Overall, the CisLunar Habitat Internal Architecture study will address the most significant questions and risks in the current CisLunar architecture, habitation, and operations concept development. This effort is achieved through definition of design criteria, evaluation criteria and process, design of the CisLunar Habitat Phase-1 internal architecture, and the development and fabrication of internal architecture concepts combined with rigorous and methodical Human-in-the-Loop (HITL) evaluations and testing of the conceptual innovations in a controlled test environment. The vision of the CisLunar Habitat Internal Architecture Study is to design, build, and test a CisLunar Phase-1 Habitat Internal Architecture that will be used for habitation (e.g. habitability and human factors) evaluations. The evaluations will mature CisLunar habitat evaluation tools, guidelines, and standards, and will interface with other projects such as the Advanced Exploration Systems (AES) Program integrated Power, Avionics, Software (iPAS), and Logistics for integrated human-in-the-loop testing. The mission of the CisLunar Habitat Internal Architecture Study is to become a forcing function to establish a common understanding of CisLunar Phase-1 Habitation Internal Architecture design criteria, processes, and tools. The scope of the CisLunar Habitat Internal Architecture study is to design, develop, demonstrate, and evaluate a Phase-1 CisLunar Habitat common module internal architecture based on design criteria agreed to by NASA, the International Partners, and Commercial Exploration teams. This task is to define the CisLunar Phase-1 Internal Architecture Government Reference Design, assist NASA in becoming a "smart buyer" for Phase-1 Habitat Concepts, and ultimately to derive standards and requirements from the Internal Architecture Design Process. The first step was to define a Habitat Internal Architecture Design Criteria and create a structured philosophy to be used by design teams as a filter by which critical aspects of consideration would be identified for the purpose of organizing and utilizing interior spaces. With design criteria in place, the team will develop a series of iterative internal architecture concept designs which will be assessed by means of an evaluation criteria and process. These assessments will successively drive and refine the design, leading to the combination and down-selection of design concepts. A single refined reference design configuration will be developed into in a medium-to-high fidelity mockup. A multi-day human-in-the-loop mission test will fully evaluate the reference design and validate its configuration. Lessons learned from the design and evaluation will enable the team to identify appropriate standards for Phase-1 CisLunar Habitat Internal Architecture and will enable NASA to develop derived requirements in support of maturing CisLunar Habitation capabilities. This paper will describe the criteria definition process, workshop event, and resulting CisLunar Phase-1 Habitat Internal Architecture Design Criteria.

Jones, R.↗

On Fast Post-Processing of Global Positioning System Simulator Truth Data and Receiver Measurements and Solutions Data

Post-processing of data, related to a GPS receiver test in a GPS simulator and test facility, is an important step towards qualifying a receiver for space flight. Although the GPS simulator provides all the parameters needed to analyze a simulation, as well as excellent analysis tools on the simulator workstation, post-processing is not a GPS simulator or receiver function alone, and it must be planned as a separate pre-flight test program requirement. A GPS simulator is a critical resource, and it is desirable to move off the pertinent test data from the simulator as soon as a test is completed. The receiver and simulator databases are used to extract the test data files for postprocessing. These files are then usually moved from the simulator and receiver systems to a personal computer (PC) platform, where post-processing is done typically using PC-based commercial software languages and tools. Because of commercial software systems generality their functions are notoriously slow and more than often are the bottleneck even for short duration simulator-based tests. There is a need to do post-processing faster and within an hour after test completion, including all required operations on the simulator and receiver to prepare and move off the post-processing files. This is especially significant in order to use the previous test feedback for the next simulation setup or to run near back-to-back simulation scenarios. Solving the post-processing timing problem is critical for a pre-flight test program success. Towards this goal an approach was developed that allows to speed-up post-processing by an order of a magnitude. It is based on improving the post-processing bottleneck function algorithm using a priory information that is specific to a GPS simulation application and using only the necessary volume of truth data. The presented postprocessing scheme was used in support of a few successful space flight missions carrying GPS receivers.

Kizhner, Semion↗

Physics-informed Bayesian machine learning case study: Integral blade rotors

This paper provides a physics-informed Bayesian machine learning (PIBML) description and case study. The PIBML approach applies three physics-based models to establish the initial beliefs before testing to determine the probability of milling stability (or prior). These include: receptance coupling substructure analysis (RCSA) prediction for the tool tip frequency response functions; finite element software prediction of the mechanistic force model coefficients; and a spindle speed-dependent power law model for process damping. Testing was then performed to identify optimal stable machining conditions using an expected improvement in material removal rate criterion. The prior probability of stability was updated using the test results to determine the posterior probability of stability. The test results were compared to the parameter recommendations provided by the endmill manufacturer. A demonstration integral blade rotor was machined at the optimal stable machining conditions for 304 stainless steel and 6061-T6 aluminum. Finally, the disagreement between manufacturer recommendations and milling performance in both materials tested emphasizes the need for broad implementation of PIBML approaches to increase machining productivity and efficiency.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Code Verification Capabilities and Assessments in Support of ASC V&V Level 2 Milestone #6035

This document provides a summary of the code verification activities supporting the FY17 Level 2 V&V milestone entitled “Deliver a Capability for V&V Assessments of Code Implementations of Physics Models and Numerical Algorithms in Support of Future Predictive Capability Framework Pegposts.” The physics validation activities supporting this milestone are documented separately. The objectives of this portion of the milestone are: 1) Develop software tools to support code verification analysis; 2) Document standard definitions of code verification test problems; and 3) Perform code verification assessments (focusing on error behavior of algorithms). This report and a set of additional standalone documents serve as the compilation of results demonstrating accomplishment of these objectives.

97 MATHEMATICS AND COMPUTING↗

Best practices in software development for robust and reproducible geoscientific models based on insights from the Global Carbon Budget's dynamic vegetation models

Computational models play an increasingly vital role in scientific research by enabling the numerical simulation of complex processes. Such models are also fundamental in geosciences. For instance, they offer critical insights into the impacts of global change on the Earth system today and in the future. Beyond their value as research tools, models are also software products and should therefore adhere to certain established software engineering standards. However, scientists are rarely trained as software developers, which can lead to potential deficiencies in software quality like unreadable, inefficient, or erroneous code. The complexity of models, coupled with their integration into broader workflows, also often makes it challenging to reproduce results, evaluate processes, and build upon them. In this paper, we review the state and current practices of the development processes of the state-of-the-art land surface models used by the Global Carbon Budget. We combine the experience of modelers from the respective research groups with the expertise of software engineers from tech companies to outline key principles and tools for improving software quality in research. We explore four main areas: (1) model testing and validation, (2) scientific, technical, and user documentation, (3) version control, continuous integration, and code review, and (4) the portability and reproducibility of workflows. Our review reveals that while modeling communities are incorporating many best practices, significant room for improvement remains in areas such as automated testing, automated documentation, and reproducibility. Therefore, we here identify and promote essential software engineering practices, including numerous examples of practices from within the community that can serve as guidelines for other models and could help streamline processes across the entire community. We conclude with an open-source example implementation of these principles, demonstrating portable and reproducible data flows, a continuous integration setup, and web-based visualizations. This example may serve as a practical resource for model developers, users, and all scientists engaged in scientific programming.

Gregor, Konstantin [Technical Univ. of Munich (Ger↗

Sensor control of robot arc welding

A basic problem in the application of robots for welding which is how to guide a torch along a weld seam using sensory information was studied. Improvement of the quality and consistency of certain Gas Tungsten Arc welds on the Space Shuttle Main Engine (SSME) that are too complex geometrically for conventional automation and therefore are done by hand was examined. The particular problems associated with space shuttle main egnine (SSME) manufacturing and weld-seam tracking with an emphasis on computer vision methods were analyzed. Special interface software for the MINC computr are developed which will allow it to be used both as a test system to check out the robot interface software and later as a development tool for further investigation of sensory systems to be incorporated in welding procedures.

Sias, F. R., Jr.↗

Model-based reasoning for power system management using KATE and the SSM/PMAD

The overall goal of this research effort has been the development of a software system which automates tasks related to monitoring and controlling electrical power distribution in spacecraft electrical power systems. The resulting software system is called the Intelligent Power Controller (IPC). The specific tasks performed by the IPC include continuous monitoring of the flow of power from a source to a set of loads, fast detection of anomalous behavior indicating a fault to one of the components of the distribution systems, generation of diagnosis (explanation) of anomalous behavior, isolation of faulty object from remainder of system, and maintenance of flow of power to critical loads and systems (e.g. life-support) despite fault conditions being present (recovery). The IPC system has evolved out of KATE (Knowledge-based Autonomous Test Engineer), developed at NASA-KSC. KATE consists of a set of software tools for developing and applying structure and behavior models to monitoring, diagnostic, and control applications.

Morris, Robert A.↗

Key Decision Record Creation and Approval Module

Retaining good key decision records is critical to ensuring the success of a project or operation. Having adequately documented decisions with supporting documents and rationale can greatly reduce the amount of rework or reinvention over a project's, vehicle's, or facility's lifecycle. Stennis Space Center developed and uses a software tool that automates the Key Decision Record (KDR) process for its engineering and test projects. It provides the ability for a user to log key decisions that are made during the course of a project. By customizing Parametric Technology Corporation's (PTC) Windchill product, the team was able to log all information about a decision, and electronically route that information for approval. Customizing the Windchill product allowed the team to directly connect these decisions to the engineering data that it might affect and notify data owners of the decision. The user interface was created in JSP and Javascript, within the OOTB (Out of the Box) Windchill product, allowing users to create KDRs. Not only does this interface allow users to create and track KDRs, but it also plugs directly into the OOTB ability to associate these decision records with other relevant engineering data such as drawings, designs, models, requirements, or specifications

Hebert, Barrt↗

2006 NASA Range Safety Annual Report

Throughout 2006, Range Safety was involved in a number of exciting and challenging activities and events, from developing, implementing, and supporting Range Safety policies and procedures-such as the Space Shuttle Launch and Landing Plans, the Range Safety Variance Process, and the Expendable Launch Vehicle Safety Program procedures-to evaluating new technologies. Range Safety training development is almost complete with the last course scheduled to go on line in mid-2007. Range Safety representatives took part in a number of panels and councils, including the newly formed Launch Constellation Range Safety Panel, the Range Commanders Council and its subgroups, the Space Shuttle Range Safety Panel, and the unmanned aircraft systems working group. Space based range safety demonstration and certification (formerly STARS) and the autonomous flight safety system were successfully tested. The enhanced flight termination system will be tested in early 2007 and the joint advanced range safety system mission analysis software tool is nearing operational status. New technologies being evaluated included a processor for real-time compensation in long range imaging, automated range surveillance using radio interferometry, and a space based range command and telemetry processor. Next year holds great promise as we continue ensuring safety while pursuing our quest beyond the Moon to Mars.

TenHaken, Ron↗

Implementation and test of an automated control hunting fault correction algorithm in a fault detection and diagnostics tool

Control hunting due to improper proportional–integral–derivative (PID) parameters in the building automation system (BAS) is one of the most common faults identified in commercial buildings. It can cause suboptimal performance and early failure of heating, ventilation, and air conditioning (HVAC) equipment. Commercial fault detection and diagnostics (FDD) software represents one of the fastest growing market segments in smart building technologies in the United States. Implementation of PID retuning procedures as an auto-correction algorithm and integration into FDD software has the potential to mitigate control hunting across a heterogeneous portfolio of buildings with different BAS in a scalable way. This paper presents the development, implementation, and field testing of an automated control hunting fault correction algorithm based on lambda tuning open-loop rules. The algorithm was developed in a commercial FDD software and successfully tested among nine variable air volume boxes in an office building in the United States. The paper shows the feasibility of using FDD tools to automatically correct control hunting faults, discusses scalability considerations, and proposes a path forward for the HVAC industry and academia to further improve this technology.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Verification Tools Secure Online Shopping, Banking

Just like rover or rocket technology sent into space, the software that controls these technologies must be extensively tested to ensure reliability and effectiveness. Ames Research Center invented the open-source Java Pathfinder (JPF) toolset for the deep testing of Java-based programs. Fujitsu Labs of America Inc., based in Sunnyvale, California, improved the capabilities of the JPF Symbolic Pathfinder tool, establishing the tool as a means of thoroughly testing the functionality and security of Web-based Java applications such as those used for Internet shopping and banking.

Source record↗

Increasing Software Testing Coverage and Portability with Spack

Adequate testing coverage is critical for successful scientific software development. For numerically intensive codes, the test matrix should cover a variety of machines, compilers, third-party library dependencies, as well as possible other features such as memory testing, parallelization paradigms, and application options that may be enabled or disabled, as some examples. Such coverage can be a major burden on developers without powerful methods for managing the matrix. Spack is a great tool developed by Lawrence Livermore National Lab for managing software on a Unix-based system and in addition to this, we are leveraging Spack to simplify application testing development and management. Here we demonstrate exploiting many of Spack's features to orchestrate the test coverage for the Nalu-Wind application we are developing under the Exascale Computing Project. By using Spack underneath our testing infrastructure, we have increased our testing coverage, portability, robustness, and software quality. While we use our Nalu-Wind application as an example, the benefits of exploiting Spack for application testing can be had for other applications as well.

97 MATHEMATICS AND COMPUTING↗

AFUE Analysis Software Tool

The Annual Fuel Utilization Efficiency (AFUE) analysis of residential and light commercial furnaces follows ANSI/ASHRAE Standard 103-2017 (i.e., Method of Testing for Annual Fuel Utilization Efficiency of Residential Central Furnaces and Boilers) . The analysis is a complex, comprehensive method based on furnace configuration and specific components equipped, requiring detailed furnace testing and measurement data. For this reason, an AFUE Analysis Tool using Microsoft Excel enabled with Visual Basic for Applications (VBA) was developed with a user-friendly interface and comprehensive coverage. The tool consists of three worksheets: unit and configuration selection, geometry and measurement data input, and AFUE plus key results. This tool can be used to estimate the AFUE of both condensing and noncondensing furnaces with single-stage, two-stage, and step-modulating functions. The tool was validated with experimental data from Oak Ridge National Laboratory’s natural gas furnace projects that are commercially available. The results indicate the tool is reasonably accurate in the evaluation of a new R&D modified furnace unit.

Gao, Zhiming [Oak Ridge National Laboratory (ORNL)↗

ON THE EFFECTIVENESS OF LLMS IN UNIT TEST GENERATION FOR STRUCTURED TEXT PROGRAMS

The reliability of industrial automation systems heavily depends on the correctness of Programmable Logic Controller (PLC) programs, which are often written in Structured Text (ST). While Large Language Models (LLMs) have shown promise in automating test generation for mainstream programming languages, their effectiveness for the syntactically strict ST language remains underexplored. This thesis presents a systematic empirical evaluation of three state-of-the-art LLMs—GPT-4o, Gemini 2.5 Pro, and Claude Sonnet 4.5—for generating ST unit tests. We examine three prompting strategies: Natural Language (NL), Code Language (CL), and Chain-of-Thought (CoT), across a curated set of 11 ST function blocks. The quality of the generated tests is assessed using Compilation Success Rate (CSR), Statement Coverage (SC), and Branch Coverage (BC). In the zero-shot setting, Claude Sonnet 4.5 achieves the highest CSR, while Gemini 2.5 Pro consistently delivers the best statement and branch coverage, particularly under CL prompts. By incorporating a one-shot CL prompt, all models exhibit substantial improvements—most notably GPT-4o, whose CSR increases from 45.45% to 90.91%, with substantial gains in both SC and BC. To further contextualize these findings, we compare GPT-4o’s one-shot results with PLCAutoTester, a state-ofthe- art ST unit test generation tool, on an additional benchmark dataset. While LLMgenerated tests approach competitive coverage levels, PLCAutoTester maintains significantly higher and more stable coverage across programs. This study provides the first comprehensive benchmark of modern LLMs for ST unit testing, highlighting their strengths, limitations, and improvements through one-shot prompting, and positioning their performance relative to specialized automated testing tools in industrial automation.

42 ENGINEERING↗