SEARCH · Engineering Papers

Results for “Software Test Report”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Fiscal Year 2023 Software Quality Assurance Activities for the ARC Software

The Argonne Reactor Code (ARC) software suite [1-17] has been developed by Argonne researchers for fast reactor design and analysis since the 1970s. With the ARC software suite, a user can quickly build a model of a proposed or existing fast spectrum reactor and carry out fuel cycle, nominal thermal analysis and flow requirements, and assess, as is appropriate, whether the core design and constraint system yield an acceptable mechanical behavior. For transient reactor analysis with SAS4A [18], the ARC software suite can be used to generate reactivity coefficients and kinetics parameters at any modeled fuel cycle time point which forms part of the input to SAS4A. The ARC suite was consistently being developed until the 1990s and followed a software QA program which was an appropriate standard for the time. In the 1990s, the DOE funding to fast reactor research and development was all but eliminated and the ARC software was put into maintenance mode. In the early 2000s, the software quality assurance (SQA) program for ARC was still in place to define an official version, but by 2005 it all but was abandoned as there were insufficient staff to fill the work roles. Since 2005, there has been a considerable increase in research and design work on fast spectrum reactors. The ARC software as a whole has since been exported to many universities and commercial companies and ANL support has been given to the various projects over the years [19-23]. Further, MC 2 -3, PERSENT, and DASSH were all developed after 2005 without any adherence to a software standard. In recent time, the DOE VTR project [22] paid for verification work to be done on the ARC software as part of the goal of making it NQA-1 complaint. The VTR project was not considered the appropriate pathway to fund and maintain a SQA program for the ARC software and while software developments (DASSH) were made and several manuals were updated and software verification work was carried out, the ARC software is not NQA-1 compliant. More recently the Advanced Reactor Development Program (ARDP [23]) has funded the creation of manuals for some ARC utility programs and funded additional software verification work on DIF3D [6, 7] and MC 2 -3 [2-5] for the purpose of commercial grade dedication. Because of the VTR and ARDP projects, software verification work was completed on MC 2 -3 and DIF3D, and detailed reports were created for each piece of software, which discuss the inputs and outputs from the codes that are covered by the verification work and link various analytic, code-to-code, and hand calculation based verification work presented in the report with verification test problems provided with the software. This is a key part of the commercial grade dedication work and constitutes the bulk of the cost to get the ARC software to commercial grade. The ARC software suite is a valuable asset as a fast reactor design and analysis tool set that has been reasonably well verified and validated with various fast reactor benchmark problems and experiments over decades. Some or all of the ARC software suite has been utilized for designing the IFR [20], PGSFR [21], VTR [22], and Natrium [23] reactors and we can expect it to continue to be used for advanced fast reactor design and/or confirmatory calculation purposes in the future. Due to increased interest by commercial companies and regulatory bodies, it is becoming more important to make the ARC software suite complete and ready-to-use in terms of its SQA pedigree and commercial grade dedication needs. This report discusses the achievements made towards building a new SQA program for the ARC software and dealing with outstanding identified QA gaps.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

FORCE Regression Testing

Via programs including the Light Water Reactor Sustainability and Integrated Energy Systems, the U.S. Department of Energy has invested in the Framework for Optimization of ResourCes and Economics (FORCE) software framework (Idaho National Laboratory 2024a) for the technical and economic analysis of nuclear-integrated energy systems (IES). Nuclear IES expand the use of nuclear from traditional baseload electricity generation to a flexible and adaptive source of combined heat and power. Nuclear heat can be used in the production of a variety of energy currencies such as hydrogen and ammonia as well as other heat applications including water desalination and district heating. FORCE is designed with the intent to provide interconnected analysis tools that enable the accurate technical and economic assessment of specific nuclear IES configurations for individual energy markets. FORCE consists of three main analysis pathways: HYBRID (Idaho National Laboratory 2024b), which contains high-resolution physical models for IES; Holistic Energy Resource Optimization Network (HERON) (Idaho National Laboratory 2024c), which analyzes IES long-term economic viability; and Optimization of Real-time Capacity Allocation (ORCA) (Idaho National Laboratory 2024d), designed for real-time control of IES via digital twins and optimal decision making, including autonomous and remote operation research. Development of the FORCE ecosystem is guided by three pillars: capability, which assures that the computational requirements of IES analysis are met by the software tools; reliability, which provides for consistent code performance and expected behaviors; and accessibility, which lowers the barrier to entry for using the software and accelerates analysis by users beyond the FORCE primary developers. Reliability of the FORCE ecosystem is established according to the American Nuclear Society?s Nuclear Quality Assurance (NQA-1) program [American Society of Mechanical Engineers 1982], with specific levels of software quality assurance (SQA) within NQA-1 applied to each software tool in FORCE. As the tools within FORCE have matured, some integration algorithms to accurately connect the software tools for holistic analysis have been developed and deployed within the FORCE software repository. In accordance with NQA-1 standards, regression tests are required to guarantee the software performs consistently even when new capabilities are added to the software. In this report, we document the deployment of both unit tests, which test the consistent behavior of small pieces of the FORCE code base, as well as integration tests, which test the consistent performance of full use cases for the FORCE integration algorithms. We further document the encapsulation of these tests within a test harness, which collectively checks for each successful test completion on demand. Finally, we document the automation of the test harness using GitHub Actions [GitHub 2024], which require all tests succeed before any new capability or other changes can be added to the FORCE integration software

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2019

Command and Control System Software Development

With the first launch of the National Aeronautics and Space Administration's Space Launch System heavy-lift expendable launch vehicle and Lockheed Martin's Orion Multi-Purpose Crew Vehicle scheduled for the year 2020, there exists a need to complete development of a new command and control system that will provide systems monitoring and launch control for NASA's Exploration Missions. One remaining task necessary for completion of this command and control system is to create and maintain comprehensive unit tests of the control system software packages. These tests should verify that the implementation of all required and desired functionality works as intended. This testing infrastructure is mostly in place, but the control system's open source automation server still reports software "bugs" (possible flaws or failures which may lead to unintended behavior) and intermittently failing unit tests. Since code correctness is of critical importance for human rated software systems, I was assigned to diagnose the root cause of failing unit tests, eliminate non-determinism in these tests, and fix bugs as reported by the automation server.

GUI↗

NASA NTRS2005

Comparison of Fiber Optic Strain Demodulation Implementations

NASA Langley Research Center is developing instrumentation based upon principles of Optical Frequency-Domain Reflectometry (OFDR) for the provision of large-scale, dense distribution of strain sensors using fiber optics embedded with Bragg gratings. Fiber Optic Bragg Grating technology enables the distribution of thousands of sensors immune to moisture and electromagnetic interference with negligible weight penalty. At Langley, this technology provides a key component for research and development relevant to comprehensive aerospace vehicle structural health monitoring. A prototype system is under development that includes hardware and software necessary for the acquisition of data from an optical network and conversion of the data into strain measurements. This report documents the steps taken to verify the software that implements the algorithm for calculating the fiber strain. Brief descriptions of the strain measurement system and the test article are given. The scope of this report is the verification of software implementations as compared to a reference model. The algorithm will be detailed along with comparison results.

Quach, Cuong C.↗

DOE OSTI2023

Active High Assurance Authentication Protocol (AHAAP)

The AHAAP Maturation Project involves maturation and evaluation of a patented zero-trust tamper-resistant high-assurance session-less dynamic and active device authentication protocol that simultaneously authenticates identity and provides integrity verification in a single step, substantially reducing the risk of cyberattack, and eliminating the need for costly and complex conventional communication security systems requirements (i.e., cryptography, Public Key Infrastructure (PKI), and key management). These cybersecurity attributes of the technology must be preserved when applying the technology to different cybersecurity solutions, including Command & Control (C&C), Over-the-Air (OTA) update, Common Access Card (CAC), and distributed energy resource (DER) implementations, among others. The technology research objective is to test and verify that the cybersecurity attributes of the technology are not degraded in different cybersecurity applications. The primary technology development objective is to build minimum viable products to demonstrate the technology addresses today’s cybersecurity threats so that prospective investors, strategic partners, regulatory agencies, and commercial customers can interact with and assess the protection assured by the technology. The AHAAP Maturation Project goal is to develop, test, and validate one or more AHAAP implementations. The AHAAP Maturation Project tasks are: (i) engineer AHAAP implementation software, (ii) build a functional prototype that implements the AHAAP software for demonstration, testing, analysis, and evaluation purposes, and (iii) generate a report detailing the results of the AHAAP C&C software and hardware implementation. The final project deliverables are: (i) AHAAP software implementation and prototype, (ii) a report from Sandia National Laboratories detailing the results of the AHAAP implementations.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1988

An experiment in software reliability: Additional analyses using data from automated replications

A study undertaken to collect software error data of laboratory quality for use in the development of credible methods for predicting the reliability of software used in life-critical applications is summarized. The software error data reported were acquired through automated repetitive run testing of three independent implementations of a launch interceptor condition module of a radar tracking problem. The results are based on 100 test applications to accumulate a sufficient sample size for error rate estimation. The data collected is used to confirm the results of two Boeing studies reported in NASA-CR-165836 Software Reliability: Repetitive Run Experimentation and Modeling, and NASA-CR-172378 Software Reliability: Additional Investigations into Modeling With Replicated Experiments, respectively. That is, the results confirm the log-linear pattern of software error rates and reject the hypothesis of equal error rates per individual fault. This rejection casts doubt on the assumption that the program's failure rate is a constant multiple of the number of residual bugs; an assumption which underlies some of the current models of software reliability. data raises new questions concerning the phenomenon of interacting faults.

Dunham, Janet R.↗

NASA NTRS2010

Random Vibrations

Ares I Zonal Random vibration environments due to acoustic impingement and combustion processes are develop for liftoff, ascent and reentry. Random Vibration test criteria for Ares I Upper Stage pyrotechnic components are developed by enveloping the applicable zonal environments where each component is located. Random vibration tests will be conducted to assure that these components will survive and function appropriately after exposure to the expected vibration environments. Methodology: Random Vibration test criteria for Ares I Upper Stage pyrotechnic components were desired that would envelope all the applicable environments where each component was located. Applicable Ares I Vehicle drawings and design information needed to be assessed to determine the location(s) for each component on the Ares I Upper Stage. Design and test criteria needed to be developed by plotting and enveloping the applicable environments using Microsoft Excel Spreadsheet Software and documenting them in a report Using Microsoft Word Processing Software. Conclusion: Random vibration liftoff, ascent, and green run design & test criteria for the Upper Stage Pyrotechnic Components were developed by using Microsoft Excel to envelope zonal environments applicable to each component. Results were transferred from Excel into a report using Microsoft Word. After the report is reviewed and edited by my mentor it will be submitted for publication as an attachment to a memorandum. Pyrotechnic component designers will extract criteria from my report for incorporation into the design and test specifications for components. Eventually the hardware will be tested to the environments I developed to assure that the components will survive and function appropriately after exposure to the expected vibration environments.

Messaro. Semma↗

NASA NTRS2014

Glenn Goddard TDRSS Waveform 1.1.3 On-Orbit Performance Report

The objective of the Space Communications and Navigation (SCaN) Testbed is to study the development, testing, and operation of software defined radios (SDRs) and their associated appliations in the operational space environment to reduce cost and risk for future space missions. This report covers the results of on-orbit performance testing completed using the Glenn Goddard Tracking and Data Relay Satellite System (TDRSS) waveform version 1.1.3 in the ground and space environments. The Glenn Goddard TDRSS (GGT) waveform, operating on the SCaN Testbed Jet Propulsion Laboratory (JPL) SDR, is capable of a variety of data rates and frequencies, operating using Binary Phase Shift Keying (BPSK).

space communications↗

NASA NTRS1999

Piloted Simulation Investigation of a Supersonic Transport Configuration (LaRC.4)

This report contains a description of the test facilities and software utilized during a joint NASA/aerospace industry study of improved control laws and desired inceptor characteristics for a candidate supersonic transport air-craft design. Details concerning the characteristics of the simulation cockpit, image generator and display systems, and motion platform are described. Depictions of the various display formats are included. The test schedule, session log, and flight cards describing the maneuvers performed is included. A brief summary of high-lights of the study is given. Modifications made to the industry-provided simulation model are described. This report is intended to serve as a reference document for industry researchers.

Jackson, E. Bruce↗

NASA NTRS2018

On-Board Battery Monitoring and Prognostics for Electric-Propulsion Aircraft

The reliability of the propulsion system of an aircraft is paramount for the aircraft safety and hence the aircraft health must be monitored continuously. In contrast to fuel- operated aircraft, electric battery-operated propulsion system poses specific problems, such as, the remaining battery power does not linearly decrease and cannot be measured directly. In this paper, we describe a combined monitoring and prognostics architecture that can continuously monitor all components of the electric propulsion system with respect to safety and performance properties as well as state of charge and rest of useful life for the battery. Our system combines a detailed electrochemical battery model for Li-ion batteries with a powerful prognostics engine based upon an Unscented Kalman Filter with the R2U2 monitoring device, which provides efficient observers for metric temporal logic and Bayesian reasoning. R2U2 is a real-time, realizable, responsive, unobtrusive unit, which continuously monitors sensor readings, outputs of the prognostics engine, as well as the ight software status for safety, performance, and security properties. We illustrate our architecture with two case studies, one reporting actual ight tests with an X8+ octocopter and the other a software-in-the-loop simulation with an unmanned Edge 540 electric aircraft model.

Kulkarni, Chetan↗

NASA NTRS2003

Experiences using OpenMP based on Computer Directed Software DSM on a PC Cluster

In this work we report on our experiences running OpenMP programs on a commodity cluster of PCs running a software distributed shared memory (DSM) system. We describe our test environment and report on the performance of a subset of the NAS Parallel Benchmarks that have been automaticaly parallelized for OpenMP. We compare the performance of the OpenMP implementations with that of their message passing counterparts and discuss performance differences.

Hess, Matthias↗

NASA NTRS2002

Experiences Using OpenMP Based on Compiler Directed Software DSM on a PC Cluster

In this work we report on our experiences running OpenMP (message passing) programs on a commodity cluster of PCs (personal computers) running a software distributed shared memory (DSM) system. We describe our test environment and report on the performance of a subset of the NAS (NASA Advanced Supercomputing) Parallel Benchmarks that have been automatically parallelized for OpenMP. We compare the performance of the OpenMP implementations with that of their message passing counterparts and discuss performance differences.

Hess, Matthias↗

DOE OSTI2025

Software Quality Assurance for the MOOSE-Based Open-Source Multiphysics Code Cardinal - An Expanded CI Testing Suite

Cardinal is a wrapping of the GPU-oriented spectral element Computational Fluid Dynamics (CFD) code NekRS and the Monte Carlo particle transport code OpenMC within the Multiphysics Object-Oriented Simulation Environment (MOOSE). Cardinal provides high-resolution thermal-hydraulics and/or radiation transport feedback to MOOSE multiphysics simulations. Multiphysics feedback is implemented in a geometry-agnostic manner which eliminates the need for rigid one-to-one mappings. A generic data transfer implementation also allows NekRS and OpenMC to couple to any MOOSE application, enabling a broad set of multiphysics capabilities. Cardinal simulations can also leverage combinations of MPI, OpenMP, and GPU resources. Cardinal continuous development and improvement efforts have led to the software being considered as a high-fidelity design and licensing tool for key areas of nuclear reactor relevant physics, including neutron transport, fluid flow, heat transfer, and mechanical processes. The fast development and expansion of the software from a pure R&D framework towards its application in the nuclear industry and regulation require a focus on developing, enhancing and, maintaining Cardinal’s software quality through strict adherence to a Software Quality Assurance (SQA) framework and SQA program. To facilitate compliance with SQA standards, the Cardinal SQA Program has been initiated during Fiscal Year 2023 (FY23). During the development of the Cardinal SQA Program, multiple gaps have been identified. These gaps are primarily related to model verification and code pedigree as they relate to the use of Cardinal as a safety analysis tool. These gaps have been captured in a report published in 2023. A second report highlighted the progress made during Fiscal Year 2024 (FY24) and described Argonne’s effort to document and integrate software verification within Cardinal’s software development process. This report documents a snapshot of the verification test cases currently available for Cardinal and NekRS in their assimilation into a Continuous Integration (CI) platform. Following the CI practice permits the integrating of source code changes frequently and ensuring that the integrated codebase clears the verification testing for the software. It should be noted that the SQA program itself, including the program plans, procedures, configuration management, and testing strategies, need to be developed in a future step of this task.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

NASA NTRS1986

Further development of the dynamic gas temperature measurement system. Volume 2: Computer program user's manual

The Dynamic Gas Temperature Measurement System compensation software accepts digitized data from two different diameter thermocouples and computes a compensated frequency response spectrum for one of the thermocouples. Detailed discussions of the physical system, analytical model, and computer software are presented in this volume and in Volume 1 of this report under Task 3. Computer program software restrictions and test cases are also presented. Compensated and uncompensated data may be presented in either the time or frequency domain. Time domain data are presented as instantaneous temperature vs time. Frequency domain data may be presented in several forms such as power spectral density vs frequency.

Stocks, Dana R.↗

NASA NTRS1996

Development and Demonstration of an Ada Test Generation System

In this project we have built a prototype system that performs Feasible Path Analysis on Ada programs: given a description of a set of control flow paths through a procedure, and a predicate at a program point feasible path analysis determines if there is input data which causes execution to flow down some path in the collection reaching the point so that tile predicate is true. Feasible path analysis can be applied to program testing, program slicing, array bounds checking, and other forms of anomaly checking. FPA is central to most applications of program analysis. But, because this problem is formally unsolvable, syntactic-based approximations are used in its place. For example, in dead-code analysis the problem is to determine if there are any input values which cause execution to reach a specified program point. Instead an approximation to this problem is computed: determine whether there is a control flow path from the start of the program to the point. This syntactic approximation is efficiently computable and conservative: if there is no such path the program point is clearly unreachable, but if there is such a path, the analysis is inconclusive, and the code is assumed to be live. Such conservative analysis too often yields unsatisfactory results because the approximation is too weak. As another example, consider data flow analysis. A du-pair is a pair of program points such that the first point is a definition of a variable and the second point a use and for which there exists a definition-free path from the definition to the use. The sharper, semantic definition of a du-pair requires that there be a feasible definition-free path from the definition to the use. A compiler using du-pairs for detecting dead variables may miss optimizations by not considering feasibility. Similarly, a program analyzer computing program slices to merge parallel versions may report conflicts where none exist. In the context of software testing, feasibility analysis plays an important role in identifying testing requirements which are infeasible. This is especially true for data flow testing and modified condition/decision coverage. Our system uses in an essential way symbolic analysis and theorem proving technology, and we believe this work represents one of the few successful uses of a theorem prover working in a completely automatic fashion to solve a problem of practical interest. We believe this work anticipates an important trend away from purely syntactic-based methods for program analysis to semantic methods based on symbolic processing and inference technology. Other results demonstrating the practical use of automatic inference is being reported in hardware verification, although there are significant differences between the hardware work and ours. However, what is common and important is that general purpose theorem provers are being integrated with more special-purpose decision procedures to solve problems in analysis and verification. We are pursuina commercial opportunities for this work, and will use and extend the work in other projects we are engaged in. Ultimately we would like to rework the system to analyze C, C++, or Java as a key step toward commercialization.

Source record↗

DOE OSTI2025

Fiscal Year 2025 Software Quality Assurance Activities for the ARC Software

The continued goal of the ARC SQA project in the Advanced Reactor Technologies program of DOE is to resolve the QA gaps for the ARC software that limit, or prevent, commercialization of the software for industry users. This project started in earnest in fiscal year 2023 which saw the entire code system moved from a SVN repository to a GitLab repository and an associated software quality assurance plan (SQAP) developed and ratified. Most of the QA gaps in the ARC software were identified in collaboration with industry partners and work begin in fiscal year 2023 and continued through 2024 and 2025. The continuous integration testing was extended to RCT, DASSH, and SE2ANL. Minor changes were required to the original continuous integration methodology to make this happen. When full confidence in the methodology is complete, a report will be created to detail the automated regression testing methodology and minor reports will be created to detail the tolerance settings that have been applied to the output for each ARC code. The primary documentation that is missing includes user manuals, user guides, software verification reports, and code coverage assessments. The DASSH, SE2ANL, and SE2RCT manuals were completed this fiscal year. A review of the SE2ANL software identified that it is unrealistic to include updated correlations or different geometry models and it was scheduled for deprecation in favor of DASSH. The SE2ANL manual is essential for SE2RCT as they are similar but quite different in purpose. The only piece of software missing a manual consistent with the source code is NUBOW-3D which is a focus of the coming year. The code coverage report for DIF3D was updated and code coverage reports were created for REBUS, RCT, PERSENT, GAMSRC, and DASSH. Minor coverage issues were identified for all of these pieces of software which did not prevent the work done to transition them to the OneAPI compiler. Because SE2ANL was scheduled for deprecation, it was not transitioned, but it was successfully tested with the OneAPI compiler. This leaves SE2RCT and NUBOW-3D as the only pieces of software not transitioned to OneAPI and further work is required to get SE2RCT to work properly. The SE2RCT software transition will begin early next year while the NUBOW-3D software requires a manual before it can begin. Software verification work has been completed for DIF3D, REBUS, GAMSOR, GAMSRC, VARPOW, EvaluateFlux, and SUMMAR. The PERSENT software verification work was completed this year which was somewhat delayed because of unexpected bugs in the software. The PERSENT manual was updated to detail some of the issues and discuss the bowing reactivity worth feature added in the previous fiscal year. The RCT, DASSH, SE2RCT, and NUBOW-3D software are the only maintained pieces of software without verification reports. The software verification work for DASSH will be a focus in the upcoming fiscal year and it is hoped that some of the test cases created can serve as verification tests for SE2RCT. The NUBOW-3D work will begin when the manual and requirements report are completed. Only minor industry partner software development funds were provided this year. The DASSH software was updated to handle general axial geometry for each assembly and the NUBOW-3D software was updated to incorporate a new input format and better output. Overall progress on resolving the QA gaps has been good this year.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2025

Fiscal Year 2025 Software Quality Assurance Activities for the ARC Software

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2008

JPL Test Effectiveness Analysis

1) The pilot study provided meaningful conclusions that are generally consistent with the earlier Test Effectiveness work done between 1992 and 1994: a) Analysis of pre-launch problem/failure reports is consistent with earlier work. b) Analysis of post-launch early mission anomaly reports indicates that there are more software issues in newer missions, and the no-test category for identification of post-launch failures is more significant than in the earlier analysis. 2) Future work includes understanding how differences in Missions effect these analyses: a) There are large variations in the number of problem reports and issues that are documented by the different Projects/Missions. b) Some missions do not have any reported environmental test anomalies, even though environmental tests were performed. 3) Each project/mission has different standards and conventions for filling out the PFR forms, the industry may wish to address this issue: a) Existing problem reporting forms are to document and track problems, failures, and issues (etc.) for the projects, to ensure high quality. b) Existing problem reporting forms are not intended for data mining.

failure cause↗