SEARCH · Engineering Papers

Results for “software testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

All sources NASA NTRS DOE OSTI

NASA NTRS2015

A Description of the Development, Capabilities, and Operational Status of the Test SLATE Data Acquisition System at the National Transonic Facility

The paper will present a brief background of the previous data acquisition system at the National Transonic Facility (NTF) and the reasoning and goals behind the upgrade to the current Test SLATE (Test Software Laboratory and Automated Testing Environments) data acquisition system. The components, performance characteristics, and layout of the Test SLATE system within the NTF control room will be discussed. The development, testing, and integration of Test SLATE within NTF operations will be detailed. The operational capabilities of the system will be outlined including: test setup, instrumentation calibration, automatic test sequencer setup, data recording, communication between data and facility control systems, real time display monitoring, and data reduction. The current operational status of the Test SLATE system and its performance during recent NTF testing will be highlighted including high-speed, frame-by-frame data acquisition with conditional sampling post-processing applied. The paper concludes with current development work on the system including the capability for real-time conditional sampling during data acquisition and further efficiency enhancements to the wind tunnel testing process.

Cramer, Christopher J.↗

NASA NTRSDate not supplied

Remotely Administered Psychoacoustic Test for sUAS Noise to Gauge Feasibility of Remote UAM Noise Study

The National Aeronautics and Space Administration (NASA) remotely administered a psychoacoustic test in fall of 2022 as the first of two phases of a cooperative Urban Air Mobility (UAM) vehicle noise human response study. The first phase, the Feasibility Test, described in this paper, determined the feasibility of the remote test method, in contrast to a previous in-person psychoacoustic test that found an annoyance response difference between small unmanned aerial system (sUAS) noise and ground vehicle noise. This paper discusses the Feasibility Test online layout, sound calibration method, remote test software development, stimuli selection, test subject recruitment, and test administration. Test performance is measured through comparison of annoyance response data with the previous in-person test results. The paper also investigates if providing a contextual cue to test subjects influenced their annoyance response. Response differences between test subjects in geographically distinct areas are analyzed. Administrative challenges that were encountered during the test are discussed. The second phase of this study, the implementation phase, will use a remote (web-based) test method and leverage the cooperation of multiple government agencies, academia, and industry to gain human response insights from a wide range of UAM vehicle sounds that would be challenging for a single organization to acquire. Improvements to administering a remote test for the implementation phase of the UAM vehicle noise human response study are recommended.

Remote Psychoacoustic Test↗

NASA NTRSDate not supplied

Remotely Administered Psychoacoustic Test for sUAS Noise to Gauge Feasibility of Remote UAM Noise Study

Remote Psychoacoustic Test↗

NASA NTRS1980

Online assessment of a distributed processor

ORT (Operational Readiness Test) software allows one engineer to test readiness of 64 minicomputers and their peripherals from single console. Software makes roll call of computers and peripherals via common data buffer to check readiness of system in morning "wake up" or at other important times. Subsystems are tested in parallel to save time. "Watchdog" terminates test of any system that does not respond in time, so one failed system does not halt test sequence. Entire rollcall is complete in about 15 minutes. Software is designed for Space Shuttle prelaunch checkout, but approach should interest users of similar equipment.

Ehrlich, L. F.↗

NASA NTRS2003

Experiments with Test Case Generation and Runtime Analysis

Software testing is typically an ad hoc process where human testers manually write many test inputs and expected test results, perhaps automating their execution in a regression suite. This process is cumbersome and costly. This paper reports preliminary results on an approach to further automate this process. The approach consists of combining automated test case generation based on systematically exploring the program's input domain, with runtime analysis, where execution traces are monitored and verified against temporal logic specifications, or analyzed using advanced algorithms for detecting concurrency errors such as data races and deadlocks. The approach suggests to generate specifications dynamically per input instance rather than statically once-and-for-all. The paper describes experiments with variants of this approach in the context of two examples, a planetary rover controller and a space craft fault protection system.

Artho, Cyrille↗

DOE OSTI2017

Simulation-based Testing of Control Software

It is impossible to adequately test complex software by examining its operation in a physical prototype of the system monitored. Adequate test coverage can require millions of test cases, and the cost of equipment prototypes combined with the real-time constraints of testing with them makes it infeasible to sample more than a small number of these tests. Model based testing seeks to avoid this problem by allowing for large numbers of relatively inexpensive virtual prototypes that operate in simulation time at a speed limited only by the available computing resources. In this report, we describe how a computer system emulator can be used as part of a model based testing environment; specifically, we show that a complete software stack including operating system and application software - can be deployed within a simulated environment, and that these simulations can proceed as fast as possible. To illustrate this approach to model based testing, we describe how it is being used to test several building control systems that act to coordinate air conditioning loads for the purpose of reducing peak demand. These tests involve the use of ADEVS (A Discrete Event System Simulator) and QEMU (Quick Emulator) to host the operational software within the simulation, and a building model developed with the MODELICA programming language using Buildings Library and packaged as an FMU (Functional Mock-up Unit) that serves as the virtual test environment.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2017

Extension of mutation testing for the requirements and design faults - 322

Mutation Testing is a fault-based software testing technique that is considered to be the most efficient in detecting faults. It focuses on the software code level and follows a set of rules which are called mutation operators to systematically seed faults into the source code. Each seeded fault results is a new version of the software, which is called a mutant. Mutation testing promises to be effective in identifying adequate test data which can be used to find faults in the source code. However, traditional mutation testing is not geared towards the detection of requirements and design faults. To cover the full spectrum of possible faults, the mutation framework has been extended from the code level to the requirements and design level. Twenty-nine defects that appear in requirements and design documents were identified to define appropriate mutants. Then corresponding mutation operators that model the defects were developed. For each mutation operator, the strategy to generate a mutant and the evaluation of the number of possible mutants were defined. Existing code level cost reduction techniques were tailored for requirements and design mutation operators that result in a large number of mutants. With the extended mutation testing framework, the full spectrum of possible faults will be tested and identified to ensure the safety and functionality of the software. (authors)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

NASA NTRS1981

A high order approach to flight software development and testing

The use of a software development facility is discussed as a means of producing a reliable and maintainable ECS software system, and as a means of providing efficient use of the ECS hardware test facility. Principles applied to software design are given, including modularity, abstraction, hiding, and uniformity. The general objectives of each phase of the software life cycle are also given, including testing, maintenance, code development, and requirement specifications. Software development facility tools are summarized, and tool deficiencies recognized in the code development and testing phases are considered. Due to limited lab resources, the functional simulation capabilities may be indispensable in the testing phase.

Steinbacher, J.↗

NASA NTRS1992

Providing an empirical basis for optimizing the verification and testing phases of software development

Applying equal testing and verification effort to all parts of a software system is not very efficient, especially when resources are limited and scheduling is tight. Therefore, one needs to be able to differentiate low/high fault density components so that the testing/verification effort can be concentrated where needed. Such a strategy is expected to detect more faults and thus improve the resulting reliability of the overall system. This paper presents an alternative approach for constructing such models that is intended to fulfill specific software engineering needs (i.e. dealing with partial/incomplete information and creating models that are easy to interpret). Our approach to classification is as follows: (1) to measure the software system to be considered; and (2) to build multivariate stochastic models for prediction. We present experimental results obtained by classifying FORTRAN components developed at the NASA/GSFC into two fault density classes: low and high. Also we evaluate the accuracy of the model and the insights it provides into the software process.

Briand, Lionel C.↗

NASA NTRS1978

X-ray enhancement software development and test

A repertoire of software to optimally analyze various X-ray imagery was successfully developed. Computer techniques are presented to solve many common problems involved in nondestructive testing X-ray analysis.

Butterfield, R. L.↗

NASA NTRS1979

To the first Shuttle launch

The major Shuttle-related testing currently in progress includes: static testing of all major elements, full scale ground vibration testing, main propulsion system testing, software verification and validation, qualification testing of many components and subsystems, and SSME verification testing. This paper reviews the current status of the Shuttle program and gives attention to the transition to operations, near-term traffic plans, user-charge policies, and payload integration. It is concluded that the transition from individual space-research projects to a fully integrated Space Transportation System, providing low-cost routine access to space for many users, involves a complete rethinking of the role NASA and the user will play.

Yardley, J. F.↗

DOE OSTI2020

Verification and Validation of PARET/ANL (V.7.6)

This report documents the software testing which has been performed for the PARET/ANL version 7.6 software. The capabilities addressed in this testing were selected based on the use of the software for analysis work under the Research and Test Reactors Department quality assurance program. The verification and validation have been performed and documented to address the steady-state capabilities of the software, as described in Chapter 2, and the transient capabilities, as described in Chapter 3. Testing based on the comparison between PARET/ANL calculations and analytical solutions, hand calculations, or another code calculations of the test cases confirms that all the identified capabilities of the software were implemented correctly.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

Summary of Documentation for DYNA3D-ParaDyn's Software Quality Assurance Regression Test Problems

The Software Quality Assurance (SQA) regression test suite for DYNA3D (Zywicz and Lin, 2015) and ParaDyn (DeGroot, et al., 2015) currently contains approximately 600 problems divided into 21 suites, and is a required component of ParaDyn’s SQA plan (Ferencz and Oliver, 2013). The regression suite allows developers to ensure that software modifications do not unintentionally alter the code response. The entire regression suite is run prior to permanently incorporating any software modification or addition. When code modifications alter test problem results, the specific cause must be determined and fully understood before the software changes and revised test answers can be incorporated. The regression suite is executed on LLNL platforms using a Python script and an associated data file. The user specifies the DYNA3D or ParaDyn executable, number of processors to use, test problems to run, and other options to the script. The data file details how each problem and its answer extraction scripts are executed. For each problem in the regression suite there exists an input deck, an eight-processor partition file, an answer file, and various extraction scripts. These scripts assemble a temporary answer file in a specific format from the simulation results. The temporary and stored answer files are compared to a specific level of numerical precision, and when differences are detected the test problem is flagged as failed. Presently, numerical results are stored and compared to 16 digits. At this accuracy level different processor types, compilers, number of partitions, etc. impact the results to various degrees. Thus, for consistency purposes the regression suite is run with ParaDyn using 8 processors on machines with a specific processor type (currently the Intel Xeon E5530 processor). For non-parallel regression problems, i.e., the two XFEM problems, DYNA3D is used instead. When environments or platforms change, executables using the current source code and the new resource are created and the regression suite is run. If differences in answers arise, the new answers are retained provided that the differences are inconsequential. This bootstrap approach allows the test suite answers to evolve in a controlled manner with a high level of confidence. Developers also run the entire regression suite with (serial) DYNA3D. While these results normally differ from the stored (parallel) answers, abnormal termination or wildly different values are strong indicators of potential issues.

42 ENGINEERING↗

NASA NTRS1994

Program Helps Design Tests Of Developmental Software

Computer program called "A Formal Test Representation Language and Tool for Functional Test Designs" (TRL) provides automatic software tool and formal language used to implement category-partition method and produce specification of test cases in testing phase of development of software. Category-partition method useful in defining input, outputs, and purpose of test-design phase of development and combines benefits of choosing normal cases having error-exposing properties. Traceability maintained quite easily by creating test design for each objective in test plan. Effort to transform test cases into procedures simplified by use of automatic software tool to create cases based on test design. Method enables rapid elimination of undesired test cases from consideration and facilitates review of test designs by peer groups. Written in C language.

Hops, Jonathan↗

NASA NTRS1985

A methodology for testing fault-tolerant software

A methodology for testing fault tolerant software is presented. There are problems associated with testing fault tolerant software because many errors are masked or corrected by voters, limiter, or automatic channel synchronization. This methodology illustrates how the same strategies used for testing fault tolerant hardware can be applied to testing fault tolerant software. For example, one strategy used in testing fault tolerant hardware is to disable the redundancy during testing. A similar testing strategy is proposed for software, namely, to move the major emphasis on testing earlier in the development cycle (before the redundancy is in place) thus reducing the possibility that undetected errors will be masked when limiters and voters are added.

Andrews, D. M.↗

NASA NTRS2011

Mars Science Laboratory Flight Software Boot Robustness Testing Project Report

On the surface of Mars, the Mars Science Laboratory will boot up its flight computers every morning, having charged the batteries through the night. This boot process is complicated, critical, and affected by numerous hardware states that can be difficult to test. The hardware test beds do not facilitate testing a long duration of back-to-back unmanned automated tests, and although the software simulation has provided the necessary functionality and fidelity for this boot testing, there has not been support for the full flexibility necessary for this task. Therefore to perform this testing a framework has been build around the software simulation that supports running automated tests loading a variety of starting configurations for software and hardware states. This implementation has been tested against the nominal cases to validate the methodology, and support for configuring off-nominal cases is ongoing. The implication of this testing is that the introduction of input configurations that have yet proved difficult to test may reveal boot scenarios worth higher fidelity investigation, and in other cases increase confidence in the robustness of the flight software boot process.

Mars Science Laboratory (MSL)↗

NASA NTRS2014

Integration and Testing of LCS Software

Kennedy Space Center is in the midst of developing a command and control system for the launch of the next generation manned space vehicle. The Space Launch System (SLS) will launch using the new Spaceport Command and Control System (SCCS). As a member of the Software Integration and Test (SWIT) Team, command scripts, and bash scripts were written to assist in integration and testing of the Launch Control System (LCS), which is a component of SCCS. The short term and midterm tasks are for the most part completed. The long term tasks if time permits will require a presentation and demonstration.

Command Script↗

DOE OSTI2014

2014 Semi-Annual Progress Letter Report on Used Nuclear Fuel Integrity Study in Transportation Environments

The objective of this project is to perform a systematic study on UNF (Used Nuclear Fuel) integrity under simulated transportation environments by using enabling hot-cell testing technology developed recently at ORNL, CIRFT (Cyclic Integrated Reversible-Bending Fatigue Tester) [1-9]. Out-of-cell study will support the ongoing ORNL hot-cell or in-cell testing. Although the use of CIRFT on H. B. Robinson UNF rods generated many interesting data, a number of important issues remain to be addressed arising from the test system and the rod itself. It has been shown that a direct examination of some of these issues in the hot-cell is prohibitive due to high cost and access limitations. The out-of-cell study has been demonstrated to be cost effective and much more feasible to provide quick turn-around solutions. A testing system with the same configuration as that of CIRFT used in hot-cell has been proposed to study the emerging issues from hot-cell testing and also to investigate the fatigue and failure mechanisms of UNF rods that cannot be identified directly with in hot-cell testing. The developments of hardware and software, test protocol, and hot-cell data analysis will be discussed in this report.

Normal Transportation↗