Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39

Digital flight control for the NASA 737 airplane

A brief description of the hardware and software for the digital flight control computers for the NASA 737 airplane is given. Software modules include the basic executive, scheduler, redundancy management, software signal selection, system test, mode logic, pitch axis flight control program and the lateral axis flight control program. A more detailed description of the software development effort is given for the digital flight control software. The software development tasks discussed are: software requirements, development, documentation, control, lab evaluation and formal lab tests. The software development costs are identified and conclusions are drawn concerning the software development effort required to support digital flight control for commercial jet transport applications.

Malcom, L. G.↗

Application of industry-standard guidelines for the validation of avionics software

The application of industry standards to the development of avionics software is discussed, focusing on verification and validation activities. It is pointed out that the procedures that guide the avionics software development and testing process are under increased scrutiny. The DO-178A guidelines, Software Considerations in Airborne Systems and Equipment Certification, are used by the FAA for certifying avionics software. To investigate the effectiveness of the DO-178A guidelines for improving the quality of avionics software, guidance and control software (GCS) is being developed according to the DO-178A development method. It is noted that, due to the extent of the data collection and configuration management procedures, any phase in the life cycle of a GCS implementation can be reconstructed. Hence, a fundamental development and testing platform has been established that is suitable for investigating the adequacy of various software development processes. In particular, the overall effectiveness and efficiency of the development method recommended by the DO-178A guidelines are being closely examined.

Hayhurst, Kelly J.↗

An experiment in software reliability

The results of a software reliability experiment conducted in a controlled laboratory setting are reported. The experiment was undertaken to gather data on software failures and is one in a series of experiments being pursued by the Fault Tolerant Systems Branch of NASA Langley Research Center to find a means of credibly performing reliability evaluations of flight control software. The experiment tests a small sample of implementations of radar tracking software having ultra-reliability requirements and uses n-version programming for error detection, and repetitive run modeling for failure and fault rate estimation. The experiment results agree with those of Nagel and Skrivan in that the program error rates suggest an approximate log-linear pattern and the individual faults occurred with significantly different error rates. Additional analysis of the experimental data raises new questions concerning the phenomenon of interacting faults. This phenomenon may provide one explanation for software reliability decay.

Dunham, J. R.↗

Collected software engineering papers, volume 3

Topics addressed include: software technology evaluation programs; collection of valid software engineering data; test processes using structural coverage; prototype expert system for software engineering management; use of an environment characteristic software metric set; software modulation; software development by analysis of change; and independent verification and validation.

Source record↗

TAXI Direct-to-Disk Interface Demultiplexes Proprietarily Formatted Data

The TAXI Direct-to-Disk interface is a special-purpose interface circuit for demultiplexing of data from a Racal Storeplex (or equivalent) multichannel recorder onto one or more hard disks that reside in, and/or are controlled by, a personal computer (PC). (The name TAXI as used here is derived from the acronym TAXI, which signifies transparent asynchronous transceiver interface.) The TAXI Direct-to-Disk interface was developed for original use in capturing data from instrumentation on a test stand in a NASA rocket-testing facility. The control, data-recording, and data-postprocessing equipment of the facility are located in a control room at a safe distance from the test stand. Heretofore, the transfer of data from the instrumentation to the postprocessing equipment has entailed post-test downloading via software, requiring many hours to days of post-test reduction before the data could be viewed in a channelized format. The installation of the TAXI Direct-to-Disk interface, in conjunction with other modifications, causes the transfer of data to take place in real time, so that the data are immediately available for review during or after the test. The instrumentation is connected to the input terminals of the signal-processing unit of multichannel recorder by standard coaxial cables. The coaxial output of the signal processing unit is converted to fiber-optic output by means of a commercial coaxial-cable/fiber-optic converter (that is, a fiber-optic transceiver) designed specifically for this application. The fiber-optic link carries the data signals to an identical fiber-optic transceiver in the control room. On the way to the TAXI Direct-to-Disk interface that is the focus of this article, the data signals are processed through a companion special purpose circuit denoted by the similar name parallel TAXI interface.

Newnan, Bruce G.↗

Diablo Test Suite: Code Coverage Analysis

An analysis of the Diablo source code covered by its Software Quality Assurance test suite was performed. Intel’s code coverage tool was used to determine the percentage of covered files, blocks, and functions. Excluding non-functional legacy files and single use developer-only files, the code coverage tool showed that 55% of blocks, the most applicable metric and closest to lines of code, are covered. For a large software code, 70-80% coverage is generally considered a reasonable target.A significant percentage of the uncovered code can be categorized as features that are not practical to test, duplicate/unused legacy code, and niche features only used by select individuals. Additionally, large segments of code dedicated to debugging and error handling are untested, as the test problems do not error out. The remaining untested code can be considered general purpose for use by the analyst/user community. These untested general purpose features are identified in the document and should be prioritized when new testing is added. Suggestions for additional test problems that would have the biggest impact on increasing the code coverage metric are presented. Finally, it is important to note that the single biggest way to increase the overall coverage metric is to delete old and unused code. While admittedly tedious to perform, future code development would also benefit markedly from this. Deleting unused code would likely push the coverage metric north of 60%, which is not too far off the 70-80% target. Overall, the current test suite covers the majority of commonly used features in Diablo, but there remains significant room for improvement. These areas for improvement have been identified.

97 MATHEMATICS AND COMPUTING↗

Optical Testing and Verification Methods for the James Webb Space Telescope Integrated Science Instrument Module Element

NASA's James Webb Space Telescope (JWST) is a 6.6m diameter, segmented, deployable telescope for cryogenic IR space astronomy (~40K). The JWST Observatory includes the Optical Telescope Element (OTE) and the Integrated Science Instrument Module (ISIM) that contains four science instruments (SI) and the fine guider. The SIs are mounted to a composite metering structure. The SI and guider units were integrated to the ISIM structure and optically tested at the NASA Goddard Space Flight Center as a suite using the Optical Telescope Element SIMulator (OSIM). OSIM is a full field, cryogenic JWST telescope simulator. SI performance, including alignment and wave front error, were evaluated using OSIM. We describe test and analysis methods for optical performance verification of the ISIM Element, with an emphasis on the processes used to plan and execute the test. The complexity of ISIM and OSIM drove us to develop a software tool for test planning that allows for configuration control of observations, associated scripts, and management of hardware and software limits and constraints, as well as tools for rapid data evaluation, and flexible re-planning in response to the unexpected. As examples of our test and analysis approach, we discuss how factors such as the ground test thermal environment are compensated in alignment. We describe how these innovative methods for test planning and execution and post-test analysis were instrumental in the verification program for the ISIM element, with enough information to allow the reader to consider these innovations and lessons learned in this successful effort in their future testing for other programs.

spaceborne telescopes↗

Acoustic Emission Analysis Applet (AEAA) Software

NASA Glenn Research and NASA White Sands Test Facility have developed software supporting an automated pressure vessel structural health monitoring (SHM) system based on acoustic emissions (AE). The software, referred to as the Acoustic Emission Analysis Applet (AEAA), provides analysts with a tool that can interrogate data collected on Digital Wave Corp. and Physical Acoustics Corp. software using a wide spectrum of powerful filters and charts. This software can be made to work with any data once the data format is known. The applet will compute basic AE statistics, and statistics as a function of time and pressure (see figure). AEAA provides value added beyond the analysis provided by the respective vendors' analysis software. The software can handle data sets of unlimited size. A wide variety of government and commercial applications could benefit from this technology, notably requalification and usage tests for compressed gas and hydrogen-fueled vehicles. Future enhancements will add features similar to a "check engine" light on a vehicle. Once installed, the system will ultimately be used to alert International Space Station crewmembers to critical structural instabilities, but will have little impact to missions otherwise. Diagnostic information could then be transmitted to experienced technicians on the ground in a timely manner to determine whether pressure vessels have been impacted, are structurally unsound, or can be safely used to complete the mission.

Nichols, Charles T.↗

A set of test problems for nonlinear optimization

The software is a set of test problems for nonlinear optimization algorithms, including subroutines such as linear algebra routines and automatic differentiation algorithms. The test problems come from chemical engineering open literature, and describe optimization tasks related to the design and operation of processes such as carbon capture, Hydrogen production, heat exchange, and distillation.

Parker, Robert↗

Assessment of Survey Results from Advanced Reactor Industry Domain Experts on Nonlinear Soil-Structure Interaction Analysis Software Verification and Validation

The seismic load case has a significant impact on the design and cost of nuclear power plants. Given the need to substantially reduce cost, advanced reactor designers are looking to leverage numerical tools that enable seismic analysis of an integrated vessel/support/structure/soil system. Modern nonlinear analysis tools provide a solution, capturing dynamic coupling between components (including soil-structure interaction (SSI)) concurrent with nonlinear behavior in one or more parts of the system. Although these methods have a rich history of technical development and implementation, software quality assurance (SQA) following nuclear industry standards remains a significant burden for those wishing to adopt such methods for advanced reactor design and licensing. To reduce the SQA burden, this project will develop the guidance for SQA verification and validation (V&V) of coupled nonlinear SSI analysis tools, to include a test matrix of software features and test problems to support commercial grade dedication (CGD). The guidance is intended to be technology-neutral: supporting designers of all advanced reactors. To maximize the value of the project to reactor designers, the project team performed focused surveys of and interviews with advanced reactor designers. Additionally, the project surveyed members of the broader industry involved in reactor design and licensing. The aggregated industry feedback consists of written survey responses, live polling responses, focused interviews, and informal feedback provided after outreach presentations, collectively referred to as “survey results”. Herein, the survey results are reported, reviewed, and assessed to inform follow-on project activities. The survey results confirmed the familiarity of the industry with the coupled nonlinear SSI analysis. They also affirm the project need based upon the expressed intent to (1) incorporate nonlinear features and (2) pursue the coupled nonlinear SSI analysis to reduce seismic demands and construction costs. All reactor designers indicated their intent to incorporate two or more nonlinear features, and all reactor designer respondents opined that the coupled nonlinear SSI analysis would allow for the optimization of the reactor design and construction. However, the programmatic challenges, whether real or perceived, present a significant barrier to reactor designers. The barrier most commonly identified by the reactor designer population was regulatory risk, with 70% citing this as a reason not to pursue the approach. The perceived regulatory risk identified by the reactor designers underscores the importance of regulator engagement and dialogue in this project. Additionally, half of the reactor designers identified cost and lack of guidance as a deterrent. The survey results also provide insights to tailor specific aspects of the guidance document and test matrix. The project plans to prepare both a formal referenceable guidance document and a collaborative, web-based test matrix and problem set. The project will focus on more complete test problem definitions for the prioritized nonlinear features over shallower problem descriptions for a larger set of features. The nonlinearities prioritized as high based upon survey feedback include fluid-structure interaction and seismic isolation and energy dissipation devices. The nonlinearities prioritized as intermediate include nonlinear geomaterials, nonlinear concrete, nonlinear steel, and interface/contact nonlinearity. Deep embedment and the associated nonlinear phenomena are assigned the lowest priority based upon survey feedback.

42 ENGINEERING↗

Seeing the Invisible: Embedding Tests in Code That Cannot be Modified

The difficulty of characterizing and observing valid software behavior during testing can be very difficult in flight systems. To address this issue, we evaluated several approaches to increasing test observability on the Shuttle Abort Flight Management (SAFM) system. To increase test observability, we added probes into the running system to evaluate the internal state and analyze test data. To minimize the impact of the instrumentation and reduce manual effort, we used Aspect-Oriented Programming (AOP) tools to instrument the source code. We developed and elicited a spectrum of properties, from generic to application specific properties, to be monitored via the instrumentation. To evaluate additional approaches, SAFM was ported to Linux, enabling the use of gcov for measuring test coverage, Valgrind for looking for memory usage errors, and libraries for finding non-normal floating point values. An in-house C++ source code scanning tool was also used to identify violations of SAFM coding standards, and other potentially problematic C++ constructs. Using these approaches with the existing test data sets, we were able to verify several important properties, confirm several problems and identify some previously unidentified issues.

O'Malley, Owen↗

Achieving dependability throughout the development process - A distributed software experiment

Distributed software engineering techniques and methods for improving the specification and testing phases are considered. With multiversion development, multiple implementations allow the use of an automated approach to testing called back-to-back (B/B) testing in which the outputs are compared to detect any discrepancies. However, a specification defect may lead to similar errors in the multiple versions and the underlying fault may not be detected with a B/B testing approach. The use of diverse formal specifications has been proposed as a solution to this problem, since defects in independently written specifications are likely to be different. To examine these issues, an experiment was performed using the design diversity approach in the specification, design, implementation, and testing of distributed software. In the experiment, three diverse formal specifications were used to produce multiple independent implementations of a distributed communication protocol in Ada. The problems encountered in building complex concurrent processing systems in Ada were also studied. Many pitfalls were discovered in mapping the formal specifications into Ada implementations.

Kelly, John P. J.↗

Automation of the Environmental Control and Life Support System

The objective of the Environmental Control and Life Support System (ECLSS) Advanced Automation Project is to recommend and develop advanced software for the initial and evolutionary Space Station Freedom (SSF) ECLS system which will minimize the crew and ground manpower needed for operations. Another objective includes capturing ECLSS design and development knowledge for future missions. This report summarizes our results from Phase I, the ECLSS domain analysis phase, which we broke down into three steps: 1) Analyze and document the baselined ECLS system, 2) envision as our goal an evolution to a fully automated regenerative life support system, built upon an augmented baseline, and 3) document the augmentations (hooks and scars) and advanced software systems which we see as necessary in achieving minimal manpower support for ECLSS operations. In addition, Phase I included development of an advanced software life cycle testing tools will be used in the development of the software. In this way, we plan in preparation for phase II and III, the development and integration phases, respectively. Automated knowledge acquisition, engineering, verification, and can capture ECLSS development knowledge for future use, develop more robust and complex software, provide feedback to the KBS tool community, and insure proper visibility of our efforts.

Dewberry, Brandon S.↗

Test facility for fission chamber assemblies - 239

The Institution -Project Center ITER- with participation of the National Research Nuclear University MEPhI are developing the neutron diagnostics -Divertor Neutron Flux Monitor- (DNFM) for the International Thermonuclear Experimental Reactor (ITER). This diagnostics shall ensure measurement of the total neutron yield and ITER fusion power. Assemblies of fission chambers (FC) with different fissile materials and sensitivities are planned to be used for the neutron flux measurement. Study of fission chambers characteristics and acceptance testing of the diagnostic system will be carried out at the testing facility of the neutronic laboratory of the 'TRINITI' Institute. The facility includes: compact powerful neutron generator with the power, control and monitoring system; remote handling system for FCs positioning and movement; novel three channels data acquisition and processing system. Each acquisition channel contains some measuring circuits with signal processing. This set of circuits provides neutron flux measurement in the range of at least 10 decades for each single FC. Data acquisition and processing system features are: signal digitizing with sample rate up to 125 MHz; real time data processing with computation and data update cycle from 0.1 ms. This paper examines the structure, composition and specification of the test facility. Application software, hardware solutions and the results of fission chambers prototypes tests are discussed. (authors)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

SAS4A/SASSYS-1 VERIFICATION AND VALIDATION TEST SUITE

SAS4A/SASSYS-1 is a safety analysis software package already under copyright by Argonne National Laboratory. There is interest from third-party users of the software to gain access to our internal verification and validation test suite. Therefore we are seeking to establish copyright over the test suite, which includes input files and reference results for hundreds of test cases that demonstrate functionality of the software. Making the test suite available to licensed users allows them to utilize the broad range of tests for commercial grade dedication activities needed to qualify the SAS4A/SASSYS-1 software under their own Software Quality Assurance program for use in a regulatory environment (e.g. with the Nuclear Regulatory Commission).

FANNING, THOMAS↗

EPICS Deployment at Fermilab

Fermilab has traditionally not been an EPICS house, as such expertise in EPICS is limited and scattered. However, PIP-II will be using EPICS for its control system. Furthermore, when PIP-II is operating, it must to interface with the existing, though modernized (see ACORN) legacy control system. We have developed and deployed a software pipeline that addresses these needs and presents to developers a tested and robust software framework, including template IOCs from which new developers can quickly gain experience. In this presentation, we will discuss the motivation for this work, the implementation of a continuous integration/continuous deployment pipeline, testing, template IOCs, and the deployment of user applications. We will also discuss how this is used with the current PIP-II teststand and lessons learned.

43 PARTICLE ACCELERATORS↗

System level verification applying the Space Shuttle experience to the Space Station

The applicability of the verification process for the Shuttle guidance, navigation and control (GNC) and data management system (DMS) for the development of the Space Station are described. Shuttle avionics hardware/software integration was delayed to finalize the hardware design before detailed definition and testing of the software. A block diagram is provided of the flight simulation laboratory used to test the GNC programs before flight data were available. The Station will have distributed computers, unlike the Orbiter, and will only be assembled fully in space. Standardized integration simulation test equipment are being defined to guide the development of hardware and software. The simulation capability may become part of nominal in-flight operations to initiate new capabilities as they are added to the Station. The Station GNC and DMS systems development will be somewhat simplified relative to those of the Shuttle because ascent and reentry will not be considered for the Station.

Gilbert, David W.↗

Challenges and Strategies for Testing Automation Practices at Sandia National Laboratories

Sandia National Laboratories is a premier United States national security laboratory which develops science-based technologies in areas such as nuclear deterrence, energy production, and climate change. Computing plays a key role in its diverse missions, and within that environment, Research Software Engineers (RSEs) and other scientific software developers utilize testing automation to ensure quality and maintainability of their work. We conducted a Participatory Action Research study to explore the challenges and strategies for testing automation through the lens of academic literature. Through the experiences collected and comparison with open literature, we identify these challenges in testing automation and then present strategies for mitigation grounded in evidence-based practice and experience reports that other, similar institutions can assess for their automation needs.

97 MATHEMATICS AND COMPUTING↗