Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software quality, testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Connecting Research and Practice: An Experience Report on Research Infusion with SAVE

NASA systems need to be highly dependable to avoid catastrophic mission failures. This calls for rigorous engineering processes including meticulous validation and verification. However, NASA systems are often highly distributed and overwhelmingly complex, making the software portion of these systems challenging to understand, maintain, change, reuse, and test. NASA's systems are long-lived and the software maintenance process typically constitutes 60-80% of the total cost of the entire lifecycle. Thus, in addition to the technical challenges of ensuring high life-time quality of NASA's systems, the post-development phase also presents a significant financial burden. Some of NASA's software-related challenges could potentially be addressed by some of the many powerful technologies that are being developed in software research laboratories. Many of these research technologies seek to facilitate maintenance and evolution by for example architecting, designing and modeling for quality, flexibility, and reuse. Other technologies attempt to detect and remove defects and other quality issues by various forms of automated defect detection, architecture analysis, and various forms of sophisticated simulation and testing. However promising, most such research technologies nevertheless do not make the transition from the research lab to the software lab. One reason the transition from research to practice seldom occurs is that research infusion and technology transfer is difficult. For example, factors related to the technology are sometimes overshadowed by other types of factors such as reluctance to change and therefore prohibits the technology from sticking. Successful infusion might also take very long time. One famous study showed that the discrepancy between the conception of the idea and its practical use was 18 years plus or minus three. Nevertheless, infusing new technology is possible. We have found that it takes special circumstances for such research infusion to succeed: 1) there must be evidence that the technology works in the practitioner's particular domain, 2) there must be a potential for great improvements and enhanced competitive edge for the practitioner, 3) the practitioner has to have strong individual curiosity and continuous interest in trying out new technologies, 4) the practitioner has to have support on multiple levels (i.e. from the researchers, from management, from sponsors etc), and 5) to remain infused, the new technology has to be integrated into the practitioner's processes so that it becomes a natural part of the daily work. NASA IV&V's Research Infusion initiative sponsored by NASA's Office of Safety & Mission Assurance (OSMA) through the Software Assurance Research Program (SARP), strives to overcome some of the problems related to research infusion.

Lindvall, Mikael↗

Statistical modeling of software reliability

This working paper discusses the statistical simulation part of a controlled software development experiment being conducted under the direction of the System Validation Methods Branch, Information Systems Division, NASA Langley Research Center. The experiment uses guidance and control software (GCS) aboard a fictitious planetary landing spacecraft: real-time control software operating on a transient mission. Software execution is simulated to study the statistical aspects of reliability and other failure characteristics of the software during development, testing, and random usage. Quantification of software reliability is a major goal. Various reliability concepts are discussed. Experiments are described for performing simulations and collecting appropriate simulated software performance and failure data. This data is then used to make statistical inferences about the quality of the software development and verification processes as well as inferences about the reliability of software versions and reliability growth under random testing and debugging.

Miller, Douglas R.↗

NASA software documentation standard software engineering program

The NASA Software Documentation Standard (hereinafter referred to as Standard) can be applied to the documentation of all NASA software. This Standard is limited to documentation format and content requirements. It does not mandate specific management, engineering, or assurance standards or techniques. This Standard defines the format and content of documentation for software acquisition, development, and sustaining engineering. Format requirements address where information shall be recorded and content requirements address what information shall be recorded. This Standard provides a framework to allow consistency of documentation across NASA and visibility into the completeness of project documentation. This basic framework consists of four major sections (or volumes). The Management Plan contains all planning and business aspects of a software project, including engineering and assurance planning. The Product Specification contains all technical engineering information, including software requirements and design. The Assurance and Test Procedures contains all technical assurance information, including Test, Quality Assurance (QA), and Verification and Validation (V&V). The Management, Engineering, and Assurance Reports is the library and/or listing of all project reports.

Source record↗

NASA Software Documentation Standard

The NASA Software Documentation Standard (hereinafter referred to as "Standard") is designed to support the documentation of all software developed for NASA; its goal is to provide a framework and model for recording the essential information needed throughout the development life cycle and maintenance of a software system. The NASA Software Documentation Standard can be applied to the documentation of all NASA software. The Standard is limited to documentation format and content requirements. It does not mandate specific management, engineering, or assurance standards or techniques. This Standard defines the format and content of documentation for software acquisition, development, and sustaining engineering. Format requirements address where information shall be recorded and content requirements address what information shall be recorded. This Standard provides a framework to allow consistency of documentation across NASA and visibility into the completeness of project documentation. The basic framework consists of four major sections (or volumes). The Management Plan contains all planning and business aspects of a software project, including engineering and assurance planning. The Product Specification contains all technical engineering information, including software requirements and design. The Assurance and Test Procedures contains all technical assurance information, including Test, Quality Assurance (QA), and Verification and Validation (V&V). The Management, Engineering, and Assurance Reports is the library and/or listing of all project reports.

Source record↗

Methodology evaluation: Effects of independent verification and intergration on one class of application

The effects of an independent verification and integration (V and I) methodology on one class of application are described. Resource profiles are discussed. The development environment is reviewed. Seven measures are presented to test the hypothesis that V and I improve the development and product. The V and I methodology provided: (1) a decrease in requirements ambiguities and misinterpretation; (2) no decrease in design errors; (3) no decrease in the cost of correcting errors; (4) a decrease in the cost of system and acceptance testing; (5) an increase in early discovery of errors; (6) no improvement in the quality of software put into operation; and (7) a decrease in productivity and an increase in cost.

Page, J.↗

Software Facilitates Sharing of Water Quality Data Worldwide

John Freighery was an environmental engineer at Johnson Space Center when a new, simplified version of the coliform bacteria test was developed for astronaut use on the International Space Station. Through his New York City-based mWater Foundation, Freighery is using the test to help rural communities monitor their water supplies for contamination. The organization has also developed a mobile phone app to make the information publicly available.

Source record↗

Real-Time Embedded Software Verification and Validation 2001

As the space applications become more complex and timing constraints on control actions are more stringent, the task of integrating and testing NASA's real-time systems (such as X-38 Crew Return Vehicle, and certain International Space Station autonomous systems) has become a great challenge. A testing environment where can preserve consistent temporal behaviors as in the target execution must be established for system-level verification and software quality assurance. Our goal is to develop an analysis suite for validation and verification of real-time systems that are used to perform human- in-the-loop control operations during safety-critical missions. The suite will be able to carry out quantitative approaches of coverage diagnostic and temporal behavior evaluation in order to measure test coverage, to optimize test utilization, and to verify timing correctness.

Lee, Yann-Hang↗

T0TEM - T0 Test Evaluation Module Development and Verification

Transition temperature testing of ferritic steels evaluates the ductile to brittle transition temperature, where the behavior of the steel changes from controlled ductile tearing to uncontrollable brittle fracture. This testing is governed by ASTM (American Society of Testing and Materials) E1921 standard. The calculations and plots required by this standard are iterative in nature and require significant effort to produce by standard hand calculations. In addition to analysis of a complete data set, intermediate analysis during testing is beneficial to determine optimal test temperatures. Given the exhaustive nature of calculating the transition temperature, being able to quickly update target temperatures is a significant benefit to not only the quality of tests, but the number of required tests as well. T0 Test Evaluation Module (T0TEM) v1.5 is a software application created under the Layered Pressure Vessel (LPV) certification effort. To reduce the analysis time of transition temperature test data sets producedfor this effort, T0TEM was created to calculate the E1921 Master Curve along with all required data and validity checks. By creating this program, hundreds of hours of analysis time were saved. In addition, the creation of a standard program provided a significant improvement in the consistency with which results were reported. T0TEM also calculates the results and plots required for the E1921 inhomogeneity annex to determine whether a material behaves in a homogenous manner. This program was created by Levi Shelton and Cameron Bosley at NASA, with input and review from the E1921 committee. The results generated by T0TEM were compared to the validation data sets provided by ASTM with good concurrence. ASTM committee members have reviewed and applied T0TEM to other data sets with satisfactory results. T0TEM was originally released to the NASA Software Repository and publicly via SourceForge in May of 2020. Version 1.5 was released in April of 2021 with minor functionality updates and additional statistical analysis methods.

Levi Shelton↗

Hypersonic Navier Stokes Comparisons to Orbiter Flight Data

Hypersonic chemical nonequilibrium simulations of low earth orbit entry flow fields are becoming increasingly commonplace as software and computational capabilities become more capable. However, development of robust and accurate software to model these environments will always encounter a significant barrier in developing a suite of high quality calibration cases. The US3D hypersonic nonequilibrium Navier Stokes analysis capability has been favorably compared to a number of wind tunnel test cases. Extension of the calibration basis for this software to Orbiter flight conditions will provide an incremental increase in confidence. As part of the Orbiter Boundary Layer Transition Flight Experiment and the Hypersonic Thermodynamic Infrared Measurements project, NASA is performing entry flight testing on the Orbiter to provide valuable aerothermodynamic heating data. An increase in interest related to orbiter entry environments is resulting from this activity. With the advent of this new data, comparisons of the US3D software to the new flight testing data is warranted. This paper will provide information regarding the framework of analyses that will be applied with the US3D analysis tool. In addition, comparisons will be made to entry flight testing data provided by the Orbiter BLT Flight Experiment and HYTHIRM projects. If data from digital scans of the Orbiter windward surface become available, simulations will also be performed to characterize the difference in surface heating between the CAD reference OML and the digitized surface provided by the surface scans.

Campbell, Charles H.↗

Rover Attitude and Pointing System Simulation Testbed

The MER (Mars Exploration Rover) Attitude and Pointing System Simulation Testbed Environment (RAPSSTER) provides a simulation platform used for the development and test of GNC (guidance, navigation, and control) flight algorithm designs for the Mars rovers, which was specifically tailored to the MERs, but has since been used in the development of rover algorithms for the Mars Science Laboratory (MSL) as well. The software provides an integrated simulation and software testbed environment for the development of Mars rover attitude and pointing flight software. It provides an environment that is able to run the MER GNC flight software directly (as opposed to running an algorithmic model of the MER GNC flight code). This improves simulation fidelity and confidence in the results. Further more, the simulation environment allows the user to single step through its execution, pausing, and restarting at will. The system also provides for the introduction of simulated faults specific to Mars rover environments that cannot be replicated in other testbed platforms, to stress test the GNC flight algorithms under examination. The software provides facilities to do these stress tests in ways that cannot be done in the real-time flight system testbeds, such as time-jumping (both forwards and backwards), and introduction of simulated actuator faults that would be difficult, expensive, and/or destructive to implement in the real-time testbeds. Actual flight-quality codes can be incorporated back into the development-test suite of GNC developers, closing the loop between the GNC developers and the flight software developers. The software provides fully automated scripting, allowing multiple tests to be run with varying parameters, without human supervision.

Vanelli, Charles A.↗

New Features in the SPOC Pipeline Release 4.0

The Science Processing Operations Center is in the process of testing and deploying Release 4.0 of the codebase in the March 2019 timeframe. This paper describes the new features of the software and their likely impact on the quality of the TESS science data products. The major goals of Release 4.0 are to improve the extraction of photometry from the pixels in light of the non-uniform pointing performance and the identification of instrumental signatures from the light curves. We also describe modifications to the FFI pipeline to allow the generation of FFI light curves, correction of the instrumental systematics therein, and planet searches, primarily for the purpose of validating the 2-min pipeline against the FFI pipeline, but also to be able to provide cotrending basis vectors (CBVs) derived directly from the FFIs to the public to aid them in their extraction and correction of photometry. We also discuss the improvements in photometric performance of the pipeline and its various components.The lapse in funding experienced between 22 December 2018 and 27 January 2019 significantly delayed our ability to conduct integration testing as planned for late December/early January, delaying the start of V&V by one month to the end of February 2019.The TESS Mission is funded by NASA's Science Mission Directorate as an Astrophysics Explorer Mission.The Science Processing Operations Center is in the process of testing and deplo!"ing Release 4.0 of thecodebase In the March 2019 tlmeframe. This paper describes the new features or the software and theirlikely impact on the quality of the TESS science data products. The major goals or Release 4.0 are to Imtheidentification of instrumental signatures from the light cuNes. We also describe modifications tothe FFI pipeline to allow the generation of FFI light curves, correction of the instrumental systematicstherein, and planet searches, primarily for the purpose of validating the 2-min pipelineagainst the FFI pipeline, but also to be able to provide cotrending basis vectors (C3Vs) ,,.~,.;.derived directly from the FFls to the public to aid them in their extractbn and correction , :-; ~'-'\'.'.of photometry. We also discuss the Improvements In photometric performance• of ~~\":;:•the pipeline and its various components. :,;.-.;The lapse In funding experienced between 22 December 2018 and 27January 2019 significantly delayed our ability to conduct Integrationtesting as planned for late December/early January, delaying thestart of V&V by one month to the end of February 2019.The TESS Mission is funded by NASA's Science Mission Directorateas an Astrophysics Explorer Mission.o.iu.c...,.....~~New Features in SPOC 4.01. Use of quatemions In photometry and centroiding.2. Use of quaternions to Identify high-motion cadences and exclude same.3. Use of the TPS detections to deemphasize pathological cadences ("skyline flattenlng"I.4. Improved CAL calculations for black and smear correction.5. PA brightness metric calculation improvements (induo'e crowding in calrulation)./ • 6. Improved POC spike goodness metric.•~• 7. Improved handling of gaps and momentum duni:>S in POC.8. Improved tuning of POC., 9. Improved attitude tweak correction in PDC.10. Improvements in PDC introduced noise and correlation goodness metrics11. Using the improved spike goodness metric to minimize overlitting in the spikeremover ' • :,''l./ 12. Enable FFI processing through planet search.,,. ,• • "« ,;,,,•.~'A , 13.1 D4.V S mtreinaim-relipnoinrgts d aartcah irveetrdie tvoa Ml aAnSdT p ersistence to database 15. Improved management of jobs on the NAS Pleiadss supercomputer

Jenkins, Jon M.↗

Space shuttle low cost/risk avionics study

All work breakdown structure elements containing any avionics related effort were examined for pricing the life cycle costs. The analytical, testing, and integration efforts are included for the basic onboard avionics and electrical power systems. The design and procurement of special test equipment and maintenance and repair equipment are considered. Program management associated with these efforts is described. Flight test spares and labor and materials associated with the operations and maintenance of the avionics systems throughout the horizontal flight test are examined. It was determined that cost savings can be achieved by using existing hardware, maximizing orbiter-booster commonality, specifying new equipments to MIL quality standards, basing redundancy on cost effective analysis, minimizing software complexity and reducing cross strapping and computer-managed functions, utilizing compilers and floating point computers, and evolving the design as dictated by the horizontal flight test schedules.

Source record↗

Assessment of Sensor Data Accuracy within Gazebo/ROS for High-Precision Autonomous In-Space Robotic Operations

Modeling high-precision in-space servicing, assembly, and manufacturing operations in a simulated environment is a critical step in the development of robotic systems that will be used to autonomously assemble large-scale structures in space. Limited facility size and high costs for manufacturing prototypes make it challenging to conduct full-scale operational testing under appropriate environmental conditions; therefore, testing in a modular, high-fidelity simulation environment is necessary for verification and validation of technology and architecture designs prior to launch. Several modeling and simulation environments exist both within NASA and industry that can be used to test robotic system design and operations, including the widely used commercial tool Gazebo integrated with Robotic Operating System software. Because the performance of autonomous robotic systems relies heavily on the quality of sensor input data, this paper focuses on assessing the accuracy of pose data from an optical sensor model in the Gazebo environment against the behavior of real hardware. The results of the tests will help developers using Gazebo for large-scale, high-precision simulation to account for modeling inaccuracies within their robotic control system algorithms.

simulation↗

Flight research simulation takes off

The simulation configurations used in research flight test, including nonreal-time, real-time all-software, hardware-in-the-loop, iron bird, and aircraft-in-the-loop, are reviewed. It is concluded that progress in simulation technology will demonstrate new concepts, evaluate designs, enhance the quality of flight test, and minimize flight risks.

Schilling, Lawrence J.↗

The Core Flight System (cFS): NASA Quality Flight Software to Power Science and Exploration Available to the World

The presentation is planned to be a continuation of two previous discussions completed at the FSW Workshop regarding the cFS Test Framework (CTF) and Engineering Data Sheets (EDS). The presentation will also cover the latest advancements in cFS, including: 1. Quick into to cFS 2. An overview of our software release process + Git repo 3. Release of EDS files for the cFE + open-source apps 4. Overview of how to independently verify EDS files using CTF + what they can be used for. 5. Future plans for cFS. This presentation complements the talk on configuration management of distributed cFS repos, to be presented by Tam Ngo from Johnson Space Center.

Dan Knutsen↗

Flight Testing and Real-Time System Identification Analysis of a UH-60A Black Hawk Helicopter with an Instrumented External Sling Load

Helicopter external air transportation plays an important role in today's world. For both military and civilian helicopters, external sling load operations offer an efficient and expedient method of handling heavy, oversized cargo. With the ability to reach areas otherwise inaccessible by ground transportation, helicopter external load operations are conducted in industries such as logging, construction, and fire fighting, as well as in support of military tactical transport missions. Historically, helicopter and load combinations have been qualified through flight testing, requiring considerable time and cost. With advancements in simulation and flight test techniques there is potential to substantially reduce costs and increase the safety of helicopter sling load certification. Validated simulation tools make possible accurate prediction of operational flight characteristics before initial flight tests. Real time analysis of test data improves the safety and efficiency of the testing programs. To advance these concepts, the U.S. Army and NASA, in cooperation with the Israeli Air Force and Technion, under a Memorandum of Agreement, seek to develop and validate a numerical model of the UH-60 with sling load and demonstrate a method of near real time flight test analysis. This thesis presents results from flight tests of a U.S. Army Black Hawk helicopter with various external loads. Tests were conducted as the U.S. first phase of this MOA task. The primary load was a container express box (CONEX) which contained a compact instrumentation package. The flights covered the airspeed range from hover to 70 knots. Primary maneuvers were pitch and roll frequency sweeps, steps, and doublets. Results of the test determined the effect of the suspended load on both the aircraft's handling qualities and its control system's stability margins. Included were calculations of the stability characteristics of the load's pendular motion. Utilizing CIFER(R) software, a method for near-real time system identification was also demonstrated during the flight test program.

McCoy, Allen H.↗

Launch Control System Software Development System Automation Testing

The Spaceport Command and Control System (SCCS) is the National Aeronautics and Space Administration's (NASA) launch control system for the Orion capsule and Space Launch System, the next generation manned rocket currently in development. This system requires high quality testing that will measure and test the capabilities of the system. For the past two years, the Exploration and Operations Division at Kennedy Space Center (KSC) has assigned a group including interns and full-time engineers to develop automated tests to save the project time and money. The team worked on automating the testing process for the SCCS GUI that would use streamed simulated data from the testing servers to produce data, plots, statuses, etc. to the GUI. The software used to develop automated tests included an automated testing framework and an automation library. The automated testing framework has a tabular-style syntax, which means the functionality of a line of code must have the appropriate number of tabs for the line to function as intended. The header section contains either paths to custom resources or the names of libraries being used. The automation library contains functionality to automate anything that appears on a desired screen with the use of image recognition software to detect and control GUI components. The data section contains any data values strictly created for the current testing file. The body section holds the tests that are being run. The function section can include any number of functions that may be used by the current testing file or any other file that resources it. The resources and body section are required for all test files; the data and function sections can be left empty if the data values and functions being used are from a resourced library or another file. To help equip the automation team with better tools, the Project Lead of the Automated Testing Team, Jason Kapusta, assigned the task to install and train an optical character recognition (OCR) tool to Brandon Echols, a fellow intern, and I. The purpose of the OCR tool is to analyze an image and find the coordinates of any group of text. Some issues that arose while installing the OCR tool included the absence of certain libraries needed to train the tool and an outdated software version. We eventually resolved the issues and successfully installed the OCR tool. Training the tool required many images and different fonts and sizes, but in the end the tool learned to accurately decipher the text in the images and their coordinates. The OCR tool produced a file that contained significant metadata for each section of text, but only the text and coordinates of the text was required for our purpose. The team made a script to parse the information we wanted from the OCR file to a different file that would be used by automation functions within the automated framework. Since a majority of development and testing for the automated test cases for the GUI in question has been done using live simulated data on the workstations at the Launch Control Center (LCC), a large amount of progress has been made. As of this writing, about 60% of all of automated testing has been implemented. Additionally, the OCR tool will help make our automated tests more robust due to the tool's text recognition being highly scalable to different text fonts and text sizes. Soon we will have the whole test system automated, allowing for more full-time engineers working on development projects.

Automation↗

Wearable Biosensor Monitor to Support Autonomous Crew Health and Readiness to Perform

For future human exploration missions, NASA needs a health monitoring system composed of hardware that is compact, fully interoperable with an integrated data management system, and requires minimal consumables. Such a system will be achieved through the integration of small, easy to use biomedical sensors that will have the ability to measure, store and transmit physiological parameters during operational and ambulatory activity. Since 2012, the Canadian Space Agency (CSA) has been active in funding the development of wearable biomonitoring sensors. The Astroskin is the first prototype and consists of a shirt-based garment and headband with embedded sensors, and associated software and technology that measure vital signs, sleep quality and activity level of the wearer. NASA and CSA have been collaborating since 2014 to test and validate this system in a lab environment at Ames Research Center and more recently in the Human Exploration Research Analog (HERA) located at Johnson Spaceflight Center. Specific objectives of the HERA study were: 1) to assess the performance of the Astroskin biosensor system for long-term health monitoring (24-hours) capabilities and during exercise as a measure of crew fitness; 2) to obtain crew feedback on comfort and usability of the Astroskin system; 3) to demonstrate performance of Bluetooth communication during real-time transmission and for verification of data in this environment; and 4) to obtain baseline data for further development of algorithms and tools that facilitate decision support for diagnosing and monitoring of a sick or injured crewmember. HERA Campaign 3 included four missions (each 30-days in duration) with four crewmembers assigned to each mission. A total of 9 men and 7 women participated in the Astroskin evaluation that included continuous physiological monitoring (24-hours) on mission days MD-11, MD1 (high workload), MD15 (low workload), MD19, MD29, and MD+7. Mission days 19 and 29 also included 30 minutes of sub-maximal exercise on a cycle ergometer. Following each 24-hour monitoring session crew physiological data were downloaded to laptops and each crewmember completed a 28 question survey on their experiences with the Astroskin hardware and software. This presentation will focus on lessons learned from the HERA missions. Specifically it will address Astroskin system performance in terms of data loss and data quality (no comparison to lab standard devices), wireless communication with the onboard mobile device, crew usability and comfort, and future development of a next generation biomonitoring system.

crew fitness↗