Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Software testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39

Increasing Cognitive Ability/Reserve Using Software – Pilot (ICARUS-Pilot)

Background: This research study was competitively awarded under the 2022 JSC Innovation Charge Account (ICA) program administered by NASA Johnson Space Center’s Joint Technology Working Group. Study period of performance was May through September 2022, with a maximum allowed procurement budget of $10K. The study sought to quantify and assess the potential benefit of using commercial-off-the-shelf (COTS) cognitive training software to improve cognitive performance in an astronaut-like terrestrial population. Methods: Five volunteer research participants were recruited from the JSC employee population to mimic certain demographic characteristics of the NASA astronaut population (age, education/discipline). Participant cognitive performance was assessed before and after executing eighteen sessions of remote cognitive training executed nominally three times per week using six exercises within an adaptive app-based COTS software package (BrainHQ, Posit Science) on study-provided tablets. Pre- and post-training cognitive performance was measured using internal assessments in BrainHQ as well as Cognition Test Battery (CTB) version ISS B01 v3 (3.0.9-201710021500), an independent software test developed specifically for NASA and used currently in research studies on astronauts. BrainHQ exercises were posited to map well or partially to several CTB sub-tests. Participants provided feedback on their study experience formally via semi-structured interview at the conclusion of testing and informally throughout the study if they encountered issues. Results: The enrolled ICARUS-Pilot study participants generally matched Artemis crew demographic characteristics. Four of five participants have completed study training and assessment activities as of the writing of this abstract. These test participants complied well with desired training session frequency and duration yielding an average cumulative active training duration of 15 hours over an average of 45 days; participants showed 78% average improvement in metric performance for the six trained exercises, with an associated overall 33%ile ranking increase against performance of the entire BrainHQ subscribing population for internal pre/post assessment, agreeing with post-study survey self-reported performance increases. CTB overall feedback scoring, not corrected for learning effects, showed an average of 19% performance improvement across its 10 performance measures over the training period for the completed participants. Detailed analyses will be conducted once participant data collection for the study is complete and the resulting dataset is fully populated. Discussion: These preliminary results provide a positive trend for the effectiveness of the training approach, but further analysis will be needed to establish significance, investigate far transfer, and suggest the needed participant pool size for subsequent efforts to achieve statistically significant outcomes given similar results. The pilot study has already been helpful by allowing the study team to learn a great deal about the capabilities and limitations of the COTS software package that will be reflected in future proposals along with revised timelines for study execution and test participant management. From participant feedback, one common thread regarding the COTS training was that it felt overly repetitive – future proposals should reassess overall training duration, available levels for each trained exercise, and the behavior of the BrainHQ internal scheduler in determining which exercises should be trained and for how long. If the final analysis of this feasibility study ultimately supports it, the study team will recommend further investigation to fully evaluate this potential countermeasure and optimize its implementation. Future proposals would cite this feasibility study’s outcome and would seek to refine the training protocol and obtain statistically significant results for cognitive performance increases as well as retention data.

cognitive training↗

A Survey of Multizone Airflow Simulation Software

H-Canyon simulates the building and process ventilation to ensure positive contamination control for any modifications as well as evaluating off-normal conditions. Simulations can also be used to identify air quality of personnel occupied areas. The models can be classified into two groups, single and multi-zone models. As the name states, a single zone model assumes that the building can be described by a single, well-mixed zone. Whereas large buildings have structures that must be described as multizone. Multizone airflow models deal with the difficulty of flows in a building by distinguishing the effects of internal flow restrictions. Several models have been developed that take internal partitions into account. CONTAM (currently used) is a multizone airflow program created by NIST to determine airflows including infiltration, exfiltration, wind pressures acting outside the building, buoyancy effects caused by the indoor and outdoor air temperature difference. COMIS is a multizone air infiltration, ventilation, and contaminant transport model that was developed at Lawrence Berkeley National Laboratory. COMIS deals with applied pressure distribution, airflow components, HVAC systems, schedules, and contaminant transport. EnergyPlus is designed by DOE. It supports flexible component level configuration of HVAC models and simulates sub-hourly time steps to control fast dynamic systems. The program features include providing accurate environmental performance data including energy consumption and carbon emissions, and reporting solar gains on surfaces. Heavent is used to model industrial ventilation systems to control airborne contaminants. It's designed based on the American Conference of Governmental Industrial Hygienists (ACGIH) Ventilation manual. DesignBuilder is the most recognized and advanced user interface to EnergyPlus. It gives access to all of the most frequently required simulation capabilities covering building fabric, thermal mass, glazing, shading renewables, and HVAC analysis. The objective of this study is to identify all relevant software available including commercially licensed, freely licensed, and open source, that can be used to simulate large, complex process facility ventilation systems. All discovered software tools are evaluated for salient features and compared to the software tool currently used to simulate the H-Canyon ventilation system. The design criteria of a ventilation system has two basic requirements: Supplying fresh air for the occupants and changing the air in the room sufficiently; therefore, the smells, fumes, and contaminants will be removed. In this study, first, whether the software simulates airflow (infiltration, natural ventilation, and interzonal) or considers them to be inputs is discussed. Second, if indoor contaminant levels are considered; and finally suggestions on how airflow analysis can be more appropriately accounted for when performing airflow simulation of a building. All of the airflow software reviewed for this study can account for constant infiltration and natural ventilation rates that are not affected by changes in indoor and outdoor conditions. Modelling infiltration in EnergyPlus or DesignBuilder are applicable for small buildings and not appropriate for larger buildings like the H-Canyon. The effect of wind on external pressures, infiltration, and natural and mechanical ventilation, can also be calculated using all of these software. These programs have control options to switch between natural to mechanical ventilation modes or both (hybrid). Interzonal airflow can be applied which means one zone can be specified to receive a fixed amount of air from another zone. The following ventilation modeling software is recommended for further testing: CONTAM, COMIS, Heavent. The next step in evaluating the software will consist of creating test facility simulations using evaluation copies of the software.

42 ENGINEERING↗

Validation and Verification of LADEE Models and Software

The Lunar Atmosphere Dust Environment Explorer (LADEE) mission will orbit the moon in order to measure the density, composition and time variability of the lunar dust environment. The ground-side and onboard flight software for the mission is being developed using a Model-Based Software methodology. In this technique, models of the spacecraft and flight software are developed in a graphical dynamics modeling package. Flight Software requirements are prototyped and refined using the simulated models. After the model is shown to work as desired in this simulation framework, C-code software is automatically generated from the models. The generated software is then tested in real time Processor-in-the-Loop and Hardware-in-the-Loop test beds. Travelling Road Show test beds were used for early integration tests with payloads and other subsystems. Traditional techniques for verifying computational sciences models are used to characterize the spacecraft simulation. A lightweight set of formal methods analysis, static analysis, formal inspection and code coverage analyses are utilized to further reduce defects in the onboard flight software artifacts. These techniques are applied early and often in the development process, iteratively increasing the capabilities of the software and the fidelity of the vehicle models and test beds.

Gundy-Burlet, Karen↗

Technical Reference Suite Addressing Challenges of Providing Assurance for Fault Management Architectural Design

Research into complexities of software systems Fault Management (FM) and how architectural design decisions affect safety, preservation of assets, and maintenance of desired system functionality has coalesced into a technical reference (TR) suite that advances the provision of safety and mission assurance. The NASA Independent Verification and Validation (IVV) Program, with Software Assurance Research Program support, extracted FM architectures across the IVV portfolio to evaluate robustness, assess visibility for validation and test, and define software assurance methods applied to the architectures and designs. This investigation spanned IVV projects with seven different primary developers, a wide range of sizes and complexities, and encompassed Deep Space Robotic, Human Spaceflight, and Earth Orbiter mission FM architectures. The initiative continues with an expansion of the TR suite to include Launch Vehicles, adding the benefit of investigating differences intrinsic to model-based FM architectures and insight into complexities of FM within an Agile software development environment, in order to improve awareness of how nontraditional processes affect FM architectural design and system health management.

Fitz, Rhonda↗

Navigation Flight Test Results from the Low Power Transceiver Communications and Navigation Demonstration on Shuttle (CANDOS) Experiment

This paper presents the Global Positioning System (GPS) navigation results from the Communications and Navigation Demonstration on Shuttle (CANDOS) experiment flown on STS-107. This experiment was the initial flight of a Low Power Transceiver (LPT) that featured high capacity space- space and space-ground communications and GPS- based navigation capabilities. The LPT also hosted the GPS Enhanced Orbit Determination Experiment (GEODE) orbit determination software. All CANDOS test data were recovered during the mission using LPT communications links via the Tracking and Data Relay Satellite System (TDRSS). An overview of the LPT s navigation software and the GPS experiment timeline is presented, along with comparisons of test results to the NASA Johnson Space Center (JSC) real-time ground navigation vectors and Best Estimate of Trajectory (BET).

Haas, Lin↗

Exploring the Use of a Test Automation Framework

It is known that software testers, more often than not, lack the time needed to fully test the delivered software product within the time period allotted to them. When problems in the implementation phase of a development project occur, it normally causes the software delivery date to slide. As a result, testers either need to work longer hours, or supplementary resources need to be added to the test team in order to meet aggressive test deadlines. One solution to this problem is to provide testers with a test automation framework to facilitate the development of automated test solutions.

TAF STAF Test Automation↗

Computer Program for Space-Shuttle Testing

Demand on Space Shuttle general-purpose computers reduced. Simulations Testbed and Scenario Pre-processor (STB&SPP) system reduces need for use of GPC's in hardware and software development and testing. System consists of computer program, SPP, and set of utility subroutines, STB, which incorporates Interface Simulator (ISIM). STB&SPP system written in FORTRAN V and Assembler.

Hyman, M. D.↗

Software development predictors, error analysis, reliability models and software metric analysis

The use of dynamic characteristics as predictors for software development was studied. It was found that there are some significant factors that could be useful as predictors. From a study on software errors and complexity, it was shown that meaningful results can be obtained which allow insight into software traits and the environment in which it is developed. Reliability models were studied. The research included the field of program testing because the validity of some reliability models depends on the answers to some unanswered questions about testing. In studying software metrics, data collected from seven software engineering laboratory (FORTRAN) projects were examined and three effort reporting accuracy checks were applied to demonstrate the need to validate a data base. Results are discussed.

Basili, Victor↗

Application of Self Nulling Eddy Current Probe Technique to the Detection of Fatigue Crack Initiation and Control of Test Procedures

A major part of fracture mechanics is concerned with studying the initiation and propagation of fatigue cracks. This typically requires constant monitoring of crack growth during fatigue cycles and the knowledge of the precise location of the crack tip at any given time. One technique currently available for measuring fatigue crack length is the Potential Drop method. The method, however, may be inaccurate if the direction of crack growth deviates considerably from what was assumed initially or the curvature of the crack becomes significant. Another popular approach is to optically view the crack using a high magnification microscope, but this entails a person constantly monitoring it. The present proposed technique uses an automated scheme, in order to eliminate the need for a person to constantly monitor the experiment. Another technique under development elsewhere is to digitize an optical image of the test specimen surface and then apply a pattern recognition algorithm to locate the crack tip. A previous publication showed that the self nulling eddy current probe successfully tracked a simulated crack in an aluminum sample. This was the impetus to develop an online real time crack monitoring system. An automated system has been developed which includes a two axis scanner mounted on the tensile testing machine, the probe and its instrumentation and a personal computer (PC) to communicate and control all the parameters. The system software controls the testing parameters as well as monitoring the fatigue crack as it propagates. This paper will discuss the experimental setup in detail and demonstrate its capabilities. A three dimensional finite element model is utilized to model the magnetic field distribution due to the probe and how the probe voltage changes as it scans the crack. Experimental data of the probe for different samples under zero load, static load and high cycle fatigue load will be discussed. The final section summarizes the major accomplishments of the present work, the elements of the future R&D needs and the advantages and disadvantages of using this system in the laboratory and field.

Namkung, M.↗

Technical Reference Suite Addressing Challenges of Providing Assurance for Fault Management Architectural Design

Research into complexities of software systems Fault Management (FM) and how architectural design decisions affect safety, preservation of assets, and maintenance of desired system functionality has coalesced into a technical reference (TR) suite that advances the provision of safety and mission assurance. The NASA Independent Verification and Validation (IV&V) Program, with Software Assurance Research Program support, extracted FM architectures across the IV&V portfolio to evaluate robustness, assess visibility for validation and test, and define software assurance methods applied to the architectures and designs. This investigation spanned IV&V projects with seven different primary developers, a wide range of sizes and complexities, and encompassed Deep Space Robotic, Human Spaceflight, and Earth Orbiter mission FM architectures. The initiative continues with an expansion of the TR suite to include Launch Vehicles, adding the benefit of investigating differences intrinsic to model-based FM architectures and insight into complexities of FM within an Agile software development environment, in order to improve awareness of how nontraditional processes affect FM architectural design and system health management. The identification of particular FM architectures, visibility, and associated IV&V techniques provides a TR suite that enables greater assurance that critical software systems will adequately protect against faults and respond to adverse conditions. Additionally, the role FM has with regard to strengthened security requirements, with potential to advance overall asset protection of flight software systems, is being addressed with the development of an adverse conditions database encompassing flight software vulnerabilities. Capitalizing on the established framework, this TR suite provides assurance capability for a variety of FM architectures and varied development approaches. Research results are being disseminated across NASA, other agencies, and the software community. This paper discusses the findings and TR suite informing the FM domain in best practices for FM architectural design, visibility observations, and methods employed for IV&V and mission assurance.

Fitz, Rhonda↗

Software Reliability 2002

In FY01 we learned that hardware reliability models need substantial changes to account for differences in software, thus making software reliability measurements more effective, accurate, and easier to apply. These reliability models are generally based on familiar distributions or parametric methods. An obvious question is 'What new statistical and probability models can be developed using non-parametric and distribution-free methods instead of the traditional parametric method?" Two approaches to software reliability engineering appear somewhat promising. The first study, begin in FY01, is based in hardware reliability, a very well established science that has many aspects that can be applied to software. This research effort has investigated mathematical aspects of hardware reliability and has identified those applicable to software. Currently the research effort is applying and testing these approaches to software reliability measurement, These parametric models require much project data that may be difficult to apply and interpret. Projects at GSFC are often complex in both technology and schedules. Assessing and estimating reliability of the final system is extremely difficult when various subsystems are tested and completed long before others. Parametric and distribution free techniques may offer a new and accurate way of modeling failure time and other project data to provide earlier and more accurate estimates of system reliability.

Wallace, Dolores R.↗

Simulation-To-Flight (STF-1): A Mission to Enable CubeSat Software-Based Validation and Verification

The Simulation-to-Flight 1 (STF-1) CubeSat mission aims to demonstrate how legacy simulation technologies may be adapted for flexible and effective use on missions using the CubeSat platform. These technologies, named NASA Operational Simulator (NOS), have demonstrated significant value on several missions such as James Webb Space Telescope, Global Precipitation Measurement, Juno, and Deep Space Climate Observatory in the areas of software development, mission operations/training, verification and validation (V&V), test procedure development and software systems check-out. STF-1 will demonstrate a highly portable simulation and test platform that allows seamless transition of mission development artifacts to flight products. This environment will decrease development time of future CubeSat missions by lessening the dependency on hardware resources. In addition, through a partnership between NASA GSFC, the West Virginia Space Grant Consortium and West Virginia University, the STF-1 CubeSat will hosts payloads for three secondary objectives that aim to advance engineering and physical-science research in the areas of navigation systems of small satellites, provide useful data for understanding magnetosphere-ionosphere coupling and space weather, and verify the performance and durability of III-V Nitride-based materials.

navigation↗

Software Verification Toolkit (SVT): Survey on Available Software Verification Tools and Future Direction

Writing software is difficult. However, writing complex, well tested and designed, and functionally correct software is incredibly difficult. An entire field of study is devoted to the validation and verification of software to address this problem, and in this paper we analyze the landscape of currently available third party software. We have divided our analyses into three separate subsections with regards to software validation: formal methods, static analysis, and test generation. Formal verification is the most complex method in which to validate software correctness, but also the most thorough as it truly validates the mathematical validity of the source code. Static analysis generally is relegated to abstract syntax tree traversal techniques to find errors related to faulty software such as memory leaks or stack overflow issues. Automatic test generation is similar in implementation to static analysis, but pushes a bit further in verifying the boundedness of function inputs and outputs with regards to annotated or parsed criteria. The crux of this report is to analyze and describe the software tools that implement these techniques to validate and verify software. Pros and cons related to installation, utilization, and capabilities of the frameworks are described, and reproducible examples are provided with a focus on usability. The initial survey concluded that the most interesting tools of note are Z3, Isabelle/HOL, and TLA+ with regards to formal verification; and Infer, Frama-C, and SonarQube with regards to static analysis. With these tools in mind, a final conjecture is provided that describes future avenues of utilizing these tools for developing a verification framework to assist in validating existing software at Sandia National Laboratories.

97 MATHEMATICS AND COMPUTING↗

A Thermal Expert System (TEXSYS) development overview - AI-based control of a Space Station prototype thermal bus

A knowledge-based control system for real-time control and fault detection, isolation and recovery (FDIR) of a prototype two-phase Space Station Freedom external thermal control system (TCS) is discussed in this paper. The Thermal Expert System (TEXSYS) has been demonstrated in recent tests to be capable of both fault anticipation and detection and real-time control of the thermal bus. Performance requirements were achieved by using a symbolic control approach, layering model-based expert system software on a conventional numerical data acquisition and control system. The model-based capabilities of TEXSYS were shown to be advantageous during software development and testing. One representative example is given from on-line TCS tests of TEXSYS. The integration and testing of TEXSYS with a live TCS testbed provides some insight on the use of formal software design, development and documentation methodologies to qualify knowledge-based systems for on-line or flight applications.

Glass, B. J.↗

Intelligent Wireless Sensor Networks for System Health Monitoring

Wireless sensor networks (WSN) based on the IEEE 802.15.4 Personal Area Network (PAN) standard are finding increasing use in the home automation and emerging smart energy markets. The network and application layers, based on the ZigBee 2007 Standard, provide a convenient framework for component-based software that supports customer solutions from multiple vendors. WSNs provide the inherent fault tolerance required for aerospace applications. The Discovery and Systems Health Group at NASA Ames Research Center has been developing WSN technology for use aboard aircraft and spacecraft for System Health Monitoring of structures and life support systems using funding from the NASA Engineering and Safety Center and Exploration Technology Development and Demonstration Program. This technology provides key advantages for low-power, low-cost ancillary sensing systems particularly across pressure interfaces and in areas where it is difficult to run wires. Intelligence for sensor networks could be defined as the capability of forming dynamic sensor networks, allowing high-level application software to identify and address any sensor that joined the network without the use of any centralized database defining the sensors characteristics. The IEEE 1451 Standard defines methods for the management of intelligent sensor systems and the IEEE 1451.4 section defines Transducer Electronic Datasheets (TEDS), which contain key information regarding the sensor characteristics such as name, description, serial number, calibration information and user information such as location within a vehicle. By locating the TEDS information on the wireless sensor itself and enabling access to this information base from the application software, the application can identify the sensor unambiguously and interpret and present the sensor data stream without reference to any other information. The application software is able to read the status of each sensor module, responding in real-time to changes of PAN configuration, providing the appropriate response for maintaining overall sensor system function, even when sensor modules fail or the WSN is reconfigured. The session will present the architecture and technical feasibility of creating fault-tolerant WSNs for aerospace applications based on our application of the technology to a Structural Health Monitoring testbed. The interim results of WSN development and testing including our software architecture for intelligent sensor management will be discussed in the context of the specific tradeoffs required for effective use. Initial certification measurement techniques and test results gauging WSN susceptibility to Radio Frequency interference are introduced as key challenges for technology adoption. A candidate Developmental and Flight Instrumentation implementation using intelligent sensor networks for wind tunnel and flight tests is developed as a guide to understanding key aspects of the aerospace vehicle design, test and operations life cycle.

networks↗

Commercial Capaciflector

A capacitive proximity/tactile sensor with unique performance capabilities ('capaciflector' or capacitive reflector) is being developed by NASA/Goddard Space Flight Center (GSFC) for use on robots and payloads in space in the interests of safety, efficiency, and ease of operation. Specifically, this sensor will permit robots and their attached payloads to avoid collisions in space with humans and other objects and to dock these payloads in a cluttered environment. The sensor is simple, robust, and inexpensive to manufacture with obvious and recognized commercial possibilities. Accordingly, NASA/GSFC, in conjunction with industry, is embarking on an effort to 'spin' this technology off into the private sector. This effort includes prototypes aimed at commercial applications. The principles of operation of these prototypes are described along with hardware, software, modelling, and test results. The hardware description includes both the physical sensor in terms of a flexible printed circuit board and the electronic circuitry. The software description will include filtering and detection techniques. The modelling will involve finite element electric field analysis and will underline techniques used for design optimization.

Vranish, John M.↗

Towards Test Driven Development for Computational Science with pFUnit

Developers working in Computational Science & Engineering (CSE)/High Performance Computing (HPC) must contend with constant change due to advances in computing technology and science. Test Driven Development (TDD) is a methodology that mitigates software development risks due to change at the cost of adding comprehensive and continuous testing to the development process. Testing frameworks tailored for CSE/HPC, like pFUnit, can lower the barriers to such testing, yet CSE software faces unique constraints foreign to the broader software engineering community. Effective testing of numerical software requires a comprehensive suite of oracles, i.e., use cases with known answers, as well as robust estimates for the unavoidable numerical errors associated with implementation with finite-precision arithmetic. At first glance these concerns often seem exceedingly challenging or even insurmountable for real-world scientific applications. However, we argue that this common perception is incorrect and driven by (1) a conflation between model validation and software verification and (2) the general tendency in the scientific community to develop relatively coarse-grained, large procedures that compound numerous algorithmic steps.We believe TDD can be applied routinely to numerical software if developers pursue fine-grained implementations that permit testing, neatly side-stepping concerns about needing nontrivial oracles as well as the accumulation of errors. We present an example of a successful, complex legacy CSE/HPC code whose development process shares some aspects with TDD, which we contrast with current and potential capabilities. A mix of our proposed methodology and framework support should enable everyday use of TDD by CSE-expert developers.

pFUnit↗

NDAS Hardware Translation Layer Development

The NASA Data Acquisition System (NDAS) project is aimed to replace all DAS software for NASA s Rocket Testing Facilities. There must be a software-hardware translation layer so the software can properly talk to the hardware. Since the hardware from each test stand varies, drivers for each stand have to be made. These drivers will act more like plugins for the software. If the software is being used in E3, then the software should point to the E3 driver package. If the software is being used at B2, then the software should point to the B2 driver package. The driver packages should also be filled with hardware drivers that are universal to the DAS system. For example, since A1, A2, and B2 all use the Preston 8300AU signal conditioners, then the driver for those three stands should be the same and updated collectively.

Nazaretian, Ryan N.↗