Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software verification and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

Rule groupings: A software engineering approach towards verification of expert systems

Currently, most expert system shells do not address software engineering issues for developing or maintaining expert systems. As a result, large expert systems tend to be incomprehensible, difficult to debug or modify and almost impossible to verify or validate. Partitioning rule based systems into rule groups which reflect the underlying subdomains of the problem should enhance the comprehensibility, maintainability, and reliability of expert system software. Attempts were made to semiautomatically structure a CLIPS rule base into groups of related rules that carry the same type of information. Different distance metrics that capture relevant information from the rules for grouping are discussed. Two clustering algorithms that partition the rule base into groups of related rules are given. Two independent evaluation criteria are developed to measure the effectiveness of the grouping strategies. Results of the experiment with three sample rule bases are presented.

Mehrotra, Mala↗

A digital flight control system verification laboratory

A NASA/FAA program has been established for the verification and validation of digital flight control systems (DFCS), with the primary objective being the development and analysis of automated verification tools. In order to enhance the capabilities, effectiveness, and ease of using the test environment, software verification tools can be applied. Tool design includes a static analyzer, an assertion generator, a symbolic executor, a dynamic analysis instrument, and an automated documentation generator. Static and dynamic tools are integrated with error detection capabilities, resulting in a facility which analyzes a representative testbed of DFCS software. Future investigations will ensue particularly in the areas of increase in the number of software test tools, and a cost effectiveness assessment.

De Feo, P.↗

Recent Developments and Challenges Implementing New and Improved Stress Intensity Factor (K) Solutions in NASGRO for Damage Tolerance Analyses

Fatigue crack growth analysis software has been available to damage tolerance analysts for many years in either commercial products or via proprietary in-house codes. The NASGRO software has been publicly available since the mid-80s (known as NASA/FLAGRO up to 1999) and since 2000 has been sustained and further developed by a collaborative effort between Southwest Research Institute® (SwRI®), the NASA Johnson Space Center (JSC), and the members of the NASGRO Industrial Consortium. Since the stress intensity factor (K) is the foundation of fracture mechanics and damage tolerance analysis of aircraft structures, a significant focus of development efforts in the past fifteen years has been geared towards enhancing legacy K solutions and developing new and efficient numerical K solutions that can handle the complicated stress gradients computed by today’s analysts using detailed finite element models of fatigue critical locations. This paper provides an overview of K solutions that have been recently implemented or improved for the analysis of geometries such as two unequal through cracks at a hole and two unequal corner cracks at a hole, as well as state-of-the-art weight function models capable of computing K in the presence of univariant and/or bivariant stress gradients and complicated residual stress distributions. Some historical background is provided to review how common K solutions have evolved over the years, including selective examples from the literature and from new research. Challenges and progress in rectifying discrepancies between older legacy solutions and newer models are reviewed as well as approaches and challenges for verification and validation of K solutions. Finally, a summary of current challenges and future research and development needs is presented. A key theme throughout the presentation of this paper will be how members of the aerospace industry have collaborated with software developers to develop a practical analysis tool that is used world-wide to support new design as well as the ongoing sustainment and airworthiness of commercial and military aircraft.

Cardinal, Joseph W.↗

Simulator verification techniques study. Integrated simulator self test system concepts

Software and hardware requirements for implementing hardware self tests are presented in support of the development of training and procedures development simulators for the space shuttle program. Self test techniques for simulation hardware and the validation of simulation performance are stipulated. The requirements of an integrated simulator self system are analyzed. Readiness tests, fault isolation tests, and incipient fault detection tests are covered.

Montoya, G.↗

Synthetic Scene Generation of Stennis Target Range for the Calibration of Remote Sensing Systems

The validation and verification (V&V) target range developed at Stennis Space Center is a useful tool for the calibration of remote sensing systems. In this paper, we present an algorithm for generating synthetic scenes or digital models of this target range. The atmospheric radiative transfer code Modtran 4 is used in simulating at-sensor radiance for a given sensor. Software is developed to generate the scene with different spatial and spectral resolutions using the at-sensor radiance values. Synthetic scenes provide a cost-effective way for in-flight validation for the spatial and radiometric accuracy of the data. Other applications include mission planning, sensor simulation, and trade-off analysis in sensor design. The radiometric and spatial accuracy of the simulation is evaluated by comparing simulated scenes with AVIRIS acquired values.

Cao, Chang-Yong↗

Proceedings of the Twenty-Third Annual Software Engineering Workshop

The Twenty-third Annual Software Engineering Workshop (SEW) provided 20 presentations designed to further the goals of the Software Engineering Laboratory (SEL) of the NASA-GSFC. The presentations were selected on their creativity. The sessions which were held on 2-3 of December 1998, centered on the SEL, Experimentation, Inspections, Fault Prediction, Verification and Validation, and Embedded Systems and Safety-Critical Systems.

Source record↗

Verifying Data Integrity of Electronically Scanned Pressure Systems at the NASA Glenn Research Center

The proper operation of the Electronically Scanned Pressure (ESP) System critical to accomplish the following goals: acquisition of highly accurate pressure data for the development of aerospace and commercial aviation systems and continuous confirmation of data quality to avoid costly, unplanned, repeat wind tunnel or turbine testing. Standard automated setup and checkout routines are necessary to accomplish these goals. Data verification and integrity checks occur at three distinct stages, pretest pressure tubing and system checkouts, daily system validation and in-test confirmation of critical system parameters. This paper will give an overview of the existing hardware, software and methods used to validate data integrity.

Panek, Joseph W.↗

Software for Statistical Analysis of Weibull Distributions with Application to Gear Fatigue Data: User Manual with Verification

The Weibull distribution has been widely adopted for the statistical description and inference of fatigue data. This document provides user instructions, examples, and verification for software to analyze gear fatigue test data. The software was developed presuming the data are adequately modeled using a two-parameter Weibull distribution. The calculations are based on likelihood methods, and the approach taken is valid for data that include type 1 censoring. The software was verified by reproducing results published by others.

Krantz, Timothy L.↗

Software for Statistical Analysis of Weibull Distributions with Application to Gear Fatigue Data: User Manual with Verification

The Weibull distribution has been widely adopted for the statistical description and inference of fatigue data. This document provides user instructions, examples, and verification for software to analyze gear fatigue test data. The software was developed presuming the data are adequately modeled using a two-parameter Weibull distribution. The calculations are based on likelihood methods, and the approach taken is valid for data that include type I censoring. The software was verified by reproducing results published by others.

Kranz, Timothy L.↗

The Planning Execution Monitoring Architecture

The Planning Execution Monitoring (PEM) architecture is a design concept for developing autonomous cockpit command and control software. The PEM architecture is designed to reduce the operations costs in the space transportation system through the use of automation while improving safety and operability of the system. Specifically, the PEM autonomous framework enables automatic performance of many vehicle operations that would typically be performed by a human. Also, this framework supports varying levels of autonomous control, ranging from fully automatic to fully manual control. The PEM autonomous framework interfaces with the core flight software to perform flight procedures. It can either assist human operators in performing procedures or autonomously execute routine cockpit procedures based on the operational context. Most importantly, the PEM autonomous framework promotes and simplifies the capture, verification, and validation of the flight operations knowledge. Through a hierarchical decomposition of the domain knowledge, the vehicle command and control capabilities are divided into manageable functional "chunks" that can be captured and verified separately. These functional units, each of which has the responsibility to manage part of the vehicle command and control, are modular, re-usable, and extensible. Also, the functional units are self-contained and have the ability to plan and execute the necessary steps for accomplishing a task based upon the current mission state and available resources. The PEM architecture has potential for application outside the realm of spaceflight, including management of complex industrial processes, nuclear control, and control of complex vehicles such as submarines or unmanned air vehicles.

Wang, Lui↗

Dynamic Stability Analysis Using Free-Flight CFD

Blunt body probes are the primary design for atmospheric entry. The blunted capsule shape, while desirable for the decreased forebody heating, typically experiences dynamic instability at low supersonic and transonic flight. At moderate to low supersonic conditions, the unsteady wake begins to adversely interact with the vehicle in free-flight resulting in attitude oscillations which can grow in time and result in tumbling. Understanding vehicle dynamics is thus important when considering vehicle entry design and guidance navigation and control (GN&C) considerations. Historically, dynamic stability has been carried out exclusively using experimental techniques. The wide range of experimental approaches provide reasonable coverage in Mach and dynamic pressure space for a given trajectory, however achieving flight similitude is often difficult or impossible. Advancements in Computational Fluid Dynamics (CFD) techniques allow for dynamic motion of the vehicle surface during the simulation with either forced/prescribed motion or free motion in response to fluid forces. Several examples of CFD software with the ability to simulate fluid dynamics and vehicle motion are available in literature. The Free-Flight CFD (FF-CFD) capability developed by NASA Ames within the US3D flow solver has shown extensive validation and verification (V&V) efforts from moderate to low supersonic flow across a range of vehicle architectures. The solver is able to simulate full rotational and translation motion resulting in full 6 degrees-of-freedom (DoF) motion. The validation and verification efforts the solver's ability to predict the motion of a ballistic range model as compared to experimental results. Similar to other computational capabilities in literature, FF-CFD relied on heritage data reduction methodologies and processes for reducing dynamic data into aerodynamic coefficients intended for use in design and trajectory codes. These heritage approaches, initially developed with the assumptions (and restrictions) of ground test facilities, resulted in the significant culling of the full 6-DoF dynamic data set to discrete values which are then fit using an aerodynamic model. Recent development of simulation and data reduction methodologies has resulted in an improved approach to FF-CFD simulations of vehicle dynamics which leverage the control of dynamic constraints within a simulation set-up. The presentation of this work will cover the constrained dynamic simulation approach for deriving dynamic coefficients for a Genesis capsule. The methodology will be outlined and a discussion of implications in motion restriction will be discussed. Finally, reconstructed vehicle trajectories using FF-CFD derived aerodynamic coefficients will be presented.

Joseph M Brock↗

NASA's Optical Program on Ascension Island: Bringing MCAT to Life as the Eugene Stansbery-Meter Class Autonomous Telescope (ES-MCAT)

In June 2015, the construction of the Meter Class Autonomous Telescope was completed and MCAT saw the light of the stars for the first time. In 2017, MCAT was newly dedicated as the Eugene Stansbery-MCAT telescope by NASA's Orbital Debris Program Office (ODPO), in honor of his inspiration and dedication to this newest optical member of the NASA ODPO. Since that time, MCAT has viewed the skies with one engineering camera and two scientific cameras, and the ODPO optical team has begun the process of vetting the entire system. The full system vetting includes verification and validation of: (1) the hardware comprising the system (e.g. the telescopes and its instruments, the dome, weather systems, all-sky camera, FLIR cloud infrared camera, etc.), (2) the custom-written Observatory Control System (OCS) master software designed to autonomously control this complex system of instruments, each with its own control software, and (3) the custom written Orbital Debris Processing software for post-processing the data. ES-MCAT is now capable of autonomous observing to include Geosynchronous survey, TLE (Two-line element) tracking of individual catalogued debris at all orbital regimes (Low-Earth Orbit all the way to Geosynchronous (GEO) orbit), tracking at specified non-sidereal rates, as well as sidereal rates for proper calibration with standard stars. Ultimately, the data will be used for validation of NASA's Orbital Debris Engineering Model, ORDEM, which aids in engineering designs of spacecraft that require knowledge of the orbital debris environment and long-term risks for collisions with Resident Space Objects (RSOs).

Lederer, S. M.↗

Bridging the Gap Between Requirements and Model Analysis : Evaluation on Ten Cyber-Physical Challenge Problems

Formal verfication and simulation are powerful tools to validate requirements against complex systems. [Problem] Requirements are developed in early stages of the software lifecycle and are typically written in ambiguous natural language. There is a gap between such requirements and formal notations that can be used by verification tools, and lack of support for proper association of requirements with software artifacts for verification. [Principal idea] We propose to write requirements in an intuitive, structured natural language with formal semantics, and to support formalization and model/code verification as a smooth, well-integrated process. [Contribution] We have developed an end-to-end, open source requirements analysis framework that checks Simulink models against requirements written in structured natural language. Our framework is built in the Formal Requirements Elicitation Tool (fret); we use fret's requirements language named fretish, and formalization of fretish requirements in temporal logics. Our proposed framework contributes the following features: 1) automatic extraction of Simulink model information and association of fretish requirements with target model signals and components; 2) translation of temporal logic formulas into synchronous dataflow cocospec specifications as well as Simulink monitors, to be used by verification tools; we establish correctness of our translation through extensive automated testing; 3) interpretation of counterexamples produced by verification tools back at requirements level. These features support a tight integration and feedback loop between high level requirements and their analysis. We demonstrate our approach on a major case study: the Ten Lockheed Martin Cyber-Physical, aerospace-inspired challenge problems.

Mavridou, Anastasia↗

Simulating Mars: Enabling Testing of the Perseverance Rover Sampling and Caching Subsystem on Earth

The development of the Sampling and Caching Subsystem (SCS) on the JPL Perseverance Rover lies at the intersection of testing, robotics, and geology. The SCS team established three primary system test campaigns and venues to aid in the development of SCS through verification and validation testing – Qualification Model Dirty Testing (QMDT) to provide a venue for testing in a Martian environment, Vehicle System Testbed (VSTB) for testing while integrated with the mobility subsystem on Martian-like terrain, and the Flight Software Testbed (FSWTB) for conducting tests using the flight motor controllers and software system on a hexapod which had the ability to simulate rover tilt. Each venue contributed a vital piece to the SCS building blocks. However, the QMDT venue operating within a 10-ft diameter Thermal Vacuum chamber to simulate Martian environment provided a sui generis opportunity to fine tune the entire sampling and caching process while building the team’s knowledge base about rock drillability, system life, and target selection. On Earth, because Martian rocks are not readily available, the development team must utilize geoanalogs to the rocks and regolith on Mars. Geologists on the team helped establish a set of standard rock types to use for Mars missions, like Basalt, Sandstone, Mudstone, Gypsum, and other related geoanalogs. These geoanalogs are characterized with a standard suite of tests for density, compressibility, and other characteristics to categorize potential drillability. This concept of drillability is what links the geoanalogs on Earth to the samples we collect on Mars. With the simulant characteristics defined, these geoanalog rocks are ready to be drilled into as we do on the Martian surface. A key aspect of interacting with the surface on Mars is rock target identification and selection. The Perseverance robotic system uses the on-board cameras, instrumentation, and software to collect enough information to identify potential scientific targets. With the targets identified, SCS can place the Corer and abrade the surface or collect a sample. For a ground test activity like QMDT, the test team did not have all of the camera and instrumentation systems that the rover does, so the team developed ground test equivalents to process a rock, build a target map, and define the target. The team constructed a Rock Scanning Station to build a 3D point cloud of the rock. This point cloud was then processed and evaluated with predefined and programmed criteria in a Target Downselect Tool. A primary output of the Target Downselect Tool is a defined target that can be uploaded directly to the robotic software system to simulate and build the robotic sequences used in tests. With these insights and programmatic definition of targets, the QMDT test team was able to make the same decisions that the Perseverance surface operations team does. In addition, valuable lessons learned from developing the target selection ground tools and using them were implemented into the tools used for surface operations.

Kim, Junggon↗

Transcriptomics Processing Pipelines for Space Biology: An Open Source and Consensus-Driven Approach

Transcriptomics holds significant value in elucidating the relationship between gene expression, experimental factors, biological factors, and various types of omics data. Enhancing our understanding of these connections is paramount for foundational biology, which plays a pivotal role in devising solutions for challenges pertinent to both space travel and terrestrial life. The NASA GeneLab project, part of the Open Science Data Repository (OSDR.nasa.gov), seeks to accelerate space biology research through cataloging and democratizing ‘omics data, including transcriptomics. Since raw omics data are largely inaccessible to non-bioinformaticians, GeneLab works with the scientific community via the Open Science Analysis Working Groups (AWGs) to develop standard processing pipelines to generate and publish processed data. Unlike raw data, processed data have greater immediate value to diverse users with varying technical backgrounds and computational capabilities. Standardizing processing workflows is essential to match the pace of raw data generation, ensure reproducibility, and enable standardized processed data for comparison across datasets. As of June 2023, transcriptomics studies comprise over half of GeneLab datasets hosted on the OSDR, including data from bulk RNA-seq and Affymetrix or Agilent 1-Channel DNA microarray assays. In collaboration with the AWGs, GeneLab developed consensus processing pipelines for these transcriptomics data types that includes quality control, background correction (microarray only), data normalization and quantification, culminating in the detection and annotation of differentially expressed genes. The work presented here describes Nextflow implementations of GeneLab’s consensus transcriptomics pipelines that automates and accelerates processing of these datasets. In addition to the core data processing, these workflows also include raw data staging and a robust verification and validation program to identify errors in real-time, stop additional downstream computation, and preserve computational resources. These workflows are used to generate GeneLab processed data hosted on the OSDR, and are publicly available as open source software for others to use at: https://github.com/nasa/GeneLab_Data_Processing.

Jonathan Oribello↗

James Webb Space Telescope Integrated Science Instrument Module Thermal Vacuum Thermal Balance Test Campaign at NASA's Goddard Space Flight Center

The James Webb Space Telescope is a large infrared telescope with a 6.5-meter primary mirror, designed as a successor to the Hubble Space Telescope when launched in 2018. Three of the four science instruments contained within the Integrated Science Instrument Module (ISIM) are passively cooled to their operational temperature range of 36K to 40K with radiators, and the fourth instrument is actively cooled to its operational temperature of approximately 6K. Thermal-vacuum testing of the flight science instruments at the ISIM element level has taken place in three separate highly challenging and extremely complex thermal tests within a gaseous helium-cooled shroud inside Goddard Space Flight Centers Space Environment Simulator. Special data acquisition software was developed for these tests to monitor over 1700 flight and test sensor measurements, track over 50 gradients, component rates, and temperature limits in real time against defined constraints and limitations, and guide the complex transition from ambient to final cryogenic temperatures and back. This extremely flexible system has proven highly successful in safeguarding the nearly $2B science payload during the 3.5-month-long thermal tests. Heat flow measurement instrumentation, or Q-meters, were also specially developed for these tests. These devices provide thermal boundaries o the flight hardware while measuring instrument heat loads up to 600 mW with an estimated uncertainty of 2 mW in test, enabling accurate thermal model correlation, hardware design validation, and workmanship verification. The high accuracy heat load measurements provided first evidence of a potentially serious hardware design issue that was subsequently corrected. This paper provides an overview of the ISIM-level thermal-vacuum tests and thermal objectives; explains the thermal test configuration and thermal balances; describes special measurement instrumentation and monitoring and control software; presents key test thermal results; lists problems encountered during testing and lessons learned.

JWST ISIM Thermal↗

Estimated Probability of a Cervical Spine Injury During an ISS Mission

Introduction: The Integrated Medical Model (IMM) utilizes historical data, cohort data, and external simulations as input factors to provide estimates of crew health, resource utilization and mission outcomes. The Cervical Spine Injury Module (CSIM) is an external simulation designed to provide the IMM with parameter estimates for 1) a probability distribution function (PDF) of the incidence rate, 2) the mean incidence rate, and 3) the standard deviation associated with the mean resulting from injury/trauma of the neck. Methods: An injury mechanism based on an idealized low-velocity blunt impact to the superior posterior thorax of an ISS crewmember was used as the simulated mission environment. As a result of this impact, the cervical spine is inertially loaded from the mass of the head producing an extension-flexion motion deforming the soft tissues of the neck. A multibody biomechanical model was developed to estimate the kinematic and dynamic response of the head-neck system from a prescribed acceleration profile. Logistic regression was performed on a dataset containing AIS1 soft tissue neck injuries from rear-end automobile collisions with published Neck Injury Criterion values producing an injury transfer function (ITF). An injury event scenario (IES) was constructed such that crew 1 is moving through a primary or standard translation path transferring large volume equipment impacting stationary crew 2. The incidence rate for this IES was estimated from in-flight data and used to calculate the probability of occurrence. The uncertainty in the model input factors were estimated from representative datasets and expressed in terms of probability distributions. A Monte Carlo Method utilizing simple random sampling was employed to propagate both aleatory and epistemic uncertain factors. Scatterplots and partial correlation coefficients (PCC) were generated to determine input factor sensitivity. CSIM was developed in the SimMechanics/Simulink environment with a Monte Carlo wrapper (MATLAB) used to integrate the components of the module. Results: The probability of generating an AIS1 soft tissue neck injury from the extension/flexion motion induced by a low-velocity blunt impact to the superior posterior thorax was fitted with a lognormal PDF with mean 0.26409, standard deviation 0.11353, standard error of mean 0.00114, and 95% confidence interval [0.26186, 0.26631]. Combining the probability of an AIS1 injury with the probability of IES occurrence was fitted with a Johnson SI PDF with mean 0.02772, standard deviation 0.02012, standard error of mean 0.00020, and 95% confidence interval [0.02733, 0.02812]. The input factor sensitivity analysis in descending order was IES incidence rate, ITF regression coefficient 1, impactor initial velocity, ITF regression coefficient 2, and all others (equipment mass, crew 1 body mass, crew 2 body mass) insignificant. Verification and Validation (V&V): The IMM V&V, based upon NASA STD 7009, was implemented which included an assessment of the data sets used to build CSIM. The documentation maintained includes source code comments and a technical report. The software code and documentation is under Subversion configuration management. Kinematic validation was performed by comparing the biomechanical model output to established corridors.

Brooker, John E.↗