Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software verification and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22

Simulator verification techniques study. Integrated simulator self test system concepts

Software and hardware requirements for implementing hardware self tests are presented in support of the development of training and procedures development simulators for the space shuttle program. Self test techniques for simulation hardware and the validation of simulation performance are stipulated. The requirements of an integrated simulator self system are analyzed. Readiness tests, fault isolation tests, and incipient fault detection tests are covered.

Montoya, G.↗

Efforts to enhance reproducibility in a human performance research project

Background: Ensuring the validity of results from funded programs is a critical concern for agencies that sponsor biological research. In recent years, the open science movement has sought to promote reproducibility by encouraging sharing not only of finished manuscripts but also of data and code supporting their findings. While these innovations have lent support to third-party efforts to replicate calculations underlying key results in the scientific literature, fields of inquiry where privacy considerations or other sensitivities preclude the broad distribution of raw data or analysis may require a more targeted approach to promote the quality of research output. Methods: We describe efforts oriented toward this goal that were implemented in one human performance research program, Measuring Biological Aptitude, organized by the Defense Advanced Research Project Agency's Biological Technologies Office. Our team implemented a four-pronged independent verification and validation (IV&V) strategy including 1) a centralized data storage and exchange platform, 2) quality assurance and quality control (QA/QC) of data collection, 3) test and evaluation of performer models, and 4) an archival software and data repository. Results: Our IV&V plan was carried out with assistance from both the funding agency and participating teams of researchers. QA/QC of data acquisition aided in process improvement and the flagging of experimental errors. Holdout validation set tests provided an independent gauge of model performance. Conclusions: In circumstances that do not support a fully open approach to scientific criticism, standing up independent teams to cross-check and validate the results generated by primary investigators can be an important tool to promote reproducibility of results.

59 BASIC BIOLOGICAL SCIENCES↗

Involute Working Group – FSI Analysis of Fuel Plates Using Finite Volume and Finite Element Methods

The three involute plate research reactors RHF, HFIR, and FRM II have expressed an interest in using computational software to carry their steady-state safety analysis. Since these tools represent a significant departure from the methods used currently (one-dimensional), the acceptability of the new approach by regulators requires thorough verification and validation of these tools. Therefore, Argonne National Laboratory and the three involute-plate reactors formed an informal group called the Involute Working Group aiming at qualifying computational tools to perform steady-state safety analysis. The present report focuses on a comparison of finite volume and finite element methods to model solids in fluid-structure interaction problems with the goal to estimate the coolant flow-induced fuel plate deflections obtained with the two methods. The finite volume method will be obsoleted in STARCCM+ by the end of 2021, nevertheless, this evaluation is important because the method was used by ANL researchers to model the response of the fuel plates, despite its drawbacks, which are discussed in the report. It was essential to check how those estimates compare to the results obtained with the finite element method that is considered superior for structural analysis. Various geometries, i.e., flat, cylindrical and circle-involute fuel plates, as well as coolant flow speed, were considered. The comparison shows that, independently of the plate geometry, the finite volume method significantly underestimates the deflection as compared to finite element method for coarser meshes. When the discretization is developed as a result of a mesh sensitivity study using finite element method, the result obtained using finite volume method can be a few times smaller than the corresponding finite element method solution. A code-to-code comparison , between STAR-CCM+ and LS-DYNA was included in the analysis. Within the LS-DYNA models, two types of finite element formulations were used: solid and shell finite elements. Mesh sensitivity study showed that both approaches converge to a similar value that was obtained with STAR-CCM+ finite element solver. The evaluation of the computational solvers was extended by adding two benchmark cases from the STAR-CCM+ Verification Suite and presented in the Appendix A. The selected cases are: (1) bending of a cantilever beam under external load, and (2) cylindrical shell deformation analysis, known in the literature as ‘Scordelis-Lo roof’. The problems were solved with finite volume, and finite element methods, and the results confirmed the previously discussed findings. The analysis shows that the finite element solver is superior to the finite volume solver in terms of representation of model geometry and estimating the structural behavior of fuel plates. Depending on the ratio of the load to the flexibility of the plate, the finite volume solver can greatly under- or overestimate the structural response if a very carefully selected mesh is not used.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Synthetic Scene Generation of Stennis Target Range for the Calibration of Remote Sensing Systems

The validation and verification (V&V) target range developed at Stennis Space Center is a useful tool for the calibration of remote sensing systems. In this paper, we present an algorithm for generating synthetic scenes or digital models of this target range. The atmospheric radiative transfer code Modtran 4 is used in simulating at-sensor radiance for a given sensor. Software is developed to generate the scene with different spatial and spectral resolutions using the at-sensor radiance values. Synthetic scenes provide a cost-effective way for in-flight validation for the spatial and radiometric accuracy of the data. Other applications include mission planning, sensor simulation, and trade-off analysis in sensor design. The radiometric and spatial accuracy of the simulation is evaluated by comparing simulated scenes with AVIRIS acquired values.

Cao, Chang-Yong↗

Proceedings of the Twenty-Third Annual Software Engineering Workshop

The Twenty-third Annual Software Engineering Workshop (SEW) provided 20 presentations designed to further the goals of the Software Engineering Laboratory (SEL) of the NASA-GSFC. The presentations were selected on their creativity. The sessions which were held on 2-3 of December 1998, centered on the SEL, Experimentation, Inspections, Fault Prediction, Verification and Validation, and Embedded Systems and Safety-Critical Systems.

Source record↗

Verifying Data Integrity of Electronically Scanned Pressure Systems at the NASA Glenn Research Center

The proper operation of the Electronically Scanned Pressure (ESP) System critical to accomplish the following goals: acquisition of highly accurate pressure data for the development of aerospace and commercial aviation systems and continuous confirmation of data quality to avoid costly, unplanned, repeat wind tunnel or turbine testing. Standard automated setup and checkout routines are necessary to accomplish these goals. Data verification and integrity checks occur at three distinct stages, pretest pressure tubing and system checkouts, daily system validation and in-test confirmation of critical system parameters. This paper will give an overview of the existing hardware, software and methods used to validate data integrity.

Panek, Joseph W.↗

Software for Statistical Analysis of Weibull Distributions with Application to Gear Fatigue Data: User Manual with Verification

The Weibull distribution has been widely adopted for the statistical description and inference of fatigue data. This document provides user instructions, examples, and verification for software to analyze gear fatigue test data. The software was developed presuming the data are adequately modeled using a two-parameter Weibull distribution. The calculations are based on likelihood methods, and the approach taken is valid for data that include type 1 censoring. The software was verified by reproducing results published by others.

Krantz, Timothy L.↗

Software for Statistical Analysis of Weibull Distributions with Application to Gear Fatigue Data: User Manual with Verification

The Weibull distribution has been widely adopted for the statistical description and inference of fatigue data. This document provides user instructions, examples, and verification for software to analyze gear fatigue test data. The software was developed presuming the data are adequately modeled using a two-parameter Weibull distribution. The calculations are based on likelihood methods, and the approach taken is valid for data that include type I censoring. The software was verified by reproducing results published by others.

Kranz, Timothy L.↗

The Planning Execution Monitoring Architecture

The Planning Execution Monitoring (PEM) architecture is a design concept for developing autonomous cockpit command and control software. The PEM architecture is designed to reduce the operations costs in the space transportation system through the use of automation while improving safety and operability of the system. Specifically, the PEM autonomous framework enables automatic performance of many vehicle operations that would typically be performed by a human. Also, this framework supports varying levels of autonomous control, ranging from fully automatic to fully manual control. The PEM autonomous framework interfaces with the core flight software to perform flight procedures. It can either assist human operators in performing procedures or autonomously execute routine cockpit procedures based on the operational context. Most importantly, the PEM autonomous framework promotes and simplifies the capture, verification, and validation of the flight operations knowledge. Through a hierarchical decomposition of the domain knowledge, the vehicle command and control capabilities are divided into manageable functional "chunks" that can be captured and verified separately. These functional units, each of which has the responsibility to manage part of the vehicle command and control, are modular, re-usable, and extensible. Also, the functional units are self-contained and have the ability to plan and execute the necessary steps for accomplishing a task based upon the current mission state and available resources. The PEM architecture has potential for application outside the realm of spaceflight, including management of complex industrial processes, nuclear control, and control of complex vehicles such as submarines or unmanned air vehicles.

Wang, Lui↗

Dynamic Stability Analysis Using Free-Flight CFD

Blunt body probes are the primary design for atmospheric entry. The blunted capsule shape, while desirable for the decreased forebody heating, typically experiences dynamic instability at low supersonic and transonic flight. At moderate to low supersonic conditions, the unsteady wake begins to adversely interact with the vehicle in free-flight resulting in attitude oscillations which can grow in time and result in tumbling. Understanding vehicle dynamics is thus important when considering vehicle entry design and guidance navigation and control (GN&C) considerations. Historically, dynamic stability has been carried out exclusively using experimental techniques. The wide range of experimental approaches provide reasonable coverage in Mach and dynamic pressure space for a given trajectory, however achieving flight similitude is often difficult or impossible. Advancements in Computational Fluid Dynamics (CFD) techniques allow for dynamic motion of the vehicle surface during the simulation with either forced/prescribed motion or free motion in response to fluid forces. Several examples of CFD software with the ability to simulate fluid dynamics and vehicle motion are available in literature. The Free-Flight CFD (FF-CFD) capability developed by NASA Ames within the US3D flow solver has shown extensive validation and verification (V&V) efforts from moderate to low supersonic flow across a range of vehicle architectures. The solver is able to simulate full rotational and translation motion resulting in full 6 degrees-of-freedom (DoF) motion. The validation and verification efforts the solver's ability to predict the motion of a ballistic range model as compared to experimental results. Similar to other computational capabilities in literature, FF-CFD relied on heritage data reduction methodologies and processes for reducing dynamic data into aerodynamic coefficients intended for use in design and trajectory codes. These heritage approaches, initially developed with the assumptions (and restrictions) of ground test facilities, resulted in the significant culling of the full 6-DoF dynamic data set to discrete values which are then fit using an aerodynamic model. Recent development of simulation and data reduction methodologies has resulted in an improved approach to FF-CFD simulations of vehicle dynamics which leverage the control of dynamic constraints within a simulation set-up. The presentation of this work will cover the constrained dynamic simulation approach for deriving dynamic coefficients for a Genesis capsule. The methodology will be outlined and a discussion of implications in motion restriction will be discussed. Finally, reconstructed vehicle trajectories using FF-CFD derived aerodynamic coefficients will be presented.

Joseph M Brock↗

NASA's Optical Program on Ascension Island: Bringing MCAT to Life as the Eugene Stansbery-Meter Class Autonomous Telescope (ES-MCAT)

In June 2015, the construction of the Meter Class Autonomous Telescope was completed and MCAT saw the light of the stars for the first time. In 2017, MCAT was newly dedicated as the Eugene Stansbery-MCAT telescope by NASA's Orbital Debris Program Office (ODPO), in honor of his inspiration and dedication to this newest optical member of the NASA ODPO. Since that time, MCAT has viewed the skies with one engineering camera and two scientific cameras, and the ODPO optical team has begun the process of vetting the entire system. The full system vetting includes verification and validation of: (1) the hardware comprising the system (e.g. the telescopes and its instruments, the dome, weather systems, all-sky camera, FLIR cloud infrared camera, etc.), (2) the custom-written Observatory Control System (OCS) master software designed to autonomously control this complex system of instruments, each with its own control software, and (3) the custom written Orbital Debris Processing software for post-processing the data. ES-MCAT is now capable of autonomous observing to include Geosynchronous survey, TLE (Two-line element) tracking of individual catalogued debris at all orbital regimes (Low-Earth Orbit all the way to Geosynchronous (GEO) orbit), tracking at specified non-sidereal rates, as well as sidereal rates for proper calibration with standard stars. Ultimately, the data will be used for validation of NASA's Orbital Debris Engineering Model, ORDEM, which aids in engineering designs of spacecraft that require knowledge of the orbital debris environment and long-term risks for collisions with Resident Space Objects (RSOs).

Lederer, S. M.↗

Bridging the Gap Between Requirements and Model Analysis : Evaluation on Ten Cyber-Physical Challenge Problems

Formal verfication and simulation are powerful tools to validate requirements against complex systems. [Problem] Requirements are developed in early stages of the software lifecycle and are typically written in ambiguous natural language. There is a gap between such requirements and formal notations that can be used by verification tools, and lack of support for proper association of requirements with software artifacts for verification. [Principal idea] We propose to write requirements in an intuitive, structured natural language with formal semantics, and to support formalization and model/code verification as a smooth, well-integrated process. [Contribution] We have developed an end-to-end, open source requirements analysis framework that checks Simulink models against requirements written in structured natural language. Our framework is built in the Formal Requirements Elicitation Tool (fret); we use fret's requirements language named fretish, and formalization of fretish requirements in temporal logics. Our proposed framework contributes the following features: 1) automatic extraction of Simulink model information and association of fretish requirements with target model signals and components; 2) translation of temporal logic formulas into synchronous dataflow cocospec specifications as well as Simulink monitors, to be used by verification tools; we establish correctness of our translation through extensive automated testing; 3) interpretation of counterexamples produced by verification tools back at requirements level. These features support a tight integration and feedback loop between high level requirements and their analysis. We demonstrate our approach on a major case study: the Ten Lockheed Martin Cyber-Physical, aerospace-inspired challenge problems.

Mavridou, Anastasia↗

Simulating Mars: Enabling Testing of the Perseverance Rover Sampling and Caching Subsystem on Earth

The development of the Sampling and Caching Subsystem (SCS) on the JPL Perseverance Rover lies at the intersection of testing, robotics, and geology. The SCS team established three primary system test campaigns and venues to aid in the development of SCS through verification and validation testing – Qualification Model Dirty Testing (QMDT) to provide a venue for testing in a Martian environment, Vehicle System Testbed (VSTB) for testing while integrated with the mobility subsystem on Martian-like terrain, and the Flight Software Testbed (FSWTB) for conducting tests using the flight motor controllers and software system on a hexapod which had the ability to simulate rover tilt. Each venue contributed a vital piece to the SCS building blocks. However, the QMDT venue operating within a 10-ft diameter Thermal Vacuum chamber to simulate Martian environment provided a sui generis opportunity to fine tune the entire sampling and caching process while building the team’s knowledge base about rock drillability, system life, and target selection. On Earth, because Martian rocks are not readily available, the development team must utilize geoanalogs to the rocks and regolith on Mars. Geologists on the team helped establish a set of standard rock types to use for Mars missions, like Basalt, Sandstone, Mudstone, Gypsum, and other related geoanalogs. These geoanalogs are characterized with a standard suite of tests for density, compressibility, and other characteristics to categorize potential drillability. This concept of drillability is what links the geoanalogs on Earth to the samples we collect on Mars. With the simulant characteristics defined, these geoanalog rocks are ready to be drilled into as we do on the Martian surface. A key aspect of interacting with the surface on Mars is rock target identification and selection. The Perseverance robotic system uses the on-board cameras, instrumentation, and software to collect enough information to identify potential scientific targets. With the targets identified, SCS can place the Corer and abrade the surface or collect a sample. For a ground test activity like QMDT, the test team did not have all of the camera and instrumentation systems that the rover does, so the team developed ground test equivalents to process a rock, build a target map, and define the target. The team constructed a Rock Scanning Station to build a 3D point cloud of the rock. This point cloud was then processed and evaluated with predefined and programmed criteria in a Target Downselect Tool. A primary output of the Target Downselect Tool is a defined target that can be uploaded directly to the robotic software system to simulate and build the robotic sequences used in tests. With these insights and programmatic definition of targets, the QMDT test team was able to make the same decisions that the Perseverance surface operations team does. In addition, valuable lessons learned from developing the target selection ground tools and using them were implemented into the tools used for surface operations.

Kim, Junggon↗

Transcriptomics Processing Pipelines for Space Biology: An Open Source and Consensus-Driven Approach

Transcriptomics holds significant value in elucidating the relationship between gene expression, experimental factors, biological factors, and various types of omics data. Enhancing our understanding of these connections is paramount for foundational biology, which plays a pivotal role in devising solutions for challenges pertinent to both space travel and terrestrial life. The NASA GeneLab project, part of the Open Science Data Repository (OSDR.nasa.gov), seeks to accelerate space biology research through cataloging and democratizing ‘omics data, including transcriptomics. Since raw omics data are largely inaccessible to non-bioinformaticians, GeneLab works with the scientific community via the Open Science Analysis Working Groups (AWGs) to develop standard processing pipelines to generate and publish processed data. Unlike raw data, processed data have greater immediate value to diverse users with varying technical backgrounds and computational capabilities. Standardizing processing workflows is essential to match the pace of raw data generation, ensure reproducibility, and enable standardized processed data for comparison across datasets. As of June 2023, transcriptomics studies comprise over half of GeneLab datasets hosted on the OSDR, including data from bulk RNA-seq and Affymetrix or Agilent 1-Channel DNA microarray assays. In collaboration with the AWGs, GeneLab developed consensus processing pipelines for these transcriptomics data types that includes quality control, background correction (microarray only), data normalization and quantification, culminating in the detection and annotation of differentially expressed genes. The work presented here describes Nextflow implementations of GeneLab’s consensus transcriptomics pipelines that automates and accelerates processing of these datasets. In addition to the core data processing, these workflows also include raw data staging and a robust verification and validation program to identify errors in real-time, stop additional downstream computation, and preserve computational resources. These workflows are used to generate GeneLab processed data hosted on the OSDR, and are publicly available as open source software for others to use at: https://github.com/nasa/GeneLab_Data_Processing.

Jonathan Oribello↗

James Webb Space Telescope Integrated Science Instrument Module Thermal Vacuum Thermal Balance Test Campaign at NASA's Goddard Space Flight Center

The James Webb Space Telescope is a large infrared telescope with a 6.5-meter primary mirror, designed as a successor to the Hubble Space Telescope when launched in 2018. Three of the four science instruments contained within the Integrated Science Instrument Module (ISIM) are passively cooled to their operational temperature range of 36K to 40K with radiators, and the fourth instrument is actively cooled to its operational temperature of approximately 6K. Thermal-vacuum testing of the flight science instruments at the ISIM element level has taken place in three separate highly challenging and extremely complex thermal tests within a gaseous helium-cooled shroud inside Goddard Space Flight Centers Space Environment Simulator. Special data acquisition software was developed for these tests to monitor over 1700 flight and test sensor measurements, track over 50 gradients, component rates, and temperature limits in real time against defined constraints and limitations, and guide the complex transition from ambient to final cryogenic temperatures and back. This extremely flexible system has proven highly successful in safeguarding the nearly $2B science payload during the 3.5-month-long thermal tests. Heat flow measurement instrumentation, or Q-meters, were also specially developed for these tests. These devices provide thermal boundaries o the flight hardware while measuring instrument heat loads up to 600 mW with an estimated uncertainty of 2 mW in test, enabling accurate thermal model correlation, hardware design validation, and workmanship verification. The high accuracy heat load measurements provided first evidence of a potentially serious hardware design issue that was subsequently corrected. This paper provides an overview of the ISIM-level thermal-vacuum tests and thermal objectives; explains the thermal test configuration and thermal balances; describes special measurement instrumentation and monitoring and control software; presents key test thermal results; lists problems encountered during testing and lessons learned.

JWST ISIM Thermal↗

AGGREGATE: dAta-driven modelinG preservinG contRollable dEr for outaGe mAnagemenT and rEsiliency (Report for Task 14: Model Verification/Validation Test Plan (Deliverable D10))

This report summarizes the technical progress and validation plan for each of AGGREGATE project modules, and integrated modules. The overall validation plan can be divided into two major parts. Firstly, it provides offline validation results of each module using the IEEE-123 node distribution system with certain modifications to support individual module results verification. The offline validation plan tentatively validated the usability and practicability of developed AGGREGATE tools. Secondly, the realworld test system - Seattle City Light (SCL) test feeder for verifying each module and the integrated modules using GE advanced distribution management system (ADMS) software.

42 ENGINEERING↗

Estimated Probability of a Cervical Spine Injury During an ISS Mission

Introduction: The Integrated Medical Model (IMM) utilizes historical data, cohort data, and external simulations as input factors to provide estimates of crew health, resource utilization and mission outcomes. The Cervical Spine Injury Module (CSIM) is an external simulation designed to provide the IMM with parameter estimates for 1) a probability distribution function (PDF) of the incidence rate, 2) the mean incidence rate, and 3) the standard deviation associated with the mean resulting from injury/trauma of the neck. Methods: An injury mechanism based on an idealized low-velocity blunt impact to the superior posterior thorax of an ISS crewmember was used as the simulated mission environment. As a result of this impact, the cervical spine is inertially loaded from the mass of the head producing an extension-flexion motion deforming the soft tissues of the neck. A multibody biomechanical model was developed to estimate the kinematic and dynamic response of the head-neck system from a prescribed acceleration profile. Logistic regression was performed on a dataset containing AIS1 soft tissue neck injuries from rear-end automobile collisions with published Neck Injury Criterion values producing an injury transfer function (ITF). An injury event scenario (IES) was constructed such that crew 1 is moving through a primary or standard translation path transferring large volume equipment impacting stationary crew 2. The incidence rate for this IES was estimated from in-flight data and used to calculate the probability of occurrence. The uncertainty in the model input factors were estimated from representative datasets and expressed in terms of probability distributions. A Monte Carlo Method utilizing simple random sampling was employed to propagate both aleatory and epistemic uncertain factors. Scatterplots and partial correlation coefficients (PCC) were generated to determine input factor sensitivity. CSIM was developed in the SimMechanics/Simulink environment with a Monte Carlo wrapper (MATLAB) used to integrate the components of the module. Results: The probability of generating an AIS1 soft tissue neck injury from the extension/flexion motion induced by a low-velocity blunt impact to the superior posterior thorax was fitted with a lognormal PDF with mean 0.26409, standard deviation 0.11353, standard error of mean 0.00114, and 95% confidence interval [0.26186, 0.26631]. Combining the probability of an AIS1 injury with the probability of IES occurrence was fitted with a Johnson SI PDF with mean 0.02772, standard deviation 0.02012, standard error of mean 0.00020, and 95% confidence interval [0.02733, 0.02812]. The input factor sensitivity analysis in descending order was IES incidence rate, ITF regression coefficient 1, impactor initial velocity, ITF regression coefficient 2, and all others (equipment mass, crew 1 body mass, crew 2 body mass) insignificant. Verification and Validation (V&V): The IMM V&V, based upon NASA STD 7009, was implemented which included an assessment of the data sets used to build CSIM. The documentation maintained includes source code comments and a technical report. The software code and documentation is under Subversion configuration management. Kinematic validation was performed by comparing the biomechanical model output to established corridors.

Brooker, John E.↗