Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software engineering software verification validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

How to Build a Rover: An Overview of the Mars 2020 Mission’s Vehicle System Testbed

While NASA’s Mars rover Perseverance continues to make groundbreaking achievements on the Red Planet, its twin is hard at work here on Earth. The Operational Perseverance Twin for the Integration of Mechanisms and Instruments Sent to Mars, or OPTIMISM, is the Mars 2020 Vehicle System Testbed (VSTB) rover operated by NASA Jet Propulsion Laboratory (JPL) in Pasadena, California. OPTIMISM’s home is the JPL Mars Yard; an outdoor field with red soil that simulates the terrain encountered by Perseverance. The VSTB is a full-scale engineering model of the flight rover, serving a number of functions to ensure mission operations can continue smoothly and on schedule. The VSTB possesses instrumentation, computers, mechanisms, cameras, and a Mobility subsystem that are nearly identical to its extraterrestrial twin. Its high fidelity allows the rover to be a highly effective tool to fully test system functionality and performance prior to commanding the flight rover. The early stages of building OPTIMISM began a few months prior to Perseverance departing JPL for Cape Canaveral, FL in early 2020. Electrical integration of the flight system avionics, and compatibility checkouts of the electrical ground support equipment ensured that the foundation of the electrical system was operational and in place. Next, the internal harnessing was installed and compatibility checks of the rover instrumentation and mechanisms were performed to confirm the system was prepared for full buildup. Finally, mechanical assembly of the rover chassis with its external components completed the integration of the system before it was moved to the Mars Yard for its initial phase of testing to perform verification & validation (V&V) of the Mobility subsystem requirements. By the time Perseverance landed at Jezero Crater in February 2021, the first phase of VSTB operations was underway. Surface guidance, navigation, and control (SGNC) testing for the Mobility subsystem ensured functionality and performance requirements were met for various capabilities such as visual odometry (VO), mapping, and automatic navigation (AutoNav). Subsequent integration of the robotic arm (RA) onto the VSTB enabled the V&V campaign for surface sampling operations (SSO) to commence. As the mission’s engineering operations (EO) have gotten underway, the VSTB has been utilized for an array of purposes including troubleshooting software anomalies, and performing dry-runs for first time activities (FTAs) prior to sending the commands to Perseverance. OPTIMISM will continue to serve mission critical functions as long as Perseverance is roving the Red Planet.

Rojas, Jose Trujillo↗

Pronghorn: A Multidimensional Coarse Mesh Application for Advanced Reactor Thermal-Hydraulics

This paper presents an overview of Pronghorn, a multiscale thermal-hydraulic (T/H) application developed by Idaho National Laboratory and the University of California, Berkeley. Pronghorn, built on the open-source finite element Multiphysics Object-Oriented Simulation Environment (MOOSE), leverages state-of-the-art physical models, numerical methods, and nonlinear solvers to deliver fast-running advanced reactor T/H simulation capabilities within a modern software engineering environment. This work summarizes the physical models, multiphysics and multiscale coupling, and numerical discretization in Pronghorn with emphasis on our initial target application to pebble bed reactors (PBRs). A diverse set of applications are shown to depressurized natural circulation in the SANA experiments, forced convection in the Pebble Bed Modular Reactor, three-dimensional (3-D)/one-dimensional coupling of Pronghorn and RELAP-7 systems T/H for loop analysis in the High Temperature Reactor Power Module, and forced convection in the Mark-1 Pebble Bed Fluoride-Salt-Cooled High-Temperature Reactor. A multiphysics coupling of Pronghorn, RELAP-7, and Griffin deterministic neutronics for a gas-cooled PBR demonstrates the capability of the MOOSE framework for reactor design calculations. These applications highlight the verification and validation underlying Pronghorn’s software development while emphasizing features that improve upon capabilities offered by legacy tools in areas such as 3-D unstructured meshing, physics modeling, and multiphysics coupling.

97 MATHEMATICS AND COMPUTING↗

A 3DHZETRN Code in a Spherical Uniform Sphere with Monte Carlo Verification

The computationally efficient HZETRN code has been used in recent trade studies for lunar and Martian exploration and is currently being used in the engineering development of the next generation of space vehicles, habitats, and extra vehicular activity equipment. A new version (3DHZETRN) capable of transporting High charge (Z) and Energy (HZE) and light ions (including neutrons) under space-like boundary conditions with enhanced neutron and light ion propagation is under development. In the present report, new algorithms for light ion and neutron propagation with well-defined convergence criteria in 3D objects is developed and tested against Monte Carlo simulations to verify the solution methodology. The code will be available through the software system, OLTARIS, for shield design and validation and provides a basis for personal computer software capable of space shield analysis and optimization.

Wilson, John W.↗

Toward Polydisperse Flows With MFIX-EXA

In the presence of large size disparities, single-grid neighbor search algorithms lead to inflated neighbor lists that significantly degrade the performance of Lagrangian particle solvers. If Eulerian–Lagrangian (EL) frameworks are to remain performant when simulating realistic systems, improved neighbor detection approaches must be adopted. To this end, we consider the application of a multigrid neighbor search (MGNS) algorithm in the mfix-exa software package, an exascale EL solver built upon the AMReX library. Here, details regarding the implementation and verification of MGNS are provided along with speedup curves for a bidisperse mixing layer. MGNS is shown to yield up to 15$\times$ speedup on CPU and 6$\times$ speedup on GPU for the problems considered here. The mfix-exa software is then validated for a variety of polydisperse flows. Finally, a brief discussion is given for how dynamic MGNS may be completed, with application to spatially varying particle size distributions.

42 ENGINEERING↗

From Sim to Real: A Pipeline for Training and Deploying Traffic Smoothing Cruise Controllers

Designing and validating controllers for connected and automated vehicles to enhance traffic flow presents significant challenges, from the complexity of replicating real-world stop-and-go traffic dynamics in simulation, to the intricacies involved in transitioning from simulation to actual deployment. In this work, we present a full pipeline from data collection to controller deployment. Specifically, we collect 772 km of driving data from the I-24 in Tennessee, and use it to build a one-lane simulator, placing simulated vehicles behind real-world trajectories. Using policy-gradient methods with an asymmetric critic, we improve fuel efficiency by over 10% when simulating congested scenarios. Our comprehensive approach includes reinforcement learning for controller training, software verification, hardware validation and setup, and navigating various sim-to-real challenges. Furthermore, we analyze the controller's behavior and wave-smoothing properties, and deploy it on four Toyota Rav4’s in a real-world validation experiment on the I-24. Lastly, we release the driving dataset, the simulator and the trained controller, to enable future benchmarking and controller design.

42 ENGINEERING↗

The Unparalleled Systems Engineering of MSL's Backup Entry, Descent, and Landing System: Second Chance

Second Chance (SECC) was a bare bones version of Mars Science Laboratory's (MSL) Entry Descent & Landing (EDL) flight software that ran on Curiosity's backup computer, which could have taken over swiftly in the event of a reset of Curiosity's prime computer, in order to land her safely on Mars. Without SECC, a reset of Curiosity's prime computer would have lead to catastrophic mission failure. Even though a reset of the prime computer never occurred, SECC had the important responsibility as EDL's guardian angel, and this responsibility would not have seen such success without unparalleled systems engineering. This paper will focus on the systems engineering behind SECC: Covering a brief overview of SECC's design, the intense schedule to use SECC as a backup system, the verification and validation of the system's "Do No Harm" mandate, the system's overall functional performance, and finally, its use on the fateful day of August 5th, 2012.

fault protection↗

The unparalleled systems engineering of MSL’s backup entry, descent, and landing system : second chance

Second Chance (SECC) was a bare bones version of Mars Science Laboratory’s (MSL) Entry Descent & Landing (EDL) flight software that ran on Curiosity’s backup computer, which could have taken over swiftly in the event of a reset of Curiosity’s prime computer, in order to land her safely on Mars. Without SECC, a reset of Curiosity’s prime computer would have lead to catastrophic mission failure. Even though a reset of the prime computer never occurred, SECC had the important responsibility as EDL’s guardian angel, and this responsibility would not have seen such success without unparalleled systems engineering. This paper will focus on the systems engineering behind SECC: Covering a brief overview of SECC’s design, the intense schedule to use SECC as a backup system, the verification and validation of the system’s “Do No Harm” mandate, the system’s overall functional performance, and finally, its use on the fateful day of August 5th, 2012.

Reeves, Glenn↗

User Guidelines and Best Practices for CASL VUQ Analysis Using Dakota

Sandia’s Dakota software (available at http://dakota.sandia.gov) supports science and engineering transformation through advanced exploration of simulations. Specifically it manages and analyzes ensembles of simulations to provide broader and deeper perspective for analysts and decision makers. This enables them to enhance understanding of risk, improve products, and assess simulation credibility. In its simplest mode, Dakota can automate typical parameter variation studies through a generic interface to a physics-based computational model. This can lend efficiency and rigor to manual parameter perturbation studies already being conducted by analysts. However, Dakota also delivers advanced parametric analysis techniques enabling design exploration, optimization, model calibration, risk analysis, and quantification of margins and uncertainty with such models. It directly supports verification and validation activities. Dakota algorithms enrich complex science and engineering models, enabling an analyst to answer crucial questions of - Sensitivity: Which are the most important input factors or parameters entering the simulation, and how do they influence key outputs?; Uncertainty: What is the uncertainty or variability in simulation output, given uncertainties in input parameters? How safe, reliable, robust, or variable is my system? (Quantification of margins and uncertainty, QMU); Optimization: What parameter values yield the best performing design or operating condition, given constraints? Calibration: What models and/or parameters best match experimental data? In general, Dakota is the Consortium for Advanced Simulation of Light Water Reactors (CASL) delivery vehicle for verification, validation, and uncertainty quantification (VUQ) algorithms. It permits ready application of the VUQ methods described above to simulation codes by CASL researchers, code developers, and application engineers.

97 MATHEMATICS AND COMPUTING↗

Independent Verification of Mars-GRAM 2010 with Mars Climate Sounder Data

The Mars Global Reference Atmospheric Model (Mars-GRAM) is an engineering-level atmospheric model widely used for diverse mission and engineering applications. Applications of Mars-GRAM include systems design, performance analysis, and operations planning for aerobraking, entry, descent and landing, and aerocapture. Atmospheric influences on landing site selection and long-term mission conceptualization and development can also be addressed utilizing Mars-GRAM. Mars-GRAM's perturbation modeling capability is commonly used, in a Monte Carlo mode, to perform high-fidelity engineering end-to-end simulations for entry, descent, and landing. Mars-GRAM is an evolving software package resulting in improved accuracy and additional features. Mars-GRAM 2005 has been validated against Radio Science data, and both nadir and limb data from the Thermal Emission Spectrometer (TES). From the surface to 80 km altitude, Mars-GRAM is based on the NASA Ames Mars General Circulation Model (MGCM). Above 80 km, Mars-GRAM is based on the University of Michigan Mars Thermospheric General Circulation Model (MTGCM). The most recent release of Mars-GRAM 2010 includes an update to Fortran 90/95 and the addition of adjustment factors. These adjustment factors are applied to the input data from the MGCM and the MTGCM for the mapping year 0 user-controlled dust case. The adjustment factors are expressed as a function of height (z), latitude and areocentric solar longitude (Ls).

Justh, Hilary L.↗

Exploring Explicit Uncertainty for Binary Analysis (EUBA)

Reverse engineering (RE) analysts struggle to address critical questions about the safety of binary code accurately and promptly, and their supporting program analysis tools are simply wrong sometimes. The analysis tools have to approximate in order to provide any information at all, but this means that they introduce uncertainty into their results. And those uncertainties chain from analysis to analysis. We hypothesize that exposing sources, impacts, and control of uncertainty to human binary analysts will allow the analysts to approach their hardest problems with high-powered analytic techniques that they know when to trust. Combining expertise in binary analysis algorithms, human cognition, uncertainty quantification, verification and validation, and visualization, we pursue research that should benefit binary software analysis efforts across the board. We find a strong analogy between RE and exploratory data analysis (EDA); we begin to characterize sources and types of uncertainty found in practice in RE (both in the process and in supporting analyses); we explore a domain-specific focus on uncertainty in pointer analysis, showing that more precise models do help analysts answer small information flow questions faster and more accurately; and we test a general population with domain-general sudoku problems, showing that adding "knobs" to an analysis does not significantly slow down performance. This document describes our explorations in uncertainty in binary analysis.

97 MATHEMATICS AND COMPUTING↗

A REDUCED ORDER MODELING APPROACH TO PROBABILISTIC CREEP-DAMAGE PREDICTIONS IN FINITE ELEMENT ANALYSIS

This paper introduces a computationally efficient Reduced Order Modeling (ROM) approach for the probabilistic prediction of creep-damage failure. Component-level probabilistic simulations are needed to assess the reliability and safety of high-temperature components. Full-scale probabilistic creep-damage modeling in finite element (FE) approach is computationally expensive requiring many hundreds of simulations to replicate the uncertainty of component failure. To that end, ROM is proposed to minimize the elevated computational cost while controlling the loss of accuracy. It is proposed that full-scale probabilistic simulations can be completed in 1D at a reduced cost, the extremum conditions extracted, and those conditions applied for lower-cost 2D/3D probabilistic simulations of components that capture the mean and uncertainty of failure. The probabilistic Sine-hyperbolic (Sinh) model is selected which in previous work was calibrated to alloy 304 stainless steel. The Sinh model includes probability density functions (pdfs) for test condition (stress and temperature), initial damage (i.e. microstructure), and material properties uncertainty. The Sinh model is programmed into ANSYS finite element software using the USERCREEP.F material subroutine. First, the Sinh model and FE code are subject to verification and validation to ensure the accuracy of the simulations. Numerous Monte Carlo simulations are executed in a 1D model to generate probabilistic creep deformation, damage, and rupture data. This data is analyzed and the probabilistic parameters corresponding to extreme creep response are extracted. The ROM concept is applied where only the extreme conditions are applied in the 2D probabilistic prediction of a component. The probabilistic predictions between the 1D and 2D geometry is compared to assess ROM for creep. The accuracy of the probabilistic prediction employing the ROM approach will potentially reduce the time and cost of simulating complex engineering systems. Future studies will introduce multi-stage Sinh, stochasticity, and spatial uncertainty for improved prediction.

36 MATERIALS SCIENCE↗

A Discussion of Time Management Concepts and Time Constraint Equations for Multi-Rate Federation Executions

The High Level Architecture (HLA) is a simulation interoperability standard developed by the Simulation Interoperability Standards Organization (SISO) and published as the international standard IEEE 1516-2010 by the Institute for Electrical and Electronics Engineers (IEEE). HLA is a widely used standard for the development and execution of collaborative distributed simulations. HLA provides a number of Management Services to simulation developers: Federation, Declaration, Object, Ownership, Data Distribution, and Time. Of those services, Time Management Services is probably one of the least understood and least used. However, Time Management Services are critical to technical simulations like those created for space systems using the Space Reference Federation Object Model (SpaceFOM). Time Management can be used to insure data coherence and execution repeatability in distributed simulations. When combined with real time execution policies, Time Management is being used to support real time execution of mixed software and hardware in the loop integration, verification, and validation simulations for active space systems development. This paper starts by providing an overview of the HLA Time Management Services. This provides the background to discuss the challenges associated with Time Management and its use, starting with simple common rate frame scheduled simulations, then simple multi-rate simulations, and ending with complex mixed rate simulations. The authors then formulate the significant time constraint relationships between identified frame scheduling parameters. The intent of the paper is to provide a concise discussion of how to use Time Management in both simple cases and in more complex mixed frame rate federation executions.

Simulation Interoperability↗

A Discussion of Time Management Concepts and Time Constraint Equations for Multi-Rate Federation Executions

The High Level Architecture (HLA) is a simulation interoperability standard developed by the Simulation Interoperability Standards Organization (SISO) and published as the international standard IEEE 1516-2010 by the Institute for Electrical and Electronics Engineers (IEEE). HLA is a widely used standard for the development and execution of collaborative distributed simulations. HLA provides a number of Management Services to simulation developers: Federation, Declaration, Object, Ownership, Data Distribution, and Time. Of those services, Time Management Services is probably one of the least understood and least used. However, Time Management Services are critical to technical simulations like those created for space systems using the Space Reference Federation Object Model (SpaceFOM). Time Management can be used to insure data coherence and execution repeatability in distributed simulations. When combined with real time execution policies, Time Management is being used to support real time execution of mixed software and hardware in the loop integration, verification, and validation simulations for active space systems development. This paper starts by providing an overview of the HLA Time Management Services. This provides the background to discuss the challenges associated with Time Management and its use, starting with simple common rate frame scheduled simulations, then simple multi-rate simulations, and ending with complex mixed rate simulations. The authors then formulate the significant time constraint relationships between identified frame scheduling parameters. The intent of the paper is to provide a concise discussion of how to use Time Management in both simple cases and in more complex mixed frame rate federation executions.

Simulation Interoperability↗

Telemetry and Science Data Software System

The Telemetry and Science Data Software System (TSDSS) was designed to validate the operational health of a spacecraft, ease test verification, assist in debugging system anomalies, and provide trending data and advanced science analysis. In doing so, the system parses, processes, and organizes raw data from the Aquarius instrument both on the ground and while in space. In addition, it provides a user-friendly telemetry viewer, and an instant pushbutton test report generator. Existing ground data systems can parse and provide simple data processing, but have limitations in advanced science analysis and instant report generation. The TSDSS functions as an offline data analysis system during I&T (integration and test) and mission operations phases. After raw data are downloaded from an instrument, TSDSS ingests the data files, parses, converts telemetry to engineering units, and applies advanced algorithms to produce science level 0, 1, and 2 data products. Meanwhile, it automatically schedules upload of the raw data to a remote server and archives all intermediate and final values in a MySQL database in time order. All data saved in the system can be straightforwardly retrieved, exported, and migrated. Using TSDSS s interactive data visualization tool, a user can conveniently choose any combination and mathematical computation of interesting telemetry points from a large range of time periods (life cycle of mission ground data and mission operations testing), and display a graphical and statistical view of the data. With this graphical user interface (GUI), the data queried graphs can be exported and saved in multiple formats. This GUI is especially useful in trending data analysis, debugging anomalies, and advanced data analysis. At the request of the user, mission-specific instrument performance assessment reports can be generated with a simple click of a button on the GUI. From instrument level to observatory level, the TSDSS has been operating supporting functional and performance tests and refining system calibration algorithms and coefficients, in sync with the Aquarius/SAC-D spacecraft. At the time of this reporting, it was prepared and set up to perform anomaly investigation for mission operations preceding the Aquarius/SAC-D spacecraft launch on June 10, 2011.

Bates, Lakesha↗

RELAP-7 Code Assessment Plan and Requirement Traceability Matrix

The RELAP-7, a safety analysis code for nuclear reactor system, is under development at Idaho National Laboratory (INL). Overall, the code development is directed towards leveraging the advancements in computer science technology, numerical solution methods and physical models over the last decades. Recently, INL has also been putting an effort to establish the code assessment plan, which aims to ensure an improved final product quality through the RELAP-7 development process. The ultimate goal of this plan is to propose a suitable way to systematically assess the wide range of software requirements for RELAP-7, including the software design, user interface, and technical requirements, etc. To this end, we first survey the literature (i.e., international/domestic reports, research articles) addressing the desirable features generally required for advanced nuclear system safety analysis codes. In addition, the V&V (verification and validation) efforts as well as the legacy issues of several recently-developed codes (e.g., RELAP5-3D, TRACE V5.0) are investigated. Lastly, this paper outlines the Requirement Traceability Matrix (RTM) for RELAP-7 which can be used to systematically evaluate and identify the code development process and its present capability.

42 ENGINEERING↗

RELAP-7 Software Verification and Validation Plan: Requirements Traceability Matrix (RTM) Update and Code Verification Strategy

This document describes the recent progress of the RELAP-7 Software Verification and Validation Plan (SVVP). The RELAP-7 RTM has been updated by adding the validation requirement items, rearranging the requirement items, and introducing the classifier to better describe the RELAP-7 V&V status at a specified point in time. Also, from the perspective of nuclear system code validation, six items from the list of LWR safety-relevant phenomena were identified as validation data gaps. Lastly, motivated by the needs revealed from the previous RELAP-7 IV&V activities, the independent verification plan for RELAP-7 has been established. The verification method, procedure, and verification test coverage suite discussed in this document will be applied to the future work for the RELAP-7 independent verification.

42 ENGINEERING↗

6DOF Testing of the SLS Inertial Navigation Unit

The Navigation System on the NASA Space Launch System (SLS) Block 1 vehicle performs initial alignment of the Inertial Navigation System (INS) navigation frame through gyrocompass alignment (GCA). Because the navigation architecture for the SLS Block 1 vehicle is a purely inertial system, the accuracy of the achieved orbit relative to mission requirements is very sensitive to initial alignment accuracy. The assessment of this sensitivity and many others via simulation is a part of the SLS Model-Based Design and Model-Based Requirements approach. As a part of the aforementioned, 6DOF Monte Carlo simulation is used in large part to develop and demonstrate verification of program requirements. To facilitate this and the GN&C flight software design process, an SLS-Program-controlled Design Math Model (DMM) of the SLS INS was developed by the SLS Navigation Team. The SLS INS model implements all of the key functions of the hardware-namely, GCA, inertial navigation, and FDIR (Fault Detection, Isolation, and Recovery)-in support of SLS GN&C design requirements verification. Despite the strong sensitivity to initial alignment, GCA accuracy requirements were not verified by test due to program cost and schedule constraints. Instead, the system relies upon assessments performed using the SLS INS model. In order to verify SLS program requirements by analysis, the SLS INS model is verified and validated against flight hardware. In lieu of direct testing of GCA accuracy in support of requirement verification, the SLS Navigation Team proposed and conducted an engineering test to, among other things, validate the GCA performance and overall behavior of the SLS INS model through comparison with test data. This paper will detail dynamic hardware testing of the SLS INS, conducted by the SLS Navigation Team at Marshall Space Flight Center's 6DOF Table Facility, in support of GCA performance characterization and INS model validation. A 6-DOF motion platform was used to produce 6DOF pad twist and sway dynamics while a simulated SLS flight computer communicated with the INS. Tests conducted include an evaluation of GCA algorithm robustness to increasingly dynamic pad environments, an examination of GCA algorithm stability and accuracy over long durations, and a long-duration static test to gather enough data for Allan Variance analysis. Test setup, execution, and data analysis will be discussed, including analysis performed in support of SLS INS model validation.

Geohagan, Kevin↗

Bison Verification and Validation Activities for TRISO

Numerical modeling and simulation (M&S) tools play a key role in the research, development, and overall safety assessments of next-generation nuclear energy systems. One such tool, Bison, is a nuclear fuel performance code that is applicable to many fuel forms (e.g., light-water reactor fuel, oxide and metallic fuel for fast reactors, tri-structural isotropic (TRISO) fuel, and plate fuel), and it uses the finite element method to model the thermo- mechanical response of nuclear fuels. One fuel form widely utilized in Generation-IV high-temperature gas-cooled and fluoride- salt-cooled nuclear reactor concepts is TRISO fuel. Recently, Bison’s capabilities were significantly expanded to enable it to model the performance of TRISO particles and compacts. It is important that Bison’s computational results be reliable and predictive, since this code is used to inform high-consequence decisions. The various processes developed to address this issue generally entail two fundamental steps: verification and validation (V&V). Verification ensures that the code functions correctly and is reliable. Code/solution verification, code benchmark, and software quality assurance exercises are examples of verification activities. On the other hand, validation is the process of assessing a code’s capability to accurately model physical problems. Comparisons between code results and experiments quantify the validation level. Application of V&V procedures is crucial to the development of computational tools that are free of coding mistakes and can accurately represent reality. The current study presents an overview of Bison V&V activities relevant to the TRISO fuel concept, which include code/solution verification exercises, CRP-6 Benchmark—a Coordinated Research Program through the International Atomic Energy Agency (IAEA)—exercises, and validation exercises with the Advanced Gas Reactor (AGR)- 1/2/3/4 experiment series.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗