Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “automated verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Program to Optimize Simulated Trajectories II (POST2) Surrogate Models for Mars Ascent Vehicle (MAV) Performance Assessment

The primary purpose of the multiPOST tool is to enable the execution of much larger sets of vehicle cases to allow for broader trade space exploration. However, this exploration is not achieved solely with the increased case throughput. The multiPOST tool is applied to carry out a Design of Experiments (DOE), which is a set of cases that have been structured to capture a maximum amount of information about the design space with minimal computational effort. The results of the DOE are then used to fit a surrogate model, ultimately enabling parametric design space exploration. The approach used for the MAV study includes both DOE and surrogate modeling. First, the primary design considerations for the vehicle were used to develop the variables and ranges for the multiPOST DOE. The final set of DOE variables were carefully selected in order to capture the desired vehicle trades and take into account any special considerations for surrogate modeling. Next, the DOE sets were executed through multiPOST. Following successful completion of the DOE cases, a manual verification trial was performed. The trial involved randomly selecting cases from the DOE set and running them by hand. The results from the human analyst's run and multiPOST were then compared to ensure that the automated runs were being executed properly. Completion of the verification trials was then followed by surrogate model fitting. After fits to the multiPOST data were successfully created, the surrogate models were used as a stand-in for POST2 to carry out the desired MAV trades. Using the surrogate models in lieu of POST2 allowed for visualization of vehicle sensitivities to the input variables as well as rapid evaluation of vehicle performance. Although the models introduce some error into the output of the trade study, they were very effective at identifying areas of interest within the trade space for further refinement by human analysts. The next section will cover all of the ground rules and assumptions associated with DOE setup and multiPOST execution. Section 3.1 gives the final DOE variables and ranges, while section 3.2 addresses the POST2 specific assumptions. The results of the verification trials are given in section 4. Section 5 gives the surrogate model fitting results, including the goodness-of-fit metrics for each fit. Finally, the MAV specific results are discussed in section 6.

Zwack, M. R.↗

Application of software technology to automatic test data analysis

The verification process for a major software subsystem was partially automated as part of a feasibility demonstration. The methods employed are generally useful and applicable to other types of subsystems. The effort resulted in substantial savings in test engineer analysis time and offers a method for inclusion of automatic verification as a part of regression testing.

Stagner, J. R.↗

Simulation-Based Verification of Livingstone Applications

AI software is viewed as a means to give greater autonomy to automated systems, capable of coping with harsh and unpredictable environments in deep space missions. Autonomous systems pose a serious challenge to traditional test-based verification approaches, because of the enormous space of possible situations that they aim to address. Before these systems are put in control of critical applications, appropriate new verification approaches need to be developed. This article describes Livingstone PathFinder (LPF), a verification tool for autonomous diagnosis applications based on NASA's Livingstone model-based diagnosis system. LPF applies state space exploration algorithms to an instrumented testbed, consisting of the Livingstone diagnosis system embedded in a simulated operating environment. The article describes different facets of LPF and reports some experimental results from applying LPF to a Livingstone model of the main propulsion feed subsystem of the X-34 space vehicle. This paper describes Livingstone PathFinder (LPF), a verification tool for autonomous diagnosis applications based on NASA's Livingstone model-based diagnosis system. LPF applies state space exploration algorithms to an instrumented testbed, consisting of the Livingstone diagnosis system embedded in a simulated operating environment. Section 2 provides an overview of Livingstone; Section 3 describes the LPF architecture; Section 4 discusses its applicability; Section 5 reviews some experimental results; Section 6 compares LPF to related verification approaches; Section 7 draws conclusions and discusses some perspectives.

Lindsey, Anthony E.↗

Development and Implementation of a CTF Code Verification Suite.

CTF is a thermal hydraulic subchannel code developed to predict light water reactor (LWR) core behavior. It is a version of Coolant Boiling in Rod Arrays (COBRA) developed by Oak Ridge National Laboratory (ORNL) and North Carolina State University (NCSU) and used in the Consortium for the Advanced Simulation of LWRs (CASL). Here, the existing CTF code verification matrix is expanded, which ensures that the code is a faithful representation of the underlying mathematical model. The suite of code verification tests are mapped to the underlying conservation equations of CTF and significant gaps are addressed. As such, five new problems are incorporated: isokinetic advection, conduction, pressure drop, convection, and pipe boiling. Convergence behavior and numerical errors are quantified for each of the tests and all tests converge at the correct rate to their corresponding analytic solution. A new verification utility that generalizes the code verification process is used to incorporate these problems into the CTF automated test suite.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Program Model Checking as a New Trend

This paper introduces a special section of STTT (International Journal on Software Tools for Technology Transfer) containing a selection of papers that were presented at the 7th International SPIN workshop, Stanford, August 30 - September 1, 2000. The workshop was named SPIN Model Checking and Software Verification, with an emphasis on model checking of programs. The paper outlines the motivation for stressing software verification, rather than only design and model verification, by presenting the work done in the Automated Software Engineering group at NASA Ames Research Center within the last 5 years. This includes work in software model checking, testing like technologies and static analysis.

Havelund, Klaus↗

Optimized Temporal Monitors for SystemC

SystemC is a modeling language built as an extension of C++. Its growing popularity and the increasing complexity of designs have motivated research efforts aimed at the verification of SystemC models using assertion-based verification (ABV), where the designer asserts properties that capture the design intent in a formal language such as PSL or SVA. The model then can be verified against the properties using runtime or formal verification techniques. In this paper we focus on automated generation of runtime monitors from temporal properties. Our focus is on minimizing runtime overhead, rather than monitor size or monitor-generation time. We identify four issues in monitor generation: state minimization, alphabet representation, alphabet minimization, and monitor encoding. We conduct extensive experimentation and identify a combination of settings that offers the best performance in terms of runtime overhead.

Tabakov, Deian↗

The DEEP2 Galaxy Redshift Survey: Design, Observations, Data Reduction, and Redshifts

We describe the design and data analysis of the DEEP2 Galaxy Redshift Survey, the densest and largest high-precision redshift survey of galaxies at z approx. 1 completed to date. The survey was designed to conduct a comprehensive census of massive galaxies, their properties, environments, and large-scale structure down to absolute magnitude MB = −20 at z approx. 1 via approx.90 nights of observation on the Keck telescope. The survey covers an area of 2.8 Sq. deg divided into four separate fields observed to a limiting apparent magnitude of R(sub AB) = 24.1. Objects with z approx. < 0.7 are readily identifiable using BRI photometry and rejected in three of the four DEEP2 fields, allowing galaxies with z > 0.7 to be targeted approx. 2.5 times more efficiently than in a purely magnitude-limited sample. Approximately 60% of eligible targets are chosen for spectroscopy, yielding nearly 53,000 spectra and more than 38,000 reliable redshift measurements. Most of the targets that fail to yield secure redshifts are blue objects that lie beyond z approx. 1.45, where the [O ii] 3727 Ang. doublet lies in the infrared. The DEIMOS 1200 line mm(exp −1) grating used for the survey delivers high spectral resolution (R approx. 6000), accurate and secure redshifts, and unique internal kinematic information. Extensive ancillary data are available in the DEEP2 fields, particularly in the Extended Groth Strip, which has evolved into one of the richest multiwavelength regions on the sky. This paper is intended as a handbook for users of the DEEP2 Data Release 4, which includes all DEEP2 spectra and redshifts, as well as for the DEEP2 DEIMOS data reduction pipelines. Extensive details are provided on object selection, mask design, biases in target selection and redshift measurements, the spec2d two-dimensional data-reduction pipeline, the spec1d automated redshift pipeline, and the zspec visual redshift verification process, along with examples of instrumental signatures or other artifacts that in some cases remain after data reduction. Redshift errors and catastrophic failure rates are assessed through more than 2000 objects with duplicate observations. Sky subtraction is essentially photon-limited even under bright OH sky lines; we describe the strategies that permitted this, based on high image stability, accurate wavelength solutions, and powerful B-spline modeling methods. We also investigate the impact of targets that appear to be single objects in ground-based targeting imaging but prove to be composite in Hubble Space Telescope data; they constitute several percent of targets at z approx. 1, approaching approx. 5%-10% at z > 1.5. Summary data are given that demonstrate the superiority of DEEP2 over other deep high-precision redshift surveys at z approx. 1 in terms of redshift accuracy, sample number density, and amount of spectral information. We also provide an overview of the scientific highlights of the DEEP2 survey thus far.

Galaxy↗

Automated Calculation of U-10Mo Fuel and Zr Cladding Thickness

Hand calculation and verification of cladding and fuel thickness in a hot-isostatically pressed uranium-molybdenum (U-10Mo) alloy can incur human errors and longer image processing time when analyzing a large set of cross-sectional images. To help alleviate both time cost and errors made in hand measurements, an automated image processing procedure was developed using Octave, an open-source MATLAB alternative, to repeatedly determine the thickness of Zr and U-10Mo layers. U-10Mo specimens were imaged using standard secondary electron and backscattered-electron imaging at 250× magnification to capture the various layers present in U-10Mo. Further image processing used pixel-by-pixel calculation of the Zr and U-10Mo layers to gather quantitative statistics on the thickness variations associated with each of the layers.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Design and Verification of a Distributed Communication Protocol

The safety of remotely operated vehicles depends on the correctness of the distributed protocol that facilitates the communication between the vehicle and the operator. A failure in this communication can result in catastrophic loss of the vehicle. To complicate matters, the communication system may be required to satisfy several, possibly conflicting, requirements. The design of protocols is typically an informal process based on successive iterations of a prototype implementation. Yet distributed protocols are notoriously difficult to get correct using such informal techniques. We present a formal specification of the design of a distributed protocol intended for use in a remotely operated vehicle, which is built from the composition of several simpler protocols. We demonstrate proof strategies that allow us to prove properties of each component protocol individually while ensuring that the property is preserved in the composition forming the entire system. Given that designs are likely to evolve as additional requirements emerge, we show how we have automated most of the repetitive proof steps to enable verification of rapidly changing designs.

Munoz, Cesar A.↗

Hybrid Verification of an Interface for an Automatic Landing

Modern commercial aircraft have extensive automation which helps the pilot by performing computations, obtaining data, and completing procedural tasks. The pilot display must contain enough information so that the pilot can correctly predict the aircraft's behavior, while not overloading the pilot with unnecessary information. Human-automation interaction is currently evaluated through extensive simulation. In this paper, using both hybrid and discrete-event system techniques, we show how one could mathematically verify that an interface contains enough information for the pilot to safely and unambiguously complete a desired maneuver. We first develop a nonlinear, hybrid model for the longitudinal dynamics of a large civil jet aircraft in an autoland/go-around maneuver. We find the largest controlled subset of the aircraft's flight envelope for which we can guarantee both safe landing and safe go-around. We abstract a discrete procedural model using this result, and verify a discrete formulation of the pilot display against it. An interface which fails this verification could result in nondeterministic or unpredictable behavior from the pilot's point of view.

Oishi, Meeko↗

Bay of Fundy verification of a system for multidate Landsat measurement of suspended sediment

A system for automated multidate Landsat CCT MSS measurement of suspended sediment concentration (S) has been implemented and verified on nine sets (108 points) of data from the Bay of Fundy, Canada. The system employs 'chromaticity analysis' to provide automatic pixel-by-pixel adjustment of atmospheric variations, permitting reference calibration data from one or several dates to be spatially and temporally extrapolated to other regions and to other dates. For verification, each data set was used in turn as test data against the remainder as a calibration set: the average absolute error was 44 percent of S over the range 1-1000 mg/l. The system can be used to measure chlorophyll (in the absence of atmospheric variations), Secchi disk depth, and turbidity.

Munday, J. C., Jr.↗

Mission operations and command assurance - Automating an operations TQM task

A long-term program is in progress at JPL to reduce cost and risk of mission operations through defect prevention and error management. A major element of this program, Mission Operations and Command Assurance (MO&CA), provides a system level function on flight projects to instill quality in mission operations. MO&CA embodies the total quality management TQM principle of continuous process improvement (CPI) and uses CPI in applying automation to mission operations to reduce risk and costs. MO&CA has led efforts to apply and has implemented automation in areas that impact the daily flight project work environment including Incident Surprise Anomaly tracking and reporting; command data verification, tracking and reporting; and command support data usage. MO&CA's future work in automation will take into account that future mission operations systems must be designed to avoid increasing error through the introduction of automation, while adapting to the demands of smaller flight teams.

Welz, Linda↗

Sim to Flight: Evaluating Flight Path Management Automation in High Density Urban Environments

Combined simulation and flight testing enable the study of single- and multi-aircraft performance of onboard automation systems for dynamic flight path management (FPM). The National Aeronautics and Space Administration (NASA) is investigating system performance and functional capabilities of such automation for immersion into complex, high density, future operations such as Urban Air Mobility (UAM). This paper provides an overview of a series of interdependent sim-to-flight research activities involving large-scale batch simulations, human-in-the-loop verification, and flight-test validation of a research prototype FPM automation system. Together, they significantly contributed to a functional assessment of FPM automation functionality in a live-virtual-constructive (LVC) operating environment characterized by two live aircraft and hundreds of virtual aircraft interacting in a modeled complex urban airspace. Initial simulation and flight test results, future work, and conclusions are presented.

Advanced Air Mobility↗

(abstract) Automated Constraint Checking of Spacecraft Command Sequences

Making certain that spacecraft command sequences do not violate any constraints is often tedious and expensive in terms of both personnel and software development. To reduce this cost, we have pursued the development of a flexible system for specifying models of spacecraft behavior in response to commands as well as constraints on that behavior. The potential need for modeling complex spacecraft behavior required that the system be designed to be usable both on a conventional workstation and a parallel supercomputer. Finally, it needed to be intuitive enough for the the intended mission operations users to easily design sets of rules and models to automate tedious, resource-consuming constraint checking of commands. We have defined a Specification And Verification Environment (SAVE) for spacecraft flight rules.

automation spacecraft command sequences constraint↗

Full-Scale Wind-Tunnel Investigation of Wing-Cooling Ducts Effects of Propeller Slipstream, Special Report

The safety of remotely operated vehicles depends on the correctness of the distributed protocol that facilitates the communication between the vehicle and the operator. A failure in this communication can result in catastrophic loss of the vehicle. To complicate matters, the communication system may be required to satisfy several, possibly conflicting, requirements. The design of protocols is typically an informal process based on successive iterations of a prototype implementation. Yet distributed protocols are notoriously difficult to get correct using such informal techniques. We present a formal specification of the design of a distributed protocol intended for use in a remotely operated vehicle, which is built from the composition of several simpler protocols. We demonstrate proof strategies that allow us to prove properties of each component protocol individually while ensuring that the property is preserved in the composition forming the entire system. Given that designs are likely to evolve as additional requirements emerge, we show how we have automated most of the repetitive proof steps to enable verification of rapidly changing designs.

Nickle, F. R.↗

KBS V and V as related to automation of space station subsystems: Rationale for a KBS lifecycle

The role of verification and validation (V and V) in software has been to support and strengthen the software lifecycle and to ensure that the resultant code meets the standards of the requirements document. Knowledge based systems (KBS) V and V should serve the same role, but the KBS lifecycle is ill-defined. Here, the rationale of the simple form of the KBS lifecycle is explained. Special KBS development requirements are accommodated where possible by modifications to the traditional software lifecycle. Research areas are suggested for those aspects which present new or unusual difficulties for V and V.

Richardson, K.↗

Software Verification of Orion Cockpit Displays

NASA's latest spacecraft Orion is in the development process of taking humans deeper into space. Orion is equipped with three main displays to monitor and control the spacecraft. To ensure the software behind the glass displays operates without faults, rigorous testing is needed. To conduct such testing, the Rapid Prototyping Lab at NASA's Johnson Space Center along with the University of Texas at Tyler employed a software verification tool, EggPlant Functional by TestPlant. It is an image based test automation tool that allows users to create scripts to verify the functionality within a program. A set of edge key framework and Common EggPlant Functions were developed to enable creation of scripts in an efficient fashion. This framework standardized the way to code and to simulate user inputs in the verification process. Moreover, the Common EggPlant Functions can be used repeatedly in verification of different displays.

Biswas, M. A. Rafe↗

Digital Twin + AI: Control Room of the Future [Slides]

The control room functions as the central brain of the grid, essential for balancing supply and demand and ensuring moment-to-moment grid reliability. Like the human brain, which processes sensory data to make decisions, control room operators analyze operational data from power generation, transmission, and distribution to make informed decisions. Currently, decision-making primarily rests with operators due to hardware and software limitations. However, with technological advancements, Digital Twins and AI are becoming high interest points in the control room's decision-making pilot programs. NREL is developing a comprehensive decision-making platform that integrates Digital Twins, AI, and advanced visualization techniques. As this integration progresses, the role of Digital Twins will evolve from conducting automated simulations to serving as a Trustworthy AI enabler, offering verification and validation of AI-generated response for power systems or providing physics-aware synthetic data of AI pre-training.

24 POWER TRANSMISSION AND DISTRIBUTION↗