Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “formal verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28

Defining and Reasoning about Model-based Safety Analysis: A Review

Model-based safety analysis (MBSA) has been around for over two decades. The benefits of MBSA have been well-documented in the literature, such as tackling complexity, introducing Formal Methods to eliminate the ambiguity in the traditional safety analysis, using automation to replace the error-prone manual safety modeling process, and ensuring consistency between the design model and the safety model. However, there is still a lack of consensus on what MBSA even is. This paper provides an approach towards developing such a consensus

model-based↗

Knowledge-Based Aircraft Automation: Managers Guide on the use of Artificial Intelligence for Aircraft Automation and Verification and Validation Approach for a Neural-Based Flight Controller

The ultimate goal of this report was to integrate the powerful tools of artificial intelligence into the traditional process of software development. To maintain the US aerospace competitive advantage, traditional aerospace and software engineers need to more easily incorporate the technology of artificial intelligence into the advanced aerospace systems being designed today. The future goal was to transition artificial intelligence from an emerging technology to a standard technology that is considered early in the life cycle process to develop state-of-the-art aircraft automation systems. This report addressed the future goal in two ways. First, it provided a matrix that identified typical aircraft automation applications conducive to various artificial intelligence methods. The purpose of this matrix was to provide top-level guidance to managers contemplating the possible use of artificial intelligence in the development of aircraft automation. Second, the report provided a methodology to formally evaluate neural networks as part of the traditional process of software development. The matrix was developed by organizing the discipline of artificial intelligence into the following six methods: logical, object representation-based, distributed, uncertainty management, temporal and neurocomputing. Next, a study of existing aircraft automation applications that have been conducive to artificial intelligence implementation resulted in the following five categories: pilot-vehicle interface, system status and diagnosis, situation assessment, automatic flight planning, and aircraft flight control. The resulting matrix provided management guidance to understand artificial intelligence as it applied to aircraft automation. The approach taken to develop a methodology to formally evaluate neural networks as part of the software engineering life cycle was to start with the existing software quality assurance standards and to change these standards to include neural network development. The changes were to include evaluation tools that can be applied to neural networks at each phase of the software engineering life cycle. The result was a formal evaluation approach to increase the product quality of systems that use neural networks for their implementation.

Broderick, Ron↗

Runtime Verification of Hard Realtime Systems With Copilot: A Tutorial

This presentation is a tutorial on RV using Copilot, a runtime verification framework for real-time embedded systems. Copilot monitors are written in a compositional, stream-based language with support for a variety of Temporal Logics (TL), which results in robust, high-level specifications that are easier to understand than their traditional counterparts. The framework translates monitor specifications into C code with static memory requirements, which can be compiled to run on embedded hardware.

runtime monitoring↗

JANNAF "Test and Evaluation Guidelines for Liquid Rocket Engines": Status and Application

For many decades, the U.S. rocket propulsion industrial base has performed remarkably in developing complex liquid rocket engines that can propel critical payloads into service for the nation, as well as transport people and hardware for missions that open the frontiers of space exploration for humanity. This has been possible only at considerable expense given the lack of detailed guidance that captures the essence of successful practices and knowledge accumulated over five decades of liquid rocket engine development. In an effort to provide benchmarks and guidance for the next generation of rocket engineers, the Joint Army Navy NASA Air Force (JANNAF) Interagency Propulsion Committee published a liquid rocket engine (LRE) test and evaluation (T&E) guideline document in 2012 focusing on the development challenges and test verification considerations for liquid rocket engine systems. This document has been well received and applied by many current LRE developers as a benchmark and guidance tool, both for government-driven applications as well as for fully commercial ventures. The USAF Space and Missile Systems Center (SMC) has taken an additional near-term step and is directing activity to adapt and augment the content from the JANNAF LRE T&E guideline into a standard for potential application to future USAF requests for proposals for LRE development initiatives and launch vehicles for national security missions. A draft of this standard was already sent out for review and comment, and is intended to be formally approved and released towards the end of 2017. The acceptance and use of the LRE T&E guideline is possible through broad government and industry participation in the JANNAF liquid propulsion committee and associated panels. The sponsoring JANNAF community is expanding upon this initial baseline version and delving into further critical development aspects of liquid rocket propulsion testing at the integrated stage level as well as engine component level, in order to advance the state of the practice. The full participation of the entire U.S. rocket propulsion industrial base is invited and expected at this opportune moment in the continuing advancement of spaceflight technology.

Parkinson, Douglas↗

Use of multiple gauges and microwave attenuation of precipitation for satellite verification

In this paper both a microwave attenuation measurement along a horizontal line and multiple point gauge measurements are analyzed as possible ground-truth designs to validate satellite precipitation retrieval algorithms at the field of view spatial level (typically about 20 km). The design consists of comparing a sequence of pairs of contemporaneous measurements taken from the ground and from space. The authors examine theoretically the variance of expected differences between the two systems. The line average measurement leads to a smaller mean-square error compared to the case of a single point gauge, since some of the small-scale variability of the rain field is smoothed away by the line integration. The multiple point gauge measurements also give smaller mean-square error than that of a single point gauge. The centroid of the line and point gauge configurations are considered to be located randomly inside the field of view for different overpasses. A space-time spectral formalism is used with a noise-forced diffusive rain field to find the mean-square error. By considering instantaneous ground and satellite measurement pairs over about 50 visits when raining, we can reduce the expected error to approximately 10% of the standard deviation of climatological variability. This is considered to be a useful level of tolerance for identifying biases in the retrieval algorithms. It is found that the multiple point gauges (especially two gauges) are the economical ground-truth design compared to the microwave attenuation based on the mean-square error comparison. The major finding of this study is that a significant improvement over the point gauge is obtained by adding a single additional piece of information; adding more gauges or extending the line of attenuation is not an important improvement.

Ha, Eunho↗

Validation and Verification of LADEE Models and Software

The Lunar Atmosphere Dust Environment Explorer (LADEE) mission will orbit the moon in order to measure the density, composition and time variability of the lunar dust environment. The ground-side and onboard flight software for the mission is being developed using a Model-Based Software methodology. In this technique, models of the spacecraft and flight software are developed in a graphical dynamics modeling package. Flight Software requirements are prototyped and refined using the simulated models. After the model is shown to work as desired in this simulation framework, C-code software is automatically generated from the models. The generated software is then tested in real time Processor-in-the-Loop and Hardware-in-the-Loop test beds. Travelling Road Show test beds were used for early integration tests with payloads and other subsystems. Traditional techniques for verifying computational sciences models are used to characterize the spacecraft simulation. A lightweight set of formal methods analysis, static analysis, formal inspection and code coverage analyses are utilized to further reduce defects in the onboard flight software artifacts. These techniques are applied early and often in the development process, iteratively increasing the capabilities of the software and the fidelity of the vehicle models and test beds.

Gundy-Burlet, Karen↗

Assessment of Survey Results from Advanced Reactor Industry Domain Experts on Nonlinear Soil-Structure Interaction Analysis Software Verification and Validation

The seismic load case has a significant impact on the design and cost of nuclear power plants. Given the need to substantially reduce cost, advanced reactor designers are looking to leverage numerical tools that enable seismic analysis of an integrated vessel/support/structure/soil system. Modern nonlinear analysis tools provide a solution, capturing dynamic coupling between components (including soil-structure interaction (SSI)) concurrent with nonlinear behavior in one or more parts of the system. Although these methods have a rich history of technical development and implementation, software quality assurance (SQA) following nuclear industry standards remains a significant burden for those wishing to adopt such methods for advanced reactor design and licensing. To reduce the SQA burden, this project will develop the guidance for SQA verification and validation (V&V) of coupled nonlinear SSI analysis tools, to include a test matrix of software features and test problems to support commercial grade dedication (CGD). The guidance is intended to be technology-neutral: supporting designers of all advanced reactors. To maximize the value of the project to reactor designers, the project team performed focused surveys of and interviews with advanced reactor designers. Additionally, the project surveyed members of the broader industry involved in reactor design and licensing. The aggregated industry feedback consists of written survey responses, live polling responses, focused interviews, and informal feedback provided after outreach presentations, collectively referred to as “survey results”. Herein, the survey results are reported, reviewed, and assessed to inform follow-on project activities. The survey results confirmed the familiarity of the industry with the coupled nonlinear SSI analysis. They also affirm the project need based upon the expressed intent to (1) incorporate nonlinear features and (2) pursue the coupled nonlinear SSI analysis to reduce seismic demands and construction costs. All reactor designers indicated their intent to incorporate two or more nonlinear features, and all reactor designer respondents opined that the coupled nonlinear SSI analysis would allow for the optimization of the reactor design and construction. However, the programmatic challenges, whether real or perceived, present a significant barrier to reactor designers. The barrier most commonly identified by the reactor designer population was regulatory risk, with 70% citing this as a reason not to pursue the approach. The perceived regulatory risk identified by the reactor designers underscores the importance of regulator engagement and dialogue in this project. Additionally, half of the reactor designers identified cost and lack of guidance as a deterrent. The survey results also provide insights to tailor specific aspects of the guidance document and test matrix. The project plans to prepare both a formal referenceable guidance document and a collaborative, web-based test matrix and problem set. The project will focus on more complete test problem definitions for the prioritized nonlinear features over shallower problem descriptions for a larger set of features. The nonlinearities prioritized as high based upon survey feedback include fluid-structure interaction and seismic isolation and energy dissipation devices. The nonlinearities prioritized as intermediate include nonlinear geomaterials, nonlinear concrete, nonlinear steel, and interface/contact nonlinearity. Deep embedment and the associated nonlinear phenomena are assigned the lowest priority based upon survey feedback.

42 ENGINEERING↗

Verification and Validation of Flight-Critical Systems

For the first time in many years, the NASA budget presented to congress calls for a focused effort on the verification and validation (V&V) of complex systems. This is mostly motivated by the results of the VVFCS (V&V of Flight-Critical Systems) study, which should materialize as a a concrete effort under the Aviation Safety program. This talk will present the results of the study, from requirements coming out of discussions with the FAA and the Joint Planning and Development Office (JPDO) to technical plan addressing the issue, and its proposed current and future V&V research agenda, which will be addressed by NASA Ames, Langley, and Dryden as well as external partners through NASA Research Announcements (NRA) calls. This agenda calls for pushing V&V earlier in the life cycle and take advantage of formal methods to increase safety and reduce cost of V&V. I will present the on-going research work (especially the four main technical areas: Safety Assurance, Distributed Systems, Authority and Autonomy, and Software-Intensive Systems), possible extensions, and how VVFCS plans on grounding the research in realistic examples, including an intended V&V test-bench based on an Integrated Modular Avionics (IMA) architecture and hosted by Dryden.

Brat, Guillaume↗

Model Checking Degrees of Belief in a System of Agents

Reasoning about degrees of belief has been investigated in the past by a number of authors and has a number of practical applications in real life. In this paper we present a unified framework to model and verify degrees of belief in a system of agents. In particular, we describe an extension of the temporal-epistemic logic CTLK and we introduce a semantics based on interpreted systems for this extension. In this way, degrees of beliefs do not need to be provided externally, but can be derived automatically from the possible executions of the system, thereby providing a computationally grounded formalism. We leverage the semantics to (a) construct a model checking algorithm, (b) investigate its complexity, (c) provide a Java implementation of the model checking algorithm, and (d) evaluate our approach using the standard benchmark of the dining cryptographers. Finally, we provide a detailed case study: using our framework and our implementation, we assess and verify the situational awareness of the pilot of Air France 447 flying in off-nominal conditions.

MAS Verification↗

Numerical computation of exponential matrices using the Cayley-Hamilton theorem

A method for computing exponential matrices, which often arise naturally in the solution of systems of linear differential equations, is developed. An exponential matrix is generated as a linear combination of a finite number (equal to the matrix order) of matrices, the coefficients of which are scalar infinite sums. The method can be generalized to apply to any formal power series of matrices. Attention is focused upon the exponential function, and the matrix exponent is assumed tri-diagonal in form. In such cases, the terms in the coefficient infinite sums can be extracted, as recursion relations, from the characteristic polynomial of the matrix exponent. Two numerical examples are presented in some detail: (1) the three dimensional infinitesimal rotation rate matrix, which is skew symmetric, and (2) an N-dimensional tri-diagonal and symmetric finite difference matrix which arises in the numerical solution of the heat conduction partial differential equation. In the second example, the known eigenvalues and eigenvectors of the finite difference matrix permit an analytical solution for the exponential matrix, through the theory of diagonalization and similarity transformations, which is used for independent verification. The convergence properties of the scalar infinite summations are investigated for finite difference matrices of various orders up to ten, and it is found that the number of terms required for convergence increases slowly with the order of the matrix.

Walden, H.↗

Calculation of the First Moment of Energy Using D-T Reactivity Formalisms Under the Maxwell-Boltzmann Distribution (Pt. 1)

Nuclear fusion science is an example of a scientific field with a rich history of expert involvement and scientific publications, which together, form an expert-knowledge base. One example of this history is the utilization of published reaction rates from a variety of authors. Investigators for Deuterium-Tritium (D-T) ion fusion can choose from using frequently cited methods: the Bosch and Hale reactivity, thermonuclear reaction rates from Caughlan and Fowler, and the reactivity evaluation from Miley, Towner & Ivich which forms the basis of the Naval Research Lab (NRL) formulary. There are other choices available. Each of the reactivity formulations considered here, are based upon the Maxwell- Boltzmann velocity distribution for D-T fusion ion reactants. Numerical methods for computer codes simulating hot, energetic plasmas, include tabulations of the reactivity, and the first moment of energy. The purpose of this study is to answer the question: what is the first moment of energy, and how has it been formulated? This report is part of a series of reports by the authors on D-T fusion-reaction formalism. The present focus is on defining the mathematical relationship of the first moment of D-T fusion ion kinetic energy, $\langle$E$\rangle$, with the fusion crosssection, fusion reactivity and its derivative with respect to ion-temperature. Three variants of the first moment $\langle$E$\rangle$ are analytically developed and explored: 1) constant cross-section, 2) a normalized first moment, and 3) a particular function of the first moment. Much of the mathematical development is relegated to the Appendices, for a concise presentation of topics and results. A research component of this discussion is our comparison of some of the reactivity and its derivative results between evaluations (1) versus (2) and (3), which the authors have not found in any other publication to date. This investigation is useful for validation, verification (V&V) and uncertainty quantification (UQ) as it compares the work of several authors.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Nuclear Forensics

Nuclear forensics is a key part of the nuclear security strategy for the United States and the international community. The NACS Division is central to the leadership role that LLNL has had in nuclear forensics over the past 25 years. Our current nuclear forensic program includes conducting R&D on provenance and attribution signatures, collaborating with international partners, training the next generation of nuclear forensic experts, developing test materials for verification and validation exercises, and providing expert advice to national and international policy makers. Key to our success in these areas has been the strong and continuing involvement in nuclear forensic casework obtained through law enforcement and intelligence channels as well as our outstanding analytical capabilities. NACS must continue to conduct forward-thinking R&D in nuclear forensics that supports operational programs and addresses intelligence gaps. To maintain leadership in nuclear forensics, we must attract, develop, and retain the best talent. The NNSA has a central leadership role in developing educational initiatives for nuclear forensics. In concert with the Glenn T. Seaborg Institute, the NACS Division will continue to participate in all nuclear forensic educational initiatives (undergraduate, graduate, postdoctoral) and incorporate LLNL’s D&I vision into its student and postdoc recruitment efforts. We will continue to build formal relationships with key academic partners and institutions with active and emerging nuclear forensic research interests. We will also strengthen relationships with NNSA DNN and SSAA funded university consortia (e.g. NSSC, ACE, CNEC, CVT, ETI, MTV), helping them conduct research of interest and relevance to nuclear forensics, while at the same time evaluating and recruiting promising talent from these programs.

07 ISOTOPE AND RADIATION SOURCES↗

Uncertainty Quantification for Data-Driven Machine Learning Models in Nuclear Engineering Applications: Where We Are and What Do We Need?

Machine learning (ML) has been leveraged to tackle a diverse range of tasks in almost all branches of nuclear engineering. Many of the successes in ML applications can be attributed to the recent performance breakthroughs in deep learning, the growing availability of computational power, data, and easy-to-use ML libraries. However, these empirical successes have often outpaced our formal understanding of the ML algorithms. An important but under-rated area is uncertainty quantification (UQ) of ML. ML-based models are subject to approximation uncertainty when they are used to make predictions, due to sources including but not limited to, data noise, data coverage, extrapolation, imperfect model architecture and the stochastic training process. The goal of this paper is to clearly explain and illustrate the importance of UQ of ML. We will elucidate the differences in the basic concepts of UQ of physics-based models and data-driven ML models. Various sources of uncertainties in physical modeling and data-driven modeling will be discussed, demonstrated, and compared. We will also present and demonstrate a few techniques to quantify the ML prediction uncertainties, including Monte Carlo dropout, deep ensemble, Bayesian neural networks, Gaussian Processes and conformal prediction. Lastly, we will discuss the need for building a verification, validation and UQ framework to establish ML credibility.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Dynamics of Superfluid Helium in Low-Gravity

This report summarizes the work performed under a contract entitled 'Dynamics of Superfluid Helium in Low Gravity'. This project performed verification tests, over a wide range of accelerations of two Computational Fluid Dynamics (CFD) codes of which one incorporates the two-fluid model of superfluid helium (SFHe). Helium was first liquefied in 1908 and not until the 1930s were the properties of helium below 2.2 K observed sufficiently to realize that it did not obey the ordinary physical laws of physics as applied to ordinary liquids. The term superfluidity became associated with these unique observations. The low temperature of SFHe and it's temperature unifonrmity have made it a significant cryogenic coolant for use in space applications in astronomical observations with infrared sensors and in low temperature physics. Superfluid helium has been used in instruments such as the Shuttle Infrared Astronomy Telescope (IRT), the Infrared Astronomy Satellite (IRAS), the Cosmic Background Observatory (COBE), and the Infrared Satellite Observatory (ISO). It is also used in the Space Infrared Telescope (SIRTF), Relativity Mission Satellite formally called Gravity Probe-B (GP-B), and the Test of the Equivalence Principle (STEP) presently under development. For GP-B and STEP, the use of SFHE is used to cool Superconducting Quantum Interference Detectors (SQUIDS) among other parts of the instruments. The Superfluid Helium On-Orbit Transfer (SHOOT) experiment flown in the Shuttle studied the behavior of SFHE. This experiment attempted to get low-gravity slosh data, however, the main emphasis was to study the low-gravity transfer of SFHE from tank to tank. These instruments carried tanks of SFHE of a few hundred liters to 2500 liters. The capability of modeling the behavior of SFHE is important to spacecraft control engineers who must design systems that can overcome disturbances created by the movement of the fluid. In addition instruments such as GP-B and STEP are very sensitive to quasi-steady changes in the mass distribution of the liquid. The CFD codes were used to model the fluid's dynamic motion. Tests in one-g were performed with the main emphasis on being able to compute the actual damping of the fluid. A series of flights on the NASA Lewis reduced gravity DC-9 aircraft were performed with the Jet Propulsion Laboratory (JPL) Low Temperature Flight Facility and a superfluid Test Cell. The data at approximately 0.04g, lg and 2g were used to determine if correct fundamental frequencies can be predicted based on the acceleration field. Tests in zero gravity were performed to evaluate zero gravity motion.

Frank, David J.↗

NASA Standard for Models and Simulations: Credibility Assessment Scale

As one of its many responses to the 2003 Space Shuttle Columbia accident, NASA decided to develop a formal standard for models and simulations (M and S)ii. Work commenced in May 2005. An interim version was issued in late 2006. This interim version underwent considerable revision following an extensive Agency-wide review in 2007 along with some additional revisions as a result of the review by the NASA Engineering Management Board (EMB) in the first half of 2008. Issuance of the revised, permanent version,hereafter referred to as the M and S Standard or just the Standard, occurred in July 2008.

verification↗

A Fuzzy Logic Approach for Separation Assurance and Collision Avoidance for Unmanned Aerial Systems

In the coming years, operations in low altitude airspace will vastly increase as the capabilities and applications of small Unmanned Aerial Systems (sUAS) continue to multiply. Therefore, solutions to managing sUAS in highly congested airspace must be explored. In this study, a Fuzzy Logic based approach was used to help mitigate the risk of collisions between aircraft using separation assurance and collision avoidance techniques. The system was evaluated for its effectiveness to mitigate the risk of mid-air collisions between aircraft. This system utilizes only current state information and can resolve potential conflicts without knowledge of intruder intent. The avoidance logic was verified using formal methods and shown to select the correct action in all instances. Additionally, the Fuzzy Logic Controllers were shown to always turn the vehicles in the correct direction. Numerical testing demonstrated that the avoidance system was able to prevent a mid-air collision between two sUAS in all tested cases. Simulations were also performed in a three-dimensional environment with a heterogenous fleet of sUAS performing a variety of realistic missions. Simulations showed that the system was 99.98 effective at preventing mid-air collisions when separation assurance was disabled (unmitigated case) and 100 effective when enabled (mitigated case).

Verification & Validation of a Fuzzy Logic Control↗

Verification and Validation of the Stakeholder Tool for Assessing Radioactive Transportation (START) – 22323

The U.S. Department of Energy (DOE) Office of Integrated Waste Management is planning for the eventual transportation, storage, and disposal of spent nuclear fuel (SNF) and high-level radioactive waste (HLW) from nuclear power plant and DOE sites. The Stakeholder Tool for Assessing Radioactive Transportation (START) is a web-based, geospatial decision-support tool developed for evaluating routing options and other aspects of transporting SNF and HLW, covering rail, truck, barge, and intermodal infrastructure and operations in the continental United States. The verification and validation (V&V) process is intended to independently assess START to provide confidence in the ability of START to accurately provide intended results. The V&V process checks the START tool using a variety of methods, ranging from independent hand calculations to comparison of START performance and results to those of other codes. The V&V activity was conducted independently from the START development team with opportunities to provide feedback and collaborate throughout the process. The V&V analyzed attributes of transportation routes produced by START, including route distance and both population and population density captured within buffer zones around routes. Population in the buffer zone, population density in the buffer zone, and route distance were all identified as crucial outputs of the START code and were subject to V&V tasks. Some of the improvements identified through the V&V process were standardizing the underlying population data in START, changing the projection of the population raster data, and changes to the methodology used for population density to improve its applicability for expected users. This collaboration also led to suggested improvements to some of the underlying shape file segments within START. A formal series of test routes went through the V&V process from START version 3.2.1 following the implementation of the recommended improvements to START. The V&V of the buffer zone population and distance reported in START were achieved by utilizing a custom workflow developed in QGIS. This platform was chosen because it is fully independent of START. The buffer zone population and route distances in START showed excellent agreement to independent V&V test results. Over 200 route test cases were run; in all cases, the percent difference in the population within the buffer zone was less than +/- 5%, and the majority of cases were below +/- 1%. These cases were also duplicated in ArcMap, which is Environmental Systems Research Institute’s (ESRI) GIS desktop application. The ArcMap results also show good agreement; however, they cannot be considered independent of START because START also uses ESRI software. The V&V tasks are currently addressing population density and estimated radiation dose in START as well as implementing further automation of the V&V process to allow larger suites of test routes to be run more easily and quickly by less experienced GIS users. These improvements to the V&V process will make it a valuable tool for the development team and the stakeholders so that each new iteration of START can easily be checked for quality and consistency. This paper and presentation will describe the V&V process, findings and implications, and ongoing V&V activities.

START, DOE, Transporation, dose, GIS↗

Bridging the Gap Between Requirements and Simulink Model Analysis

Formal veri fication and simulation are powerful tools for the veri fication of requirements against complex systems. Requirements are developed in early stages of the software lifecycle and are typically expressed in natural language. There is a gap between such requirements and their software implementations. We present a framework that bridges this gap by supporting a tight integration and feedback loop between high-level requirements and their analysis against software artifacts. Our framework implements an analysis portal within the fret requirements elicitation tool, thus forming an end-to-end, open-source environment where requirements are written in an intuitive, structured natural language, and are veri fied automatically against Simulink models.

FRET↗