Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Formal verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29

NASA's Space Launch System: Systems Engineering Approach for Affordability and Mission Success

NASA is working toward the first launch of the Space Launch System, a new, unmatched capability for deep space exploration with launch readiness planned for 2019. Since program start in 2011, SLS has passed several major formal design milestones, and every major element of the vehicle has produced test and flight hardware. The SLS approach to systems engineering has been key to the program's success. Key aspects of the SLS SE&I approach include: 1) minimizing the number of requirements, 2) elimination of explicit verification requirements, 3) use of certified models of subsystem capability in lieu of requirements when appropriate and 4) certification of capability beyond minimum required capability.

Hutt, John J.↗

Establishing the Assurance Efficacy of Automated Risk Mitigation Strategies

Verification and validation of increasingly autonomous aviation systems is a major challenge. Traditional techniques for the assurance of high-confidence, safety-critical systems are not equipped to handle the complexity, uncertainty, and lack of predictability inherent in non-deterministic systems. Techniques such as run time monitoring, formal methods, and testing and simulation have been applied to some effect, but it is difficult to properly assess the success of such measures. The authors propose the concept of Assurance Efficacy to address this gap. Assurance Efficacy is seen as a parameter, criteria, or perspective by which to evaluate, identify and explore safety risk mitigation strategies and operational assurance architectures. Validation of the utility of this concept through flight testing is a first step in determining its potential role in assessing the overall safety of complex, increasingly autonomous systems that cannot be fully assured in the design phase.

system safety↗

NASA's M and S Accreditation Process Plan and Specification for Space Exploration

NASA's Exploration Systems Mission Directorate (ESMD) is implementing a management approach for modeling and simulation (M&S) that will provide decision-makers information on the model's fidelity, credibility, and quality. This information will allow the decision-maker to understand the risks involved in using a model's results in the decision-making process. This presentation will discuss NASA's overall approach to achieving formal accreditation of its models or simulations supporting space exploration. The development of a formal Accreditation Plan is a key component in the preliminary activities for modeling and simulation (M&S) assessment. This presentation will describe NASA's process for identifying risks associated with M&S use and the associated M&S assessments that will dictate the level of data certification and M&S verification and validation (V&V) activities required to support the decision-making process. The M&S Accreditation Plan and Report templates for ESMD will also be illustrated.

O'Neil, David↗

EM Program Expectations on Implementation of DOE O 458: Radiation Protection of Public/Environment - 19553

DOE Order 458.1, Radiation Protection of the Public and Environment, was issued in 2011. DOE sites were given 18 months to implement. The revised Order promulgates requirements for property clearance actions and independent verification. DOE Order 458.1 allows for preapproved authorized limits for the clearance of property. However, when property with the potential to contain residual radioactive material does not meet preapproved authorized limits, an authorized limits package must be developed. The Order contains a range of requirements for demonstrating with reasonable assurance that dose constraints are met for any actual or likely future use of the property to be released. EM has developed a draft guidance document that provides the elements of an authorized limits package. These elements are used to review all authorized limits packages that are submitted to EM. Expected elements include property description and process knowledge documentation, current and likely future use, the conceptual site model, dose modeling methodologies, specific authorized limits, dose- or concentration-based constraints, collective dose estimate to the potentially exposed population, stakeholder communications, description of the As Low As Reasonably Achievable (ALARA) analysis, implementation plan, evaluation of difficult to assess surfaces, and independent verification plan. The draft guidance document also provides further explanation of the desired elements of an independent verification (IV) plan. An IV plan should describe the planning phase, implementation phase, assessment phase, and decision phase of an activity. The planning phase includes scope and cost, concurrent with cleanup contractor procurement, and review and approval of site radiological survey clearance program and IV program plans. The implementation phase assures sufficient collection of data and adjustment can be made as necessary. During the assessment phase, data and reports are reviewed, and coordination between the clean-up contractor and IV personnel occurs. A decision of whether or not the clearance actions need to be altered occurs during the decision phase. IV certification that the clearance action is complete also occurs during the decision phase. Issuance of the final guidance document is meant to improve the consultation process and minimize iterative interactions with EM. Standardizing and formalizing property clearance and IV actions also helps assure that cleanup actions are transparent and sustainable for future scrutiny. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

A BrachyPhantom for verification of dose calculation of HDR brachytherapy planning system

Purpose: To develop a calibration phantom for {sup 192}Ir high dose rate (HDR) brachytherapy units that renders possible the direct measurement of absorbed dose to water and verification of treatment planning system.Methods: A phantom, herein designated BrachyPhantom, consists of a Solid Water™ 8-cm high cylinder with a diameter of 14 cm cavity in its axis that allows the positioning of an A1SL ionization chamber with its reference measuring point at the midheight of the cylinder's axis. Inside the BrachyPhantom, at a 3-cm radial distance from the chamber's reference measuring point, there is a circular channel connected to a cylindrical-guide cavity that allows the insertion of a 6-French flexible plastic catheter from the BrachyPhantom surface. The PENELOPE Monte Carlo code was used to calculate a factor, P{sub sw}{sup lw}, to correct the reading of the ionization chamber to a full scatter condition in liquid water. The verification of dose calculation of a HDR brachytherapy treatment planning system was performed by inserting a catheter with a dummy source in the phantom channel and scanning it with a CT. The CT scan was then transferred to the HDR computer program in which a multiple treatment plan was programmed to deliver a total dose of 150 cGy to the ionization chamber. The instrument reading was then converted to absorbed dose to water using the N{sub gas} formalism and the P{sub sw}{sup lw} factor. Likewise, the absorbed dose to water was calculated using the source strength, S{sub k}, values provided by 15 institutions visited in this work.Results: A value of 1.020 (0.09%, k= 2) was found for P{sub sw}{sup lw}. The expanded uncertainty in the absorbed dose assessed with the BrachyPhantom was found to be 2.12% (k= 1). To an associated S{sub k} of 27.8 cGy m{sup 2} h{sup −1}, the total irradiation time to deliver 150 cGy to the ionization chamber point of reference was 161.0 s. The deviation between the absorbed doses to water assessed with the BrachyPhantom and those calculated by the treatment plans and using the S{sub k} values did not exceed ±3% and ±1.6%, respectively.Conclusions: The BrachyPhantom may be conveniently used for quality assurance and/or verification of HDR planning system with a priori threshold level to spot problems of 2% and ±3%, respectively, and in the long run save time for the medical physicist.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

MBSE Validation and Verification: Case Study for LADEE

The Lunar Atmosphere Dust Environment Explorer (LADEE) mission orbited the moon in order to measure the density, composition, and time variability of the lunar dust environment. The successful mission launched September 7, 2013 and was de-orbited and impacted the moon's surface on April 17, 2014. The ground-side and onboard flight software for the mission was developed using a “Model-Based Software Engineering” (MBSE) methodology combined with strong reuse of Government and Commercial Off-The Shelf (G/COTS) components. Models of the spacecraft and flight software were developed in a graphical dynamics modeling package. Flight Software requirements were prototyped and refined using the simulated models. After the model was shown to work as desired in the simulation framework, C-code software was automatically generated from the models. The auto-generated software was then tested in real-time Processor-in-the-Loop and Hardware-in-the-Loop test beds. “Traveling Road Show” test beds were used for early integration tests with payloads and other subsystems. Traditional techniques for verifying computational sciences models were used to characterize the spacecraft simulation. A lightweight set of formal methods analysis, static analysis, formal inspection, and code coverage analyses were utilized to further reduce defects in the onboard flight software artifacts. These techniques were applied early and often in the development process, iteratively increasing the capabilities of software and fidelity of vehicle models and test beds.

Model-Based Software Engineering, Validation and V↗

An error covariance model for sea surface topography and velocity derived from TOPEX/POSEIDON altimetry

In order to facilitate the use of satellite-derived sea surface topography and velocity oceanographic models, methodology is presented for deriving the total error covariance and its geographic distribution from TOPEX/POSEIDON measurements. The model is formulated using a parametric model fit to the altimeter range observations. The topography and velocity modeled with spherical harmonic expansions whose coefficients are found through optimal adjustment to the altimeter range residuals using Bayesian statistics. All other parameters, including the orbit, geoid, surface models, and range corrections are provided as unadjusted parameters. The maximum likelihood estimates and errors are derived from the probability density function of the altimeter range residuals conditioned with a priori information. Estimates of model errors for the unadjusted parameters are obtained from the TOPEX/POSEIDON postlaunch verification results and the error covariances for the orbit and the geoid, except for the ocean tides. The error in the ocean tides is modeled, first, as the difference between two global tide models and, second, as the correction to the present tide model, the correction derived from the TOPEX/POSEIDON data. A formal error covariance propagation scheme is used to derive the total error. Our global total error estimate for the TOPEX/POSEIDON topography relative to the geoid for one 10-day period is found tio be 11 cm RMS. When the error in the geoid is removed, thereby providing an estimate of the time dependent error, the uncertainty in the topography is 3.5 cm root mean square (RMS). This level of accuracy is consistent with direct comparisons of TOPEX/POSEIDON altimeter heights with tide gauge measurements at 28 stations. In addition, the error correlation length scales are derived globally in both east-west and north-south directions, which should prove useful for data assimilation. The largest error correlation length scales are found in the tropics. Errors in the velocity field are smallest in midlatitude regions. For both variables the largest errors caused by uncertainty in the geoid. More accurate representations of the geoid await a dedicated geopotential satellite mission. Substantial improvements in the accuracy of ocean tide models are expected in the very near future from research with TOPEX/POSEIDON data.

Tsaoussi, Lucia S.↗

What Sensing Tells Us: Towards a Formal Theory of Testing for Dynamical Systems

Just as actions can have indirect effects on the state of the world, so too can sensing actions have indirect effects on an agent's state of knowledge. In this paper, we investigate "what sensing actions tell us", i.e., what an agent comes to know indirectly from the outcome of a sensing action, given knowledge of its actions and state constraints that hold in the world. To this end, we propose a formalization of the notion of testing within a dialect of the situation calculus that includes knowledge and sensing actions. Realizing this formalization requires addressing the ramification problem for sensing actions. We formalize simple tests as sensing actions. Complex tests are expressed in the logic programming language Golog. We examine what it means to perform a test, and how the outcome of a test affects an agent's state of knowledge. Finally, we propose automated reasoning techniques for test generation and complex-test verification, under certain restrictions. The work presented in this paper is relevant to a number of application domains including diagnostic problem solving, natural language understanding, plan recognition, and active vision.

McIlraith, Sheila↗

A Test Generation Framework for Distributed Fault-Tolerant Algorithms

Heavyweight formal methods such as theorem proving have been successfully applied to the analysis of safety critical fault-tolerant systems. Typically, the models and proofs performed during such analysis do not inform the testing process of actual implementations. We propose a framework for generating test vectors from specifications written in the Prototype Verification System (PVS). The methodology uses a translator to produce a Java prototype from a PVS specification. Symbolic (Java) PathFinder is then employed to generate a collection of test cases. A small example is employed to illustrate how the framework can be used in practice.

Goodloe, Alwyn↗

Fusing Quantitative Requirements Analysis with Model-based Systems Engineering

A vision is presented for fusing quantitative requirements analysis with model-based systems engineering. This vision draws upon and combines emergent themes in the engineering milieu. "Requirements engineering" provides means to explicitly represent requirements (both functional and non-functional) as constraints and preferences on acceptable solutions, and emphasizes early-lifecycle review, analysis and verification of design and development plans. "Design by shopping" emphasizes revealing the space of options available from which to choose (without presuming that all selection criteria have previously been elicited), and provides means to make understandable the range of choices and their ramifications. "Model-based engineering" emphasizes the goal of utilizing a formal representation of all aspects of system design, from development through operations, and provides powerful tool suites that support the practical application of these principles. A first step prototype towards this vision is described, embodying the key capabilities. Illustrations, implications, further challenges and opportunities are outlined.

optimization↗

Mesoscale to microscale wind farm flow modeling and evaluation

The increasing size of wind turbines, with rotors already spanning more than 150 m diameter and hub heights above 100 m, requires proper modeling of the atmospheric boundary layer (ABL) from the surface to the free atmosphere. Furthermore, large wind farm arrays create their own boundary layer structure with unique physics. This poses significant challenges to traditional wind engineering models that rely on surface-layer theories and engineering wind farm models to simulate the flow in and around wind farms. However, adopting an ABL approach offers the opportunity to better integrate wind farm design tools and meteorological models. The challenge is how to build the bridge between atmospheric and wind engineering model communities and how to establish a comprehensive evaluation process that identifies relevant physical phenomena for wind energy applications with modeling and experimental requirements. A framework for model verification, validation, and uncertainty quantification is established to guide this process by a systematic evaluation of the modeling system at increasing levels of complexity. In terms of atmospheric physics, ‘building the bridge’ means developing models for the so-called ‘terra incognita,’ a term used to designate the turbulent scales that transition from mesoscale to microscale. This range of scales within atmospheric research deals with the transition from parameterized to resolved turbulence and the improvement of surface boundary-layer parameterizations. The coupling of meteorological and wind engineering flow models and the definition of a formal model evaluation methodology, is a strong area of research for the next generation of wind conditions assessment and wind farm and wind turbine design tools. Some fundamental challenges are identified in order to direct future research in this area.

17 WIND ENERGY↗

Hierarchical Modeling and Statistical Calibration for Photometric Redshifts

The cosmological exploitation of modern photometric galaxy surveys requires both accurate (unbiased) and precise (narrow) redshift probability distributions derived from broadband photometry. Existing methodologies do not meet these requirements. Standard template fitting delivers interpretable models and errors, but lacks the flexibility to learn inaccuracies in the observed photometry or spectral templates. Machine learning addresses those issues, but requires representative training data, and the resulting models and uncertainties cannot be interpreted in the context of a physical model or outside of the training data. We present a hierarchical modeling approach simultaneously addressing the issues of flexibility, interpretability, and generalization. It combines template fitting with flexible (machine-learning-like) models to correct the spectral templates, model their redshift distributions, and recalibrate the photometric observations. By optimizing the full posterior distribution of the model and solving for its (thousands of) parameters, one can perform a global statistical calibration of the data and the spectral energy distribution (SED) model. We apply this approach to the public Dark Energy Survey Science Verification data and show that it provides more accurate and compact redshift posterior distributions than existing methods, as well as insights into residual photometric and SED systematics. Here, the model is causal and makes predictions for future data (e.g., additional photometric bandpasses), and its internal parameters and components are interpretable. This approach does not formally require the training data to be complete or representative; in principle, it can even work in regimes in which few or no spectroscopic redshifts are available.

79 ASTRONOMY AND ASTROPHYSICS↗

What FM can offer DFCS design

The results of aircrafts and spacecrafts flight tests are reported. It is shown that the problems of Digital Flight Control Systems (DFCS) are the problems of systems whose complexity has exceeded the reach of the intellectual tools employed. It is also shown that intuition, experience, and techniques derived from mechanical and analog systems are insufficient for complex, integrated, digital systems. Formal Methods (FM) of computer science can offer DFCS systematic techniques for the construction of trustworthy software, including: techniques for the precise specification of requirements and the development of designs; systematic approaches to the design and structuring of distributed and concurrent systems; fault tolerance algorithms; and systematic methods of testing and analytic methods of verification.

Rushby, John↗

A Provably Correct Floating-Point Implementation of Well Clear Avionics Concepts

The NASA DAIDALUS library provides formal definitions for Detect-and-Avoid avionics concepts such as when an aircraft is well-clear with respect to the surrounding air traffic, i.e., it does not operate in such proximity to create a collision hazard. While several properties are proven correct for DAIDALUS assuming ideal real number arithmetic, an actual implementation that uses floating-point numbers may behave unexpectedly because of round-off errors and run-time exceptions. This paper presents an experience report on the application of a formal methods toolchain to extract and verify floating-point C code from a real-valued specification of the well-clear module of DAIDALUS. This toolchain comprises the PVS theorem prover, the PRECiSA floating-point analyzer and code generator, and the Frama-C analysis suite. The generated code is automatically instrumented to detect when the control flow of the floating-point program may diverge from the ideal real number specification, and it is annotated with contracts that state the maximum accumulated round-off error. The absence of overflows is also formally verified for the generated code. In order to apply the toolchain to an industrial case study such as DAIDALUS, a formally verified pre-processing of the input specification is performed, which includes a program slicing and several semantic-preserving simplifications.

Program verification↗

Closing the Certification Gaps in Adaptive Flight Control Software

Over the last five decades, extensive research has been performed to design and develop adaptive control systems for aerospace systems and other applications where the capability to change controller behavior at different operating conditions is highly desirable. Although adaptive flight control has been partially implemented through the use of gain-scheduled control, truly adaptive control systems using learning algorithms and on-line system identification methods have not seen commercial deployment. The reason is that the certification process for adaptive flight control software for use in national air space has not yet been decided. The purpose of this paper is to examine the gaps between the state-of-the-art methodologies used to certify conventional (i.e., non-adaptive) flight control system software and what will likely to be needed to satisfy FAA airworthiness requirements. These gaps include the lack of a certification plan or process guide, the need to develop verification and validation tools and methodologies to analyze adaptive controller stability and convergence, as well as the development of metrics to evaluate adaptive controller performance at off-nominal flight conditions. This paper presents the major certification gap areas, a description of the current state of the verification methodologies, and what further research efforts will likely be needed to close the gaps remaining in current certification practices. It is envisioned that closing the gap will require certain advances in simulation methods, comprehensive methods to determine learning algorithm stability and convergence rates, the development of performance metrics for adaptive controllers, the application of formal software assurance methods, the application of on-line software monitoring tools for adaptive controller health assessment, and the development of a certification case for adaptive system safety of flight.

Jacklin, Stephen A.↗

Parsing-based Approaches for Verification and Recognition of Hierarchical Plans

Hierarchical Task Networks were proposed as a method todescribe plans by decomposition of tasks to sub-tasks untilprimitive tasks, actions, are obtained. Plan verification assumesa complete plan as input, and the objective is findinga task that decomposes to this plan. In plan recognition, aprefix of the plan is given and the objective is finding a taskthat decomposes to the (shortest) plan with the given prefix.This paper describes how to verify and recognize plans usinga common method known from formal grammars, by parsing.

Bartak, Roman↗

A Game-Theoretic Quantum Algorithm for Solving Magic Squares

Variational quantum algorithms (VQAs) offer a promising near-term approach to finding optimal quantum strategies for playing non-local games. These games test quantum correlations beyond classical limits and enable entanglement verification. In this work, we present a variational framework for the Magic Square Game (MSG), a two-player non-local game with perfect quantum advantage. We construct a value Hamiltonian that encodes the game’s parity and consistency constraints, then optimize parameterize quantum circuits to minimize this cost. Our approach build on the stabilizer formalism, leverages commutation structure for circuit design, and is hardware-efficient. Compared to existing work, our contribution emphasizes algebraic structure an interpretability. We validate our method through numerical experiments and outline generalizations to larger games.

Chehade, Sarah [ORNL]↗

Optimal Experimental Design for Inverse Problems in the Presence of Observation Correlations

Optimal experimental design (OED) is the general formalism of sensor placement and decisions on the data collection strategy for engineered or natural experiments. This problem is prevalent in many critical fields such as battery design, numerical weather prediction, geosciences, environmental and urban studies. State-of-the-art computational methods for experimental design do not accommodate correlation structure in observational errors produced by many expensive-to-operate devices such as X-ray machines, radars, and satellites. Discarding evident data correlations leads to biased results, higher expenses, and waste of valuable resources. We present a general formulation of the OED formalism for model-constrained large-scale Bayesian linear inverse problems, where measurement errors are generally correlated. The proposed approach utilizes the Hadamard product of matrices to formulate the weighted-likelihood, and is valid for both finite as well as infinite-dimensional Bayesian inverse problems. Extensive numerical experiments are carried out for empirical verification of the proposed approach using an advection-diffusion model, where the objective is to optimally place a small set of sensors, under a limited budget, to predict the concentration of a contaminant in a closed and bounded domain.

Correlated observations↗