Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Formal Verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

A framework for monitoring-based commissioning: Identifying variables that act as barriers and enablers to the process

The practice of monitoring-based commissioning (MBCx) using energy management and information systems (EMIS) has been shown to enable and help sustain up to 20% energy savings in buildings. Despite research that has quantified the costs, benefits, and energy savings of MBCx, the process remains under-utilized. To understand why MBCx is not more frequently adopted and how to encourage its use, this research synthesizes qualitative data from over 40 organizations, currently engaging in MBCx. The outcome of this research is a framework containing variables that emerged from the qualitative data, marked as barriers or enablers, organized by phases of the MBCx process. The framework is comprised of 51 emergent variables that fall within 13 different categories. The variables that most frequently act as barriers are data configuration, measurement & verification (M&V), developing specifications for EMIS, and data architecture. Although some variables that act as barriers for one organization were identified as enablers for another. For example, payback/ROI was considered a barrier 7 times and an enabler 3 times. One organization had difficulty making the business case for the initial investment for MBCx due to lack of cost information, while another was able to justify large investments with documented savings of previously implemented measures identified through MBCx. The framework formally validates barriers found in previous research, and can be used by practitioners to better understand common experiences with MBCx. This research also highlights the need for a similar collective data set to validate common enablers to MBCx and also the need for empirical research to determine relationships between variables.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
NASA NTRS1995

Sea surface topography fields of the tropical Pacific from data assimilation

Time series of maps of monthly tropical Pacific dynamic topography anomalies from 1979 through 1985 were constructed by means of assimilation of the tide gauge and expendable bathythermograph (XBT) data into a linear model driven by observed winds. Estimates of error statistics were calculated and compared to actual differences between hindcasts and observations. Four experiements were performed as follows: one with no assimilation, one with assimiation of sea level anomaly data from eight selected island tide gauge stations, one with assimilation of dynamic height anomalies derived from XBT data, and one with both XBT and tide gauge data assimilated. Data from seven additional tide gauge stations were withheld from the assimilation process and used for verification in all four experiements. Statistical objective maps based on data alone were also constructed for comparison purposes. The dynamic response of the model without assimilation was, in general, weaker than the observed response. Assimilation resulted in enhanced signal amplitude in all three assimilation experiments. With few exceptions the error estimated generated by the Kalman filter appeared quite reasonable. Since the error processes cannot be assumed to be white or stationary, we could find no straightforward way to test the formal statistical hypothesis that the time series of differences between the filter ouput and the actual observations were drawn from a population with statistics given by the Kalman filter estimaes. We found that the best filter we could devise was still short of the goal of producing a white innovation sequence. Extensive changes in the assumed error statistics make only marginal differences. The same is true for long time and space scale behavior of different models with richer physics and finer resolution. Better data assimilation results will probably require relaxation of the assumptions of stationarity and serial independence of the errors. Formation of such detailed noise models will require longer time series, with the attendant problems of matching very different data sets.

Miller, Robert N.↗
NASA NTRS2020

Europa Clipper Mission: Onward to Implementation!

Europa, the fourth largest moon of Jupiter, is believed to be one of the best places in the solar system to look for extant life beyond Earth, and the Europa Clipper Mission is preparing to investigate. Now entering the fifth year of life as a formal project, the Europa Clipper Mission has completed its major design phase and is preparing for implementation. The mission envisions sending a flight system, consisting of a spacecraft equipped with a payload of NASA-selected scientific instruments, to execute numerous flybys of Europa while in Jupiter orbit. The mission is being formulated and implemented by a joint Jet Propulsion Laboratory (JPL) and Applied Physics Laboratory (APL) Project team. At the Project Preliminary Design Review (PDR) in August 2018, it was determined that the unique integration of the REASON instrument’s antennas with the spacecraft’s solar arrays required additional time for maturity and review. The required maturity was demonstrated, and in June 2019 the Clipper Mission passed its Delta Preliminary Design Review, and formal confirmation for entry into Phase C was announced by NASA in August 2019. The earliest launch opportunity is now in July 2023, one year later than originally planned. The Project is now working toward its Critical Design Review in May 2020. This paper will describe the progress of the Europa Clipper Mission since January 2019, including maturation of the spacecraft, subsystem and instrument designs, the beginning of fabrication of engineering models and the first flight models, planning for the verification & validation phase, and planning for operations.

Jackson, Maddalena↗
NASA NTRS1939

Full-Scale Wind-Tunnel Investigation of Wing-Cooling Ducts Effects of Propeller Slipstream, Special Report

The safety of remotely operated vehicles depends on the correctness of the distributed protocol that facilitates the communication between the vehicle and the operator. A failure in this communication can result in catastrophic loss of the vehicle. To complicate matters, the communication system may be required to satisfy several, possibly conflicting, requirements. The design of protocols is typically an informal process based on successive iterations of a prototype implementation. Yet distributed protocols are notoriously difficult to get correct using such informal techniques. We present a formal specification of the design of a distributed protocol intended for use in a remotely operated vehicle, which is built from the composition of several simpler protocols. We demonstrate proof strategies that allow us to prove properties of each component protocol individually while ensuring that the property is preserved in the composition forming the entire system. Given that designs are likely to evolve as additional requirements emerge, we show how we have automated most of the repetitive proof steps to enable verification of rapidly changing designs.

Nickle, F. R.↗
DOE OSTI2015

SU-F-BRA-07: Dosimetric Verification of the Valencia Skin Applicator Using Gafchromic EBT3 Film

Purpose: The Valencia applicators have recently been introduced for HDR treatment of small and shallow superficial skin lesions (< 20 mm diameter and 3-mm depth). Per AAPM TG 56, any HDR applicator internal dimensions must be verified prior to clinical use. However radiographic and tomographic imaging to validate the Valencia applicators is impractical due to the Tungsten alloy housing and flattening filter. In this study, we propose to use EBT3 film to indirectly confirm the physical integrity of the Valencia applicators. Methods: Treatment plans were created using the Oncentra MasterPlan TPS v4.5 for the H2 (20-mm dia.) and H3 (30-mm dia.) Valencia Applicators. A virtual CT phantom (2-mm slice thickness) was created with one source position in water. The published effective depth method was used for each applicator to delivery 500 cGy to a 3-mm depth using the TG-43 formalism. Film measurements (n=3) at 3-mm depth and vertical plane in solid water were performed for each applicator to verify the prescribed dose calculated by the TPS. Percent depth dose curves and off-axis profiles (phantom surface and 3-mm depth) were measured and compared to published data. Films were analyzed using an in-house written program and RIT113 v6 software. Film calibration was performed per TG-55 protocol using the Ir-192 source with NIST-traceable calibration. Results: The prescription absolute dose difference was 1% for the Valencia H2 applicator and 4% for the Valencia H3 applicator. The measured percent depth dose curves and off-axis dose profiles measured for the H2 and H2 Valencia applicators are in excellent agreement with the Granero et al. Monte Carlo results{sup 1}. Conclusion: Gafchromic EBT3 film can be used to indirectly verify the internal components of special HDR skin applicators constructed from high Z materials.{sup 1}Granero et al. “Design and evaluation of a HDR skin applicator with flattening filter”, Med. Phys. 35(2), 495–503, 2008.

60 APPLIED LIFE SCIENCES↗
NASA NTRS1977

Documentation requirements for Applications Systems Verification and Transfer projects (ASVTs)

NASA's Application Systems Verification and Transfer Projects (ASVTs) are deliberate efforts to facilitate the transfer of applications of NASA-developed space technology to users such as federal agencies, state and local governments, regional planning groups, public service institutions, and private industry. This study focused on the role of documentation in facilitating technology transfer both to primary users identified during project planning and to others with similar information needs. It was understood that documentation can be used effectively when it is combined with informal (primarily verbal) communication within each user community and with other formal techniques such as organized demonstrations and training programs. Documentation examples from eight ASVT projects and one potential project were examined to give scope to the investigation.

Suchy, J. T.↗
DOE OSTI2016

SU-G-BRB-06: Commissioning and Evaluation of EPID-Based in Vivo Dosimetry Software Using a Tissue-Maximum Ratio Approach

Purpose: To commission and evaluate an in vivo EPID-based transit dosimetry software (EPIgray, DOSIsoft, Cachan, France) using simple fields and TG119-based IMRT treatment plans. Methods: EPIgray was commissioned on a Truebeam based on finite tissue-maximum ratio (fTMR) measurements with solid water blocks of thicknesses between 0 and 37 cm. Field sizes varied from 2×2 to 20×20 cm{sup 2}. Subsequently, treatment plans of single and opposed beams with field sizes between 4×4 and 15×15 cm{sup 2} as well as IMRT plans were measured to evaluate the dose reconstruction accuracy. Single field dose predictions were made for anterior-posterior and lateral beams. IMRT plans were created based on TG119 recommendations. The reconstructed dose was compared to the planned dose for selected points at isocenter and away from isocenter. Results: For single square fields, the dose in EPIgray was reconstructed within 3% accuracy at isocenter relative to the planned dose. Similarly, the relative deviation of the total dose was accurately reconstructed within 3% for all IMRT plans with points placed inside a high dose region near the isocenter. Predictions became less accurate than 5% when the evaluation point was outside the majority of IMRT beam segments. Additionally, points 5 cm or more away from the isocenter or within an avoidance structure were predicted less reliably. Conclusion: EPIgray formalism accuracy is adequate for an efficient error detection system. It provides immediate intra-fractional feedback on the delivery of treatment plans without affecting the treatment beam. Besides the EPID, no additional hardware is required, which makes it accessible to all clinics. The software evaluates point dose measurements to verify treatment plan delivery and patient positioning within 5% accuracy, depending on the placement of evaluation points. EPIgray is not intended to replace patient-specific quality assurance but should be utilized as an additional layer of safety for continuous patient treatment verification. This research is supported by DOSIsoft.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2012

Towards a tactical nuclear weapons treaty? Is There a Role of IAEA Tools of Safeguards?

In recent years, there is growing interest in formal negotiations on non-strategic or tactical nuclear weapons. With the negotiations of New START, there has been much speculation that a tactical nuclear weapons treaty should be included in the follow on to New START. This paper examines the current policy environment related to tactical weapons and some of the issues surrounding the definition of tactical nuclear weapons. We then map out the steps that would need to be taken in order to begin discussions on a tactical nuclear weapons treaty. These steps will review the potential role of the IAEA in verification of a tactical nuclear weapons treaty. Specifically, does IAEA involvement in various arms control treaties serve as a useful roadmap on how to overcome some of the issues pertaining to a tactical nuclear weapons treaty?

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
NASA NTRS2020

IV&V Assurance Case Design for Artemis II

As human-rated missions like those in NASA’s Artemis program continue to grow in both size and complexity, and the role of software in achieving mission objectives expands dramatically, NASA’s Independent Verification and Validation (IV&V) Teams face evolving challenges in assuring the safety and performance of the safety- and mission-critical embedded software that is essential to landing astronauts on the surface of the Moon by 2024. Key among these challenges is IV&V’s desire to present a cohesive, integrated assurance statement to its stakeholders that encapsulates and summarizes our assurance positions across the integrated Artemis systems and their combined role in support of a safe and successful flight. In order to meet this challenge, the IV&V Teams have begun a transition to using formal assurance case concepts and documentation in the Goal Structuring Notation (GSN) to build an argument in support of software assurance. IV&V recognizes significant benefits to the logical argumentation structure provided by assurance cases and GSN over our current practices for documenting and managing assurance claims. In order to reap these benefits, IV&V is integrating the use of assurance case concepts with our paradigm of follow-the-risk capability based assurance. Because of this, assurance cases created and used by IV&V are distinct from the sort of assurance case created by a development project or embedded software assurance organization. IV&V’s assurance cases depend much less upon standards and regulations, and more on evidence captured by IV&V regarding the environment, requirements, design, and implementation. IV&V constructs an independent network of claims based on an independent decomposition of arguments. Based upon the risk posture of these claims and their associated software and software artifacts, IV&V then develops and executes engineering analyses and testing, which provide evidence to either support or refute the claim. This emerging risk-informed assurance case methodology is being put into practice as IV&V plans for support of the Artemis II mission, the first flight of the Orion capsule and Space Launch System with astronauts on board.

Gerek Whitman↗
NASA NTRS2020

IV&V Assurance Case Design for Artemis II

As human-rated missions like those in NASA's Artemis program continue to grow in both size and complexity, and the role of software in achieving mission objectives expands dramatically, NASA's Independent Verification and Validation (IV&V) Teams face evolving challenges in assuring the safety and performance of the safety- and mission-critical embedded software that is essential to landing astronauts on the surface of the Moon by 2024. Key among these challenges is IV&V's desire to present a cohesive, integrated assurance statement to its stakeholders that encapsulates and summarizes our assurance positions across the integrated Artemis systems and their combined role in support of a safe and successful flight. In order to meet this challenge, the IV&V Teams have begun a transition to using formal assurance case concepts and documentation in the Goal Structuring Notation (GSN) to build an argument in support of software assurance. IV&V recognizes significant benefits to the logical argumentation structure provided by assurance cases and GSN over our current practices for documenting and managing assurance claims. In order to reap these benefits, IV&V is integrating the use of assurance case concepts with our paradigm of follow-the-risk capability based assurance. Because of this, assurance cases created and used by IV&V are distinct from the sort of assurance case created by a development project or embedded software assurance organization. IV&V's assurance cases depend much less upon standards and regulations, and more on evidence captured by IV&V regarding the environment, requirements, design, and implementation. IV&V constructs an independent network of claims based on an independent decomposition of arguments. Based upon the risk posture of these claims and their associated software and software artifacts, IV&V then develops and executes engineering analyses and testing, which provide evidence to either support or refute the claim. This emerging risk-informed assurance case methodology is being put into practice as IV&V plans for support of the Artemis II mission, the first flight of the Orion capsule and Space Launch System with astronauts on board.

Whitman, Gerek↗
DOE OSTI2015

The evaluation of a 2D diode array in “magic phantom” for use in high dose rate brachytherapy pretreatment quality assurance

Purpose: High dose rate (HDR) brachytherapy is a treatment method that is used increasingly worldwide. The development of a sound quality assurance program for the verification of treatment deliveries can be challenging due to the high source activity utilized and the need for precise measurements of dwell positions and times. This paper describes the application of a novel phantom, based on a 2D 11 × 11 diode array detection system, named “magic phantom” (MPh), to accurately measure plan dwell positions and times, compare them directly to the treatment plan, determine errors in treatment delivery, and calculate absorbed dose. Methods: The magic phantom system was CT scanned and a 20 catheter plan was generated to simulate a nonspecific treatment scenario. This plan was delivered to the MPh and, using a custom developed software suite, the dwell positions and times were measured and compared to the plan. The original plan was also modified, with changes not disclosed to the primary authors, and measured again using the device and software to determine the modifications. A new metric, the “position–time gamma index,” was developed to quantify the quality of a treatment delivery when compared to the treatment plan. The MPh was evaluated to determine the minimum measurable dwell time and step size. The incorporation of the TG-43U1 formalism directly into the software allows for dose calculations to be made based on the measured plan. The estimated dose distributions calculated by the software were compared to the treatment plan and to calibrated EBT3 film, using the 2D gamma analysis method. Results: For the original plan, the magic phantom system was capable of measuring all dwell points and dwell times and the majority were found to be within 0.93 mm and 0.25 s, respectively, from the plan. By measuring the altered plan and comparing it to the unmodified treatment plan, the use of the position–time gamma index showed that all modifications made could be readily detected. The MPh was able to measure dwell times down to 0.067 ± 0.001 s and planned dwell positions separated by 1 mm. The dose calculation carried out by the MPh software was found to be in agreement with values calculated by the treatment planning system within 0.75%. Using the 2D gamma index, the dose map of the MPh plane and measured EBT3 were found to have a pass rate of over 95% when compared to the original plan. Conclusions: The application of this magic phantom quality assurance system to HDR brachytherapy has demonstrated promising ability to perform the verification of treatment plans, based upon the measured dwell positions and times. The introduction of the quantitative position–time gamma index allows for direct comparison of measured parameters against the plan and could be used prior to patient treatment to ensure accurate delivery.

60 APPLIED LIFE SCIENCES↗
NASA NTRS1992

Software design as a problem in learning theory (a research overview)

Our interest in automating software design has come out of our research in automated reasoning, inductive inference, learnability, and algebraic machine theory. We have investigated these areas extensively, in connection with specific problems of language representation, acquisition, processing, and design. In the case of formal context-free (CF) languages we established existence of finite learnable models ('behavioral realizations') and procedures for constructing them effectively. We also determined techniques for automatic construction of the models, inductively inferring them from finite examples of how they should 'behave'. These results were obtainable due to appropriate representation of domain knowledge, and constraints on the domain that the representation defined. It was when we sought to generalize our results, and adapt or apply them, that we began investigating the possibility of determining similar procedures for constructing correct software. Discussions with other researchers led us to examine testing and verification processes, as they are related to inference, and due to their considerable importance in correct software design. Motivating papers by other researchers, led us to examine these processes in some depth. Here we present our approach to those software design issues raised by other researchers, within our own theoretical context. We describe our results, relative to those of the other researchers, and conclude that they do not compare unfavorably.

Fass, Leona F.↗
DOE OSTI2012

IAEA CRP on HTGR Uncertainty Analysis: Benchmark Definition and Test Cases

Uncertainty and sensitivity studies are essential elements of the reactor simulation code verification and validation process. Although several international uncertainty quantification activities have been launched in recent years in the LWR, BWR and VVER domains (e.g. the OECD/NEA BEMUSE program, from which the current OECD/NEA LWR Uncertainty Analysis in Modelling (UAM) benchmark effort was derived), the systematic propagation of uncertainties in cross-section, manufacturing and model parameters for High Temperature Reactor (HTGR) designs has not been attempted yet. This paper summarises the scope, objectives and exercise definitions of the IAEA Coordinated Research Project (CRP) on HTGR UAM. Note that no results will be included here, as the HTGR UAM benchmark was only launched formally in April 2012, and the specification is currently still under development.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
NASA NTRS2021

Defining and Reasoning about Model-based Safety Analysis: A Review

Model-based safety analysis (MBSA) has been around for over two decades. The benefits of MBSA have been well-documented in the literature, such as tackling complexity, introducing Formal Methods to eliminate the ambiguity in the traditional safety analysis, using automation to replace the error-prone manual safety modeling process, and ensuring consistency between the design model and the safety model. However, there is still a lack of consensus on what MBSA even is. This paper provides an approach towards developing such a consensus

model-based↗
NASA NTRS1997

Knowledge-Based Aircraft Automation: Managers Guide on the use of Artificial Intelligence for Aircraft Automation and Verification and Validation Approach for a Neural-Based Flight Controller

The ultimate goal of this report was to integrate the powerful tools of artificial intelligence into the traditional process of software development. To maintain the US aerospace competitive advantage, traditional aerospace and software engineers need to more easily incorporate the technology of artificial intelligence into the advanced aerospace systems being designed today. The future goal was to transition artificial intelligence from an emerging technology to a standard technology that is considered early in the life cycle process to develop state-of-the-art aircraft automation systems. This report addressed the future goal in two ways. First, it provided a matrix that identified typical aircraft automation applications conducive to various artificial intelligence methods. The purpose of this matrix was to provide top-level guidance to managers contemplating the possible use of artificial intelligence in the development of aircraft automation. Second, the report provided a methodology to formally evaluate neural networks as part of the traditional process of software development. The matrix was developed by organizing the discipline of artificial intelligence into the following six methods: logical, object representation-based, distributed, uncertainty management, temporal and neurocomputing. Next, a study of existing aircraft automation applications that have been conducive to artificial intelligence implementation resulted in the following five categories: pilot-vehicle interface, system status and diagnosis, situation assessment, automatic flight planning, and aircraft flight control. The resulting matrix provided management guidance to understand artificial intelligence as it applied to aircraft automation. The approach taken to develop a methodology to formally evaluate neural networks as part of the software engineering life cycle was to start with the existing software quality assurance standards and to change these standards to include neural network development. The changes were to include evaluation tools that can be applied to neural networks at each phase of the software engineering life cycle. The result was a formal evaluation approach to increase the product quality of systems that use neural networks for their implementation.

Broderick, Ron↗
NASA NTRS2024

Runtime Verification of Hard Realtime Systems With Copilot: A Tutorial

This presentation is a tutorial on RV using Copilot, a runtime verification framework for real-time embedded systems. Copilot monitors are written in a compositional, stream-based language with support for a variety of Temporal Logics (TL), which results in robust, high-level specifications that are easier to understand than their traditional counterparts. The framework translates monitor specifications into C code with static memory requirements, which can be compiled to run on embedded hardware.

runtime monitoring↗
NASA NTRS2017

JANNAF "Test and Evaluation Guidelines for Liquid Rocket Engines": Status and Application

For many decades, the U.S. rocket propulsion industrial base has performed remarkably in developing complex liquid rocket engines that can propel critical payloads into service for the nation, as well as transport people and hardware for missions that open the frontiers of space exploration for humanity. This has been possible only at considerable expense given the lack of detailed guidance that captures the essence of successful practices and knowledge accumulated over five decades of liquid rocket engine development. In an effort to provide benchmarks and guidance for the next generation of rocket engineers, the Joint Army Navy NASA Air Force (JANNAF) Interagency Propulsion Committee published a liquid rocket engine (LRE) test and evaluation (T&E) guideline document in 2012 focusing on the development challenges and test verification considerations for liquid rocket engine systems. This document has been well received and applied by many current LRE developers as a benchmark and guidance tool, both for government-driven applications as well as for fully commercial ventures. The USAF Space and Missile Systems Center (SMC) has taken an additional near-term step and is directing activity to adapt and augment the content from the JANNAF LRE T&E guideline into a standard for potential application to future USAF requests for proposals for LRE development initiatives and launch vehicles for national security missions. A draft of this standard was already sent out for review and comment, and is intended to be formally approved and released towards the end of 2017. The acceptance and use of the LRE T&E guideline is possible through broad government and industry participation in the JANNAF liquid propulsion committee and associated panels. The sponsoring JANNAF community is expanding upon this initial baseline version and delving into further critical development aspects of liquid rocket propulsion testing at the integrated stage level as well as engine component level, in order to advance the state of the practice. The full participation of the entire U.S. rocket propulsion industrial base is invited and expected at this opportune moment in the continuing advancement of spaceflight technology.

Parkinson, Douglas↗