Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 667 records · Page 37
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

A Verification and Validation Approach for COMSOL Multiphysics to Support High Flux Isotope Reactor (HFIR)

Over the last several decades, many reactors have successfully been converted from high enriched uranium (HEU) to low enriched uranium (LEU) fuels in United States in support of its global non-proliferation objectives. Of the reactors slated for conversion, five high-performance research reactors (HPRRs) remain. The high-flux isotope reactor (HFIR) at Oak Ridge National Laboratory (ORNL) is one of the five. Conversion of HFIR requires the qualification of a new fuel. To aid in the conversion process as well as improve safety margins, COMSOL Multiphysics was chosen to support and eventually supplement the steady-state heat-transfer code (SSHTC). In order to use COMSOL as a supplementary tool, i.e. one used for safety basis calculations, the code needs to undergo a verification and validation process. Verification and validation are important processes for the use of any software; however, physics-based solvers have a unique challenge, in that the validation is much more rigorous and its domain is only applicable to the general scope of the problem. This document details the verification and validation of COMOSL for analysis of HFIR. This report is intended for two different users: (1) current and future users of COMSOL for HFIR and (2) for individuals interested in assessing the scope of the validation of COMSOL for HFIR. This report is divided into six sections. The first of which details the process of verification and validation. Section one provides an overview of key concepts from ASME V&V 10, 20, and 40 and how these concepts are used to provide a validation of any multiphysics software. Section two covers the approach needed in order to validate multiphysics software for a given HFIR analysis. Parts three through five split up the key concepts highlighted in section two. The final section provides concluding remarks which demonstrate the scope and limitations of the validation study and how these may be expanded and improved.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2015

Verification and Validation of Digitally Upgraded Control Rooms

As nuclear power plants undertake main control room modernization, a challenge is the lack of a clearly defined human factors process to follow. Verification and validation (V&V) as applied in the nuclear power community has tended to involve efforts such as integrated system validation, which comes at the tail end of the design stage. To fill in guidance gaps and create a step-by-step process for control room modernization, we have developed the Guideline for Operational Nuclear Usability and Knowledge Elicitation (GONUKE). This approach builds on best practices in the software industry, which prescribe an iterative user-centered approach featuring multiple cycles of design and evaluation. Nuclear regulatory guidance for control room design emphasizes summative evaluation—which occurs after the design is complete. In the GONUKE approach, evaluation is also performed at the formative stage of design—early in the design cycle using mockups and prototypes for evaluation. The evaluation may involve expert review (e.g., software heuristic evaluation at the formative stage and design verification against human factors standards like NUREG-0700 at the summative stage). The evaluation may also involve user testing (e.g., usability testing at the formative stage and integrated system validation at the summative stage). An additional, often overlooked component of evaluation is knowledge elicitation, which captures operator insights into the system. In this report we outline these evaluation types across design phases that support the overall modernization process. The objective is to provide industry-suitable guidance for steps to be taken in support of the design and evaluation of a new human-machine interface (HMI) in the control room. We suggest the value of early-stage V&V and highlight how this early-stage V&V can help improve the design process for control room modernization. We argue that there is a need to overcome two shortcomings of V&V in current practice—the propensity for late-stage V&V and the use of increasingly complex psychological assessment measures for V&V.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
NASA NTRS1995

Verification and Validation of Simulation Models for High-Speed Flow Fields

This paper is largely based on the paper titled "Guide to Credible Computational Fluid Dynamics Simulations (Invited)" (AIAA Paper 95-2225). The significance of computational fluid dynamics (CFD) simulations depends solely on their credibility. A customer of CFD products -- simulations and software -- expects that these products are credible for the intended use. Simulation model verification and validation are critical in establishing the credibility of simulations and in certifying simulation software. The only thing that matters in establishing the credibility is uncertainty, not veracity or validity. The sensitivity-uncertainty analysis is the key to the establishment of this credibility. Assessing the credibility of complex simulation results poses a significant challenge. Terminology, concepts, framework, and guidelines are presented for addressing this challenge. Verification assesses whether the problem is solved correctly and estimates the level of computational accuracy of simulations; validation determines whether the right problem is solved and assesses the level of validity of the simulation model by estimating the degree to which simulations accurately represent reality. These concepts and the related guidelines are discussed with examples from high-speed flow fields.

Mehta, Unmeel B.↗
DOE OSTI2023

Developing Ultrahigh-Resolution E3SM Land Model for GPU Systems

Designing and refactoring complex scientific code, such as the E3SM land model (ELM), for new computing architectures is challenging. This paper presents design strategies and technical approaches to develop a data-oriented, GPU-ready ELM model using compiler directives (OpenACC/OpenMP). We first analyze the datatypes and processes in the original ELM code. Then we present design considerations for ultrahigh-resolution ELM (uELM) development for massive GPU systems. These techniques include the global data-oriented simulation workflow, domain partition, code porting and data copy, memory reduction, parallel loop restructure and flattening, and race condition detection. We implemented the first version of uELM using OpenACC targeting the NVidia GPUs in the Summit supercomputer at Oak Ridge National Laboratory. During the implementation, we developed a software tool (named SPEL) to facilitate code generation, verification, and performance tuning using these techniques. The first uELM implementation for Nvidia GPUs on Summit delivered promising results: 1) over 98% of the ELM code was automatically generated and tuned by scripts. Most ELM modules had better computational performances than the original ELM code for CPUs. The GPU-ready uELM is more scalable than the CPU code on fully-loaded Summit nodes. Example profiling results from several modules are also presented to illustrate the performance improvements and race condition detection. The lessons learned and toolkit developed in the study are also suitable for further uELM deployment using OpenMP on the first US exascale computer, Frontier, equipped with AMD CPUs and GPUs.

Schwartz, Peter↗
DOE OSTI2015

TU-G-BRD-01: Quantifying the Effectiveness of the Physics Pre-Treatment Plan Review for Detecting Errors in Radiation Therapy

Purpose: Physics pre-treatment plan review is crucial to safe radiation oncology treatments. Studies show that most errors originate in treatment planning, which underscores the importance of physics plan review. As a QA measure the physics review is of fundamental importance and is central to the profession of medical physics. However, little is known about its effectiveness. More hard data are needed. The purpose of this study was to quantify the effectiveness of physics review with the goal of improving it. Methods: This study analyzed 315 “potentially serious” near-miss incidents within an institutional incident learning system collected over a two-year period. 139 of these originated prior to physics review and were found at the review or after. Incidents were classified as events that: 1)were detected by physics review, 2)could have been detected (but were not), and 3)could not have been detected. Category 1 and 2 events were classified by which specific check (within physics review) detected or could have detected the event. Results: Of the 139 analyzed events, 73/139 (53%) were detected or could have been detected by the physics review; although, 42/73 (58%) were not actually detected. 45/73 (62%) errors originated in treatment planning, making physics review the first step in the workflow that could detect the error. Two specific physics checks were particularly effective (combined effectiveness of >20%): verifying DRRs (8/73) and verifying isocenter (7/73). Software-based plan checking systems were evaluated and found to have potential effectiveness of 40%. Given current data structures, software implementations of some tests such as isocenter verification check would be challenging. Conclusion: Physics plan review is a key safety measure and can detect majority of reported events. However, a majority of events that potentially could have been detected were NOT detected in this study, indicating the need to improve the performance of physics review.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2021

Suitability of Altair FEKO's Computational Electromagnetic Techniques for Tree Model Scattering and Penetration at Ku-band

Foliage penetration (FOPEN) radar at lower frequencies (VHF, UHF) is a well-studied area with many contributions. However, there is growing interest in using higher Ku-band frequencies (12-18 GHz) for FOPEN. Specifically, the reduced wavelength sizes provide some key saliencies for developing more optimized detection solutions. The disadvantage is that exploiting Ku-band for FOPEN is complicated because higher frequencies have pronounced scattering effects due to their smaller wavelengths. A methodology h as been developed to model and simulate FOPEN problems that characterize the phenomenology of Ku-band electromagnetic ( EM ) wave transmissions through moderate foliage. The details of this research (i.e. the realistic tree models, simulation setup and results) are documented in multiple reports. The main focus of this report is to describe the preliminary validation and verification of Altair FEKO, the computational EM (CEM) software used for this research, as well as present a simplified symmetrical tree model and an introductory CAD tree model.

42 ENGINEERING↗
DOE OSTI2020

FAUST - Benchmark and Validation Framework [Slides]

This presentation discusses Python packages for nuclear data applications and for benchmarking. It also discusses the goals and objectives for FAUST which include providing input and output processing for different calculation codes, allowing for exchanging results between different applications and codes, running benchmarks and processing the results, automating and simplify plot and report generation, and providing a basis for developing applications useful for nuclear data evaluators. The presentation also discusses how the packages can be used both inside and outside LANL.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
NASA NTRS1985

Some key considerations in evolving a computer system and software engineering support environment for the space station program

The space station data management system involves networks of computing resources that must work cooperatively and reliably over an indefinite life span. This program requires a long schedule of modular growth and an even longer period of maintenance and operation. The development and operation of space station computing resources will involve a spectrum of systems and software life cycle activities distributed across a variety of hosts, an integration, verification, and validation host with test bed, and distributed targets. The requirement for the early establishment and use of an apporopriate Computer Systems and Software Engineering Support Environment is identified. This environment will support the Research and Development Productivity challenges presented by the space station computing system.

Mckay, C. W.↗
NASA NTRS1985

Collected software engineering papers, volume 3

Topics addressed include: software technology evaluation programs; collection of valid software engineering data; test processes using structural coverage; prototype expert system for software engineering management; use of an environment characteristic software metric set; software modulation; software development by analysis of change; and independent verification and validation.

Source record↗
NASA NTRS1998

Framework for Small-Scale Experiments in Software Engineering: Guidance and Control Software Project: Software Engineering Case Study

Software is becoming increasingly significant in today's critical avionics systems. To achieve safe, reliable software, government regulatory agencies such as the Federal Aviation Administration (FAA) and the Department of Defense mandate the use of certain software development methods. However, little scientific evidence exists to show a correlation between software development methods and product quality. Given this lack of evidence, a series of experiments has been conducted to understand why and how software fails. The Guidance and Control Software (GCS) project is the latest in this series. The GCS project is a case study of the Requirements and Technical Concepts for Aviation RTCA/DO-178B guidelines, Software Considerations in Airborne Systems and Equipment Certification. All civil transport airframe and equipment vendors are expected to comply with these guidelines in building systems to be certified by the FAA for use in commercial aircraft. For the case study, two implementations of a guidance and control application were developed to comply with the DO-178B guidelines for Level A (critical) software. The development included the requirements, design, coding, verification, configuration management, and quality assurance processes. This paper discusses the details of the GCS project and presents the results of the case study.

Hayhurst, Kelly J.↗
NASA NTRS2003

Foundations of the Bandera Abstraction Tools

Current research is demonstrating that model-checking and other forms of automated finite-state verification can be effective for checking properties of software systems. Due to the exponential costs associated with model-checking, multiple forms of abstraction are often necessary to obtain system models that are tractable for automated checking. The Bandera Tool Set provides multiple forms of automated support for compiling concurrent Java software systems to models that can be supplied to several different model-checking tools. In this paper, we describe the foundations of Bandera's data abstraction mechanism which is used to reduce the cardinality (and the program's state-space) of data domains in software to be model-checked. From a technical standpoint, the form of data abstraction used in Bandera is simple, and it is based on classical presentations of abstract interpretation. We describe the mechanisms that Bandera provides for declaring abstractions, for attaching abstractions to programs, and for generating abstracted programs and properties. The contributions of this work are the design and implementation of various forms of tool support required for effective application of data abstraction to software components written in a programming language like Java which has a rich set of linguistic features.

Hatcliff, John↗
NASA NTRS2004

Using Automated Theorem Provers to Certify Auto-Generated Aerospace Software

We describe a system for the automated certification of safety properties of NASA software. The system uses Hoare-style program verification technology to generate proof obligations which are then processed by an automated first-order theorem prover (ATP). For full automation, however, the obligations must be aggressively preprocessed and simplified We describe the unique requirements this places on the ATP and demonstrate how the individual simplification stages, which are implemented by rewriting, influence the ability of the ATP to solve the proof tasks. Experiments on more than 25,000 tasks were carried out using Vampire, Spass, and e-setheo.

Denney, Ewen↗
NASA NTRS2004

An Empirical Evaluation of Automated Theorem Provers in Software Certification

We describe a system for the automated certification of safety properties of NASA software. The system uses Hoare-style program verification technology to generate proof obligations which are then processed by an automated first-order theorem prover (ATP). We discuss the unique requirements this application places on the ATPs, focusing on automation, proof checking, and usability. For full automation, however, the obligations must be aggressively preprocessed and simplified, and we demonstrate how the individual simplification stages, which are implemented by rewriting, influence the ability of the ATPs to solve the proof tasks. Our results are based on 13 certification experiments that lead to more than 25,000 proof tasks which have each been attempted by Vampire, Spass, e-setheo, and Otter. The proofs found by Otter have been proof-checked by IVY.

Denney, Ewen↗
NASA NTRS2001

Incorporating cost-benefit analyses into software assurance planning

The objective is to use cost-benefit analyses to identify, for a given project, optimal sets of software assurance activities. Towards this end we have incorporated cost-benefit calculations into a risk management framework.

investment software quality software process impro↗
NASA NTRS2010

Verification and Validation for Flight-Critical Systems (VVFCS)

On March 31, 2009 a Request for Information (RFI) was issued by NASA s Aviation Safety Program to gather input on the subject of Verification and Validation (V & V) of Flight-Critical Systems. The responses were provided to NASA on or before April 24, 2009. The RFI asked for comments in three topic areas: Modeling and Validation of New Concepts for Vehicles and Operations; Verification of Complex Integrated and Distributed Systems; and Software Safety Assurance. There were a total of 34 responses to the RFI, representing a cross-section of academic (26%), small & large industry (47%) and government agency (27%).

Graves, Sharon S.↗
NASA NTRS2012

Certification of COTS Software in NASA Human Rated Flight Systems

Adoption of commercial off-the-shelf (COTS) products in safety critical systems has been seen as a promising acquisition strategy to improve mission affordability and, yet, has come with significant barriers and challenges. Attempts to integrate COTS software components into NASA human rated flight systems have been, for the most part, complicated by verification and validation (V&V) requirements necessary for flight certification per NASA s own standards. For software that is from COTS sources, and, in general from 3rd party sources, either commercial, government, modified or open source, the expectation is that it meets the same certification criteria as those used for in-house and that it does so as if it were built in-house. The latter is a critical and hidden issue. This paper examines the longstanding barriers and challenges in the use of 3rd party software in safety critical systems and cover recent efforts to use COTS software in NASA s Multi-Purpose Crew Vehicle (MPCV) project. It identifies some core artifacts that without them, the use of COTS and 3rd party software is, for all practical purposes, a nonstarter for affordable and timely insertion into flight critical systems. The paper covers the first use in a flight critical system by NASA of COTS software that has prior FAA certification heritage, which was shown to meet the RTCA-DO-178B standard, and how this certification may, in some cases, be leveraged to allow the use of analysis in lieu of testing. Finally, the paper proposes the establishment of an open source forum for development of safety critical 3rd party software.

Goforth, Andre↗
NASA NTRS2013

FPGA Flash Memory High Speed Data Acquisition

The purpose of this research is to design and implement a VHDL ONFI Controller module for a Modular Instrumentation System. The goal of the Modular Instrumentation System will be to have a low power device that will store data and send the data at a low speed to a processor. The benefit of such a system will give an advantage over other purchased binary IP due to the capability of allowing NASA to re-use and modify the memory controller module. To accomplish the performance criteria of a low power system, an in house auxiliary board (Flash/ADC board), FPGA development kit, debug board, and modular instrumentation board will be jointly used for the data acquisition. The Flash/ADC board contains four, 1 MSPS, input channel signals and an Open NAND Flash memory module with an analog to digital converter. The ADC, data bits, and control line signals from the board are sent to an Microsemi/Actel FPGA development kit for VHDL programming of the flash memory WRITE, READ, READ STATUS, ERASE, and RESET operation waveforms using Libero software. The debug board will be used for verification of the analog input signal and be able to communicate via serial interface with the module instrumentation. The scope of the new controller module was to find and develop an ONFI controller with the debug board layout designed and completed for manufacture. Successful flash memory operation waveform test routines were completed, simulated, and tested to work on the FPGA board. Through connection of the Flash/ADC board with the FPGA, it was found that the device specifications were not being meet with Vdd reaching half of its voltage. Further testing showed that it was the manufactured Flash/ADC board that contained a misalignment with the ONFI memory module traces. The errors proved to be too great to fix in the time limit set for the project.

Gonzalez, April↗