SEARCH · Engineering Papers

Results for “Software Verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 973 records · Page 54

All sources NASA NTRS DOE OSTI

NASA NTRS2017

Security Vulnerability Profiles of NASA Mission Software: Empirical Analysis of Security Related Bug Reports

NASA develops, runs, and maintains software systems for which security is of vital importance. Therefore, it is becoming an imperative to develop secure systems and extend the current software assurance capabilities to cover information assurance and cybersecurity concerns of NASA missions. The results presented in this report are based on the information provided in the issue tracking systems of one ground mission and one flight mission. The extracted data were used to create three datasets: Ground mission IVV issues, Flight mission IVV issues, and Flight mission Developers issues. In each dataset, we identified the software bugs that are security related and classified them in specific security classes. This information was then used to create the security vulnerability profiles (i.e., to determine how, why, where, and when the security vulnerabilities were introduced) and explore the existence of common trends. The main findings of our work include:- Code related security issues dominated both the Ground and Flight mission IVV security issues, with 95 and 92, respectively. Therefore, enforcing secure coding practices and verification and validation focused on coding errors would be cost effective ways to improve mission's security. (Flight mission Developers issues dataset did not contain data in the Issue Category.)- In both the Ground and Flight mission IVV issues datasets, the majority of security issues (i.e., 91 and 85, respectively) were introduced in the Implementation phase. In most cases, the phase in which the issues were found was the same as the phase in which they were introduced. The most security related issues of the Flight mission Developers issues dataset were found during Code Implementation, Build Integration, and Build Verification; the data on the phase in which these issues were introduced were not available for this dataset.- The location of security related issues, as the location of software issues in general, followed the Pareto principle. Specifically, for all three datasets, from 86 to 88 the security related issues were located in two to four subsystems.- The severity levels of most security issues were moderate, in all three datasets.- Out of 21 primary security classes, five dominated: Exception Management, Memory Access, Other, Risky Values, and Unused Entities. Together, these classes contributed from around 80 to 90 of all security issues in each dataset. This again proves the Pareto principle of uneven distribution of security issues, in this case across CWE classes, and supports the fact that addressing these dominant security classes provides the most cost efficient way to improve missions' security. The findings presented in this report uncovered the security vulnerability profiles and identified the common trends and dominant classes of security issues, which in turn can be used to select the most efficient secure design and coding best practices compiled by the part of the SARP project team associated with the NASA's Johnson Space Center. In addition, these findings provide valuable input to the NASA IVV initiative aimed at identification of the two 25 CWEs of ground and flight missions.

vulnerability↗

DOE OSTI2017

EOSlib, Version 3

Equilibrium thermodynamics underpins many of the technologies used throughout theoretical physics, yet verification of the various theoretical models in the open literature remains challenging. EOSlib provides a single, consistent, verifiable implementation of these models, in a single, easy-to-use software package. It consists of three parts: a software library implementing various published equation-of-state (EOS) models; a database of fitting parameters for various materials for these models; and a number of useful utility functions for simplifying thermodynamic calculations such as computing Hugoniot curves or Riemann problem solutions. Ready availability of this library will enable reliable code-to- code testing of equation-of-state implementations, as well as a starting point for more rigorous verification work. EOSlib also provides a single, consistent API for its analytic and tabular EOS models, which simplifies the process of comparing models for a particular application.

Woods, Nathan↗

DOE OSTI2017

EOSlib, Version 3

Woods, Nathan↗

DOE OSTI2015

Time-resolved dosimetry using a pinpoint ionization chamber as quality assurance for IMRT and VMAT

Purpose: To develop a method to verify the dose delivery in relation to the individual control points of intensity modulated radiotherapy (IMRT) and volumetric modulated arc therapy (VMAT) using an ionization chamber. In addition to more effective problem solving during patient-specific quality assurance (QA), the aim is to eventually map out the limitations in the treatment chain and enable a targeted improvement of the treatment technique in an efficient way. Methods: Pretreatment verification was carried out for 255 treatment plans that included a broad range of treatment indications in two departments using the equipment of different vendors. In-house developed software was used to enable calculation of the dose delivery for the individual beamlets in the treatment planning system (TPS), for data acquisition, and for analysis of the data. The observed deviations were related to various delivery and measurement parameters such as gantry angle, field size, and the position of the detector with respect to the field edge to distinguish between error sources. Results: The average deviation of the integral fraction dose during pretreatment verification of the planning target volume dose was −2.1% ± 2.2% (1 SD), −1.7% ± 1.7% (1 SD), and 0.0% ± 1.3% (1 SD) for IMRT at the Radboud University Medical Center (RUMC), VMAT (RUMC), and VMAT at the Wellington Blood and Cancer Centre, respectively. Verification of the dose to organs at risk gave very similar results but was generally subject to a larger measurement uncertainty due to the position of the detector at a high dose gradient. The observed deviations could be related to limitations of the TPS beam models, attenuation of the treatment couch, as well as measurement errors. The apparent systematic error of about −2% in the average deviation of the integral fraction dose in the RUMC results could be explained by the limitations of the TPS beam model in the calculation of the beam penumbra. Conclusions: This study showed that time-resolved dosimetry using an ionization chamber is feasible and can be largely automated which limits the required additional time compared to integrated dose measurements. It provides a unique QA method which enables identification and quantification of the contribution of various error sources during IMRT and VMAT delivery.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2018

PySFD: comprehensive molecular insights from significant feature differences detected among many simulated ensembles

Motivation Many modeling analyses of molecular dynamics (MD) simulations are based on a definition of states that can be (groups of) clusters of simulation frames in a feature space composed of molecular coordinates. With increasing dimension of this feature space (due to the increasing size or complexity of a simulated molecule), it becomes very difficult to cluster the underlying MD data and estimate a statistically robust model. To mitigate this “curse of dimensionality”, one can reduce the feature space, e.g., with principal component or time-lagged independent component analysis transformations, focusing the analysis on the most important modes of transitions. In practice, however, all these reduction strategies may neglect important molecular details that are susceptible to experimental verification. Results To recover such molecular details, I have developed PySFD (Significant Feature Differences analyzer for Python), a multi-processing software package that efficiently selects significantly different features of any user-defined feature type among potentially many different simulated state ensembles, such as meta-stable states of a Markov State Model (MSM). Applying PySFD on MSMs of an aggregate of 300 microseconds MD simulations recently performed on the major histocompatibility complex class II (MHCII) protein, I demonstrate how this toolkit can extract and visualize valuable mechanistic information from big MD simulation data, e.g., in form of networks of dynamic interaction changes connecting functionally relevant sites of a protein complex.

Biochemistry & Molecular Biology↗

NASA NTRS1987

A formal approach to validation and verification for knowledge-based control systems

As control systems become more complex in response to desires for greater system flexibility, performance and reliability, the promise is held out that artificial intelligence might provide the means for building such systems. An obstacle to the use of symbolic processing constructs in this domain is the need for verification and validation (V and V) of the systems. Techniques currently in use do not seem appropriate for knowledge-based software. An outline of a formal approach to V and V for knowledge-based control systems is presented.

Castore, Glen↗

NASA NTRS2003

Developing An Autonomy Infusion Infrastructure for Robotic Exploration

Future robotic exploration missions will require autonomy in order to accomplish mission goals for operational efficiency and science return. For example, it will require three communication cycles for the Mars Exploration Rovers, Spirit and Opportunity, to place an instrument on a science target. Reducing this time necessitates highly accurate navigation, obstacle avoidance, target tracking, target analysis, manipulation, and fault diagnosis. Technologies to address these and other operational elements are currently being developed at NASA and within academia. However, infusion into missions has always been a difficult task for researchers. In order to keep risk down, mission managers are reluctant to include new technologies unless they have undergone extensive testing and verification under flight-realistic conditions. Furthermore, infusion of new technologies into missions is made more difficult by the variety of software frameworks under which these technologies are developed. Missions would like to see competing solutions demonstrated on a common platform so that they can compare performance and choose the solution best suited to their application.

Bualat, Maria G.↗

NASA NTRS2015

Lessons Learned from Optical Payload for Lasercomm Science (OPALS) Mission Operations

This paper provides an overview of Optical Payload for Lasercomm Science (OPALS) activities and lessons learned during mission operations. Activities described cover the periods of commissioning, prime, and extended mission operations, during which primary and secondary mission objectives were achieved for demonstrating space-to-ground optical communications. Lessons learned cover Mission Operations System topics in areas of: architecture verification and validation, staffing, mission support area, workstations, workstation tools, interfaces with support services, supporting ground stations, team training, procedures, flight software upgrades, post-processing tools, and public outreach.

Sindiy, Oleg V.↗

DOE OSTI2021

Modeling fission product diffusion in TRISO fuel particles with BISON

Diffusion of fission products in intact TRISO particles depends on particle geometry, fission product source rates, time, temperature, and temperature-dependent diffusion coefficients. Simulating this diffusion process requires models for source rates and diffusion coefficients, plus computation of the temperature field if not prescribed. In addition, simulation quality depends on discretization of the geometry, appropriate time stepping, and the accuracy of the solution method. In this paper, we explore the simulation of fission product diffusion in TRISO fuel particles using the finite element method via the fuel performance code Bison. Recent material model development has occurred in Bison for each material present in tri-structural isotropic (TRISO) fuel particles: the buffer, inner pyrolytic carbon, silicon carbide, and outer pyrolytic carbon layers, as well as the fuel kernel. Also, new mesh generation and fission product release fraction capabilities have been added. Diffusion capabilities are shown to converge to the correct solution via formal verification tests. A large number of code benchmarking problems are also given, with good results, showing that Bison’s computed release fractions closely match those of other software tools. Finally, a significant validation effort is detailed in which fission product release, measured as part of the AGR-1 capsule experiments, is compared to Bison outputs. Bison outputs compare very well to the experimental data and to PARFUME results.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

NASA NTRS1993

Cooperative GN&C development in a rapid prototyping environment

The Navigation, Control and Aeronautics Division (NCAD) at NASA-JSC is exploring ways of producing Guidance, Navigation and Control (GN&C) flight software faster, better, and cheaper. To achieve these goals NCAD established two hardware/software facilities that take an avionics design project from initial inception through high fidelity real-time hardware-in-the-loop testing. Commercially available software products are used to develop the GN&C algorithms in block diagram form and then automatically generate source code from these diagrams. A high fidelity real-time hardware-in-the-loop laboratory provides users with the capability to analyze mass memory usage within the targeted flight computer, verify hardware interfaces, conduct system level verification, performance, acceptance testing, as well as mission verification using reconfigurable and mission unique data. To evaluate these concepts and tools, NCAD embarked on a project to build a real-time 6 DOF simulation of the Soyuz Assured Crew Return Vehicle flight software. To date, a productivity increase of 185 percent has been seen over traditional NASA methods for developing flight software.

Bordano, Aldo↗

NASA NTRS2006

Inspection and Verification of Domain Models with PlanWorks and Aver

When developing a domain model, it seems natural to bring the traditional informal tools of inspection and verification, debuggers and automated test suites, to bear upon the problems that will inevitably arise. Debuggers that allow inspection of registers and memory and stepwise execution have been a staple of software development of all sorts from the very beginning. Automated testing has repeatedly proven its considerable worth, to the extent that an entire design philosophy (Test Driven Development) has been developed around the writing of tests. Unfortunately, while not entirely without their uses, the limitations of these tools and the nature of the complexity of models and the underlying planning systems make the diagnosis of certain classes of problems and the verification of their solutions difficult or impossible. Debuggers provide a good local view of executing code, allowing a fine-grained look at algorithms and data. This view is, however, usually only at the level of the current scope in the implementation language, and the data-inspection capabilities of most debuggers usually consist of on-line print statements. More modem graphical debuggers offer a sort of tree view of data structures, but even this is too low-level and is often inappropriate for the kinds of structures created by planning systems. For instance, god or constraint networks are at best awkward when visualized as trees. Any any non-structural link between data structures, as through a lookup table, isn't captured at all. Further, while debuggers have powerful breakpointing facilities that are suitable for finding specific algorithmic errors, they have little use in the diagnosis of modeling errors.

Bedrax-Weiss, Tania↗

NASA NTRS1997

Knowledge-Based Aircraft Automation: Managers Guide on the use of Artificial Intelligence for Aircraft Automation and Verification and Validation Approach for a Neural-Based Flight Controller

The ultimate goal of this report was to integrate the powerful tools of artificial intelligence into the traditional process of software development. To maintain the US aerospace competitive advantage, traditional aerospace and software engineers need to more easily incorporate the technology of artificial intelligence into the advanced aerospace systems being designed today. The future goal was to transition artificial intelligence from an emerging technology to a standard technology that is considered early in the life cycle process to develop state-of-the-art aircraft automation systems. This report addressed the future goal in two ways. First, it provided a matrix that identified typical aircraft automation applications conducive to various artificial intelligence methods. The purpose of this matrix was to provide top-level guidance to managers contemplating the possible use of artificial intelligence in the development of aircraft automation. Second, the report provided a methodology to formally evaluate neural networks as part of the traditional process of software development. The matrix was developed by organizing the discipline of artificial intelligence into the following six methods: logical, object representation-based, distributed, uncertainty management, temporal and neurocomputing. Next, a study of existing aircraft automation applications that have been conducive to artificial intelligence implementation resulted in the following five categories: pilot-vehicle interface, system status and diagnosis, situation assessment, automatic flight planning, and aircraft flight control. The resulting matrix provided management guidance to understand artificial intelligence as it applied to aircraft automation. The approach taken to develop a methodology to formally evaluate neural networks as part of the software engineering life cycle was to start with the existing software quality assurance standards and to change these standards to include neural network development. The changes were to include evaluation tools that can be applied to neural networks at each phase of the software engineering life cycle. The result was a formal evaluation approach to increase the product quality of systems that use neural networks for their implementation.

Broderick, Ron↗

DOE OSTI2016

In Situ Data Infrastructure for Scientific Unit Testing Platform

Testing is a significant software development process for the management of software systems and scientific code. However, as the complexity of scientific codes increases, extra checks are needed to monitor impacts to dependent models and to verify system constraints. The software complexity also impedes the efforts of module developers and software engineers to rapidly develop and extend their code. Recently, we have developed an automatic methodology and prototype platform to facilitate scientific verification of individual functions within complex scientific codes. With this system, the scientific module builders are able to track variables conveniently in one module or track variables’ changes among different modules. In this paper, we present a procedure for automatic unit testing generation. For the interest of a general audience of this conference, we are emphasizing the technical details of integrating the In Situ data infrastructure into our platform. At the end of this paper, we have included an implementation of unit testing for the ACME Land Model (ALM) to demonstrate the usefulness and correctness of the platform. We have also used single- and multipoint checks to demonstrate the efficient variable tracking capability of this platform.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

BISON TRISO Modeling Advancements and Validation to AGR-1 Data

BISON is a finite element-based nuclear fuel performance code. Among its unique characteristics are its ability to model 1D, 2D, and 3D geometries and its applicability to a wide variety of nuclear fuels. For the last eight years, BISON has included a beginning capability to model tri-structural isotropic (TRISO) fuel. Recently, interest in TRISO fuel has grown, and a significant effort has been made to improve BISON’s capabilities in this area. Capability development has occurred for each material present in TRISO fuel particles: the buffer, inner pyrolytic carbon, silicon carbide, and outer pyrolytic carbon layers, as well as the fuel kernel. New elastic, creep, swelling, thermal expansion, thermal conductivity, and fission gas release (FGR) models are available. New models for the graphite matrix are also now available. Another important addition is the ability to perform statistical failure analysis of large samples of fuel particles. This new capability, which continues to grow, enables evaluation of failure due to pressure or crack formation by analyzing many thousands of particles. This enables realistic calculations of fission product release from the many particles in a TRISO-fueled reactor. These capabilities were checked via regression and verification tests. A large number of code benchmarking problems were also run, showing that BISON’s results closely match those of other software tools. Finally, a significant validation effort was completed in which fission product release, measured as part of the AGR-1 capsule experiments, was compared to BISON outputs. BISON outputs compared very well to the experimental data and to PARFUME results. Interest in BISON’s TRISO capabilities is growing, with the U.S. Nuclear Regulatory Commission (NRC) and Westinghouse Electric Company receiving training during the past year. Multiple other entities have expressed interest in or are actively using BISON. Kairos Power, LLC, has a strong partnership with Idaho National Laboratory (INL) regarding the use of BISON for TRISO analysis. While its capabilities still continue to grow, BISON has already become a powerful tool for TRISO analysis.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2023

Requirements Description of VARPOW

This report presents the modeling and simulation capabilities of Argonne National Laboratory’s VARPOW code [1] that is used in present reactor analysis activities. These capabilities will then be used to establish the set of VARPOW verification tasks necessary to verify VARPOW for usage on commercial projects. A similar approach was taken for the PERSENT, REBUS and DIF3D software packages. The VARPOW program is a post-processing utility program for DIF3D, specifically DIF3DVARIANT. As covered in [1], the VARPOW program was built to provide input for follow-on steady state thermal-hydraulic analysis. Its primary purpose is to act as the interface between DASSH and DIF3D-VARIANT. The outputs from VARPOW are contained in three output files: Output.VARPOW, VariantMonoExponents.out, and MaterialPower.out.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2023

Requirements Description of VARPOW

This report presents the modeling and simulation capabilities of Argonne National Laboratory’s VARPOW code that is used in present reactor analysis activities. These capabilities will then be used to establish the set of VARPOW verification tasks necessary to verify VARPOW for usage on commercial projects. A similar approach was taken for the PERSENT, REBUS and DIF3D software packages. The VARPOW program is a post-processing utility program for DIF3D, specifically DIF3D-VARIANT. As covered in [1], the VARPOW program was built to provide input for follow-on steady state thermal-hydraulic analysis. Its primary purpose is to act as the interface between DASSH and DIF3D-VARIANT. The outputs from VARPOW are contained in three output files: Output.VARPOW, VariantMonoExponents.out, and MaterialPower.out.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2026

Requirements Description of VARPOW

Ramey, Kyle [Argonne National Laboratory (ANL), Ar↗

NASA NTRS1980

Tools to aid the specification and design of flight software, appendix B

The tasks that are normally performed during the specification and architecture design stages of software development are identified. Ways that tools could perform, or aid the performance, of such tasks are also identified. Much of the verification and analysis that is suggested is currently rarely performed during these early stages, but it is believed that this analysis should be done as early as possible so as to detect errors as early as possible.

Bristow, G.↗