Introduction to Complex Fault Protection Software Testing
This paper describes how fault protection software verification testing has been addressed for the Attitude and Articulation Control Subsystem on the Saturn-bound Cassini spacecraft.
fault↗
SEARCH · Engineering Papers
Results for “Software Verification”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
2021 Small Turbine Certification Awardee: Intergrid LLC
Currently, there are no U.S., fully certified inverters for wind turbines in the 10-15-kilowatt (kW) range. The federal government's Competitiveness Improvement Project (CIP) and Small Business Innovation Research programs present the most viable paths for manufacturers in the distributed wind energy industry to conduct research, develop their products, and, ultimately, commercialize them. Through CIP, Intergrid LLC is reducing the cost, size, and weight of its legacy inverter for distributed wind turbines while increasing efficiency, reliability, and serviceability. To accomplish this, Intergrid will address three areas related to power electronics for distributed wind turbines: (1) Software verification and functional safety certification; (2) Simulation for grid-interconnecting testing; (3) Component alternatives that reduce cost and improve reliability and serviceability. The first two areas relate to long-term management of inverter software, which, for grid-connected inverters, must comply with three standards - one issued by Underwriters Laboratories (UL) and two issued by the Institute of Electrical and Electronics Engineers.
CIP↗
Verification and Validation Challenges for Adaptive Flight Control of Complex Autonomous Systems
Autonomy of aerospace systems requires the ability for flight control systems to be able to adapt to complex uncertain dynamic environment. In spite of the five decades of research in adaptive control, the fact still remains that currently no adaptive control system has ever been deployed on any safety-critical or human-rated production systems such as passenger transport aircraft. The problem lies in the difficulty with the certification of adaptive control systems since existing certification methods cannot readily be used for nonlinear adaptive control systems. Research to address the notion of metrics for adaptive control began to appear in the recent years. These metrics, if accepted, could pave a path towards certification that would potentially lead to the adoption of adaptive control as a future control technology for safety-critical and human-rated production systems. Development of certifiable adaptive control systems represents a major challenge to overcome. Adaptive control systems with learning algorithms will never become part of the future unless it can be proven that they are highly safe and reliable. Rigorous methods for adaptive control software verification and validation must therefore be developed to ensure that adaptive control system software failures will not occur, to verify that the adaptive control system functions as required, to eliminate unintended functionality, and to demonstrate that certification requirements imposed by regulatory bodies such as the Federal Aviation Administration (FAA) can be satisfied. This presentation will discuss some of the technical issues with adaptive flight control and related V&V challenges.
Nguyen, Nhan T.↗
Quantitative Analysis Guidance in Safety Software Validation
Quantitative analysis in software development has been a challenging topic over the years in safety software. While qualitative processes for vigorous safety software verification and validation (V and V) have been provided through standard such as IEEE 1012, quantitative analysis for V and V tasks such as correctness, completeness or accuracy evaluation have not been formally defined. In practice, there are quantitative analyses in terms of using software metrics. Many of these software metrics are useful in analyzing the test results. Some of them are used for providing guidance in code evaluation activities e.g. numbers of line of codes not exceeding certain limits, recursive calls are not allowed, etc. However, few can be used for providing quantitative analysis guidance for the V and V tasks such as completeness evaluation. AREVA Federal Services (AFS) has provided services for many nuclear projects in US. Through compliance with IEEE 1012, AFS V and V processes have provided vigorous qualitative processes ensuring quality level of the software products used in the project meeting the required nuclear software level. In addition, quantitative analysis is developed to provide structured validation framework ensuring 'zero defect' in the software products. This paper describes some aspects of the quantitative approaches using a sample safety application. (authors)
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
Verification and Validation of the PLTEMP/ANL Code for Thermal-Hydraulic Analysis of Experimental and Test Reactors, Volume 1
This is Volume 1 of a two-volume document that collects the verification and validation (V&V) works done for the PLTEMP/ANL code during the years of its development and improvement. Volume 1 describes the V&V of sixteen capabilities of the PLTEMP/ANL code that were identified by research reactor analysts as frequently used in their thermal-hydraulic analysis. Volume 2 describes the V&V of developments and improvements since the release of PLTEMP/ANL Version 4.3. Each chapter of the document focuses on verifying or validating a specific part of the software that calculates a specific phenomenon, e.g., channel flow calculation, coolant property calculation, heat transfer calculation, and flow instability calculation. Software verification is performed by comparing the code with hand calculation, Microsoft spreadsheet calculation, Mathematica calculation, or MATLAB calculation. The software validation is done by comparing the code with experimental data or a widely tested code like the RELAP5 code. In addition, some PLTEMP/ANL V&V works that are available in the open literature are simply cited in Volume 1 of the document. PLTEMP/ANL has been used in the safety analysis reports of several US and foreign research reactors licensed and converted from highly enriched uranium fuel to low-enriched uranium fuel. A list of such reactors is given in Volume 1 of the document.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
Investigation of a Verification and Validation Tool with a Turbofan Aircraft Engine Application
The development of more advanced control architectures for turbofan aircraft engines can yield gains in performance and efficiency over the lifetime of an engine. However, the implementation of these increasingly complex controllers is contingent on their ability to provide safe, reliable engine operation. Therefore, having the means to verify the safety of new control algorithms is crucial. As a step towards this goal, CoCoSim, a publicly available verification tool for Simulink, is used to analyze C-MAPSS40k, a 40,000 lbf class turbo-fan engine model developed at NASA for testing new control algorithms. Due to current limitations of the verification software, several modifications are made to C-MAPSS40k to achieve compatibility with CoCoSim. Some of these modifications sacrifice fidelity to the original model. Several safety and performance requirements typical for turbofan engines are identified and constructed into a verification framework. Preliminary results using an industry standard baseline controller for these requirements are presented. While verification capabilities are demonstrated, a truly comprehensive analysis will require further development of the verification tool.
V&↗
Development and Implementation of Dynamic Scripts to Support Local Model Verification at National Weather Service Weather Forecast Offices
Local modeling with a customized configuration is conducted at National Weather Service (NWS) Weather Forecast Offices (WFOs) to produce high-resolution numerical forecasts that can better simulate local weather phenomena and complement larger scale global and regional models. The advent of the Environmental Modeling System (EMS), which provides a pre-compiled version of the Weather Research and Forecasting (WRF) model and wrapper Perl scripts, has enabled forecasters to easily configure and execute the WRF model on local workstations. NWS WFOs often use EMS output to help in forecasting highly localized, mesoscale features such as convective initiation, the timing and inland extent of lake effect snow bands, lake and sea breezes, and topographically-modified winds. However, quantitatively evaluating model performance to determine errors and biases still proves to be one of the challenges in running a local model. Developed at the National Center for Atmospheric Research (NCAR), the Model Evaluation Tools (MET) verification software makes performing these types of quantitative analyses easier, but operational forecasters do not generally have time to familiarize themselves with navigating the sometimes complex configurations associated with the MET tools. To assist forecasters in running a subset of MET programs and capabilities, the Short-term Prediction Research and Transition (SPoRT) Center has developed and transitioned a set of dynamic, easily configurable Perl scripts to collaborating NWS WFOs. The objective of these scripts is to provide SPoRT collaborating partners in the NWS with the ability to evaluate the skill of their local EMS model runs in near real time with little prior knowledge of the MET package. The ultimate goal is to make these verification scripts available to the broader NWS community in a future version of the EMS software. This paper provides an overview of the SPoRT MET scripts, instructions for how the scripts are run, and example use cases.
Zavordsky, Bradley↗
Development and Implementation of Dynamic Scripts to Support Local Model Verification at National Weather Service Weather Forecast Offices
Local modeling with a customized configuration is conducted at National Weather Service (NWS) Weather Forecast Offices (WFOs) to produce high-resolution numerical forecasts that can better simulate local weather phenomena and complement larger scale global and regional models. The advent of the Environmental Modeling System (EMS), which provides a pre-compiled version of the Weather Research and Forecasting (WRF) model and wrapper Perl scripts, has enabled forecasters to easily configure and execute the WRF model on local workstations. NWS WFOs often use EMS output to help in forecasting highly localized, mesoscale features such as convective initiation, the timing and inland extent of lake effect snow bands, lake and sea breezes, and topographically-modified winds. However, quantitatively evaluating model performance to determine errors and biases still proves to be one of the challenges in running a local model. Developed at the National Center for Atmospheric Research (NCAR), the Model Evaluation Tools (MET) verification software makes performing these types of quantitative analyses easier, but operational forecasters do not generally have time to familiarize themselves with navigating the sometimes complex configurations associated with the MET tools. To assist forecasters in running a subset of MET programs and capabilities, the Short-term Prediction Research and Transition (SPoRT) Center has developed and transitioned a set of dynamic, easily configurable Perl scripts to collaborating NWS WFOs. The objective of these scripts is to provide SPoRT collaborating partners in the NWS with the ability to evaluate the skill of their local EMS model runs in near real time with little prior knowledge of the MET package. The ultimate goal is to make these verification scripts available to the broader NWS community in a future version of the EMS software. This paper provides an overview of the SPoRT MET scripts, instructions for how the scripts are run, and example use cases.
Zavodsky, Bradley↗
Verification and Validation of the PLTEMP/ANL Code for Thermal-Hydraulic Analysis of Experimental and Test Reactors, Volume 2
This is Volume 2 of a two-volume document that collects the verification and validation (V&V) works done for the PLTEMP/ANL code during the years of its development and improvement. Volume 1 describes the V&V of sixteen capabilities of the PLTEMP/ANL code that were identified by research reactor analysts as frequently used in their thermal-hydraulic analysis. Volume 2 describes the V&V of developments and improvements since the release of PLTEMP/ANL Version 4.3. Each chapter of the document focuses on verifying or validating a specific part of the software that calculates a particular phenomenon, e.g., channel flow calculation, coolant property calculation, heat transfer calculation, and flow instability calculation. Software verification is performed by comparing the code with a hand calculation, Microsoft spreadsheet calculation, Mathematica calculation, or MATLAB calculation. The software validation is achieved by comparing the code with experimental data or a widely tested code like the RELAP5 code. In addition, some PLTEMP/ANL V&V works that are available in the open literature are simply referenced in Volume 1 of the document. PLTEMP/ANL has been used in safety analysis reports of several US and foreign research reactors licensed and converted from highly enriched uranium fuel to low-enriched uranium fuel. A list of such reactors is given in Volume 1 of the document.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
An integrated user-oriented laboratory for verification of digital flight control systems: Features and capabilities
A Digital Flight Control Systems Verification Laboratory (DFCSVL) has been established at NASA Ames Research Center. This report describes the major elements of the laboratory, the research activities that can be supported in the area of verification and validation of digital flight control systems (DFCS), and the operating scenarios within which these activities can be carried out. The DFCSVL consists of a palletized dual-dual flight-control system linked to a dedicated PDP-11/60 processor. Major software support programs are hosted in a remotely located UNIVAC 1100 accessible from the PDP-11/60 through a modem link. Important features of the DFCSVL include extensive hardware and software fault insertion capabilities, a real-time closed loop environment to exercise the DFCS, an integrated set of software verification tools, and a user-oriented interface to all the resources and capabilities.
Defeo, P.↗
Space Shuttle Ascent Flight Design Process: Evolution and Lessons Learned
The Space Shuttle Ascent Flight Design team is responsible for defining a launch to orbit trajectory profile that satisfies all programmatic mission objectives and defines the ground and onboard reconfiguration requirements for this high-speed and demanding flight phase. This design, verification and reconfiguration process ensures that all applicable mission scenarios are enveloped within integrated vehicle and spacecraft certification constraints and criteria, and includes the design of the nominal ascent profile and trajectory profiles for both uphill and ground-to-ground aborts. The team also develops a wide array of associated training, avionics flight software verification, onboard crew and operations facility products. These key ground and onboard products provide the ultimate users and operators the necessary insight and situational awareness for trajectory dynamics, performance and event sequences, abort mode boundaries and moding, flight performance and impact predictions for launch vehicle stages for use in range safety, and flight software performance. These products also provide the necessary insight to or reconfiguration of communications and tracking systems, launch collision avoidance requirements, and day of launch crew targeting and onboard guidance, navigation and flight control updates that incorporate the final vehicle configuration and environment conditions for the mission. Over the course of the Space Shuttle Program, ascent trajectory design and mission planning has evolved in order to improve program flexibility and reduce cost, while maintaining outstanding data quality. Along the way, the team has implemented innovative solutions and technologies in order to overcome significant challenges. A number of these solutions may have applicability to future human spaceflight programs.
Picka, Bret A.↗
Code Coverage Status of ARC Code-DIF3D
The Argonne Reactor Code (ARC) software system supports users in their fast reactor design goals by providing neutronic, thermal-hydraulic, and structural analysis capabilities. DIF3D plays a pivotal role in the ARC system as the primary homogenized assembly neutronic calculation methodology for fast reactor problems. Over its 40 years history, ARC software usage with DIF3D has been applied to numerous fast and thermal spectrum reactor analysis projects with good to excellent comparison against experiments. With continued improvement of computation resources, many of the geometry modeling capabilities in DIF3D that were primarily used in low order schemes are not really needed anymore. Today, the diffusion and transport capabilities of DIF3D-VARIANT are primarily used in the reactor design process with some scattered usage of DIF3D-FD and DIF3D-Nodal. In recent work, the DIF3D software verification was completed for DIF3D-FD and DIF3D-VARIANT on the geometry options used in the Versatile Test Reactor project. While we can be confident that these capabilities of DIF3D are well used and thus trusted, it does not demonstrate that all possible input options of DIF3D are actually working, but just those that were tested as part of VTR are and that they are correct. Thus, the purpose of the present work is to identify a set of test problems for DIF3D and assess the code coverage of DIF3D for those test problems. The goal is to document what parts of the existing DIF3D code are touched by the set of test problems and which are not. Because the verification work done on DIF3D-VARIANT and DIF3D-FD was focused on the most common uses of DIF3D for fast reactor analysis, the code coverage assessment of those capabilities is the highest priority. This will ensure that nothing is being missed by the existing verification test problems that DIF3D relies upon. The DIF3D-Nodal capability will also be inspected for code coverage as part of this work to further ensure that regular regression testing of DIF3D will trap any likely errors the end user might experience with the DIF3D software. The code coverage analysis of DIF3D was performed with the Code Coverage Tool of the Intel Fortran compiler which requires modifications to the compilation of DIF3D. The detailed coverage tables are given for each submodule of DIF3D separately, and for the submodules which are primarily developed for DIF3D, most of the source files could be at least partially touched. Most of the uncovered parts/files could be easily ignored, because they are either for error message and debugging output or obviously not needed by DIF3D. Out of the entire source codes of DIF3D, only a few uncovered modules deserve further investigation.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
Code Coverage Status of the ARC Code DIF3D
The Argonne Reactor Code (ARC) software system supports users in their fast reactor design goals by providing neutronic, thermal-hydraulic, and structural analysis capabilities. DIF3D plays a pivotal role in the ARC system as the primary homogenized assembly neutronic calculation methodology for fast reactor problems. Over its 40 years history, ARC software usage with DIF3D has been applied to numerous fast and thermal spectrum reactor analysis projects with good to excellent comparison against experiments. With continued improvement of computation resources, many of the geometry modeling capabilities in DIF3D that were primarily used in low order schemes are not really needed anymore. Today, the diffusion and transport capabilities of DIF3D-VARIANT are primarily used in the reactor design process with some scattered usage of DIF3D-FD and DIF3D-Nodal. In recent work, the DIF3D software verification was completed for DIF3D-FD and DIF3D-VARIANT on the geometry options used in the Versatile Test Reactor project. While we can be confident that these capabilities of DIF3D are well used and thus trusted, it does not demonstrate that all possible input options of DIF3D are actually working, but just those that were tested as part of VTR are and that they are correct. Thus, the purpose of the present work is to identify a set of test problems for DIF3D and assess the code coverage of DIF3D for those test problems. The goal is to document what parts of the existing DIF3D code are touched by the set of test problems and which are not. Because the verification work done on DIF3D-VARIANT and DIF3D-FD was focused on the most common uses of DIF3D for fast reactor analysis, the code coverage assessment of those capabilities is the highest priority. This will ensure that nothing is being missed by the existing verification test problems that DIF3D relies upon. The DIF3D-Nodal capability will also be inspected for code coverage as part of this work to further ensure that regular regression testing of DIF3D will trap any likely errors the end user might experience with the DIF3D software. The code coverage analysis of DIF3D was performed with the Code Coverage Tool of the Intel Fortran compiler which requires modifications to the compilation of DIF3D. The detailed coverage tables are given for each submodule of DIF3D separately, and for the submodules which are primarily developed for DIF3D, most of the source files could be at least partially touched. Most of the uncovered parts/files could be easily ignored, because they are either for error message and debugging output or obviously not needed by DIF3D. Out of the entire source codes of DIF3D, only a few uncovered modules deserve further investigation.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
Technology Transfer Challenges for High-Assurance Software Engineering Tools
In this paper, we describe our experience with the challenges thar we are currently facing in our effort to develop advanced software verification and validation tools. We categorize these challenges into several areas: cost benefits modeling, tool usability, customer application domain, and organizational issues. We provide examples of challenges in each area and identrfj, open research issues in areas which limit our ability to transfer high-assurance software engineering tools into practice.
John Penix↗
Kodiak: An Implementation Framework for Branch and Bound Algorithms
Recursive branch and bound algorithms are often used to refine and isolate solutions to several classes of global optimization problems. A rigorous computation framework for the solution of systems of equations and inequalities involving nonlinear real arithmetic over hyper-rectangular variable and parameter domains is presented. It is derived from a generic branch and bound algorithm that has been formally verified, and utilizes self-validating enclosure methods, namely interval arithmetic and, for polynomials and rational functions, Bernstein expansion. Since bounds computed by these enclosure methods are sound, this approach may be used reliably in software verification tools. Advantage is taken of the partial derivatives of the constraint functions involved in the system, firstly to reduce the branching factor by the use of bisection heuristics and secondly to permit the computation of bifurcation sets for systems of ordinary differential equations. The associated software development, Kodiak, is presented, along with examples of three different branch and bound problem types it implements.
Smith, Andrew P.↗
An operation verification strategy against network attack of DCS in nuclear power plant - 323
Digital Instrumentation and control (I and C) system serves as the 'senses and nervous system' of a nuclear power plant (NPP). It plays an important role in the safe and economic operation in the NPPs. Digital I and C system has a series of advantages compared with analogue ones, but the introduction of the operation system platform, the application software, and the network communication structure are causing security risk to the power plant. Different kinds of procedures have already been adopted against these risks such as the communication isolation, but the security attacks still occur from time to time in the industrial fields around the world. This paper presents two computer security methods based on the thought of software verification and validation with step-by-step operation identification. First, three scenarios of security attack are proposed according to the structure of a distributed control system (DCS). Then a method with two network bypasses and a security computer are adopted to obtain the operation commands, the actuation signal, and the operation feedback in case of the first two scenarios by identifying the consistency of the obtained data. Furthermore, the method is optimized by combining the operation procedure and the camera shot to identify the anomaly of the information flow, thus to improve the effectiveness against the computer security issues. In the end, the influence and feasibility of this method is discussed. (authors)
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
SU-E-T-49: A Multi-Institutional Study of Independent Dose Verification for IMRT
Purpose: AAPM TG114 does not cover the independent verification for IMRT. We conducted a study of independent dose verification for IMRT in seven institutes to show the feasibility. Methods: 384 IMRT plans in the sites of prostate and head and neck (HN) were collected from the institutes, where the planning was performed using Eclipse and Pinnacle3 with the two techniques of step and shoot (S&S) and sliding window (SW). All of the institutes used a same independent dose verification software program (Simple MU Analysis: SMU, Triangle Product, Ishikawa, JP), which is Clarkson-based and CT images were used to compute radiological path length. An ion-chamber measurement in a water-equivalent slab phantom was performed to compare the doses computed using the TPS and an independent dose verification program. Additionally, the agreement in dose computed in patient CT images between using the TPS and using the SMU was assessed. The dose of the composite beams in the plan was evaluated. Results: The agreement between the measurement and the SMU were −2.3±1.9 % and −5.6±3.6 % for prostate and HN sites, respectively. The agreement between the TPSs and the SMU were −2.1±1.9 % and −3.0±3.7 for prostate and HN sites, respectively. There was a negative systematic difference with similar standard deviation and the difference was larger in the HN site. The S&S technique showed a statistically significant difference between the SW. Because the Clarkson-based method in the independent program underestimated (cannot consider) the dose under the MLC. Conclusion: The accuracy would be improved when the Clarkson-based algorithm should be modified for IMRT and the tolerance level would be within 5%.
60 APPLIED LIFE SCIENCES↗
SU-E-T-229: Craniospinal Radiotherapy Planning with VMAT, Two First Years Experience
Purpose: To describe how we moved to VMAT in the craniospinal radiotherapy planning process, the actual procedure details, and the results for the patients treated. Methods: Twelve patients underwent craniospinal irradiation with the new procedure, based on the paper by Lee et al. (IJROBP 82, 2012), with some additional modifications. Patients were treated in supine position in Varian Clinac iX linacs with 6 MV RapidArc; prescription doses ranged from 23.4 to 40 Gy (13 to 20 fractions); depending on the PTV length, 2 or 3 isocenters were used, all coordinates being equal except the longitudinal one, setting a few centimeter-long overlapping region; 2 arcs (RA) sharing isocentre for the cranial region, RA1 encompassing cranium and superior spinal region, and RA2 intended to improve conformity, only for cranium; for spine, 1 or 2 isocenters were employed; optimization was performed with Eclipse (V 13.0) using AAA algorithm, establishing sets of optimization parameters to give high conformity while sparing OAR. In pediatric patients, homogeneous irradiation of the vertebrae was also required.Conformity (CI) and heterogeneity (HI) indices (same as Lee et al.), and mean and maximum doses for OAR were calculated. Several pre-treatment verification methods were used: Octavius4D (PTW) for each isocentre, point dose at the junction region, Portal Dosimetry (when possible), and independent MU verification software (Diamond, PTW). Results: CI median value was 1.02 (0.99–1.07) and HI, 1.07 (1.06–1.09); a great reduction was observed for CI and OAR mean doses with respect to Lee et al. data; median maximum eye lens dose was 7.3 Gy (4.0–12.0); mean LungV20Gy was 1.9%; in children, vertebrae were homogeneously irradiated (D95%=20.8 Gy, Dmean= 23.2 Gy).All pre-treatment verifications were found within our action levels except for Portal Dosimetry. Conclusion: A RapidArc planning process for craniospinal axis irradiation has been implemented with significant advantages on conformity, homogeneity, feasibility and efficiency. and increase brain tissue sparing. Variations in volume decrease may be related to shape or location of the tumor.
60 APPLIED LIFE SCIENCES↗