SEARCH · Engineering Papers

Results for “software failure model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

DOE OSTI2022

An Application of a Modified Beta Factor Method for the Analysis of Software Common Cause Failures

This paper presents an approach for modeling software common cause failures (CCFs) within digital instrumentation and control (I&C) systems. CCFs consist of a concurrent failure between two or more components due to a shared failure cause and coupling mechanism. This work emphasizes the importance of identifying software-centric attributes related to the coupling mechanisms necessary for simultaneous failures of redundant software components. The groups of components that share coupling mechanisms are called common cause component groups (CCCGs). Most CCF models rely on operational data as the basis for establishing CCCG parameters and predicting CCFs. This work is motivated by two primary concerns: (1) a lack of operational and CCF data for estimating software CCF model parameters; and (2) the need to model single components as part of multiple CCCGs simultaneously. A hybrid approach was developed to account for these concerns by leveraging existing techniques: a modified beta factor model allows single components to be placed within multiple CCCGs, while a second technique provides software-specific model parameters for each CCCG. This hybrid approach provides a means to overcome the limitations of conventional methods while offering support for design decisions under the limited data scenario.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2022

Application of Orthogonal Defect Classification for Software Reliability Analysis

The modernization of existing and new nuclear power plants with digital instrumentation and control systems (DI&C) is a recent and highly trending topic. However, there lacks strong consensus on best-estimate reliability methodologies by both the United States (U.S.) Nuclear Regulatory Commission (NRC) and the industry. This has resulted in hesitation for further modernization projects until a more unified methodology is realized. In this work, we develop an approach called Orthogonal-defect Classification for Assessing Software Reliability (ORCAS) to quantify probabilities of various software failure modes in a DI&C system. The method utilizes accepted industry methodologies for software quality assurance that are also verified by experimental or mathematical formulations. In essence, the approach combines a semantic failure classification model with a reliability growth model to predict (and quantify) the potential failure modes of a DI&C software system. The semantic classification model is used to address the question: How do latent defects in software contribute to different software failure root causes? The use of reliability growth models is then used to address the question: Given the connection between latent defects and software failure root causes, how can we quantify the reliability of the software? A case study was conducted on a representative I&C platform (ChibiOS) running a smart sensor acquisition software developed by Virginia Commonwealth University (VCU). The testing and evidence collection guidance in ORCAS was applied, and defects were uncovered in the software. Qualitative evidence, such as condition coverage, was used to gauge the completeness and trustworthiness of the assessment while quantitative evidence was used to determine the software failure probabilities. The reliability of the software was then estimated and compared to existing operational data of the sensor device. It is demonstrated that by using ORCAS, a semantic reasoning framework can be developed to justify if the software is reliable (or unreliable) while still leveraging the strength of the existing methods.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2022

Application of Orthogonal Defect Classification for Software Reliability Analysis

The modernization of existing and new nuclear power plants with digital instrumentation and control systems (DI&C) is a recent and highly trending topic. However, there lacks strong consensus on best-estimate reliability methodologies by both the United States (U.S.) Nuclear Regulatory Commission (NRC) and the industry. This has resulted in hesitation for further modernization projects until a more unified methodology is realized. In this work, we develop an approach called Orthogonal-defect Classification for Assessing Software Reliability (ORCAS) to quantify probabilities of various software failure modes in a DI&C system. The method utilizes accepted industry methodologies for software quality assurance that are also verified by experimental or mathematical formulations. In essence, the approach combines a semantic failure classification model with a reliability growth model to predict (and quantify) the potential failure modes of a DI&C software system. The semantic classification model is used to address the question: How do latent defects in software contribute to different software failure root causes? The use of reliability growth models is then used to address the question: Given the connection between latent defects and software failure root causes, how can we quantify the reliability of the software? A case study was conducted on a representative I&C platform (ChibiOS) running a smart sensor acquisition software developed by Virginia Commonwealth University (VCU). The testing and evidence collection guidance in ORCAS was applied, and defects were uncovered in the software. Qualitative evidence, such as condition coverage, was used to gauge the completeness and trustworthiness of the assessment while quantitative evidence was used to determine the software failure probabilities. The reliability of the software was then estimated and compared to existing operational data of the sensor device. It is demonstrated that by using ORCAS, a semantic reasoning framework can be developed to justify software reliability (or unreliability) while still leveraging the strength of the existing methods.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2017

Application of a bayesian belief network model to reliability assessment of nuclear safety-related software - 207

As the instrumentation and control (I and C) systems in nuclear power plants (NPPs) have been replaced with digital-based systems, the need to incorporate software failures into NPP probabilistic risk assessments has arisen. In order to assess the probability of software failure on demand, a Bayesian belief network (BBN) model was developed which estimates the number of defects and the resulting probability of software failure on demand in nuclear safety-related software. To assess the feasibility of the BBN framework, the BBN model was applied to the prototype Integrated Digital Protection System-Reactor Protection System (IDiPS-RPS) to estimate the number of remaining faults and the software failure probability of a target software. The developmental- and V and V-activities carried out during the IDiPS-RPS development process were evaluated based on the well-defined checklist derived by the V and V team and were estimated based on expert elicitation. In addition, the attribute evaluations and the number of FPs of the target software is provided as the inputs for the BBN model. The application results showed the feasibility of using BBNs for quantifying software failure probabilities and several insights were gained from the applications of the BBN model. The proposed BBN framework can be applied to estimate the software failure probability for other safety-related NPP software and provide an insight on modeling the software development process that involves iterations between different development phases. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2017

Development of a bayesian belief network model for the software reliability assessment of nuclear digital I and C safety systems - 206

Since the digital instrumentation and control systems are expected to play an important role for the safety systems in nuclear power plants (NPPs), the need has emerged to not only establish a basis for incorporating software behavior into digital I and C system reliability models, but also to quantify the failure probability of the software used in NPP digital protection systems. In this study, a Bayesian belief network (BBN) model is developed to quantitatively assess software reliability by estimating the number of faults in a software program considering its software development life cycle (SDLC). The model structure and parameters are established based on the information applicable to NPP safety-related systems and the evidence used to construct and quantify the BBN model was collected from three stages of expert elicitation. The software failure probability is estimated from the number of residual defects in a software program at the end of SDLC phase. As a case study, the BBN model was applied to quantify the software reliability of a typical digital protection software having the size of 50 function points and having the Medium development and validation and verification (V and V) qualities. The developed model can be applied to estimate the failure probability for both developing and deployed safety-related NPP software, and such results can be used to evaluate the quality of the digital I and C systems in addition to estimating potential reactor risk due to software failure. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2023

Analyzing Hardware and Software Common Cause Failures in Digital Instrumentation and Control Systems using Dual Error Propagation Method

This paper develops a methodology for quantifying software common cause failures (CCFs) in digital instrumentation and control (I&C) systems of nuclear power plants. To support the transition of analog I&C systems to digital in nuclear power plants, probabilistic risk assessment (PRA) techniques are used. The hardware components of the I&C systems have reliability databases that can be used in the PRA studies. However, the failure data for redundant software components of the systems is sparse. Failure of components constitutes a CCF, wherein two or more components or systems fail due to a single shared cause and coupling mechanism. This paper proposes a quantification approach that can simultaneously model hardware and software components, incorporate the CCFs of software systems in the models, and bridge the gap between the failure quantification of models and the development of CCF parametric databases. We demonstrate the dual error propagation method (DEPM) by developing I&C systems failure models for a representative digital reactor trip system. The DEPM models are built to simulate the control and data flows within the systems and can accommodate failure states. By expanding DEPM to software CCFs, we generated alpha factor parameter estimates for each of the modeled error propagation mechanisms.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2024

Design Methods, Tools, and Data for Ceramic Solar Receivers

This report presents the development of tools and methods for evaluating the reliability and performance of ceramic materials in high temperature solar receivers. As Concentrating Solar Power (CSP) technologies aim for higher operating temperatures to enhance efficiency and meet industrial process heat requirements, current high temperature metallic materials face challenges in maintaining structural integrity. This report explores advanced ceramics as a promising alternative, given their superior high temperature strength and lower thermal expansion, compared to metals. To address the need for effective ceramic receiver design tools, this report integrates statistical failure models of ceramics into the existing srlife tool: an open-source software package designed to estimate the life of high temperature CSP components. These failure models account for the inherent variability and flaw distribution in ceramics, as well as the impact of subcritical crack growth under high temperature cyclic loads. The report also presents experimental data collected for a commercially available ceramic material, SiC, and details the process of estimating reliability model parameters from these data. A comparative design analysis is then performed between ceramic (SiC) and metallic (current nickel-based superalloys A740H and A282) receiver. This comparison demonstrates that SiC receivers can achieve service life exceeding 30 years under high incident heat flux conditions, compared to just a few years for metallic receivers.

14 SOLAR ENERGY↗

DOE OSTI2022

An Advanced Meso-Scale Peridynamic Modeling Technology using High-Performance Computing for Cost-Effective Product Design and Testing of Carbon Fiber Reinforced Polymer Composites in Light-weight Vehicles

We study a peridynamic composite modeling technology based on the discontinuous Galerkin finite element method, implemented in the commercial LS-DYNA software, for modeling and prediction of failure in carbon fiber reinforced polymer composites. The proposed technology is developed for the material failure analysis at the meso-scale, which provides the prevailing fiber-matrix interaction mechanism, without adoption of the representative volume element method and thus avoiding complicated numerical calibration procedures. Three types of experimental tests—in-plane coupon test, out-of-plane coupon test, and a component crash test—are simulated in a high-performance computing environment to assess the performance of the proposed peridynamic composite modeling technology.

36 MATERIALS SCIENCE↗

DOE OSTI2024

DECIDER

This software offers methods and functions for building failure detectors for deep image classification models with the aid of vision-language models and LLMs. It includes functionalities for training baseline image classifiers, debiasing classifiers using vision-language models and LLMs, evaluating failure between models along with baselines. Developed using PyTorch, this software is compatible with standard neural network architectures used for imaging data. Additionally, it provides capabilities to compute evaluation metrics for assessing the performance and quality of the detectors.

Narayanaswamy, Vivek Sivaraman↗

DOE OSTI2016

A Statistical Testing Approach for Quantifying Software Reliability; Application to an Example System

The U.S. Nuclear Regulatory Commission (NRC) encourages the use of probabilistic risk assessment (PRA) technology in all regulatory matters, to the extent supported by the state-of-the-art in PRA methods and data. Although much has been accomplished in the area of risk-informed regulation, risk assessment for digital systems has not been fully developed. The NRC established a plan for research on digital systems to identify and develop methods, analytical tools, and regulatory guidance for (1) including models of digital systems in the PRA's of nuclear power plants (NPPs), and, (2) incorporating digital systems in the NRC's risk-informed licensing and oversight activities. Under NRC's sponsorship, Brookhaven National Laboratory (BNL) explored approaches for addressing the failures of digital instrumentation and control (I and C) systems in the current NPP PRA framework. Specific areas investigated included PRA modeling digital hardware , development of a philosophical basis for defining software failure , and identification of desirable attributes of quantitative software reliability methods . Based on the earlier research, statistical testing is considered a promising method for quantifying software reliability. It is widely recognized that software failures are due to the triggering of pre-existing defects by the software's operational environment. Software defects can arise from errors made in user requirements or coding errors introduced during the developmental process. The software's operational environment includes factors such as the time history of digital system inputs, communication interfaces, the internal state of the digital system, and external conditions. Thus, software reliability is a function of both the number of pre-existing defects and the presence of a triggering condition caused by the manner in which the software is used. In this paper, we describe a statistical software-testing approach for quantifying software reliability and applied it to the loop-operating control system (LOCS) of an experimental loop of the Advanced Test Reactor (ATR) at Idaho National Laboratory (INL). The work involved collaboration between BNL and INL. The objectives of the study include: (1) Development of a statistical testing approach for estimating software failure probability on demand, the results of which are suitable for including in a probabilistic risk assessment (PRA); and, (2) Application of the approach to the LOCS to estimate its failure probability, and obtain insights into the feasibility, practicality, and usefulness of the estimation in models of digital systems for inclusion in nuclear power plants' PRAs. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2016

Finite-Test-Case Development to Demonstrate Logical Integrity of Software in DRPS

Reactor protection system (RPS) in nuclear power plant (NPP) is in the process of digitalization in response to advanced features of digital systems. In this context, software integrity in the RPS becomes an important issue since it may cause common-cause failure. Therefore, to guarantee the safety of NPPs, the integrity of the software need to be properly demonstrated or assessed. There are three typical approaches for software reliability quantification: software reliability growth model (SRGM), Bayesian belief network (BBN), and test-based method. Basically, SRGM is difficult to apply to the safety-critical software because not only sufficient failure sets are unavailable but consequence of system failure is also unacceptable. Then the result obtained through the BBN model may have large uncertainty, so it produces difficulties in integrity demonstration of safety-critical software. Thus, the BBN result need to be complemented or verified by test-based method. Currently, for the integrity demonstration of safety-critical software, black-box test which feeds inputs and examines just outputs whether they succeeded or failed is considered based on trajectory-form-of-input (a series of successive values for the input variables that occur during the operation of the software over time). Usually these trajectory-inputs are randomly sampled from operational profile. In this approach, however, there are some practical difficulties. Therefore, we proposed a new test method for safety-critical software in digitalized reactor protection system (DRPS). The main contents described in this paper are about basic concepts under development of finite-test-cases by which the integrity of the software logic can be verified. It should be noted that the influence of the environments, such as operating system and other hardware, are not considered since the method focuses on software logic only. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2016

A Statistical Testing Approach for Quantifying Software Reliability; Application to an Example System

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2021

Prediction of Thermal Conditions of DED With FEA Metal Additive Simulation

This paper presents the integration of wire-arc additive manufacturing (WAAM) using Gas Metal Arc Welding (GMAW) into a machine tool to create a retrofit hybrid computer numeric control (CNC) machine tool. GMAW, along with other direct energy deposition systems, has the capacity to deposit material faster than the excess thermal energy can dissipate. This results in the need to allow the part to cool between consecutive layers, which is the most time-consuming part of the additive process. Finite element analysis (FEA) was used in conjunction with monitored build plate surface temperatures during deposition samples to improve adequate dwell time prediction and to develop a cooling system. A deposition was completed where no dwell time was used and the build plate along with the machine table temperatures were monitored. A second deposition was completed where only one bead was deposited and the traverse speed was increased. The GMAW welder was mounted on a 3-axis CNC machine where two square deposition samples were completed. A FEA model was designed and verified using the monitored samples. The model will be used to determine improved depositions speeds and whether forced cooling would allow for an increased deposition rate without structural failure. It was determined the FEA software can be used to accurately model and predict the thermal response of WAAM AM components.

Heinrich, Lauren↗

DOE OSTI2018

Recurrent Neural Network Attention Mechanisms for Interpretable System Log Anomaly Detection

Deep learning has recently demonstrated state-of-the art performance on key tasks related to the maintenance of computer systems, such as intrusion detection, denial of service attack detection, hardware and software system failures, and malware detection. In these contexts, model interpretability is vital for administrator and analyst to trust and act on the automated analysis of machine learning models. Deep learning methods have been criticized as black box oracles which allow limited insight into decision factors. In this work we seek to "bridge the gap'' between the impressive performance of deep learning models and the need for interpretable model introspection. To this end we present recurrent neural network (RNN) language models augmented with attention for anomaly detection in system logs. Our methods are generally applicable to any computer system and logging source. By incorporating attention variants into our RNN language models we create opportunities for model introspection and analysis without sacrificing state-of-the art performance. We demonstrate model performance and illustrate model interpretability on an intrusion detection task using the Los Alamos National Laboratory (LANL) cyber security dataset, reporting upward of 0.99 area under the receiver operator characteristic curve despite being trained only on a single day's worth of data.

Anomaly detection, Attention, Recurrent Neural Net↗

DOE OSTI2020

Dynamic PRA-Based Estimation of PWR Coping Time Using a Surrogate Model for Accident Tolerant Fuel

In this study, we propose an interpolation-based response surface surrogate methodology to manage a large number of scenarios in dynamic probabilistic risk assessment. It adopts the shape Dynamic Time Warping algorithm to cluster the interpolation neighborhood from time series sample data. The interpolation method was adapted from Taylor Kriging to allow a reduced-order model of the Taylor series. In order to demonstrate its applicability to complex issues in risk assessment for nuclear engineering, an example risk response surface to estimate emergency core cooling system (ECCS) criteria for triplex silicon carbide (SiC) accident-tolerant fuel was constructed. The response surface was exploited to estimate the cumulative failure probability of the fuel cladding structure due to the uncertainties in operator actions and safety systems. The functional failures were assessed based on a combination of individual layer failures computed by coupling Risk Analysis Virtual Environment software with a pressurized water reactor 1000-MW(electric) RELAP5 model and the in-house fuel performance assessment module. Results showed that SiC cladding failure probability spiked less than 1 min after a large-break loss-of- coolant accident whenever the current ECCS criteria for Zircaloy-4 (Zr-4) cladding was used. However, it still provides an increased safety margin of three orders of magnitude compared to Zr-4. This positive margin could be utilized to relax active ECCS requirements by allowing deviations of up to 450 s in its actuation time. The proposed surrogate methodology generated a response surface of SiC cladding failure probability reasonably well, with a significant savings of computation time. This methodology is expected to be useful in the analysis of system response with complex uncertainty sources.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2020

Exploratory Investigation of Disengagements and Crashes in Autonomous Vehicles Under Mixed Traffic: An Endogenous Switching Regime Framework

Autonomous Vehicles (AVs) have a large potential to improve traffic safety but also pose some critical challenges. While AVs may help reduce crashes caused by human error, they still may experience failures of technologies and sensing, as well as decision-making errors in a mixed traffic environment. A disengagement refers to an AV transitioning control from autonomous systems to the trained test driver. The safety critical nature of disengagements makes it imperative to analyze disengagements and crashes together. In this study, we analyze both crashes and disengagements from real-world AV driving in California to fill the knowledge gap regarding the relationship between disengagements and crashes in a mixed traffic environment. A nested logit model was calibrated using three different outcomes: (1) disengagement with a crash, (2) disengagement with no crash, and (3) no disengagement with a crash. Furthermore, endogenous switching regime models were also calibrated to draw distinctions between the relation of disengagements and crashes while accounting for endogeneity effects. The results show that factors related to AV systems (such as software failures) and other roadway participants increase the propensity of a disengagement without a crash. Furthermore, AVs were observed to disengage less often as the technology matured over time. Marginal effects revealed an 8% decrease. The results thus suggest that disengagements are a part of AVs' safe performance and disengagement alerts may need to be triggered in order to avoid certain failures with current technology.

42 ENGINEERING↗

DOE OSTI2023

Advanced Monitoring and Control in the ANL METL Facility Using an Engineering Digital Twin

The potential benefits of using an engineering digital twin to achieve greater autonomy for monitoring and control functions in advanced reactors was investigated for the Mechanisms Engineering Test Loop (METL) liquid sodium facility at Argonne National Laboratory. The METL sodium purification system served as a representative system as it requires significant human in-the-loop interaction to accomplish its design function. The objective was to demonstrate how real-time operation could be automated while preserving oversight of the operator for ensuring that the system design functions are met. A digital twin model of the purification system was developed for both the cold trap purification loop and plugging meter diagnostic loop using information from the METL piping and instrumentation diagram (P&ID). Automated monitoring and diagnosis of component degradation in the METL facility was demonstrated in tests using the PRO-AID health monitoring software with the digital twin model incorporated in the library of components. Component failures were introduced and were successfully diagnosed in real time. These tests serve to demonstrate an advanced monitoring capability able to differentiate sensor degradation from component degradation, to generate a rank ordering of probabilities of different failure mechanisms that serves to circumvent the false alarm problem with current anomaly detection methods, and how facility monitoring can be transformed from anomaly detection to identification of a specific fault. Automated control of the purification system was demonstrated through simulations that exercised a model predictive controller designed using the digital twin model. Results of these simulations compared favorably with experimental data showing very good reference tracking response with negligible overshoot. In conclusion, these pilot tests and simulations successfully demonstrated the use of a digital twin for improved automation of monitoring and control. It was shown how the digital twin enables switching between control modes from cold trap operation where impurities are removed to plugging meter operation where impurity concentrations are measured.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2019

Surrogate Models for Estimating Failure in Brittle and Quasi-Brittle Materials

In brittle fracture applications, failure paths, regions where the failure occurs and damage statistics, are some of the key quantities of interest (QoI). High-fidelity models for brittle failure that accurately predict these QoI exist but are highly computationally intensive, making them infeasible to incorporate in upscaling and uncertainty quantification frameworks. The goal of this paper is to provide a fast heuristic to reasonably estimate quantities such as failure path and damage in the process of brittle failure. Towards this goal, we first present a method to predict failure paths under tensile loading conditions and low-strain rates. The method uses a k-nearest neighbors algorithm built on fracture process zone theory, and identifies the set of all possible pre-existing cracks that are likely to join early to form a large crack. The method then identifies zone of failure and failure paths using weighted graphs algorithms. We compare these failure paths to those computed with a high-fidelity fracture mechanics model called the Hybrid Optimization Software Simulation Suite (HOSS). A probabilistic evolution model for average damage in a system is also developed that is trained using 150 HOSS simulations and tested on 40 simulations. A non-parametric approach based on confidence intervals is used to determine the damage evolution over time along the dominant failure path. For upscaling, damage is the key QoI needed as an input by the continuum models. This needs to be informed accurately by the surrogate models for calculating effective moduli at continuum-scale. We show that for the proposed average damage evolution model, the prediction accuracy on the test data is more than 90%. In terms of the computational time, the proposed models are ≈ O ( 10 6 ) times faster compared to high-fidelity fracture simulations by HOSS. These aspects make the proposed surrogate model attractive for upscaling damage from micro-scale models to continuum models. We would like to emphasize that the surrogate models are not a replacement of physical understanding of fracture propagation. The proposed method in this paper is limited to tensile loading conditions at low-strain rates. This loading condition corresponds to a dominant fracture perpendicular to tensile direction. The proposed method is not applicable for in-plane shear, out-of-plane shear, and higher strain rate loading conditions.

36 MATERIALS SCIENCE↗