Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software reliability”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

A Statistical Testing Approach for Quantifying Software Reliability; Application to an Example System

The U.S. Nuclear Regulatory Commission (NRC) encourages the use of probabilistic risk assessment (PRA) technology in all regulatory matters, to the extent supported by the state-of-the-art in PRA methods and data. Although much has been accomplished in the area of risk-informed regulation, risk assessment for digital systems has not been fully developed. The NRC established a plan for research on digital systems to identify and develop methods, analytical tools, and regulatory guidance for (1) including models of digital systems in the PRA's of nuclear power plants (NPPs), and, (2) incorporating digital systems in the NRC's risk-informed licensing and oversight activities. Under NRC's sponsorship, Brookhaven National Laboratory (BNL) explored approaches for addressing the failures of digital instrumentation and control (I and C) systems in the current NPP PRA framework. Specific areas investigated included PRA modeling digital hardware , development of a philosophical basis for defining software failure , and identification of desirable attributes of quantitative software reliability methods . Based on the earlier research, statistical testing is considered a promising method for quantifying software reliability. It is widely recognized that software failures are due to the triggering of pre-existing defects by the software's operational environment. Software defects can arise from errors made in user requirements or coding errors introduced during the developmental process. The software's operational environment includes factors such as the time history of digital system inputs, communication interfaces, the internal state of the digital system, and external conditions. Thus, software reliability is a function of both the number of pre-existing defects and the presence of a triggering condition caused by the manner in which the software is used. In this paper, we describe a statistical software-testing approach for quantifying software reliability and applied it to the loop-operating control system (LOCS) of an experimental loop of the Advanced Test Reactor (ATR) at Idaho National Laboratory (INL). The work involved collaboration between BNL and INL. The objectives of the study include: (1) Development of a statistical testing approach for estimating software failure probability on demand, the results of which are suitable for including in a probabilistic risk assessment (PRA); and, (2) Application of the approach to the LOCS to estimate its failure probability, and obtain insights into the feasibility, practicality, and usefulness of the estimation in models of digital systems for inclusion in nuclear power plants' PRAs. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2016

A Statistical Testing Approach for Quantifying Software Reliability; Application to an Example System

The U.S. Nuclear Regulatory Commission (NRC) encourages the use of probabilistic risk assessment (PRA) technology in all regulatory matters, to the extent supported by the state-of-the-art in PRA methods and data. Although much has been accomplished in the area of risk-informed regulation, risk assessment for digital systems has not been fully developed. The NRC established a plan for research on digital systems to identify and develop methods, analytical tools, and regulatory guidance for (1) including models of digital systems in the PRAs of nuclear power plants (NPPs), and (2) incorporating digital systems in the NRC's risk-informed licensing and oversight activities. Under NRC's sponsorship, Brookhaven National Laboratory (BNL) explored approaches for addressing the failures of digital instrumentation and control (I and C) systems in the current NPP PRA framework. Specific areas investigated included PRA modeling digital hardware, development of a philosophical basis for defining software failure, and identification of desirable attributes of quantitative software reliability methods. Based on the earlier research, statistical testing is considered a promising method for quantifying software reliability. This paper describes a statistical software testing approach for quantifying software reliability and applies it to the loop-operating control system (LOCS) of an experimental loop of the Advanced Test Reactor (ATR) at Idaho National Laboratory (INL).

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2022

Application of Orthogonal Defect Classification for Software Reliability Analysis

The modernization of existing and new nuclear power plants with digital instrumentation and control systems (DI&C) is a recent and highly trending topic. However, there lacks strong consensus on best-estimate reliability methodologies by both the United States (U.S.) Nuclear Regulatory Commission (NRC) and the industry. This has resulted in hesitation for further modernization projects until a more unified methodology is realized. In this work, we develop an approach called Orthogonal-defect Classification for Assessing Software Reliability (ORCAS) to quantify probabilities of various software failure modes in a DI&C system. The method utilizes accepted industry methodologies for software quality assurance that are also verified by experimental or mathematical formulations. In essence, the approach combines a semantic failure classification model with a reliability growth model to predict (and quantify) the potential failure modes of a DI&C software system. The semantic classification model is used to address the question: How do latent defects in software contribute to different software failure root causes? The use of reliability growth models is then used to address the question: Given the connection between latent defects and software failure root causes, how can we quantify the reliability of the software? A case study was conducted on a representative I&C platform (ChibiOS) running a smart sensor acquisition software developed by Virginia Commonwealth University (VCU). The testing and evidence collection guidance in ORCAS was applied, and defects were uncovered in the software. Qualitative evidence, such as condition coverage, was used to gauge the completeness and trustworthiness of the assessment while quantitative evidence was used to determine the software failure probabilities. The reliability of the software was then estimated and compared to existing operational data of the sensor device. It is demonstrated that by using ORCAS, a semantic reasoning framework can be developed to justify software reliability (or unreliability) while still leveraging the strength of the existing methods.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Application of Orthogonal Defect Classification for Software Reliability Analysis

The modernization of existing and new nuclear power plants with digital instrumentation and control systems (DI&C) is a recent and highly trending topic. However, there lacks strong consensus on best-estimate reliability methodologies by both the United States (U.S.) Nuclear Regulatory Commission (NRC) and the industry. This has resulted in hesitation for further modernization projects until a more unified methodology is realized. In this work, we develop an approach called Orthogonal-defect Classification for Assessing Software Reliability (ORCAS) to quantify probabilities of various software failure modes in a DI&C system. The method utilizes accepted industry methodologies for software quality assurance that are also verified by experimental or mathematical formulations. In essence, the approach combines a semantic failure classification model with a reliability growth model to predict (and quantify) the potential failure modes of a DI&C software system. The semantic classification model is used to address the question: How do latent defects in software contribute to different software failure root causes? The use of reliability growth models is then used to address the question: Given the connection between latent defects and software failure root causes, how can we quantify the reliability of the software? A case study was conducted on a representative I&C platform (ChibiOS) running a smart sensor acquisition software developed by Virginia Commonwealth University (VCU). The testing and evidence collection guidance in ORCAS was applied, and defects were uncovered in the software. Qualitative evidence, such as condition coverage, was used to gauge the completeness and trustworthiness of the assessment while quantitative evidence was used to determine the software failure probabilities. The reliability of the software was then estimated and compared to existing operational data of the sensor device. It is demonstrated that by using ORCAS, a semantic reasoning framework can be developed to justify if the software is reliable (or unreliable) while still leveraging the strength of the existing methods.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2016

Development of an Automated Software Reliability Tester for Digital I and C

Existing nuclear facilities are in the process of upgrading analog systems to new, software- based digital components; however, the qualification of these new systems has been a significant hurdle for their implementation in the United States. This paper describes the development of a Software Reliability Tester (SRT) that will provide a means to quantify the reliability and fault tolerance of digital instrumentation and control (I and C) systems used in nuclear facilities. The SRT is comprised of hardware and software that exercise digital equipment inputs and compare the outputs to expected values to quantify system reliability. Incorporation of fault injection techniques enables the SRT to evaluate the robustness of digital I and C in the presence of abnormal conditions. The integration of both quantitative reliability and fault tolerance measures will provide the nuclear industry with an automated and systematic approach to digital I and C qualification. This will enable safer, more reliable, and more cost effective implementations of a broad spectrum of digital I and C equipment in existing and next generation nuclear facilities. This paper includes a description of the design of the SRT, including software modules and the hardware interface. In addition, an example of using the SRT to automate functional testing of a digital timer relay is presented. The SRT described in this paper provides a means to conduct efficient and robust testing for a variety of digital devices. Future plans for the SRT include integrating fault injection testing so that devices can be evaluated in the presence of off-normal conditions. (authors)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2017

Development of a bayesian belief network model for the software reliability assessment of nuclear digital I and C safety systems - 206

Since the digital instrumentation and control systems are expected to play an important role for the safety systems in nuclear power plants (NPPs), the need has emerged to not only establish a basis for incorporating software behavior into digital I and C system reliability models, but also to quantify the failure probability of the software used in NPP digital protection systems. In this study, a Bayesian belief network (BBN) model is developed to quantitatively assess software reliability by estimating the number of faults in a software program considering its software development life cycle (SDLC). The model structure and parameters are established based on the information applicable to NPP safety-related systems and the evidence used to construct and quantify the BBN model was collected from three stages of expert elicitation. The software failure probability is estimated from the number of residual defects in a software program at the end of SDLC phase. As a case study, the BBN model was applied to quantify the software reliability of a typical digital protection software having the size of 50 function points and having the Medium development and validation and verification (V and V) qualities. The developed model can be applied to estimate the failure probability for both developing and deployed safety-related NPP software, and such results can be used to evaluate the quality of the digital I and C systems in addition to estimating potential reactor risk due to software failure. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Bayesian And Human Reliability Analysis (hra)-aided Method For The Reliability Analysis Of Software (bahamas)

The purpose of the BAHAMAS code is to provide a simplified process for performing quantitative evaluations of software reliability. The Bayesian and Human Reliability Analysis (HRA)-Aided method for the Reliability Analysis of software (BAHAMAS) was developed specifically to perform quantification under limited data conditions, i.e., when limited testing or operational data are available, such as during early development stages. BAHAMAS essentially examines the quality of a software development life cycle to determine the probability of specific types of software failure. BAHAMAS will have modules to support user input for detailed and simplified analyses. The user interface will also support software common cause failure analysis.

Wang, Congjian (0000000207789927)↗
DOE OSTI2016

Finite-Test-Case Development to Demonstrate Logical Integrity of Software in DRPS

Reactor protection system (RPS) in nuclear power plant (NPP) is in the process of digitalization in response to advanced features of digital systems. In this context, software integrity in the RPS becomes an important issue since it may cause common-cause failure. Therefore, to guarantee the safety of NPPs, the integrity of the software need to be properly demonstrated or assessed. There are three typical approaches for software reliability quantification: software reliability growth model (SRGM), Bayesian belief network (BBN), and test-based method. Basically, SRGM is difficult to apply to the safety-critical software because not only sufficient failure sets are unavailable but consequence of system failure is also unacceptable. Then the result obtained through the BBN model may have large uncertainty, so it produces difficulties in integrity demonstration of safety-critical software. Thus, the BBN result need to be complemented or verified by test-based method. Currently, for the integrity demonstration of safety-critical software, black-box test which feeds inputs and examines just outputs whether they succeeded or failed is considered based on trajectory-form-of-input (a series of successive values for the input variables that occur during the operation of the software over time). Usually these trajectory-inputs are randomly sampled from operational profile. In this approach, however, there are some practical difficulties. Therefore, we proposed a new test method for safety-critical software in digitalized reactor protection system (DRPS). The main contents described in this paper are about basic concepts under development of finite-test-cases by which the integrity of the software logic can be verified. It should be noted that the influence of the environments, such as operating system and other hardware, are not considered since the method focuses on software logic only. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2016

Software Dependability for Digital Instrumentation and Control Systems in Nuclear Power Plants - Lessons Learnt from a DOE Sponsored Project

This paper discusses the lessons learnt from a sponsored study undertaken under the sponsorship of the Department of Energy by a team of scientists at the Ohio State University in collaboration with Technology Resources and experts around the world. This paper makes recommendations for future research and studies based on these lessons learnt. The purpose of the research was to provide a software dependability quantification framework that would allow its evaluation along the software development life cycle for safety critical instrumentation and control systems (I and C) such as those found in nuclear power plants. A typical example of safety critical system is the reactor protection system. The research funded by the Department of Energy built on preliminary research carried out for the US NRC that had focused solely on software reliability quantification. Quantification of software dependability is important for safety critical I and C because the regulatory framework in use in the United States for nuclear power plants is risk informed. Quantification of risk requires quantification of all contributors to risk and those include instrumentation and control systems. The purpose of the study was to help developers of software-based I and C with an early evaluation of the potential risk an application would represent at different stages of the software development life cycle by building gates at the end of each life cycle phase. Life cycle phases of software development comprehend phases such as requirements, design, coding and test. The gates' function would be to stop development from moving forward if the assessed dependability objective for that phase were not met. Gates thresholds were to be defined based on measurements collected during the life cycle phase under evaluation and phases that preceded it and an understanding of the mechanisms that would lead to threatening the dependability objectives. The information contained in the gates would in addition help steer the development towards areas that needed improvement. The following are four high level conclusions to our study. First the study demonstrates a lack of consensus on mechanisms that will threaten the dependability attributes. This lack of consensus should be resolved. The lack of consensus appears for all measures to the exception of reliability. Second, measures should be developed for software availability and safety. Third, for all dependability attributes (with the exception of reliability) consistency of the measurement threads should evaluated and measurement-based development gates derived. Finally our ability to perform such type of research depends in a significant manner on access to vendor and/or utility data. The licensing environment and the first-of-a-kind nature of current software-based applications for nuclear I and C make it difficult to collect the relevant data. An approach which would either increase access to data by providing 'immunity' to the vendor or utility or the development of a software-based I and C for the sole purpose of data collection for research should be investigated. (authors)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

A Bayesian and HRA-Aided Method for the Novel Reliability Analysis of Software

Technological advancements and nuclear power plant modernization has inspired considerable research in the areas of safety and reliability, yet there remains a lack of consensus for the reliability assessment of digital instrumentation and control (I&C) systems. Motivated by the lack of consensus for reliability analysis methods, this work employs a novel framework that incorporates Bayesian, human reliability, and common-cause failure (CCF) modeling techniques. The novel framework allows the use of state-of-the-art or classical modeling techniques when accounting for human and CCF effects on system reliability. The Bayesian and HRA-Aided Method for the Reliability Analysis of Software (BAHAMAS) is demonstrated by a case study for the quantification of software hazards found in a previous analysis of a digital reactor trip system. The results demonstrate the ability of BAHAMAS to account for human activities during the software development life cycle and their influence on software reliability. BAHAMAS is a flexible tool for extending the coverage of conventional probabilistic risk assessments to include modernized digital I&C systems.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2017

Application of quantitative methods for reliability testing of a nuclear power plant digital rod position indication diagnostic system - 391

Obsolescence, aging, reliability, and performance issues are driving nuclear facilities to replace conventional analog Instrumentation and Control (I and C) systems with digital technologies. In addition, the designs of the next generation of nuclear reactors, including Small Modular Reactors (SMRs), will incorporate digital I and C for most, if not all, of their safety and non-safety related functions. As part of an on-going research and development (R and D) effort under the auspices of the U.S. Department of Energy (DOE), Analysis and Measurement Services Corporation (AMS) is developing a platform to provide automated testing of digital I and C systems and create a standard method of evaluation for reliability assessments. This hands-on R and D effort has produced a Software Reliability Tester (SRT), which is a set of software and hardware tools designed to automate the testing of digital I and C systems to measure and quantify how well the systems perform under normal operating conditions and in the presence of faults. Furthermore, the SRT provides the foundation for a practical tool to automate verification and validation (V and V) activities and reduce the amount of testing time of digital I and C systems. When combined with its capabilities of integrating both reliability and fault tolerance quantification, the SRT can be used to ensure that digital I and C implementations are both safe and cost-effective for the nuclear industry. This paper describes the application of the SRT to perform V and V of a new Digital Rod Position Indication (DRPI) diagnostic system. The DRPI diagnostic system is a digital system that monitors the rod address and rod position data bus voltages of typical DRPI systems in nuclear power plants to detect and diagnose faults. For this application, the SRT was configured to exercise the inputs of the DRPI diagnostic system, and test cases were generated for common system faults such as address and data bit errors, bus voltage, and timing problems. These test cases were applied by the SRT hardware as voltage inputs to the DRPI diagnostic system and the outputs were compared to expected values generated by a model of the DRPI diagnostic system. Included in the paper is a description of the overall design of the SRT including the hardware and software architectures. Methodologies for automating test cases are also described. V and V using the SRT demonstrates the benefits of automated testing and qualification to provide a quantitative assessment of reliability and cost effective implementation of digital I and C in existing and next generation nuclear power plants. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2019

Software predicts disaster-driven grid failures

A new, free, open-source software reliably predicts how damage from hurricanes, ice storms, earthquakes, and other extreme events will restrict power delivery from utility grids. The Severe Contingency Solver for Electric Power Transmission is the only software available—commercially or open-source—that reliably supports analysis of extreme events that cause widespread damage.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2016

Importance Ranking of Software Dependability Attributes in the Nuclear Industry

With the current transition from analog to digital instrumentation and control systems in nuclear power plants, the number and variety of software-based systems has significantly increased. Due to the sophisticated nature and the increasing complexity of software, a significant challenge faced by the nuclear industry is the trust one can place in software-based systems once they have been developed. The trust placed in software is typically termed software dependability, a qualifier which encompasses different attributes such as software reliability, safety, security, availability and maintainability. It is recognized that these attributes relate to each other in different ways under various scenarios. The detailed dependencies between software dependability attributes can be found in a previous study. With limited time and budget resources, which dependability attribute should be given the highest priority when certifying software-based systems in the nuclear industry is a fundamental question that needs to be answered. This paper addresses this problem using stakeholders' opinion elicitation. A comprehensive questionnaire was designed to elicit the goals that should be fulfilled by the nuclear industry and to collect the relative importance of the various dependability attributes for each goal. We then used the Analytical Hierarchy Process (AHP) to analyze the data and obtained the relative importance ranking of software dependability attributes in the context of a reactor protection system. The results will provide guidance to resource allocation for the certification of software dependability in the nuclear industry. (authors)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2019

Using simulation to quantify the reliability of control software

More than two decades ago, Butler and Finelli examined the problem of experimentally demonstrating the reliability of safety critical software and concluded that it was impractical. We revisit this conclusion in the light of recent advances in computer system virtualization technology and the capability to link virtualization tools to simulation models of physical environments. A specific demonstration of testing for reliability is offered using software that is part of a building control system. Extrapolating the results of this demonstration, we conclude that experimental demonstrations of high reliability may now be feasible for some applications.

Nutaro, James J.↗