SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
This paper proposes the use of Bi-directional Analysis (BDA), an integrated extension of SFMEA and SFTA, as a core assessment technique by which saftey-critical software can be certified.
In its 2013 recommendations to the International Atomic Energy Agency (IAEA), the Technical Working Group on Nuclear Power Plant Instrumentation and Control (TWG-NPPIC) identified assessment and evaluation of software dependability as a topic of interest for IAEA Member States. To implement the recommendation of the TWG-NPPIC, the IAEA Scientific Secretary for the TWG initiated a consultancy activity in 2014 to develop a technical report describing a dependability assessment framework suitable for safety instrumentation and control (I and C) systems in nuclear power plants (NPPs). The development of the software dependability assessment report has involved three consultancy meetings at IAEA headquarters in Vienna, Austria. The conduct of these meetings allowed consultation among subject matter experts to generate consensus technical information that serves as the content of the report. The technical contributors consist of international experts from academia, industry, regulatory bodies and research organizations. In September 2014, a technical meeting was held in Daejeon, Republic of Korea, to present interim findings of the consultancy effort and to solicit input from representatives of IAEA Member States. The information and technical comments contributed during the technical meeting were addressed in subsequent development of the report. This paper gives an overview of key technical elements of the IAEA report. The IAEA report presents a framework by which a dependability assessment of safety software can be performed. The specified guiding principles provide the basis for establishing an assessment strategy and implementing the assessment process. The assessment framework provides a systematic, coherent structure within which to organize technical and process evidence to satisfy arguments in support of claims about software dependability properties. During the consultancy activity, the participating subject matter experts focused on high level principles, a structured approach, and identification of suitable methods and techniques. However, it was recognized that what remains is to concretize the framework through specific examples and usage experience. Thus, following publication of the report, it is recommended that case studies be performed to illustrate and more precisely define specific phases and elements of the assessment framework. For example, current national assessment approaches could be mapped into the assessment framework to serve as case studies and help to identify any aspects of the assessment framework that needs further definition or adjustment. Additionally, a challenge problem could be defined to engage stakeholders to apply the assessment framework with a common focus. This could help to better illustrate strategic options and approaches to the assessment phases. It also could serve to suggest other considerations and potential sources of evidence not fully covered in the report. Based on findings from case studies, challenge problems and country experience, it should be possible to develop generalized templates to support application of the assessment framework. These findings can be collected to serve as the basis for a User's Guide on dependability assessment of safety software. (authors)
The goal of this review is to enable application of codes or software packages for safety assessment of advanced sodium-cooled fast reactor (SFR) designs. To address near-term programmatic needs, the authors have focused on two objectives. First, the authors have focused on identification of requirements for software QA that must be satisfied to enable the application of software to future safety analyses. Second, the authors have collected best practices applied by other code development teams to minimize cost and time of initial code qualification activities and to recommend a path to the stated goal.
Paper examines issue of software safety. Presents four case histories of software-safety analysis. Concludes that, to be safe, software, for all practical purposes, must be free of errors. Backup systems still needed to prevent catastrophic software failures.
The application of assurance engineering techniques improves the duration of failure-free performance of software. The totality of features and characteristics of a software product are what determine its ability to satisfy customer needs. Software in safety-critical systems is very important to NASA. We follow the System Safety Working Groups definition for system safety software as: 'The optimization of system safety in the design, development, use and maintenance of software and its integration with safety-critical systems in an operational environment. 'If it is not safe, say so' has become our motto. This paper goes over methods that have been used by NASA to make software design improvements by focusing on software quality and the design and inspection process.
NASA's Aeronautics Blueprint lays out a research agenda for the Agency s aeronautics program. The word software appears only four times in this Blueprint, but the critical importance of safe and correct software to the fulfillment of the proposed research is evident on almost every page. Most of the technology solutions proposed to address challenges in aviation are software dependent technologies. Of the fifty-two specific technology solutions described in the Blueprint, forty-one depend, at least in part, on software for success. For thirty-five of these forty-one, software is not only critical to success, but also to human safety. That is, implementing the technology solutions will require using software in such a way that it may, if not specified, designed, and implemented properly, lead to fatal accidents. These results have at least two implications for the research based on the Blueprint: (1) knowledge about the current state-of-the-art and state-of-the-practice in software engineering and software system safety is essential, and (2) research into current unsolved problems in these software disciplines is also essential.
Digitalization of the instrumentation and control systems in nuclear power plant entails some new features which do not exist in analog systems. When the new features are applied in safety critical systems, the risk stemming from them should be quantified properly to assure the reliability of the entire power plant. Among the new features, software is considered as the most important factor because it can cause common cause failure to many redundant systems. Therefore, in this work, a test based reliability assessment method for a safety-critical software is developed. An output of a software is determined by not only inputs but also the internal state of the software at the time. In the case of safety critical software developed for usage in nuclear power plant, it is possible to directly investigate probable internal states thanks to detailed design specification and programing features, and the probable inputs also can be identified based on the physical linearity of each process parameter and hardware characteristics related to data acquisition. In this work, a development method for practical exhaustive test case consisting of investigation of internal state and probable input sets to a specific internal state are suggested. In addition to this, software logic simulator for execution of software logic test and the reliability quantification method based on the test result are developed. The feasibility of the suggested method is demonstrated via a case study. (authors)
With the current transition from analog to digital instrumentation and control systems in nuclear power plants, the number and variety of software-based systems have significantly increased. The sophisticated nature and increasing complexity of software raises trust in these systems as a significant challenge. The trust placed in a software system is typically termed software dependability. Software dependability analysis faces uncommon challenges since software systems’ characteristics differ from those of hardware systems. The lack of systematic science-based methods for quantifying the dependability attributes in software-based instrumentation as well as control systems in safety critical applications has proved itself to be a significant inhibitor to the expanded use of modern digital technology in the nuclear industry. Dependability refers to the ability of a system to deliver a service that can be trusted. Dependability is commonly considered as a general concept that encompasses different attributes, e.g., reliability, safety, security, availability and maintainability. Dependability research has progressed significantly over the last few decades. For example, various assessment models and/or design approaches have been proposed for software reliability, software availability and software maintainability. Advances have also been made to integrate multiple dependability attributes, e.g., integrating security with other dependability attributes, measuring availability and maintainability, modeling reliability and availability, quantifying reliability and security, exploring the dependencies between security and safety and developing integrated analysis models. However, there is still a lack of understanding of the dependencies between various dependability attributes as a whole and of how such dependencies are formed. To address the need for quantification and give a more objective basis to the review process -- therefore reducing regulatory uncertainty -- measures and methods are needed to assess dependability attributes early on, as well as throughout the life-cycle process of software development. In this research, extensive expert opinion elicitation is used to identify the measures and methods for assessing software dependability. Semi-structured questionnaires were designed to elicit expert knowledge. A new notation system, Causal Mechanism Graphing, was developed to extract and represent such knowledge. The Causal Mechanism Graphs were merged, thus, obtaining the consensus knowledge shared by the domain experts. In this report, we focus on how software contributes to dependability. However, software dependability is not discussed separately from the context of systems or socio-technical systems. Specifically, this report focuses on software dependability, reliability, safety, security, availability, and maintainability. Our research was conducted in the sequence of stages found below. Each stage is further examined in its corresponding chapter. Stage 1 (Chapter 2): Elicitation of causal maps describing the dependencies between dependability attributes. These causal maps were constructed using expert opinion elicitation. This chapter describes the expert opinion elicitation process, the questionnaire design, the causal map construction method and the causal maps obtained. Stage 2 (Chapter 3): Elicitation of the causal map describing the occurrence of the event of interest for each dependability attribute. The causal mechanisms for the “event of interest” were extracted for each of the software dependability attributes. The “event of interest” for a dependability attribute is generally considered to be the “attribute failure”, e.g. security failure. The extraction was based on the analysis of expert elicitation results obtained in Stage 1. Stage 3 (Chapter 4): Identification of relevant measurements. Measures for the “events of interest” and their causal mechanisms were obtained from expert opinion elicitation for each of the software dependability attributes. The measures extracted are presented in this chapter. Stage 4 (Chapter 5): Assessment of the coverage of the causal maps via measures. Coverage was assessed to determine whether the measures obtained were sufficient to quantify software dependability, and what measures are further required. Stage 5 (Chapter 6): Identification of “missing” measures and measurement approaches for concepts not covered. New measures, for concepts that had not been covered sufficiently as determined in Stage 4, were identified using supplementary expert opinion elicitation as well as literature reviews. Stage 6 (Chapter 7): Building of a detailed quantification model based on the causal maps and measurements obtained. Ability to derive such a quantification model shows that the causal models and measurements derived from the previous stages (Stage 1 to Stage 5) can form the technical basis for developing dependability quantification models. Scope restrictions have led us to prioritize this demonstration effort. The demonstration was focused on a critical system, i.e. the reactor protection system. For this system, a ranking of the software dependability attributes by nuclear stakeholders was developed. As expected for this application, the stakeholder ranking identified safety as the most critical attribute to be quantified. A safety quantification model limited to the requirements phase of development was built. Two case studies were conducted for verification. A preliminary control gate for software safety for the requirements stage was proposed and applied to the first case study. The control gate allows a cost effective selection of the duration of the requirements phase.
This paper is a work of fiction, but it is fiction with a very real purpose: to stimulate careful thought and friendly discussion about some questions for which thought is often careless and discussion is often unfriendly. To accomplish this purpose, the paper creates a fictional legal case. The most important issue in this fictional case is whether certain proffered expert testimony about software engineering for safety critical systems should be admitted. Resolving this issue requires deciding the extent to which current practices and research in software engineering, especially for safety-critical systems, can rightly be considered based on knowledge, rather than opinion.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
The National Aeronautics and Space Administration (NASA) is asking more of its human spaceflight programs than ever before through the collective Artemis Missions. The NASA Independent Verification and Validation (IV&V) Program contributes to NASA’s human spaceflight goals by providing IV&V services for NASA’s critical spacecraft and ground software. The IV&V Program is tasked with providing assurance from both individual and integrated mission software perspectives. The Artemis IV&V organization is actively supporting six distinct development efforts: Orion, the Space Launch System (SLS), Exploration Ground Systems (EGS), Mission Control Center (MCC), the Lunar Gateway, and the Human Landing System (HLS), representing a wide diversity of developer organizations, management structures, and development approaches. With much of this extremely complex flight and ground software being essential to human safety both on the ground and in space, Artemis IV&V is likewise challenged to provide more value-added assurance to future Artemis missions within a constrained budget. To meet this challenge, Artemis IV&V employs a variety of novel and evolving “Adaptive IV&V” approaches for planning and executing IV&V analysis to increase both the efficiency and effectiveness of the IV&V Program’s assurance activities, and to address the difficulties imposed by assuring software for a large, highly integrated, multi-mission enterprise managed and executed by physically and organizationally distinct programs. Instilling agile principles like iterative planning cycles, self-organizing teams, and regular retrospectives, into IV&V planning and execution has led to a more rapid turnaround of a minimum viable assurance product and allowed for increased alignment of assurance activities with development progress. Adopting an assurance case methodology has led to greater consistency and clearer communication of assurance design and provided a foundation for long-term maintenance of assurance plans, products, and results across missions. The IV&V-developed Assurance / Safety Case Analytical Network (A-SCAN) framework and tool has enabled the quantification and tracking of system/software risk and confidence. These confidence measures provide a means to repeatedly express the impact of planned and completed assurance work and the remaining residual risk. Applied as part of a “Follow-the-Risk” organizational ethos, this allows consistent rightsizing of analysis rigor and intensity commensurate with the perceived risk of defects, as well as appropriate targeting of the highest risk areas of the software to find safety issues before they can manifest. Finally, the development of the IV&V Advanced Risk Reduction Integrated Software Test and Operations Tri-program Lightweight Environment (ARRISTOTLE), an integrated software-only simulation of Orion, SLS, and EGS systems, has made it possible to independently test integrated pad and flight scenarios and inject faults to observe how the Artemis multi-program, mission software behaves in degraded modes and in response to hazards. These adaptive IV&V investments have enabled Artemis IV&V to become more efficient and effective in IV&V planning and execution and respond more readily to changes in the risk landscape, increasing the breadth and depth of risk reduction possible within the available resources. Residual risk tracking allows IV&V to communicate more effectively with stakeholders, both internal and external at all levels, and inform key decision-making personnel. This evolving assurance design approach provides IV&V surety that work is performed in the highest risk, most value-added areas of the software, to keep our astronauts and ground crews safe and ensure mission success.
The National Aeronautics and Space Administration (NASA) is asking more of its human spaceflight programs than ever before through the collective Artemis Missions. The NASA Independent Verification and Validation (IV&V) Program contributes to NASA’s human spaceflight goals by providing IV&V services for NASA’s critical spacecraft and ground software. The IV&V Program is tasked with providing assurance from both individual and integrated mission software perspectives. The Artemis IV&V organization is actively supporting six distinct development efforts: Orion, the Space Launch System (SLS), Exploration Ground Systems (EGS), Mission Control Center (MCC), the Lunar Gateway, and the Human Landing System (HLS), representing a wide diversity of developer organizations, management structures, and development approaches. With much of this extremely complex flight and ground software being essential to human safety both on the ground and in space, Artemis IV&V is likewise challenged to provide more value-added assurance to future Artemis missions within a constrained budget. To meet this challenge, Artemis IV&V employs a variety of novel and evolving “Adaptive IV&V” approaches for planning and executing IV&V analysis to increase both the efficiency and effectiveness of the IV&V Program’s assurance activities, and to address the difficulties imposed by assuring software for a large, highly integrated, multi-mission enterprise managed and executed by physically and organizationally distinct programs. Instilling agile principles like iterative planning cycles, self-organizing teams, and regular retrospectives, into IV&V planning and execution has led to a more rapid turnaround of a minimum viable assurance product and allowed for increased alignment of assurance activities with development progress. Adopting an assurance case methodology has led to greater consistency and clearer communication of assurance design and provided a foundation for long-term maintenance of assurance plans, products, and results across missions. The IV&V-developed Assurance / Safety Case Analytical Network (A-SCAN) framework and tool has enabled the quantification and tracking of system/software risk and confidence. These confidence measures provide a means to repeatedly express the impact of planned and completed assurance work and the remaining residual risk. Applied as part of a “Follow-the-Risk” organizational ethos, this allows consistent rightsizing of analysis rigor and intensity commensurate with the perceived risk of defects, as well as appropriate targeting of the highest risk areas of the software to find safety issues before they can manifest. Finally, the development of the IV&V Advanced Risk Reduction Integrated Software Test and Operations Tri-program Lightweight Environment (ARRISTOTLE), an integrated software-only simulation of Orion, SLS, and EGS systems, has made it possible to independently test integrated pad and flight scenarios and inject faults to observe how the Artemis multi-program, mission software behaves in degraded modes and in response to hazards. These adaptive IV&V investments have enabled Artemis IV&V to become more efficient and effective in IV&V planning and execution and respond more readily to changes in the risk landscape, increasing the breadth and depth of risk reduction possible within the available resources. Residual risk tracking allows IV&V to communicate more effectively with stakeholders, both internal and external at all levels, and inform key decision-making personnel. This evolving assurance design approach provides IV&V surety that work is performed in the highest risk, most value-added areas of the software, to keep our astronauts and ground crews safe and ensure mission success.
System safety is primarily concerned with epistemic questions, that is, questions concerning knowledge and the degree of confidence that can be placed in that knowledge. For systems with which human experience is long, such as roads, bridges, and mechanical devices, knowledge about what is required to make the systems safe is deep and detailed. High confidence can be placed in the validity of that knowledge. For other systems, however, with which human experience is comparatively short, such as those that rely in part or in whole on software, knowledge about what is required to ensure safety tends to be shallow and general. The confidence that can be placed in the validity of that knowledge is consequently low. In a previous paper, we enumerated a collection of foundational epistemic questions concerning software system safety. In this paper, we review and refine the questions, discuss some difficulties that attend to answering the questions today, and speculate on possible research to improve the situation.
This paper describes the role of requirements discovery during the testing of a safety-critical sofiware system.
The NASA Langley Research Center (LaRC), Cockpit Motion Facility (CMF) motion base design has taken advantage of inherent hydraulic characteristics to implement safety features using hardware solutions only. Motion system safety has always been a concern and its implementation is addressed differently by each organization. Some approaches rely heavily on software safety features. Software which performs safety functions is subject to more scrutiny making its approval, modification, and development time consuming and expensive. The NASA LaRC's CMF motion system is used for research and, as such, requires that the software be updated or modified frequently. The CMF's customers need the ability to update the simulation software frequently without the associated cost incurred with safety critical software. This paper describes the CMF engineering team's approach to achieving motion base safety by designing and implementing all safety features in hardware, resulting in applications software (including motion cueing and actuator dynamic control) being completely independent of the safety devices. This allows the CMF safety systems to remain intact and unaffected by frequent research system modifications.
We describe a generic approach for automatically integrating the output generated from a formal method/tool into a software safety assurance case, as an evidence argument, by (a) encoding the underlying reasoning as a safety case pattern, and (b) instantiating it using the data produced from the method/tool. We believe this approach not only improves the trustworthiness of the evidence generated from a formal method/tool, by explicitly presenting the reasoning and mechanisms underlying its genesis, but also provides a way to gauge the suitability of the evidence in the context of the wider assurance case. We illustrate our work by application to a real example-an unmanned aircraft system- where we invoke a formal code analysis tool from its autopilot software safety case, automatically transform the verification output into an evidence argument, and then integrate it into the former.
SAS4A/SASSYS-1 (SAS) is a fast-running simulation tool used to perform deterministic analysis of anticipated events as well as design basis and beyond design basis accidents for advanced liquid-metal-cooled nuclear reactors. It is a critical element of safety analysis capabilities for the U.S. Department of Energy and is utilized within industry to perform the transient safety analyses required to support the licensing of Liquid Metal-cooled Fast Reactors (LMFRs). Although SAS is exceptionally fast for most transient scenarios, fuel performance calculations, along with the associated pre-transient characterization of the fuel pin, may be required for transient scenarios where fuel pin failure is hypothesized. Both the pre-transient characterization and the transient fuel performance calculation are necessary to properly quantify margins to potential fuel failure and assess the time spent potentially exceeding such margins during events. While safety analysis calculations with fuel performance models provide a more detailed characterization of the reactor during a transient, the pre-transient characterization can be time-consuming and computationally expensive. Often, large numbers of fuel pins have been exposed to similar pre-transient irradiation conditions. Similarly, the same pre-transient fuel characterization may be applicable to numerous transient conditions. This provides an opportunity to optimize the SAS computational framework such that pre-transient fuel characterization can be shared across multiple channels (fuel pins) and across multiple simulations, thus dramatically reducing overall computational costs. This report summarizes progress toward enhancing the SAS computational framework to support shared, multiple channel fuel performance characterizations intended to significantly reduce computational costs. Preliminary testing has shown that the computational time saved by using the pre-transient sharing capability is approximately equal to the time it takes to perform the pre-transient characterization.