Issues and Methods for Assessing COTS Reliability, Maintainability, and Availability
This discussion takes the viewpoint of how the customer can ensure the quality of COTS components.
COTS↗
SEARCH · Engineering Papers
Results for “software reliability”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Experiences in improving the state of the practice in verification and validation of knowledge-based systems
Knowledge-based systems (KBS's) are in general use in a wide variety of domains, both commercial and government. As reliance on these types of systems grows, the need to assess their quality and validity reaches critical importance. As with any software, the reliability of a KBS can be directly attributed to the application of disciplined programming and testing practices throughout the development life-cycle. However, there are some essential differences between conventional software and KBSs, both in construction and use. The identification of these differences affect the verification and validation (V&V) process and the development of techniques to handle them. The recognition of these differences is the basis of considerable on-going research in this field. For the past three years IBM (Federal Systems Company - Houston) and the Software Technology Branch (STB) of NASA/Johnson Space Center have been working to improve the 'state of the practice' in V&V of Knowledge-based systems. This work was motivated by the need to maintain NASA's ability to produce high quality software while taking advantage of new KBS technology. To date, the primary accomplishment has been the development and teaching of a four-day workshop on KBS V&V. With the hope of improving the impact of these workshops, we also worked directly with NASA KBS projects to employ concepts taught in the workshop. This paper describes two projects that were part of this effort. In addition to describing each project, this paper describes problems encountered and solutions proposed in each case, with particular emphasis on implications for transferring KBS V&V technology beyond the NASA domain.
Culbert, Chris↗
The process group approach to reliable distributed computing
The difficulty of developing reliable distribution software is an impediment to applying distributed computing technology in many settings. Experience with the ISIS system suggests that a structured approach based on virtually synchronous process groups yields systems that are substantially easier to develop, exploit sophisticated forms of cooperative computation, and achieve high reliability. Six years of research on ISIS, describing the model, its implementation challenges, and the types of applications to which ISIS has been applied are reviewed.
Birman, Kenneth P.↗
Difficult Decisions Made Easier
NASA missions are extremely complex and prone to sudden, catastrophic failure if equipment falters or if an unforeseen event occurs. For these reasons, NASA trains to expect the unexpected. It tests its equipment and systems in extreme conditions, and it develops risk-analysis tests to foresee any possible problems. The Space Agency recently worked with an industry partner to develop reliability analysis software capable of modeling complex, highly dynamic systems, taking into account variations in input parameters and the evolution of the system over the course of a mission. The goal of this research was multifold. It included performance and risk analyses of complex, multiphase missions, like the insertion of the Mars Reconnaissance Orbiter; reliability analyses of systems with redundant and/or repairable components; optimization analyses of system configurations with respect to cost and reliability; and sensitivity analyses to identify optimal areas for uncertainty reduction or performance enhancement.
Source record↗
Graphical workstation capability for reliability modeling
In addition to computational capabilities, software tools for estimating the reliability of fault-tolerant digital computer systems must also provide a means of interfacing with the user. Described here is the new graphical interface capability of the hybrid automated reliability predictor (HARP), a software package that implements advanced reliability modeling techniques. The graphics oriented (GO) module provides the user with a graphical language for modeling system failure modes through the selection of various fault-tree gates, including sequence-dependency gates, or by a Markov chain. By using this graphical input language, a fault tree becomes a convenient notation for describing a system. In accounting for any sequence dependencies, HARP converts the fault-tree notation to a complex stochastic process that is reduced to a Markov chain, which it can then solve for system reliability. The graphics capability is available for use on an IBM-compatible PC, a Sun, and a VAX workstation. The GO module is written in the C programming language and uses the graphical kernal system (GKS) standard for graphics implementation. The PC, VAX, and Sun versions of the HARP GO module are currently in beta-testing stages.
Bavuso, Salvatore J.↗
Assembly and Integration Status of a High Fidelity Ground Test Bed for the Water Processor Assembly
The Water Recovery System (WRS) is a critical component of life support aboard the International Space Station (ISS) and will play an essential role in future missions beyond Low Earth Orbit (LEO). Its primary functional units – the Urine Processor Assembly (UPA), Brine Processor Assembly (BPA), and Water Processor Assembly (WPA) – must be evaluated for extended operation, dormancy resilience, material obsolescence, and reliability under exploration-driven constraints. Ground testing is vital for developing these technologies and generating statistically relevant reliability assessments, which requires extended runtime under integrated, Flight-like conditions. Currently, no high-fidelity, fully integrated WPA ground test bed exists to support these objectives. To address this gap, NASA is developing a WPA test bed at Marshall Space Flight Center (MSFC) that combines downgraded ISS flight hardware with functionally flight-like components in a cost-effective configuration while maintaining priority hardware investigations. This paper describes the current status of hardware assembly and integration, outlines key challenges such as simulating microgravity effects and mitigating obsolescence, and presents future test objectives including software development, reliability assessments, dormancy studies, and exploration-oriented upgrades.
Mary-Elizabeth Davis↗
Assembly and Integration Status of a High Fidelity Ground Test Bed for the Water Processor Assembly
The Water Recovery System (WRS) is a critical component of life support aboard the International Space Station (ISS) and will play an essential role in future missions beyond Low Earth Orbit (LEO). Its primary functional units – the Urine Processor Assembly (UPA), Brine Processor Assembly (BPA), and Water Processor Assembly (WPA) – must be evaluated for extended operation, dormancy resilience, material obsolescence, and reliability under exploration-driven constraints. Ground testing is vital for developing these technologies and generating statistically relevant reliability assessments, which requires extended runtime under integrated, Flight-like conditions. Currently, no high-fidelity, fully integrated WPA ground test bed exists to support these objectives. To address this gap, NASA is developing a WPA test bed at Marshall Space Flight Center (MSFC) that combines downgraded ISS flight hardware with functionally flight-like components in a cost-effective configuration while maintaining priority hardware investigations. This paper describes the current status of hardware assembly and integration, outlines key challenges such as simulating microgravity effects and mitigating obsolescence, and presents future test objectives including software development, reliability assessments, dormancy studies, and exploration-oriented upgrades.
Water Processor Assembly↗
Investigation of the Use of Dynamic Probabilistic Risk Assessment Methodologies for Identifying Digital I&C System Common Cause Failures
Digital Instrumentation and Control (I&C) systems have a key role in nuclear power plants in the upgrade of aging analog systems. Digital systems improve plant safety and reliability through features such as increased hardware reliability and stability and improved failure detection capability. There is no consensus on which of the current probabilistic risk assessment methods are most suitable for use in the reliability analysis of digital I&C systems. While the traditional event-tree/fault-tree (ET/FT) approach is still used for their reliability modeling, there are concerns regarding this approach in properly accounting for dynamic interactions among system components since potentially significant dependencies among failure events may not be identified and/or their likelihood may not be properly quantified. Dynamic methodologies are expected to provide a much more accurate representation of probabilistic evolution of the I&C systems in time due to their capability to more properly account for complex interactions than the static approach. The applicability of dynamic PRA methodologies for digital I&C system is investigated using the criteria presented in the NUREG/CR-6901, and the comparisons made in NUREG/CR-6901 are updated in light of the latest studies. The Dynamic Event Tree (DET) approach has been identified as one of the top dynamic methods when evaluated against the requirements for the reliability modeling of digital I&C systems. The DET method is a strong candidate for integration into existing PRA studies, as it bears many similarities to the traditional ET approach. In this study, the DET approach has been applied to the Plant Protection System of the APR1400 design, and the results are compared to results from its available traditional ET/FT analysis. Possible approaches to evaluate and quantify the effects of common cause failures on system safety using dynamic methods are also examined.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
On requirements for software fault tolerance for flight controls
The need for the application of software fault tolerance techniques in digital flight control systems is argued to follow from the requirements derivable from the safety constraints of such systems, requirements which can be stated in terms of minimum acceptable system reliability levels and, moreover, stated quantitatively. It is argued further that, while fault tolerance appears to be a viable mechanism in general, individual fault tolerance schemes need to be analyzed to ensure that they are adequate to the task and being properly utilized, that such analysis is essentially an exercise in software 'reliability' estimation involving software characteristics not currently included in software 'reliability' modeling (most especially, the degree of correlation of malfunctions among redundant, dissimilar software modules), and that, consequently, further research and studies in the characterization of software behavior and malfunctions is required.
Migneault, G. E.↗
Towards understanding the formation of internal fragments generated by collisionally activated dissociation for top-down mass spectrometry
Top-down mass spectrometry (TD-MS) generates fragment ions that returns information on the polypeptide amino acid sequence. In addition to terminal fragments, internal fragments that result from multiple cleavage events can also be formed. Traditionally, internal fragments are largely ignored due to a lack of available software to reliably assign them, mainly caused by a poor understanding of their formation mechanism. To accurately assign internal fragments, their formation process needs to be better understood. Here we applied a statistical method to compare fragmentation patterns of internal and terminal fragments of peptides and proteins generated by collisionally activated dissociation (CAD). Internal fragments share similar fragmentation propensities with terminal fragments (e.g., enhanced cleavages N-terminal to proline and C-terminal to acidic residues), suggesting that their formation follows conventional CAD pathways. Internal fragments should be generated by subsequent cleavages of terminal fragments and their formation can be explained by the well-known mobile proton model. Additionally, internal fragments can be coupled with terminal fragments to form complementary product ions that span the entire protein sequence. These enhance our understanding of internal fragment formation and can help improve sequencing algorithms to accurately assign internal fragments, which will ultimately lead to more efficient and comprehensive TD-MS analysis of proteins and proteoforms.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
In Their Shoes: Persona-Based Approaches to Software Quality Practice Incentivization
Many teams struggle to adapt and right-size software engineering best practices for quality assurance to fit their context. Introducing software quality is not usually framed in a way that motivates teams to take action, thus resulting in it becoming a “check the box for compliance” activity instead of a cultural practice that values software quality and the effort to achieve it. When and how can we provide effective incentives for software teams to adopt and integrate meaningful and enduring software quality practices? Here, we explored this question through a persona-based ideation exercise at the 2021 Collegeville Workshop on Scientific Software in which we created three unique personas that represent different scientific software developer perspectives.
97 MATHEMATICS AND COMPUTING↗
Machine learning based job status prediction in scientific clusters
Large high-performance computing systems are built with increasing number of components with more CPU cores, more memory, and more storage space. At the same time, scientific applications have been growing in complexity. Together, they are leading to more frequent unsuccessful job statuses on HPC systems. From measured job statuses, 23.4% of CPU time was spent to the unsuccessful jobs. Here, we set out to study whether these unsuccessful job statuses could be anticipated from known job characteristics. To explore this possibility, we have developed a job status prediction method for the execution of jobs on scientific clusters. The Random Forests algorithm was applied to extract and characterize the patterns of unsuccessful job statuses. Experimental results show that our method can predict the unsuccessful job statuses from the monitored ongoing job executions in 99.8% the cases with 83.6% recall and 94.8% precision. Lastly, this prediction accuracy can be sufficiently high that it can be used to mitigation procedures of predicted failures.
97 MATHEMATICS AND COMPUTING↗
Advanced High-Performance Computational Modeling of the Seismic Response of High-Hazard and/or Nuclear Facilities and Critical Infrastructure at the NNSS
New methods for predicting the amplitude and variability of ground shaking from earthquakes (and explosions) are needed for seismic hazard analysis for buildings, nuclear power plants, and critical infrastructure at the NNSS. We are comparing existing 1-D and new 3-D geophysical methods for estimating the shear-wave velocity structure in the upper 30 meters of the ground surface (Vs30), which plays a major role in ground motion amplification and seismic response of buildings. We evaluate the performance of these methodologies at the U1a Complex at the NNSS and develop simple 1-D and high-resolution 3-D Vs30 models. We then emplace these high-resolution models into a background seismic velocity model. We will collaborate with Lawrence Livermore National Laboratory (LLNL) to conduct numerical modeling of the ground shaking at the NNSS using their high-performance computing technology and state-of-the-art ground motion simulation methodology. The primary work that was completed in FY 2019 was to acquire the seismic systems and familiarize staff at the NNSS with their use. We also worked on developing a collection plan with the Device Assembly Facility (DAF) at the NNSS, but due to time constraints and other ongoing projects at the DAF, we had to use U1a as a backup. We were able to coordinate the seismic survey, and we will complete the collection of seismic data in FY 2020. Additionally, during FY 2019, we completed the geologic framework model (GFM) for the U1a Complex and modeled the Yucca fault. LLNL worked with us through FY 2019 to prepare the data files for their modeling software and tested the software for reliability. In FY 2020 we will develop the end-to-end capability so that any facility could easily be modeled and the expected shaking from a local earthquake understood. The work in FY 2020 will include building fault models from the GFM and finalizing the velocity model analysis. The final simulations will be run for multiple rupture models, and final assessments will demonstrate the seismic hazard at the U1a Complex.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
LWRS Efforts on Qualitative and Quantitative Risk Assessment of Digital Software Systems [Slides]
Presentation slides for distribution to interested party as part of ongoing collaboration with industry. Slides present existing LWRS efforts in the qualitative and quantitative risk assessment of digital software systems. Specifically, a discussion on the RESHA and ORCAS methodologies is presented.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
ComPort: Rigorous Testing Methods to Safeguard Software Porting (Final Technical Report)
This is a technical report from the lead institution – University of Utah, Kahlert School of Computing – funded under the Department of Energy, Office of Science, Office of Advanced Scientific Computing Research under award number DE-SC0022252. We summarize our work done over the three years of funding received. The relevant papers and software have already been uploaded at the DOE site.
97 MATHEMATICS AND COMPUTING↗
The STAR /self-testing and repairing/ computer - An investigation of the theory and practice of fault-tolerant computer design.
This paper presents the results obtained in a continuing investigation of fault-tolerant computing which is being conducted at the Jet Propulsion Laboratory. Initial studies led to the decision to design and construct an experimental computer with dynamic (standby) redundancy, including replaceable subsystems and a program rollback provision to eliminate transient errors. This system, called the STAR computer, began operation in 1969. The following aspects of the STAR system are described: architecture, reliability analysis, software, automatic maintenance of peripheral systems, and adaptation to serve as the central computer of an outer-planet exploration spacecraft.
A Avizienis↗
HAL/S - The programming language for Shuttle
HAL/S is a higher order language and system, now operational, adopted by NASA for programming Space Shuttle on-board software. Program reliability is enhanced through language clarity and readability, modularity through program structure, and protection of code and data. Salient features of HAL/S include output orientation, automatic checking (with strictly enforced compiler rules), the availability of linear algebra, real-time control, a statement-level simulator, and compiler transferability (for applying HAL/S to additional object and host computers). The compiler is described briefly.
Martin, F. H.↗
Payload operations control center network (POCCNET) systems definition phase study report
The results of the studies performed during the systems definition phase of POCCNET are presented. The concept of POCCNET as a system of standard POCCs is described and an analysis of system requirements is also included. Alternative systems concepts were evaluated as well as various methods for development of reliable reusable software. A number of POCC application areas, such as command management, on board computer support, and simulation were also studied. Other areas of investigation included the operation of POCCNET systems, the facility requirements and usage.
Desjardins, R.↗