Practical Issues in Implementing Software Reliability Measurement
Many ways of estimating software systems' reliability, or reliability-related quantities, have been developed over the past several years.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Many ways of estimating software systems' reliability, or reliability-related quantities, have been developed over the past several years.
The Glenn Research Center Communication Analysis Suite (GCAS) includes many analysis tools that can be used to support a wide range of scenarios. It includes a visualization tool that implements the three.js graphics library to display a three-dimensional (3D) representation of its results. This software will enable researchers, engineers, and mission planners to interact intuitively with and understand the results of their analyses, which might not be apparent from raw data. With NASA’s efforts to return humans to the Moon as a part of the Artemis missions, the GCAS has been used extensively for lunar terrain and landing system development analysis, which is vital to ensuring mission achievability and safety. This has created the need to add several significant features to the visualization tool, such as the ability to display the terrain of the lunar surface accurately and to provide information demonstrating how a given region might impact mission objectives. Many of the changes made to the visualization tool can be separated into one of three general advancements: code restructuring to adhere to modern coding standards and practices, new user camera controls for first-person and third-person perspective views, and a terrain generation feature to enable rendering highly accurate terrains based on any celestial body’s digital elevation model (DEM) in GeoTIFF format. These improvements notably elevate the visualization tool's functionality, accuracy, and user interaction while providing a robust foundation for future development.
With NASA’s goal to land the next human on the lunar surface in the next few years, it has become vitally important that we have a better understanding of how future landing spacecraft will interact with the unique properties of regolith¬¬––the layer of loose, unconsolidated dust and rock on the lunar surface¬¬––which can cause hazards like visual obstructions, particulate clouds, and cratering of the landing zone. Researchers from the Fluid Dynamics Branch at NASA’s Marshall Space Flight Center are performing plume-surface interaction (PSI) simulations between lander engine plumes and unprepared regolith surfaces, and have developed new tools to provide predictive PSI environments for various NASA projects and missions, including the Human Lander System (HLS), Commercial Lunar Payload Services (CLPS), and future Mars landers. These tools allow the researchers to determine how to best meet the simulation and time requirements for each project by varying model fidelity. The highest fidelity tool is the Gas Granular Flow Solver (Loci/GGFS) that models gas-particle multi-phase interactions to predict regolith cratering and ejection of particles into the immediate surroundings of the lander. At its highest fidelity, it can model microscopic regolith particle interactions with a particle size/shape distribution that statistically replicates actual regolith, however, to be most effective with today’s computing resources, it is currently run using only one to three equivalent particle sizes/shapes. The team also incorporated engineering models into their software suite to create production-ready hybrid tools with reduced fidelity. At the lowest fidelity, the computational fluid dynamics (CFD) code Loci/CHEM+DIGGEM can predict crater depth over time by relating local CFD-predicted surface shear stresses to a model of erosion mass flux.
This report describes the development of the system identification (SysID) rack custom application code. This code is used to automate the system identification experimental processes for the National Aeronautics and Space Administration (NASA) Combined Cycle Engine Large-Scale Inlet for Mode Transition Experiments testbed. This series of experiments took place in the NASA Glenn Research Center (GRC) 10 foot by 10 foot (10x10) Supersonic Wind Tunnel (SWT) test facility. The SysID code was developed to apply command signals to testbed actuators, receive actuator feedback signals, and acquire sensor data which is used to identify the dynamics of the experimental processes. This code development and the process identification experiments were conducted during Phase-2 testing at various operating points. These points were predefined from data collected during the Phase-1 testbed characterization experiments. The developed SysID code was executed on a NASA designed and built real-time data acquisition (DAQ) and controls (SysID) rack. This report focuses on the development of the SysID rack real-time DAQ and control hardware and the custom application code used to automate the system identification process. The implementation of this reliable and efficient data acquisition and controls development tool is demonstrated by presenting an example of experimental data collected to be used for system identification.
Computational fluid dynamics (CFD) code developed for use in design analyses of flow instabilities associated with combustion of sprayed liquid propellants in rocket engines. Code also contributes to design of improved commercial sprayed-fuel combustors in furnaces and jet engines. Proves robust, user-friendly software tool with comprehensive analysis capability. Enables characterization of stability or instability of engine in terms of such physically meaningful parameters as initial conditions of spray, spatial distribution of ratio between concentrations of fuel and oxidizer at injector faces, geometry of combustor, and configurations of baffles.
This paper describes the application of formal methods and object-oriented modeling to reverse engineering, in which formal specifications are developed for existing, or legacy, code. In this project, several layers of formal specifications were constructed for a portion of the NASA Space Shuttle Digital Auto Pilot (DAP), a software module that is used to control the position of the spacecraft through appropriate jet firings.
A semi-empirical theoretical model and a C++ computer program that implements the model have been developed for use in predicting the noise generated by a rocket or jet engine. The computer program, entitled the Realtime Rocket and Jet Engine Noise Analysis and Prediction Software, is one of two main subsystems of the Acoustic Prediction/Measurement Tool, which comprises software, acoustic instrumentation, and electronic hardware combined to afford integrated capabilities for real-time prediction and measurement of noise emitted by rocket and jet engines. [The other main subsystem, consisting largely of acoustic instrumentation and electronic hardware, is described in Wireless Acoustic Measurement System, which appears elsewhere in this section.] The theoretical model was derived from the fundamental laws of fluid mechanics, as first was done by M. J. Lighthill in his now famous theory of aerodynamically generated sound. The far-field approximation of the Lighthill theory is incorporated into this model. Many other contributions from various researchers have also been introduced into the model. The model accounts for two noise components: shear noise and self noise. The final result of the model is expressed in terms of a volume integral of the acoustic intensities attributable to these two components, subject to various directivity coefficients. The computer program was written to solve the volume integral. The inputs required by the program are two data files from a computational fluid dynamics (CFD) simulation of the flow of interest: the computational-grid file and the solution file. The CFD solution should be one that has been obtained for conditions that closely approximate those of an experimental test that is yet to be performed. In the current state of development of the model and software, it is recommended that the observation points lie along a radius at an angle >60 from the jet axis. The software provides, and is driven via, a graphical user interface, which facilitates its use. Optionally, the program accepts additional input in the form of data on the measured sound pressure level as a function of frequency at a given far-field location, preferably at an angle of 90 from the jet axis. The user is prompted to use default empirical constants or to choose constants based the measurement data. The user can view the results and compare them with other computational or experimental data. Once satisfied with the results, the user can save a graph of the results in a file that can be imported into documents.
We describe an approach and existing software tool support for effective interactions between human factors engineers and systems engineers in early analysis activities during system acquisition. We examine the tasks performed during this stage, emphasizing those tasks where system engineers and human engineers interact. The Concept of Operations (ConOps) document is an important product during this phase, and particular attention is paid to its influences on subsequent acquisition activities. Understanding this influence helps ConOps authors describe a complete system concept that guides subsequent acquisition activities. We identify commonly used system engineering and human engineering tools and examine how they can support the specific tasks associated with system definition. We identify possible gaps in the support of these tasks, the largest of which appears to be creating the ConOps document itself. Finally, we outline the goals of our future empirical investigations of tools to support system concept definition.
We describe an approach and existing software tool support for effective interactions between human factors engineers and systems engineers in early analysis activities during system acquisition. We examine the tasks performed during this stage, emphasizing those tasks where system engineers and human engineers interact. The Concept of Operations (ConOps) document is an important product during this phase, and particular attention is paid to its influences on subsequent acquisition activities. Understanding this influence helps ConOps authors describe a complete system concept that guides subsequent acquisition activities. We identify commonly used system engineering and human engineering tools and examine how they can support the specific tasks associated with system definition. We identify possible gaps in the support of these tasks, the largest of which appears to be creating the ConOps document itself. Finally, we outline the goals of our future empirical investigations of tools to support system concept definition.
The Integrated Modeling Environment (IME) is a software system that establishes a centralized Web-based interface for integrating people (who may be geographically dispersed), processes, and data involved in a common engineering project. The IME includes software tools for life-cycle management, configuration management, visualization, and collaboration.
The Eli system, a knowledge-based reusable software synthesis system, is being developed for NASA Langley under a Phase 2 SBIR contract. Named after Eli Whitney, the inventor of interchangeable parts, Eli assists engineers of large-scale software systems in reusing components while they are composing their software specifications or designs. Eli will identify reuse potential, search for components, select component variants, and synthesize components into the developer's specifications. The Eli project began as a Phase 1 SBIR to define a reusable software synthesis methodology that integrates reusabilityinto the top-down development process and to develop an approach for an expert system to promote and accomplish reuse. The objectives of the Eli Phase 2 work are to integrate advanced technologies to automate the development of reusable components within the context of large system developments, to integrate with user development methodologies without significant changes in method or learning of special languages, and to make reuse the easiest operation to perform. Eli will try to address a number of reuse problems including developing software with reusable components, managing reusable components, identifying reusable components, and transitioning reuse technology. Eli is both a library facility for classifying, storing, and retrieving reusable components and a design environment that emphasizes, encourages, and supports reuse.
Hardware and software systems developed to further research and education in interactive computer graphics were described, as well as several of the ongoing application-oriented projects, educational graphics programs, and graduate research projects. The software system consists of a FORTRAN 4 subroutine package, in conjunction with a PDP 11/40 minicomputer as the primary computation processor and the Imlac PDS-1 as an intelligent display processor. The package comprises a comprehensive set of graphics routines for dynamic, structured two-dimensional display manipulation, and numerous routines to handle a variety of input devices at the Imlac.
Vencore is a proven information solutions, engineering, and analytics company that helps our customers solve their most complex challenges. For more than 40 years, we have designed, developed and delivered mission-critical solutions as our customers' trusted partner. The Engineering Services Contract, or ESC, provides engineering and design services to the NASA organizations engaged in development of new technologies at the Kennedy Space Center. Vencore is the ESC prime contractor, with teammates that include Stinger Ghaffarian Technologies, Sierra Lobo, Nelson Engineering, EASi, and Craig Technologies. The Vencore team designs and develops systems and equipment to be used for the processing of space launch vehicles, spacecraft, and payloads. We perform flight systems engineering for spaceflight hardware and software; develop technologies that serve NASA's mission requirements and operations needs for the future. Our Flight Payload Support (FPS) team at Kennedy Space Center (KSC) provides engineering, development, and certification services as well as payload integration and management services to NASA and commercial customers. Our main objective is to assist principal investigators (PIs) integrate their science experiments into payload hardware for research aboard the International Space Station (ISS), commercial spacecraft, suborbital vehicles, parabolic flight aircrafts, and ground-based studies. Vencore's FPS team is AS9100 certified and a recognized implementation partner for the Center for Advancement of Science in Space (CASIS
Design of large precision optical systems requires multi-disciplinary analysis, modeling, and design. Thermal, structural and optical characteristics of the hardware must be accurately understood in order to design a system capable of accomplishing the performance requirements. The interactions between each of the disciplines become stronger as systems are designed lighter weight for space applications. This coupling dictates a concurrent engineering design approach. In the past, integrated modeling tools have been developed that attempt to integrate all of the complex analysis within the framework of a single model. This often results in modeling simplifications and it requires engineering specialist to learn new applications. The software described in this presentation addresses the concurrent engineering task using a different approach. The software tool, Integrated Optical Design Analysis (IODA), uses data fusion technology to enable a cross discipline team of engineering experts to concurrently design an optical system using their standard validated engineering design tools.
The past several decades have seen the rise of large, highly interactive systems that are on the forward edge of technology. As a result of this growth and the increased usage of digital systems (computers and software), the concept of systems engineering has gained increasing attention. Some of this attention is no doubt due to large program failures which possibly could have been avoided, or at least mitigated, through the use of systems engineering principles. The complexity of modern day weapon systems requires conscious application of systems engineering concepts to ensure producible, operable and supportable systems that satisfy mission requirements. Although many authors have traced the roots of systems engineering to earlier dates, the initial formalization of the systems engineering process for military development began to surface in the mid-1950s on the ballistic missile programs. These early ballistic missile development programs marked the emergence of engineering discipline 'specialists' which has since continued to grow. Each of these specialties not only has a need to take data from the overall development process, but also to supply data, in the form of requirements and analysis results, to the process. A number of technical instructions, military standards and specifications, and manuals were developed as a result of these development programs. In particular, MILSTD-499 was issued in 1969 to assist both government and contractor personnel in defining the systems engineering effort in support of defense acquisition programs. This standard was updated to MIL-STD499A in 1974, and formed the foundation for current application of systems engineering principles to military development programs.
The study, formulation, and generation of structures for Ada (GRASP/Ada) are discussed in this second phase report of a three phase effort. Various graphical representations that can be extracted or generated from source code are described and categorized with focus on reverse engineering. The overall goal is to provide the foundation for a CASE (computer-aided software design) environment in which reverse engineering and forward engineering (development) are tightly coupled. Emphasis is on a subset of architectural diagrams that can be generated automatically from source code with the control structure diagram (CSD) included for completeness.
Project Constellation implements NASA's vision for space exploration to expand human presence in our solar system. The engineering focus of this project is developing a system of systems architecture. This architecture allows for the incremental development of the overall program. Systems can be built and connected in a "Lego style" manner to generate configurations supporting various mission objectives. The development of the avionics or control systems of such a massive project will result in concurrent engineering. Also, each system will have software and the need to communicate with other (possibly heterogeneous) systems. Fortunately, this design problem has already been solved during the creation and evolution of systems such as the Internet and the Department of Defense's successful effort to standardize distributed simulation (now IEEE 1516). The solution relies on the use of a standard layered software framework and a communication protocol. A standard framework and communication protocol is suggested for the development and maintenance of Project Constellation systems. The ARINC 653 standard is a great start for such a common software framework. This paper proposes a common system software framework that uses the Real Time Publish/Subscribe protocol for framework-to-framework communication to extend ARINC 653. It is highly recommended that such a framework be established before development. This is important for the success of concurrent engineering. The framework provides an infrastructure for general system services and is designed for flexibility to support a spiral development effort.
The NASA OPAD spectrometer system relies heavily on extensive software which repetitively extracts spectral information from the engine plume and reports the amounts of metals which are present in the plume. The development of this software is at a sufficiently advanced stage where it can be used in actual engine tests to provide valuable data on engine operation and health. This activity will continue and, in addition, the OPAD system is planned to be used in flight aboard space vehicles. The two implementations, test-stand and in-flight, may have some differing requirements. For example, the data stored during a test-stand experiment are much more extensive than in the in-flight case. In both cases though, the majority of the requirements are similar. New data from the spectrograph is generated at a rate of once every 0.5 sec or faster. All processing must be completed within this period of time to maintain real-time performance. Every 0.5 sec, the OPAD system must report the amounts of specific metals within the engine plume, given the spectral data. At present, the software in the OPAD system performs this function by solving the inverse problem. It uses powerful physics-based computational models (the SPECTRA code), which receive amounts of metals as inputs to produce the spectral data that would have been observed, had the same metal amounts been present in the engine plume. During the experiment, for every spectrum that is observed, an initial approximation is performed using neural networks to establish an initial metal composition which approximates as accurately as possible the real one. Then, using optimization techniques, the SPECTRA code is repetitively used to produce a fit to the data, by adjusting the metal input amounts until the produced spectrum matches the observed one to within a given level of tolerance. This iterative solution to the original problem of determining the metal composition in the plume requires a relatively long period of time to execute the software in a modern single-processor workstation, and therefore real-time operation is currently not possible. A different number of iterations may be required to perform spectral data fitting per spectral sample. Yet, the OPAD system must be designed to maintain real-time performance in all cases. Although faster single-processor workstations are available for execution of the fitting and SPECTRA software, this option is unattractive due to the excessive cost associated with very fast workstations and also due to the fact that such hardware is not easily expandable to accommodate future versions of the software which may require more processing power. Initial research has already demonstrated that the OPAD software can take advantage of a parallel computer architecture to achieve the necessary speedup. Current work has improved the software by converting it into a form which is easily parallelizable. Timing experiments have been performed to establish the computational complexity and execution speed of major components of the software. This work provides the foundation of future work which will create a fully parallel version of the software executing in a shared-memory multiprocessor system.