Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software reliability software measurement software metrics software measurement”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2003

Developing Fault Models for Space Mission Software

A viewgraph presentation on the development of fault models for space mission software is shown. The topics include: 1) Goal: Improve Understanding of Technology Fault Generation Process; 2) Required Measurement; 3) Measuring Structural Evolution; 4) Module Attributes; 5) Principal Components of Raw Metrics; 6) The Measurement Process; 7) View of Structural Evolution at the System and Module Level; 8) Identifying and Counting Faults; 9) Fault Enumeration; 10) Modeling Fault Content; 11) Modeling Results; 12) Current and Future Work; and 13) Discussion and Conclusions.

software measurements↗
NASA NTRS2003

Understanding the nature of software evolution

In this paper, we analyze the measurements of structural evolution and fault counts obtained from the JPL flight software technology development effort.

software evolution software measurement software m↗
NASA NTRS1998

Software Evolution and the Fault Process

In developing a software system, we would like to estimate the way in which the fault content changes during its development, as well determine the locations having the highest concentration of faults.

software↗
NASA NTRS1990

Multiversion software reliability through fault-avoidance and fault-tolerance

In this project we have proposed to investigate a number of experimental and theoretical issues associated with the practical use of multi-version software in providing dependable software through fault-avoidance and fault-elimination, as well as run-time tolerance of software faults. In the period reported here we have working on the following: We have continued collection of data on the relationships between software faults and reliability, and the coverage provided by the testing process as measured by different metrics (including data flow metrics). We continued work on software reliability estimation methods based on non-random sampling, and the relationship between software reliability and code coverage provided through testing. We have continued studying back-to-back testing as an efficient mechanism for removal of uncorrelated faults, and common-cause faults of variable span. We have also been studying back-to-back testing as a tool for improvement of the software change process, including regression testing. We continued investigating existing, and worked on formulation of new fault-tolerance models. In particular, we have partly finished evaluation of Consensus Voting in the presence of correlated failures, and are in the process of finishing evaluation of Consensus Recovery Block (CRB) under failure correlation. We find both approaches far superior to commonly employed fixed agreement number voting (usually majority voting). We have also finished a cost analysis of the CRB approach.

Vouk, Mladen A.↗
NASA NTRS1992

Using software metrics and software reliability models to attain acceptable quality software for flight and ground support software for avionic systems

This paper is concerned with methods of measuring and developing quality software. Reliable flight and ground support software is a highly important factor in the successful operation of the space shuttle program. Reliability is probably the most important of the characteristics inherent in the concept of 'software quality'. It is the probability of failure free operation of a computer program for a specified time and environment.

Lawrence, Stella↗
NASA NTRS2004

Software Quality Assurance Metrics

Software Quality Assurance (SQA) is a planned and systematic set of activities that ensures conformance of software life cycle processes and products conform to requirements, standards and procedures. In software development, software quality means meeting requirements and a degree of excellence and refinement of a project or product. Software Quality is a set of attributes of a software product by which its quality is described and evaluated. The set of attributes includes functionality, reliability, usability, efficiency, maintainability, and portability. Software Metrics help us understand the technical process that is used to develop a product. The process is measured to improve it and the product is measured to increase quality throughout the life cycle of software. Software Metrics are measurements of the quality of software. Software is measured to indicate the quality of the product, to assess the productivity of the people who produce the product, to assess the benefits derived from new software engineering methods and tools, to form a baseline for estimation, and to help justify requests for new tools or additional training. Any part of the software development can be measured. If Software Metrics are implemented in software development, it can save time, money, and allow the organization to identify the caused of defects which have the greatest effect on software development. The summer of 2004, I worked with Cynthia Calhoun and Frank Robinson in the Software Assurance/Risk Management department. My task was to research and collect, compile, and analyze SQA Metrics that have been used in other projects that are not currently being used by the SA team and report them to the Software Assurance team to see if any metrics can be implemented in their software assurance life cycle process.

McRae, Kalindra A.↗
NASA NTRS1989

Multi-version software reliability through fault-avoidance and fault-tolerance

A number of experimental and theoretical issues associated with the practical use of multi-version software to provide run-time tolerance to software faults were investigated. A specialized tool was developed and evaluated for measuring testing coverage for a variety of metrics. The tool was used to collect information on the relationships between software faults and coverage provided by the testing process as measured by different metrics (including data flow metrics). Considerable correlation was found between coverage provided by some higher metrics and the elimination of faults in the code. Back-to-back testing was continued as an efficient mechanism for removal of un-correlated faults, and common-cause faults of variable span. Software reliability estimation methods was also continued based on non-random sampling, and the relationship between software reliability and code coverage provided through testing. New fault tolerance models were formulated. Simulation studies of the Acceptance Voting and Multi-stage Voting algorithms were finished and it was found that these two schemes for software fault tolerance are superior in many respects to some commonly used schemes. Particularly encouraging are the safety properties of the Acceptance testing scheme.

Vouk, Mladen A.↗
NASA NTRS1990

Rule groupings in expert systems

Currently, expert system shells do not address software engineering issues for developing or maintaining expert systems. As a result, large expert systems tend to be incomprehensible, difficult to debug or modify, and almost impossible to verify or validate Partitioning rule-based systems into rule groups which reflect the underlying subdomains of the problem should enhance the comprehensibility, maintainability, and reliability of expert-system software. In this paper, we investigate methods to semi-automatically structure a CLIPS rule base e into groups of rules that carry related information. We discuss three different distance metrics for measuring the relatedness of rules and describe two clustering algorithms based on these distance metrics. The results of our experiment with three sample rule bases are also presented.

Mehrotra, Mala↗
NASA NTRS1975

Automatic documentation system extension to multi-manufacturers' computers and to measure, improve, and predict software reliability

The DOMONIC system has been modified to run on the Univac 1108 and the CDC 6600 as well as the IBM 370 computer system. The DOMONIC monitor system has been implemented to gather data which can be used to optimize the DOMONIC system and to predict the reliability of software developed using DOMONIC. The areas of quality metrics, error characterization, program complexity, program testing, validation and verification are analyzed. A software reliability model for estimating program completion levels and one on which to base system acceptance have been developed. The DAVE system which performs flow analysis and error detection has been converted from the University of Colorado CDC 6400/6600 computer to the IBM 360/370 computer system for use with the DOMONIC system.

Simmons, D. B.↗
NASA NTRS1994

Understanding software faults and their role in software reliability modeling

This study is a direct result of an on-going project to model the reliability of a large real-time control avionics system. In previous modeling efforts with this system, hardware reliability models were applied in modeling the reliability behavior of this system. In an attempt to enhance the performance of the adapted reliability models, certain software attributes were introduced in these models to control for differences between programs and also sequential executions of the same program. As the basic nature of the software attributes that affect software reliability become better understood in the modeling process, this information begins to have important implications on the software development process. A significant problem arises when raw attribute measures are to be used in statistical models as predictors, for example, of measures of software quality. This is because many of the metrics are highly correlated. Consider the two attributes: lines of code, LOC, and number of program statements, Stmts. In this case, it is quite obvious that a program with a high value of LOC probably will also have a relatively high value of Stmts. In the case of low level languages, such as assembly language programs, there might be a one-to-one relationship between the statement count and the lines of code. When there is a complete absence of linear relationship among the metrics, they are said to be orthogonal or uncorrelated. Usually the lack of orthogonality is not serious enough to affect a statistical analysis. However, for the purposes of some statistical analysis such as multiple regression, the software metrics are so strongly interrelated that the regression results may be ambiguous and possibly even misleading. Typically, it is difficult to estimate the unique effects of individual software metrics in the regression equation. The estimated values of the coefficients are very sensitive to slight changes in the data and to the addition or deletion of variables in the regression equation. Since most of the existing metrics have common elements and are linear combinations of these common elements, it seems reasonable to investigate the structure of the underlying common factors or components that make up the raw metrics. The technique we have chosen to use to explore this structure is a procedure called principal components analysis. Principal components analysis is a decomposition technique that may be used to detect and analyze collinearity in software metrics. When confronted with a large number of metrics measuring a single construct, it may be desirable to represent the set by some smaller number of variables that convey all, or most, of the information in the original set. Principal components are linear transformations of a set of random variables that summarize the information contained in the variables. The transformations are chosen so that the first component accounts for the maximal amount of variation of the measures of any possible linear transform; the second component accounts for the maximal amount of residual variation; and so on. The principal components are constructed so that they represent transformed scores on dimensions that are orthogonal. Through the use of principal components analysis, it is possible to have a set of highly related software attributes mapped into a small number of uncorrelated attribute domains. This definitively solves the problem of multi-collinearity in subsequent regression analysis. There are many software metrics in the literature, but principal component analysis reveals that there are few distinct sources of variation, i.e. dimensions, in this set of metrics. It would appear perfectly reasonable to characterize the measurable attributes of a program with a simple function of a small number of orthogonal metrics each of which represents a distinct software attribute domain.

Munson, John C.↗
DOE OSTI2026

An interactive machine learning platform for analyzing multi-particle coincidence data from cold target recoil ion momentum spectroscopy

We present SCULPT (Supervised Clustering and Uncovering Latent Patterns with Training), a comprehensive software platform for analyzing tabulated high-dimensional multi-particle coincidence data from Cold Target Recoil Ion Momentum Spectroscopy (COLTRIMS) experiments. The software addresses critical challenges in modern momentum spectroscopy by integrating advanced machine learning techniques with physics-informed analysis in an interactive web-based environment. SCULPT implements uniform manifold approximation and projection for non-linear dimensionality reduction to reveal correlations in high-dimensional data. We also discuss potential extensions to deep autoencoders for feature learning and genetic programming for automated discovery of physically meaningful observables. A novel adaptive confidence scoring system provides quantitative reliability assessments by evaluating user-selected clustering quality metrics with predefined weights that reflect each metric’s robustness. The platform features configurable molecular profiles for different experimental systems, interactive visualization with selection tools, and comprehensive data filtering capabilities. Utilizing a subset of SCULPT’s capabilities, we analyze photo-double-ionization data measured using the COLTRIMS method for three-body dissociation of the D 2 O molecule, revealing distinct fragmentation channels and their correlations with physics parameters. The software’s modular architecture and web-based implementation make it accessible to the broader atomic and molecular physics community, significantly reducing the time required for complex multi-dimensional analyses. This opens the door to finding and isolating rare events exhibiting non-linear correlations on the fly during experimental measurements, which can help steer exploration and improve the efficiency of experiments.

Artificial neural networks↗
NASA NTRS1991

Rule groupings: A software engineering approach towards verification of expert systems

Currently, most expert system shells do not address software engineering issues for developing or maintaining expert systems. As a result, large expert systems tend to be incomprehensible, difficult to debug or modify and almost impossible to verify or validate. Partitioning rule based systems into rule groups which reflect the underlying subdomains of the problem should enhance the comprehensibility, maintainability, and reliability of expert system software. Attempts were made to semiautomatically structure a CLIPS rule base into groups of related rules that carry the same type of information. Different distance metrics that capture relevant information from the rules for grouping are discussed. Two clustering algorithms that partition the rule base into groups of related rules are given. Two independent evaluation criteria are developed to measure the effectiveness of the grouping strategies. Results of the experiment with three sample rule bases are presented.

Mehrotra, Mala↗
NASA NTRS2019

Airborne Radar for sUAS Sense and Avoid

A primary challenge for the safe integration of small UAS operations into the National Airspace System (NAS) is traffic deconfliction, both from manned and unmanned aircraft. The UAS Traffic Management (UTM) project being conducted at the National Aeronautics and Space Administration (NASA) considers a layered approach to separation provision, ranging from segregation of operations through airspace volumes (geofences) to autonomous sense and avoid (SAA) technologies for higher risk, densely occupied airspace. Cooperative SAA systems, such as Automatic Dependent Surveillance-Broadcast (ADS-B) and/or vehicle-to-vehicle communication systems provide significant additional risk mitigation but they fail to adequately mitigate collision risks for non-cooperative (non-transponder equipped) airborne aircraft. The RAAVIN (Radar on Autonomous Aircraft to Verify ICAROUS Navigation) flight test being conducted by NASA and the Mid-Atlantic Aviation Partnership (MAAP) was designed to investigate the applicability and performance of a prototype, commercially available sUAS radar to detect and track non-cooperative airborne traffic, both manned and unmanned. The radar selected for this research was a Frequency Modulated Continuous Wave (FMCW) radar with 120 degree azimuth and 80 degree elevation field of view operating at 24.55GHz center frequency with a 200 MHz bandwidth. The radar transmits 2 watts of power thru a Metamaterial Electronically Scanning Array antenna in horizontal polarization. When the radar is transmitting, personnel must be at least 1 meter away from the active array to limit nonionizing radiation exposure. The radar physical dimensions are 18.7cm by 12.1cm by 4.1cm and it weighs less than 820 grams making it well suited for installation on small UASs. The onboard, SAA capability, known as ICAROUS, (Independent Configurable Architecture for Reliable Operations of Unmanned Systems), developed by NASA to support sUAS operations, will provide autonomous guidance using the traffic radar tracks from the onboard radar. The RAAVIN set of studies will be conducted in three phases. The first phase included outdoor, ground-based radar evaluations performed at the Virginia Tech’s Kentland Farm testing range in Blacksburg, VA. The test was designed to measure how well the radar could detect and track a small UAS flying in the radar’s field of view. The radar was used to monitor 5 test flights consisting of outbound, inbound and crossing routes at different ranges and altitudes. The UAS flown during the ground test was the Inspire 2, a quad copter weighing less than 4250 grams (10 pounds) at maximum payload. The radar was set up to scan and track targets over its full azimuthal field of view from 0 to 40 degrees in elevation. The radar was configured to eliminate tracks generated from any targets located beyond 2000 meters from the radar and moving at velocities under 1.45 meters per second. For subsequent phases of the study the radar will be integrated with a sUAS platform to evaluate its performance in flight for SAA applications ranging from sUAS to manned GA aircraft detections and tracking. Preliminary data analysis from the first outdoor ground tests showed the radar performed well at tracking the vehicle as it flew outbound and repeatedly maintained a track out to 1000 meters (maximum 1387 meters) until the vehicle slowed to a stop to reverse direction to fly inbound. As the Inspire flew inbound tracks from beyond 800 meters, a reacquisition time delay was consistently observed between when the Inspire exceeds a speed of 1.45 meters per second and when the radar indicated an inbound target was present and maintained its track. The time delay varied between 6 seconds to over 37 seconds for the inbound flights examined, and typically resulted in about a 200 meter closure distance before the Inspire track was maintained. The radar performed well at both acquiring and tracking the vehicle as it flew crossing routes out past 400 meters across the azimuthal field of view. The radar and ICAROUS software will be integrated and flown on a BFD-1400-SE8-E UAS during the next phase of the RAAVIN project. The main goal at the conclusion of this effort is to determine if this radar technology can reliably support minimum requirements for SAA applications of sUAS. In particular, the study will measure the range of vehicle detections, lateral and vertical angular errors, false and missed/late detections, and estimated distance at closest point of approach after an avoidance maneuver is executed. This last metric is directly impacted by sensor performance and indicates its suitability for the task.

Szatkowski, George N.↗