Empirically based analysis of failures in software systems

Abstract

An empirical analysis of software-system failures is used to study several specific issues in software testing, reliability analysis, and reuse. Failure data from a large software manufacturer and a NASA production environment were collected and analyzed. The systems ranged in size from 30,000 to over 100,000 lines. The results show that (1) the first 15 percent of the test cases detected 67 percent of the high-severity failures and 50 percent of all failures; (2) multiple fault-detection and testing phases may result in a significant increase in reliability or none at all; (3) composite measures of system reliability did not adequately reflect reliability at the function or component level; (4) developers were biased toward portions of systems that would be heavily tested; (5) fault-proneness of reused or modified components was 74 percent less than that of newly developed components; and (6) systems with more reused software had lower component development effort, but not lower component fault-proneness.