Engineering Papers⌕ Search

Engineering topics

Nikora, Allen P.

Publications and source records attributed to Nikora, Allen P..

18 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2016

Topic Modeling of NASA Space System Problem Reports: Research in Practice

Problem reports at NASA are similar to bug reports: they capture defects found during test, post-launch operational anomalies, and document the investigation and corrective action of the issue. These artifacts are a rich source of lessons learned for NASA, but are expensive to analyze since problem reports are comprised primarily of natural language text. We apply topic modeling to a corpus of NASA problem reports to extract trends in testing and operational failures. We collected 16,669 problem reports from six NASA space flight missions and applied Latent Dirichlet Allocation topic modeling to the document corpus. We analyze the most popular topics within and across missions, and how popular topics changed over the lifetime of a mission. We find that hardware material and flight software issues are common during the integration and testing phase, while ground station software and equipment issues are more common during the operations phase. We identify a number of challenges in topic modeling for trend analysis: 1) that the process of selecting the topic modeling parameters lacks definitive guidance, 2) defining semantically-meaningful topic labels requires nontrivial effort and domain expertise, 3) topic models derived from the combined corpus of the six missions were biased toward the larger missions, and 4) topics must be semantically distinct as well as cohesive to be useful. Nonetheless,topic modeling can identify problem themes within missions and across mission lifetimes, providing useful feedback to engineers and project managers.

Data Mining↗
NASA NTRS2013

Tool Use Within NASA Software Quality Assurance

As space mission software systems become larger and more complex, it is increasingly important for the software assurance effort to have the ability to effectively assess both the artifacts produced during software system development and the development process itself. Conceptually, assurance is a straightforward idea - it is the result of activities carried out by an organization independent of the software developers to better inform project management of potential technical and programmatic risks, and thus increase management's confidence in the decisions they ultimately make. In practice, effective assurance for large, complex systems often entails assessing large, complex software artifacts (e.g., requirements specifications, architectural descriptions) as well as substantial amounts of unstructured information (e.g., anomaly reports resulting from testing activities during development). In such an environment, assurance engineers can benefit greatly from appropriate tool support. In order to do so, an assurance organization will need accurate and timely information on the tool support available for various types of assurance activities. In this paper, we investigate the current use of tool support for assurance organizations within NASA, and describe on-going work at JPL for providing assurance organizations with the information about tools they need to use them effectively.

software assurance↗
NASA NTRS2012

Toward Baseline Software Anomalies in NASA Missions

In this fast abstract, we provide preliminary findings an analysis of 14,500 spacecraft anomalies from unmanned NASA missions. We provide some baselines for the distributions of software vs. non-software anomalies in spaceflight systems, the risk ratings of software anomalies, and the corrective actions associated with software anomalies.

anonaly↗
NASA NTRS2006

Developing Formal Correctness Properties from Natural Language Requirements

This viewgraph presentation reviews the rationale of the program to transform natural language specifications into formal notation.Specifically, automate generation of Linear Temporal Logic (LTL)correctness properties from natural language temporal specifications. There are several reasons for this approach (1) Model-based techniques becoming more widely accepted, (2) Analytical verification techniques (e.g., model checking, theorem proving) significantly more effective at detecting types of specification design errors (e.g., race conditions, deadlock) than manual inspection, (3) Many requirements still written in natural language, which results in a high learning curve for specification languages, associated tools and increased schedule and budget pressure on projects reduce training opportunities for engineers, and (4) Formulation of correctness properties for system models can be a difficult problem. This has relevance to NASA in that it would simplify development of formal correctness properties, lead to more widespread use of model-based specification, design techniques, assist in earlier identification of defects and reduce residual defect content for space mission software systems. The presentation also discusses: potential applications, accomplishments and/or technological transfer potential and the next steps.

requirements engineering↗
NASA NTRS2003

Increasing software testability with standard access and control interfaces

Testing is the most common method of determining whether a software system satisfies its requirements. Traditionally, testing starts with the detailed examination of individual functions or methods, progresses through the integration of functions or methods into subsystems, and ends with testing the functionality and behavior of the completely integrated system. At each stage of testing, the amount of functionality and behavior of the artifact being tested is increasingly limited. One reason for this is that it becomes impossible to test all paths through the system within a reasonable amount of time. However, another reason for this progressive decrease of test coverage has to do with increasingly limited control of and visibility into the state of the artifact being tested. During unit test, it is rather simple to control the inputs of individual functions or methods or view their internal state - modem development environments provide adequate facilities for doing so. However, these facilities do not scale up to the testing of partially or completely integrated systems. Control of and visibility into the system's state is then limited to the input and output facilities provided by the software itself as well as the hardware on which the software is hosted during the test. These facilities are usually insufficient to precisely control the state of individual components or sets of components of the system; they are also inadequate to the task of displaying on demand the state of specific components. We describe an approach to improving the testability of complex software systems with software constructs modeled after the hardware JTAG bus, used to provide visibility and controllability in testing digital circuits.

Tamir, Yuval↗
NASA NTRS2003

Developing Fault Models for Space Mission Software

A viewgraph presentation on the development of fault models for space mission software is shown. The topics include: 1) Goal: Improve Understanding of Technology Fault Generation Process; 2) Required Measurement; 3) Measuring Structural Evolution; 4) Module Attributes; 5) Principal Components of Raw Metrics; 6) The Measurement Process; 7) View of Structural Evolution at the System and Module Level; 8) Identifying and Counting Faults; 9) Fault Enumeration; 10) Modeling Fault Content; 11) Modeling Results; 12) Current and Future Work; and 13) Discussion and Conclusions.

software measurements↗
NASA NTRS1999

Software Evolution and the Fault Process

In developing a software system, we would like to estimate the way in which the fault content changes during its development, as well determine the locations having the highest concentration of faults. In the phases prior to test, however, there may be very little direct information regarding the number and location of faults. This lack of direct information requires developing a fault surrogate from which the number of faults and their location can be estimated. We develop a fault surrogate based on changes in the fault index, a synthetic measure which has been successfully used as a fault surrogate in previous work. We show that changes in the fault index can be used to estimate the rates at which faults are inserted into a system between successive revisions. We can then continuously monitor the total number of faults inserted into a system, the residual fault content, and identify those portions of a system requiring the application of additional fault detection and removal resources.

Nikora, Allen P.↗
NASA NTRS1999

Practical Issues in Implementing Software Reliability Measurement

Many ways of estimating software systems' reliability, or reliability-related quantities, have been developed over the past several years. Of particular interest are methods that can be used to estimate a software system's fault content prior to test, or to discriminate between components that are fault-prone and those that are not. The results of these methods can be used to: 1) More accurately focus scarce fault identification resources on those portions of a software system most in need of it. 2) Estimate and forecast the risk of exposure to residual faults in a software system during operation, and develop risk and safety criteria to guide the release of a software system to fielded use. 3) Estimate the efficiency of test suites in detecting residual faults. 4) Estimate the stability of the software maintenance process.

Nikora, Allen P.↗
NASA NTRS1998

Issues and Methods for Assessing COTS Reliability, Maintainability, and Availability

Many vendors produce products that are not domain specific (e.g., network server) and have limited functionality (e.g., mobile phone). In contrast, many customers of COTS develop systems that am domain specific (e.g., target tracking system) and have great variability in functionality (e.g., corporate information system). This discussion takes the viewpoint of how the customer can ensure the quality of COTS components. In evaluating the benefits and costs of using COTS, we must consider the environment in which COTS will operate. Thus we must distinguish between using a non-mission critical application like a spreadsheet program to produce a budget and a mission critical application like military strategic and tactical operations. Whereas customers will tolerate an occasional bug in the former, zero tolerance is the rule in the latter. We emphasize the latter because this is the arena where there are major unresolved problems in the application of COTS. Furthermore, COTS components may be embedded in the larger customer system. We refer to these as embedded systems. These components must be reliable, maintainable, and available, and must be with the larger system in order for the customer to benefit from the advertised advantages of lower development and maintenance costs. Interestingly, when the claims of COTS advantages are closely examined, one finds that to a great extent these COTS components consist of hardware and office products, not mission critical software [1]. Obviously, COTS components are different from custom components with respect to one or more of the following attributes: source, development paradigm, safety, reliability, maintainability, availability, security, and other attributes. However, the important question is whether they should be treated differently when deciding to deploy them for operational use; we suggest the answer is no. We use reliability as an example to justify our answer. In order to demonstrate its reliability, a COTS component must pass the same reliability evaluations as the custom components, otherwise the COTS components will be the weakest link in the chain of components and will be the determinant of software system reliability. The challenge is that there will be less information available for evaluating COTS components than for custom components but this does not mean we should despair and do nothing. Actually, there is a lot we can do even in the absence of documentation on COTS components because the customer will have information about how COTS components are to be used in the larger system. To illustrate our approach, we will consider the reliability, maintainability, and availability (RMA) of COTS components as used in larger systems. Finally, COTS suppliers might consider increasing visibility into their products to assist customers in determining the components' fitness for use in a particular application. We offer ideas of information that would be useful to customers, and what vendors might do to provide it.

Schneidewind, Norman F.↗
NASA NTRS1997

Finding Fault with Faults: A Case Study

We describe our effort in extending this work beyond the initial software contruction. Our area of focus is determining the rate of fault injection over a sequence of successive builds, first observing that software faults may be seen to fall into two distinct classes some faults are incorporated during the initial coding effort, while others are added in successive software builds.

Software Systems Software faults↗
NASA NTRS1995

An Experiment in Determining Software Reliability Model Applicability

There have been few reports on the behavior of software reliability models under controlled conditions. That is to say, most of the reported experience with the models is during the testing phase of actual projects, during which researchers have little or no control over the data with which they work. Give that failure data for actual projects can be noisy, distorted, and uncertain, reported procedures for determining model applicability may be incomplete.

Software Reliability↗