Establishing a Framework and Testbed for Evaluating and Infusing Software Assurance Tools
No abstract available
tool evaluation↗
SEARCH · Engineering Papers
Results for “software assurance tools”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Establishing a Framework and Testbed for Evaluating and Infusing Software Assurance Tools
No abstract available
software assurance tools↗
Technology Transfer Challenges for High-Assurance Software Engineering Tools
In this paper, we describe our experience with the challenges thar we are currently facing in our effort to develop advanced software verification and validation tools. We categorize these challenges into several areas: cost benefits modeling, tool usability, customer application domain, and organizational issues. We provide examples of challenges in each area and identrfj, open research issues in areas which limit our ability to transfer high-assurance software engineering tools into practice.
John Penix↗
Advanced techniques in reliability model representation and solution
The current tendency of flight control system designs is towards increased integration of applications and increased distribution of computational elements. The reliability analysis of such systems is difficult because subsystem interactions are increasingly interdependent. Researchers at NASA Langley Research Center have been working for several years to extend the capability of Markov modeling techniques to address these problems. This effort has been focused in the areas of increased model abstraction and increased computational capability. The reliability model generator (RMG) is a software tool that uses as input a graphical object-oriented block diagram of the system. RMG uses a failure-effects algorithm to produce the reliability model from the graphical description. The ASSURE software tool is a parallel processing program that uses the semi-Markov unreliability range evaluator (SURE) solution technique and the abstract semi-Markov specification interface to the SURE tool (ASSIST) modeling language. A failure modes-effects simulation is used by ASSURE. These tools were used to analyze a significant portion of a complex flight control system. The successful combination of the power of graphical representation, automated model generation, and parallel computation leads to the conclusion that distributed fault-tolerant system architectures can now be analyzed.
Palumbo, Daniel L.↗
Generation and analysis of large reliability models
An effort has been underway for several years at NASA's Langley Research Center to extend the capability of Markov modeling techniques for reliability analysis to the designers of highly reliable avionic systems. This effort has been focused in the areas of increased model abstraction and increased computational capability. The reliability model generator (RMG), a software tool which uses as input a graphical, object-oriented block diagram of the system, is discussed. RMG uses an automated failure modes-effects analysis algorithm to produce the reliability model from the graphical description. Also considered is the ASSURE software tool, a parallel processing program which uses the ASSIST modeling language and SURE semi-Markov solution technique. An executable failure modes-effects analysis is used by ASSURE. The successful combination of the power of graphical representation, automated model generation, and parallel computation leads to the conclusion that large system architectures can now be analyzed.
Aerospace electronics↗
Training, Quality Assurance Factors, and Tools Investigation: a Work Report and Suggestions on Software Quality Assurance
Previously, several research tasks have been conducted, some observations were obtained, and several possible suggestions have been contemplated involving software quality assurance engineering at NASA Johnson. These research tasks are briefly described. Also, a brief discussion is given on the role of software quality assurance in software engineering along with some observations and suggestions. A brief discussion on a training program for software quality assurance engineers is provided. A list of assurance factors as well as quality factors are also included. Finally, a process model which can be used for searching and collecting software quality assurance tools is presented.
Lee, Pen-Nan↗
Tool Use Within NASA Software Quality Assurance
As space mission software systems become larger and more complex, it is increasingly important for the software assurance effort to have the ability to effectively assess both the artifacts produced during software system development and the development process itself. Conceptually, assurance is a straightforward idea - it is the result of activities carried out by an organization independent of the software developers to better inform project management of potential technical and programmatic risks, and thus increase management's confidence in the decisions they ultimately make. In practice, effective assurance for large, complex systems often entails assessing large, complex software artifacts (e.g., requirements specifications, architectural descriptions) as well as substantial amounts of unstructured information (e.g., anomaly reports resulting from testing activities during development). In such an environment, assurance engineers can benefit greatly from appropriate tool support. In order to do so, an assurance organization will need accurate and timely information on the tool support available for various types of assurance activities. In this paper, we investigate the current use of tool support for assurance organizations within NASA, and describe on-going work at JPL for providing assurance organizations with the information about tools they need to use them effectively.
software assurance↗
Are we ready for the first EASA guidance on the use of ML in Aviation?
NASA has been working for the past 12 years on software tools for the assurance of software in Aviation critical systems. For now two years, NASA has focused more on the use of AI-based techniques in Aviation than the traditional software systems used in the past. The primary focus has been on machine learning (ML), and more specifically, on supervised off-line learning ML systems. NSA’s research has been driven by case studies such as a vision-based centerline tracking system (implemented using deep neural networks) and the new generation of collision avoidance systems developed under the FAA guidance, i.e., the family of ACAS-X products. Since EASA has recently released its first usable guidance for Level 1 machine learning applications, it is opportunity to see how the research done at NASA is mapping to this first guidance for ML. In this talk I will use the EASA guidance document as a guide to present the past, present, and future tools and techniques being developed at NASA. The intent is to not only provide an overview of the research effort at NASA but also to see how this effort is addressing the concerns listed in the EASA first usable guidance for ML.
Guillaume Brat↗
Advanced Transmission Technologies – GETs and HPCs Session 2: Advanced Power Flow Control and Transmission Topology Optimization
The INL TADA GETs Cohort Session 2, held on November 7, 2025, conducted in collaboration with ScottMadden, focused on two core Advanced Transmission Technologies (ATTs): Advanced Power Flow Control (APFC) and Transmission Topology Optimization (TTO). These technologies are pivotal in enhancing grid flexibility, reliability, and cybersecurity resilience. APFC, particularly through modular FACTS devices like Modular Static Synchronous Series Compensators (M-SSSCs), enables dynamic voltage injection to reroute power flows. The session highlighted the deployment benefits of APFC, such as rapid installation, minimal civil works, and re-deployability. Regulatory drivers like FERC Order 2023 mandate the inclusion of Grid-Enhancing Technologies (GETs) in interconnection studies. Case studies from Central Hudson, CAISO, and National Grid (UK) demonstrated APFC’s effectiveness in congestion relief and cost savings. The session also addressed cybersecurity concerns, including firmware vulnerabilities, SCADA integration risks, and supply chain dependencies. Participants engaged in interactive exercises to rank cybersecurity and supply chain risks, emphasizing the need for robust digital assurance strategies. TTO involves software-based reconfiguration of transmission networks to optimize power flow without new infrastructure. The session showcased its operational value, with examples from SPP, PJM, and MISO showing significant congestion cost reductions. Cybersecurity vulnerabilities were discussed, particularly in API security and software supply chains, referencing incidents like SolarWinds and attacks on Danish utilities. Digital assurance exercises explored worst-case scenarios, attack paths, and mitigation responsibilities between vendors and utilities. Reliability challenges such as algorithm stability, vendor dependency, and operator trust were also examined. Cross-cutting themes emphasized the importance of digital assurance tools, including Software Bills of Materials (SBOMs) and hardware-in-loop testing. Human performance, training, and operational confidence were identified as critical enablers of technology adoption. The session concluded with a preview of Session 3, which will focus on High Performance Conductors (HPCs) and risk-based cybersecurity tools. Session 2 of 3.
24 - POWER TRANSMISSION AND DISTRIBUTION↗
Initial flight qualification and operational maintenance of X-29A flight software
This paper is predominantly a nontechnical discussion of some significant aspects of the initial flight qualification and operational maintenance of the flight control system software for the X-29A technology demonstrator. Flight qualification and maintenance of complex, embedded flight control system software poses unique problems. The X-29A technology demonstrator aircraft has a digital flight control system which incorporates functions generally considered too complex for analog systems. Organizational responsibilities, software assurance issues, tools, and facilities are discussed.
Earls, Michael R.↗
Initial flight qualification and operational maintenance of X-29A flight software
A discussion is presented of some significant aspects of the initial flight qualification and operational maintenance of the flight control system softward for the X-29A technology demonstrator. Flight qualification and maintenance of complex, embedded flight control system software poses unique problems. The X-29A technology demonstrator aircraft has a digital flight control system which incorporates functions generally considered too complex for analog systems. Organizational responsibilities, software assurance issues, tools, and facilities are discussed.
Earls, Michael R.↗
The NASA Software Research Infusion Initiative: Successful Technology Transfer for Software Assurance
New processes, methods and tools are constantly appearing in the field of software engineering. Many of these augur great potential in improving software development processes, resulting in higher quality software with greater levels of assurance. However, there are a number of obstacles that impede their infusion into software development practices. These are the recurring obstacles common to many forms of research. Practitioners cannot readily identify the emerging techniques that may most benefit them, and cannot afford to risk time and effort in evaluating and experimenting with them while there is still uncertainty about whether they will have payoff in this particular context. Similarly, researchers cannot readily identify those practitioners whose problems would be amenable to their techniques and lack the feedback from practical applications necessary to help them to evolve their techniques to make them more likely to be successful. This paper describes an ongoing effort conducted by a software engineering research infusion team, and the NASA Research Infusion Initiative, established by NASA s Software Engineering Initiative, to overcome these obstacles.
Hinchey, Michael G.↗
Software quality: Process or people
This paper will present data related to software development processes and personnel involvement from the perspective of software quality assurance. We examine eight years of data collected from six projects. Data collected varied by project but usually included defect and fault density with limited use of code metrics, schedule adherence, and budget growth information. The data are a blend of AFSCP 800-14 and suggested productivity measures in Software Metrics: A Practioner's Guide to Improved Product Development. A software quality assurance database tool, SQUID, was used to store and tabulate the data.
Palmer, Regina↗
Production and quality assurance automation in the Goddard Space Flight Center Flight Dynamics Facility
The Flight Dynamics Facility (FDF) at the NASA Goddard Space Flight Center (GSFC) generates numerous products for NASA-supported spacecraft, including the Tracking and Data Relay Satellites (TDRS's), the Hubble Space Telescope (HST), the Extreme Ultraviolet Explorer (EUVE), and the space shuttle. These products include orbit determination data, acquisition data, event scheduling data, and attitude data. In most cases, product generation involves repetitive execution of many programs. The increasing number of missions supported by the FDF has necessitated the use of automated systems to schedule, execute, and quality assure these products. This automation allows the delivery of accurate products in a timely and cost-efficient manner. To be effective, these systems must automate as many repetitive operations as possible and must be flexible enough to meet changing support requirements. The FDF Orbit Determination Task (ODT) has implemented several systems that automate product generation and quality assurance (QA). These systems include the Orbit Production Automation System (OPAS), the New Enhanced Operations Log (NEOLOG), and the Quality Assurance Automation Software (QA Tool). Implementation of these systems has resulted in a significant reduction in required manpower, elimination of shift work and most weekend support, and improved support quality, while incurring minimal development cost. This paper will present an overview of the concepts used and experiences gained from the implementation of these automation systems.
Chapman, K. B.↗
Sensitivity Analysis for the Component Design App: Analysis of Success Assured Data
A new tool has been developed to perform variance-based global sensitivity analysis (VBGSA) on data from a set-based concurrent engineering software called Success Assured (SA). The tool is part of a digital component design app, which is currently in production as an Accelerated Digital Engineering Pathfinder at Sandia National Laboratories. When working with complex digital models, it is important to understand relationships between inputs and outputs, i.e., how “sensitive” model outputs are to changes in model inputs. After extensive research and trials of various sensitivity analysis methods, it was determined that estimation of Sobol’ indices for VBGSA with Monte Carlo simulation, paired with simple surrogate models, produces the best results for SA data. This tool increases understanding of SA models and streamlines the creation of SA datasets. This report details the methodology and implementation of this sensitivity analysis tool so others can understand it and implement it.
97 MATHEMATICS AND COMPUTING↗
Technology Infusion of CodeSonar into the Space Network Ground Segment (RII07): Software Assurance Symposium Technical Summary
Presents a source code analysis tool (CodeSonar) for use in the Space Network Ground Segment. The Space Network requires 99.9% proficiency and 97.0% availability of systems. Software has historically accounted for an annual average of 28% of the Space Network loss of availability and proficiency. CSCI A and CSCI B account for 42% of the previous eight months of software data loss. The technology infusion of CodeSonar into the Space Network Ground segment is meant to aid in determining the impact of the technology on the project both in the expenditure of effort and the technical results of the technology. Running a CodeSonar analysis and performing a preliminary review of the results averaged 3.5 minutes per finding (approximately 20 hours total). An additional 40 hours is estimated to analyze the 37 findings deemed too complex for the initial review. Using CodeSonar's tools to suppress known non-problems, delta tool runs will not repeat findings that have been marked as non-problems, further reducing the time needed for review. The 'non-interesting' finding rate of 70% is a large number, but filtering, search, and detailed contextual features of CodeSonar reduce the time per finding. Integration of the tool into the build process may also provide further savings by preventing developers from having to configure and operate the tool separately. These preliminary results show the tool to be easy to use and incorporate into the engineering process. These findings also provide significant potential improvements in proficiency and availability on the part of the software. As time-to-fix data become available a better cost trade can be made on person hours saved versus tool cost. Selective factors may be necessary to determine where best to apply CodeSonar to balance cost and benefits.
Benson, Markland J.↗
Model Based Engineering for Software Assurance
NASA's successful development of next generation space vehicles, habitats, and robotic systems will require reliable hardware and software systems. The aim of this initiative is to develop modeling methodology and tools to support Model-Based Systems Engineering (MBSE) for software assurance and reliability analysis. This effort expands the Unified Modeling Language (UML) software design models to include fault data for the extraction of Failure Modes and Effects Criticality Analysis (FMECA) and Fault Tree Analysis (FTA) for software. We explored different modeling approaches to integrate the UML software design models with the Systems Modeling Language (SysML) system models to generate an integrated model and reliability tools that take into account software and hardware interfaces.The benefits of this concept directly affect the safety community with quick turnarounds to produce software assurance and reliability analysis artifacts and the ability to visualize failure effects, both hardware and software. The result is enhanced system design integrity and early identification of system risks. This initiative will enable software assurance activities early in the system design lifecycle, facilitating the discovery of design weaknesses and enhancing the capability to produce safe, hazard-free systems
Wang, Lui↗