Defect measurement and analysis of JPL ground software: a case study
Explore the source record for details and available documents.
metrics↗
SEARCH · Engineering Papers
Results for “defects metrics”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Modeling defect trends for iterative development
The Employment of Defects (EoD) approach to measuring and analyzing defects seeks to identify and capture trends and phenomena that are critical to managing software quality in the iterative software development lifecycle at JPL.
modeling defects metrics↗
Modeling defect trends for iterative development at JPL
Explore the source record for details and available documents.
defects metrics↗
Employing software defect measurement for improvement at JPL
Explore the source record for details and available documents.
defects metrics↗
Defect measurement and analysis of JPL ground software: a case study
Ground software systems at JPL must meet high assurance standards while remaining on schedule due to relatively immovable launch dates for spacecraft that will be controlled by such systems. Toward this end, the Software Quality Improvement (SQI) project's Measurement and Benchmarking (M&B) team is collecting and analyzing defect data of JPL ground system software projects to build software defect prediction models. The aim of these models is to improve predictability with regard to software quality activities. Predictive models will quantitatively define typical trends for JPL ground systems as well as Critical Discriminators (CDs) to provide explanations for atypical deviations from the norm at JPL. CDs are software characteristics that can be estimated or foreseen early in a software project's planning. Thus, these CDs will assist in planning for the predicted degree to which software quality activities for a project are likely to deviation from the normal JPL ground system based on pasted experience across the lab.
defects↗
Leveraging Automated Fiber Placement Computer Aided Process Planning Framework for Defect Validation and Dynamic Layup Strategies
Process planning represents an essential stage of the Automated Fiber Placement (AFP) workflow. It develops useful and efficient machine processes based upon the working material, composite design, and manufacturing resources. The current state of process planning requires a high degree of interaction from the process planner and could greatly benefit from increased automation. Therefore, a list of key steps and functions are created to identify the more difficult and time-consuming phases of process planning. Additionally, a set of metrics must exist by which to evaluate the effectiveness of the manufactured laminate from the machine code created during the Process Planning stage. Layup strategies, in addition to dog ears, stagger shifts, steering constraints, and starting points, represented the group of functions labeled as process optimization and ranked the highest in terms of priority for automation. The laminates resulting from the selected parameters are evaluated through the occurrences of principal defect metrics such as fiber gaps, overlaps, angle deviation and steering violations. This document presents an automated software solution to the layup strategy and starting point selection phase of process planning. A series of ply scenarios are generated with variations of these ply parameters and evaluated according to a set of metrics entered by the Process Planner. These metrics are generated through use of the Analytical Hierarchy Process (AHP), where relative importance between each of the fiber features are defined. The ply scenarios are selected which reduce the overall fiber feature scores based on the defects the Process Planner wishes to minimize.
HiCAM↗
Defect measurement and analysis of JPL software: a case study
Explore the source record for details and available documents.
defects↗
Single Kerr-Schild metric for Taub-NUT instanton
It is shown that a complex coordinate transformation maps the Taub-Newman-Unti-Tamburino instanton metric to a Kerr-Schild metric. This metric involves a semi-infinite line defect as the gravitational analog of the Dirac string, much like the original metric. Moreover, it facilitates three versions of classical double copy correspondence with the self-dual dyon in electromagnetism, one of which involves a nonlocal operator. The relevance to the Newman-Janis algorithm is briefly noted. Published by the American Physical Society 2025
Kim, Joon-Hwi (ORCID:000000025474123X)↗
Bayesian SegNet for Semantic Segmentation with Improved Interpretation of Microstructural Evolution During Irradiation of Materials
Understanding the relationship between the evolution of microstructures of irradiated LiAlO2pellets and tritium diffusion, retention and release could improve predictions of tritium performance. Given expert-labeled segmented images of irradiated and unirradiated pellets, we trained Deep Convolutional Neural Networks to segment images into defect, grain, and boundary classes. Qualitative microstructural information was calculated from these segmented images to facilitate the comparison of unirradiated and irradiated pellets. We tested modifications to improve the sensitivity of the model, including incorporating meta-data into the model and utilizing uncertainty quantification. The predicted segmentation was similar to the expert-labeled segmentation for most methods of microstructural qualification, including pixel proportion, defect area, and defect density. Overall, the high performance metrics for the best models for both irradiated and unirradiated images shows that utilizing neural network models is a viable alternative to expert-labeled images.
Oostrom, Marjolein T.↗
Additive Manufacturing Model-Based Process Metrics: Reduced Order Modeling of the Laser Powder Bed Fusion Process
The multi-scale and complex process of printing additively manufactured (AM) parts can have unexpected, but predictable, build conditions that result in material microstructure variability. In this work, we describe a fully parallel reduced order modeling approach that has been developed to evaluate the evolution of AM processes, termed the AM moment measure method. This method couples the known sequence of the AM process with a physically informed nearest neighbors’ calculation to map the conditions of a part-scale build. The result is a map of the build that is derived directly from build files or in-situ process monitoring sensors. The methodology and terminology of the approach will be described, and computed build maps will be calculated and compared for various laser powder bed fusion (LPBF) builds of Ti-6Al-4V. Such comparative results develop understanding of how the sequential process actions can affect the LPBF-AM build quality and microstructure variability.
Laser Powder Bed Fusion↗
How simple is software defect detection?
This study benchmarks several FSS techniques and reports several studies where a large set metrics were reduced to a handful with little loss of detection accuracy. This result raises the possibility that software defect detection may be much simpler than previously believed.
feature subset selection software fault model soft↗
Advances in Integrally Formed Metallic Structures
NASA Langley Research Center (LaRC) is applying flow forming technology to aerospace aluminum alloys for the manufacture of integrally stiffened cylinders that meet aircraft fuselage or launch vehicle specifications. High-strength 2xxx series alloys, in various temper/product form conditions, are being compared with medium-strength, very formable AA 6061 as the benchmark. Screening of candidate materials traditionally relies on empirical forming trials, but more recently on metrics derived from constant nominal strain rate tensile testing. In this work, the established hierarchy of formability metrics is adopted as the baseline; tensile area reduction, work hardening exponent, and modulus of resilience. The suite of metrics can be expanded based on phenomenological and mechanistic considerations of plastic flow instabilities to incorporate anisotropy ratio, rate of work hardening, and dynamic strain aging effects. Separate research thrusts at LaRC are aimed at formulating a protocol for scalable production of fully formed, defect-free components: 1) quantifying the extended set of formability metrics from constant and jump strain rate tensile tests with digital image correlation (DIC); 2) using statistically designed experiments on a vertical spin/flow forming facility (WF VUD-600®) to isolate the effect of convenient process variables, such as axial feed rate and thickness reduction; and 3) predicting the effect of feed ratio (roller translation relative to mandrel rotation speeds) on material response via process modeling using the DEFORM® simulation software. Current results will be discussed in terms of correlating intrinsic material properties with the bulk workability characteristics to successfully fabricate cylinders with complex cross-sections.
Aluminum↗
NASA Software Engineering Benchmarking Study
To identify best practices for the improvement of software engineering on projects, NASA's Offices of Chief Engineer (OCE) and Safety and Mission Assurance (OSMA) formed a team led by Heather Rarick and Sally Godfrey to conduct this benchmarking study. The primary goals of the study are to identify best practices that: Improve the management and technical development of software intensive systems; Have a track record of successful deployment by aerospace industries, universities [including research and development (R&D) laboratories], and defense services, as well as NASA's own component Centers; and Identify candidate solutions for NASA's software issues. Beginning in the late fall of 2010, focus topics were chosen and interview questions were developed, based on the NASA top software challenges. Between February 2011 and November 2011, the Benchmark Team interviewed a total of 18 organizations, consisting of five NASA Centers, five industry organizations, four defense services organizations, and four university or university R and D laboratory organizations. A software assurance representative also participated in each of the interviews to focus on assurance and software safety best practices. Interviewees provided a wealth of information on each topic area that included: software policy, software acquisition, software assurance, testing, training, maintaining rigor in small projects, metrics, and use of the Capability Maturity Model Integration (CMMI) framework, as well as a number of special topics that came up in the discussions. NASA's software engineering practices compared favorably with the external organizations in most benchmark areas, but in every topic, there were ways in which NASA could improve its practices. Compared to defense services organizations and some of the industry organizations, one of NASA's notable weaknesses involved communication with contractors regarding its policies and requirements for acquired software. One of NASA's strengths was its software assurance practices, which seemed to rate well in comparison to the other organizational groups and also seemed to include a larger scope of activities. An unexpected benefit of the software benchmarking study was the identification of many opportunities for collaboration in areas including metrics, training, sharing of CMMI experiences and resources such as instructors and CMMI Lead Appraisers, and even sharing of assets such as documented processes. A further unexpected benefit of the study was the feedback on NASA practices that was received from some of the organizations interviewed. From that feedback, other potential areas where NASA could improve were highlighted, such as accuracy of software cost estimation and budgetary practices. The detailed report contains discussion of the practices noted in each of the topic areas, as well as a summary of observations and recommendations from each of the topic areas. The resulting 24 recommendations from the topic areas were then consolidated to eliminate duplication and culled into a set of 14 suggested actionable recommendations. This final set of actionable recommendations, listed below, are items that can be implemented to improve NASA's software engineering practices and to help address many of the items that were listed in the NASA top software engineering issues. 1. Develop and implement standard contract language for software procurements. 2. Advance accurate and trusted software cost estimates for both procured and in-house software and improve the capture of actual cost data to facilitate further improvements. 3. Establish a consistent set of objectives and expectations, specifically types of metrics at the Agency level, so key trends and models can be identified and used to continuously improve software processes and each software development effort. 4. Maintain the CMMI Maturity Level requirement for critical NASA projects and use CMMI to measure organizations developing software for NASA. 5.onsolidate, collect and, if needed, develop common processes principles and other assets across the Agency in order to provide more consistency in software development and acquisition practices and to reduce the overall cost of maintaining or increasing current NASA CMMI maturity levels. 6. Provide additional support for small projects that includes: (a) guidance for appropriate tailoring of requirements for small projects, (b) availability of suitable tools, including support tool set-up and training, and (c) training for small project personnel, assurance personnel and technical authorities on the acceptable options for tailoring requirements and performing assurance on small projects. 7. Develop software training classes for the more experienced software engineers using on-line training, videos, or small separate modules of training that can be accommodated as needed throughout a project. 8. Create guidelines to structure non-classroom training opportunities such as mentoring, peer reviews, lessons learned sessions, and on-the-job training. 9. Develop a set of predictive software defect data and a process for assessing software testing metric data against it. 10. Assess Agency-wide licenses for commonly used software tools. 11. Fill the knowledge gap in common software engineering practices for new hires and co-ops.12. Work through the Science, Technology, Engineering and Mathematics (STEM) program with universities in strengthening education in the use of common software engineering practices and standards. 13. Follow up this benchmark study with a deeper look into what both internal and external organizations perceive as the scope of software assurance, the value they expect to obtain from it, and the shortcomings they experience in the current practice. 14. Continue interactions with external software engineering environment through collaborations, knowledge sharing, and benchmarking.
Rarick, Heather L.↗
The Inspectability Metric: A Formalized System Of Measurement Enabling The Design For Inspection Framework
Nondestructive evaluation (NDE) engineers are often confronted with structural design choices that present challenges to meeting inspection requirements. These challenges, at best, increase the resources needed to design an inspection solution and, at worst, require resource intensive redesign of the structure. If the inspectability of the structure can be determined early in the design cycle, these challenging inspection scenarios can be avoided. The emergence of additive manufacturing has further compounded this problem by enabling the creation of highly optimized structures with no regard to inspection constraints. Design for inspection (DFI) offers a framework to integrate nondestructive evaluation (NDE) into the design process to alleviate the mechanisms that produce uninspectable designs. DFI is the concept of including inspectability in a multi-objective optimization framework so that it can be considered in parallel to other metrics such as mass and manufacturability. This allows rapid evaluation of the trade-off between design metrics to find solutions that meet the inspection needs of a particular material system, structural concept, or vehicle program. To enable DFI, there must be a system by which the inspectability of a structure can be measured. This system must be agile to produce results quickly, it must be versatile to work with the type of incomplete information one would encounter early in the design process (such as lack of inspection requirements), and it must be delivered in a form that is easily understood by designers. To meet this need, this presentation introduces the novel inspectability metric as a system to measure inspectability. The inspectability metric is a standardized, automation friendly procedure that uses simulations to determine inspectability. Along with guidelines to properly process designs and integrate with existing workflows, the inspectability metric provides a suite of simulation tests to interrogate the ability to find defects and the sensitivity to variability. The testing rubric is designed to maximize the coverage of the parameter space while minimizing the number of simulations needed. The inspectability metric has been in development in collaboration with industry partners to ensure compatibility with modern simulation tools and aerospace design workflows. In this study, we will demonstrate how the inspectability metric is able to determine the inspectability of multiple types of structures, including aerospace composites and additively manufactured parts. We will then show how the inspectability score can be plugged into existing design optimization tasks, such as structural sizing algorithms or design for manufacturing (DFM) frameworks.
Design for inspection↗
Software quality: Process or people
This paper will present data related to software development processes and personnel involvement from the perspective of software quality assurance. We examine eight years of data collected from six projects. Data collected varied by project but usually included defect and fault density with limited use of code metrics, schedule adherence, and budget growth information. The data are a blend of AFSCP 800-14 and suggested productivity measures in Software Metrics: A Practioner's Guide to Improved Product Development. A software quality assurance database tool, SQUID, was used to store and tabulate the data.
Palmer, Regina↗
Design for Vibration Monitoring: A Methodology for Reliable and Cost-Effective Vibration Monitoring
The purpose of health monitoring systems is to detect failures or defects for increased safety and performance and to provide on-condition maintenance with reduced costs. The problems associated with health monitoring systems include high rates of false alarms and missed failures, which make monitoring an unreliable and costly task. The reason for this is that unaccounted variations invalidate signal modeling assumptions. Our approach was to focus on vibration monitoring of rotating components. We analyzed baseline signals to determine statistical variations, identify and model factors that influence vibrations (pre-production vs. post-production variations), determine hit and false alarm rates with baseline flight data, model and predict effects of defects and variations on vibrations, and develop algorithms and metrics for failure and anomaly detection in the presence of variations.
Tumer, Irem Y.↗
PUMA:POWDER UTILIZATION MODELING APPLICATION
SF-25-084 PUMA a high performance modeling framework to simulate powder processing. It provides a scalable tool for manufacturers to simulate powder pre- and post-processing. The tool can predict the distortion, residual stress, and (for reactive processes) reaction completion fraction of complex parts after curing/debinding, sintering, and infiltration processes. These predictions are key metrics industry uses to optimize these processes to produce dense, defect-free, stable components.
HU, TIANCHEN (GARY)↗
Software Quality Assurance Metrics
Software Quality Assurance (SQA) is a planned and systematic set of activities that ensures conformance of software life cycle processes and products conform to requirements, standards and procedures. In software development, software quality means meeting requirements and a degree of excellence and refinement of a project or product. Software Quality is a set of attributes of a software product by which its quality is described and evaluated. The set of attributes includes functionality, reliability, usability, efficiency, maintainability, and portability. Software Metrics help us understand the technical process that is used to develop a product. The process is measured to improve it and the product is measured to increase quality throughout the life cycle of software. Software Metrics are measurements of the quality of software. Software is measured to indicate the quality of the product, to assess the productivity of the people who produce the product, to assess the benefits derived from new software engineering methods and tools, to form a baseline for estimation, and to help justify requests for new tools or additional training. Any part of the software development can be measured. If Software Metrics are implemented in software development, it can save time, money, and allow the organization to identify the caused of defects which have the greatest effect on software development. The summer of 2004, I worked with Cynthia Calhoun and Frank Robinson in the Software Assurance/Risk Management department. My task was to research and collect, compile, and analyze SQA Metrics that have been used in other projects that are not currently being used by the SA team and report them to the Software Assurance team to see if any metrics can be implemented in their software assurance life cycle process.
McRae, Kalindra A.↗