Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software flaw detection”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Using Neural Architecture Search for Improving Software Flaw Detection in Multimodal Deep Learning Models

Software flaw detection using multimodal deep learning models has been demonstrated as a very competitive approach on benchmark problems. In this work, we demonstrate that even better performance can be achieved using neural architecture search (NAS) combined with multimodal learning models. We adapt a NAS framework aimed at investigating image classification to the problem of software flaw detection and demonstrate improved results on the Juliet Test Suite, a popular benchmarking data set for measuring performance of machine learning models in this problem domain.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Using Neural Architecture Search for Improving Software Flaw Detection in Multimodal Deep Learning Models

Software flaw detection using multimodal deep learning models has been demonstrated as a very competitive approach on benchmark problems. In this work, we demonstrate that even better performance can be achieved using neural architecture search (NAS) combined with multimodal learning models. We adapt a NAS framework aimed at investigating image classification to the problem of software flaw detection and demonstrate improved results on the Juliet Test Suite, a popular benchmarking data set for measuring performance of machine learning models in this problem domain.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Multimodal Deep Learning for Flaw Detection in Software Programs

We explore the use of multiple deep learning models for detecting flaws in software programs. Current, standard approaches for flaw detection rely on a single representation of a software program (e.g., source code or a program binary). We illustrate that, by using techniques from multimodal deep learning, we can simultaneously leverage multiple representations of software programs to improve flaw detection over single representation analyses. Specifically, we adapt three deep learning models from the multimodal learning literature for use in flaw detection and demonstrate how these models outperform traditional deep learning models. We present results on detecting software flaws using the Juliet Test Suite and Linux Kernel.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Joint Analysis of Program Data Representations using Machine Learning for Improved Software Assurance and Development Capabilities

We explore the use of multiple deep learning models for detecting flaws in software programs. Current, standard approaches for flaw detection rely on a single representation of a software program (e.g., source code or a program binary). We illustrate that, by using techniques from multimodal deep learning, we can simultaneously leverage multiple representations of software programs to improve flaw detection over single representation analyses. Specifically, we adapt three deep learning models from the multimodal learning literature for use in flaw detection and demonstrate how these models outperform traditional deep learning models. We present results on detecting software flaws using the Juliet Test Suite and Linux Kernel.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2017

Optimizing Probability of Detection Point Estimate Demonstration

Probability of detection (POD) analysis is used in assessing reliably detectable flaw size in nondestructive evaluation (NDE). MIL-HDBK-18231and associated mh18232POD software gives most common methods of POD analysis. Real flaws such as cracks and crack-like flaws are desired to be detected using these NDE methods. A reliably detectable crack size is required for safe life analysis of fracture critical parts. The paper provides discussion on optimizing probability of detection (POD) demonstration experiments using Point Estimate Method. POD Point estimate method is used by NASA for qualifying special NDE procedures. The point estimate method uses binomial distribution for probability density. Normally, a set of 29 flaws of same size within some tolerance are used in the demonstration. The optimization is performed to provide acceptable value for probability of passing demonstration (PPD) and achieving acceptable value for probability of false (POF) calls while keeping the flaw sizes in the set as small as possible.

Koshti, Ajay M.↗
NASA NTRS2003

Low Temperature SQUID for NDE Applications

We have developed a low temperature SuperConducting Quantum Interference Device - SQUID measurement system for detection of defects deep under the surface of aluminum structures using eddy current techniques. The system uses a two dimensional planar inducer with two different excitation frequencies to induce a current in the sample. We have developed a data analysis software program that enabled us to distinguish between round defects (holes), straight defects (slots) and slots close to holes simulating cracks starting from rivets in aluminum structures. We were able to detect defects that are 8mm below the surface. We have also measured the change in phase of the detected signal as a function of depth of the defect. This relationship can be used to determine the depth of hidden flaws. Using this analysis software with the high temperature SQUID system at NASA Langley we were able to detect slots close to holes in layered aluminum sample.

Wincheski, Buzz↗
NASA NTRS2017

Reliably Detectable Flaw Size for NDE Methods that Use Calibration

Probability of detection (POD) analysis is used in assessing reliably detectable flaw size in nondestructive evaluation (NDE). MIL-HDBK-1823 and associated mh1823 POD software gives most common methods of POD analysis. In this paper, POD analysis is applied to an NDE method, such as eddy current testing, where calibration is used. NDE calibration standards have known size artificial flaws such as electro-discharge machined (EDM) notches and flat bottom hole (FBH) reflectors which are used to set instrument sensitivity for detection of real flaws. Real flaws such as cracks and crack-like flaws are desired to be detected using these NDE methods. A reliably detectable crack size is required for safe life analysis of fracture critical parts. Therefore, it is important to correlate signal responses from real flaws with signal responses form artificial flaws used in calibration process to determine reliably detectable flaw size.

Koshti, Ajay M.↗
DOE OSTI2023

Machine Learning Enabled Sensor Fusion for In-Situ Defect Detection in Laser Powder Bed Fusion

Laser Powder Bed Fusion (L-PBF) Additive Manufacturing (AM) is among the metal 3D printing technologies most broadly adopted by the manufacturing industry. The current industry qualification paradigm for critical-application L-PBF parts relies heavily on expensive non-destructive inspection techniques such as X-Ray Computed Tomography (XCT), which significantly limits the use-cases of L-PBF. In situ monitoring of the process promises a less expensive alternative to ex situ testing, but existing sensor technologies and data analysis techniques struggle to detect sub-surface flaws (e.g., porosity and cracking) on production-scale L-PBF printers. RTX Technologies Research Center (RTRC) has licensed ORNL’s Peregrine software package – a printer- and camera-agnostic data analytics tool designed specifically for detecting process anomalies using in situ data collected during powder bed printing. The goal of this project was to feed temporally rich, multi-modal sensor data, including visible light, integrated near infrared (NIR), and spatially mapped co-axial melt pool thermal emission data into Peregrine to enable detection of subsurface flaws. XCT data was used as ground truth training data to allow Peregrine’s deep learning algorithms to recognize anomalies in these complex data streams in both test artifacts and industrially relevant geometries. Completion of this program has seen the successful implementation of multi-modal, multi-layer sensor data footprints for training of machine learning models in Peregrine. Flaws detected in XCT data have been successfully detected directly from this in situ data footprint, and initial analyses of the in situ probability-of-detection has been conducted, showing performance levels commensurate with traditional non-destructive evaluation (NDE) methods. The in situ monitoring methodology was then applied to an industrially relevant component that was using post-build NDE, highlighting the utility of the proposed method for hard-to-inspect AM components. As a direct result of this program, two journal manuscripts [1], [2] have been published in Additive Manufacturing, with additional manuscripts planned following program completion.

36 MATERIALS SCIENCE↗
NASA NTRS2001

Electronic Nose: Evaluation of Kamina Prototype Unit

The Kamina, Sam and Cyranose electronic nose systems were evaluated and partially trained. Much work was performed on the Kamina as it has the ability to respond to low (less than 10 ppb) concentrations of hydrazine compounds. We were able to tell the difference between Hydrazine (Hz) and Monomethylhydrazine (MMH) in standard clean humid air. We were able to detect MMH in reduced pressure (1/3 atm) at about 250 ppb, however the training set was to far from the real situation to be useful now. Various engineering and usability aspects of both the noses was noted, especially the software. One serious physical engineering flaw was remedied in the Kamina system. A gas flow manifold was created for the Sam system. Different chips were evaluated for the Kamina system. It is still unclear if they can be exchanged without retraining the software.The Sam Detect commercial unit was evaluated for solvent detection and evaluation. It was able to successfully identify some solvents. The Cyranose, was observed and evaluated for two days. It has the ability to detect gasses in the 100 parts per million level but not the 10 parts per billion level. It is very sensitive to humidity changes; there is software to partially handle this.

Schattke, Nathan↗
NASA NTRS2019

Assessing Reliability of NDE Flaw Detection Using Smaller Number of Demonstration Data Points

The paper provides an engineering analysis approach for assessing reliability of NDE flaw detection using smaller number of demonstration data points. It explores dependence of probability of detection (POD), probability of false positive (POF), on contrast-to-noise ratio, and net decision threshold-to-noise ratio in a simulated data; and draws some generically applicable inferences to devise the approach. ASTM nondestructive evaluation standards provide requirements on signal-to-noise ratio and/or contrast-to-noise ratio in order to provide reliable flaw detection and limit false positive calls. POD analysis of inspection test data results in an estimated flaw size, denoted by 𝑎90/95. This flaw size has 90% POD and minimum 95% confidence. POF is also estimated in the analysis. POD demonstration requires specimens with flaws of known size. In many situations, it is very expensive to produce the large number of flaws required for the POD analysis. In some situations, only real flaws can truly represent the flaws for demonstration. Real flaws of correct size and location in part configuration specimen may be difficult to produce, if not impossible. Here, an engineering analysis approach is devised using simulation to assess reliability of NDE technique when a limited number of flaws are available for demonstration. In this simulation, a technique is considered reliable, if it provides flaw detectability size equal to or better than the theoretical 𝑎90𝑡ℎ used in simulation and also provides a POF less than or equal to a chosen value. The paper uses simulated signal response versus flaw size data to devise the approach. Linear correlation is used between the signal response data and flaw size. POD software mh1823 uses generalized linear model (GLM) in POD analysis after transforming the flaw size and signal response, if needed, using logarithm. Therefore, this approach is in agreement with the linear signal correlation used in mh1823. Using the POD analysis of data, generic conditions on contrast-to-noise ratio and net decision threshold-to-noise ratio are derived for reliable flaw detection. In order to assess technique reliability using the engineering approach, signal response-to-flaw size correlation about the flaw size of concern is needed. In addition, measurement of noise is also needed. If the technique meets the above requirements, assumption of linear signal-to-flaw size correlation and conditions on noise, then the technique can be assessed using this analysis as it fits the underlying POD model used here. The approach is conservative and is designed to provide a larger flaw size compared to the POD approach. Such NDE technique assessment approach, although, not as rigorous as POD, can be cost effective if the larger flaw size can be tolerated. Typically, this is a situation for all quality control NDE inspections. Here, an NDE technique needs to be reliable and 𝑎90/95 is not estimated, but the assessed flaw size is assumed to be larger than the unknown a90 due to conservative factors or margins. Applicability of the approach for assessing reliability of flaw detection in x-ray radiography and 2D imaging in general is also explored.

Koshti, Ajay M.↗
NASA NTRS2012

Research and Development of Automated Eddy Current Testing for Composite Overwrapped Pressure Vessels

Eddy current testing (ET) was used to scan bare metallic liners used in the fabrication of composite overwrapped pressure vessels (COPVs) for flaws which could result in premature failure of the vessel. The main goal of the project was to make improvements in the areas of scan signal to noise ratio, sensitivity of flaw detection, and estimation of flaw dimensions. Scan settings were optimized resulting in an increased signal to noise ratio. Previously undiscovered flaw indications were observed and investigated. Threshold criteria were determined for the system software's flaw report and estimation of flaw dimensions were brought to an acceptable level of accuracy. Computer algorithms were written to import data for filtering and a numerical derivative filtering algorithm was evaluated.

Carver, Kyle L.↗
DOE OSTI2025

Demonstrating Advanced Sensors for In-Situ Monitoring Towards Qualification of Nuclear Relevant Components

The U.S. Department of Energy’s Office of Nuclear Energy Advanced Materials and Manufacturing Technologies (AMMT) program is pursuing qualification of laser powder bed fusion (LPBF) components for nuclear applications. A major focus of this effort is the use of in situ process monitoring and machine learning–based tools to establish real-time quality assurance. The primary objective of this report is to identify and evaluate the most relevant in situ sensor systems for LPBF, and to document the deployment of these systems across platforms critical to the AMMT program. This work demonstrates how in situ monitoring can detect process anomalies, track geometry-dependent flaws, and identify limiting combinations of processing parameters—particularly those related to energy density and complex geometries (e.g., overhanging structures). To support this goal, a diverse suite of sensor modalities was evaluated across LPBF platforms, including visible and near-infrared (NIR) imaging, fringe projection profilometry, long-wavelength infrared (LWIR) thermography, and high-speed photodiode/pyrometry systems. These sensor streams were integrated with Peregrine, a machine-agnostic software platform that, among other capabilities, can generate real-time process anomaly classification. This report documents sensor deployments on multiple AMMT flagship platforms, including the Concept Laser M2 and Renishaw AM400/AM250 systems. Calibration builds with complex, flaw-prone geometries such as unsupported overhangs, stepped features, and thin walls, were used to evaluate how well Peregrine and its associated sensors could detect process anomalies and other instabilities under varied energy densities. It will be shown how Peregrine reliably identifies common process anomalies such as recoater streaking, superelevation, etc., and can be used in post-build analysis for anomaly spatial distributions throughout the build height to better understand the impact of geometry and processing parameter choice on the build. This work demonstrates measurable progress toward the vision that components can be born-qualified by establishing a real-time monitoring framework, identifying limiting process conditions, and laying the foundation for sensor fusion–enabled prediction pipelines that are scalable across platforms and applicable to nuclear-relevant components.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS2007

Air Data Report Improves Flight Safety

NASA's Aviation Safety Program in the NASA Aeronautics Research Mission Directorate, which seeks to make aviation safer by developing tools for flight data analysis and interpretation and then by transferring these tools to the aviation industry, sponsored the development of Morning Report software. The software, created at Ames Research Center with the assistance of the Pacific Northwest National Laboratory, seeks to detect atypicalities without any predefined parameters-it spots deviations and highlights them. In 2004, Sagem Avionics Inc. entered a licensing agreement with NASA for the commercialization of the Morning Report software, and also licensed the NASA Aviation Data Integration System (ADIS) tool, which allows for the integration of data from disparate sources into the flight data analysis process. Sagem Avionics incorporated the Morning Report tool into its AGS product, a comprehensive flight operations monitoring system that helps users detect irregular or divergent practices, technical flaws, and problems that might develop when aircraft operate outside of normal procedures. Sagem developed AGS in collaboration with airlines, so that the system takes into account their technical evolutions and needs, and each airline is able to easily perform specific treatments and to build its own flight data analysis system. Further, the AGS is designed to support any aircraft and flight data recorders.

Source record↗
NASA NTRS1992

Fracture mechanics expert system

Attention is given to fracture mechanics, an analytical method used extensively in the National Space Transportation System to conservatively predict the remaining service life of an article when a flaw or a material defect is detected. These analyses are performed on hardware containing material defects that have been detected by various nondestructive inspection techniques. An expert system being developed to streamline the process so that hardware dispositions may be obtained in a timely and consistent manner is discussed. The expert system reduces the potential for errors due to the manual transcription between the various software programs involved in completing a fracture mechanics analysis. NEXPERT Object, the expert system development shell selected for this purpose, allows the various software programs used in fracture mechanics analyses to be accessed and manipulated from the same platform.

Powers, E.↗
NASA NTRS2012

In-Situ Nondestructive Evaluation of Kevlar(Registered Trademark)and Carbon Fiber Reinforced Composite Micromechanics for Improved Composite Overwrapped Pressure Vessel Health Monitoring

NASA has been faced with recertification and life extension issues for epoxy-impregnated Kevlar 49 (K/Ep) and carbon (C/Ep) composite overwrapped pressure vessels (COPVs) used in various systems on the Space Shuttle and International Space Station, respectively. Each COPV has varying criticality, damage and repair histories, time at pressure, and pressure cycles. COPVs are of particular concern due to the insidious and catastrophic burst-before-leak failure mode caused by stress rupture (SR) of the composite overwrap. SR life has been defined [1] as the minimum time during which the composite maintains structural integrity considering the combined effects of stress level(s), time at stress level(s), and associated environment. SR has none of the features of predictability associated with metal pressure vessels, such as crack geometry, growth rate and size, or other features that lend themselves to nondestructive evaluation (NDE). In essence, the variability or surprise factor associated with SR cannot be eliminated. C/Ep COPVs are also susceptible to impact damage that can lead to reduced burst pressure even when the amount of damage to the COPV is below the visual detection threshold [2], thus necessitating implementation of a mechanical damage control plan [1]. Last, COPVs can also fail prematurely due to material or design noncompliance. In each case (SR, impact or noncompliance), out-of-family behavior is expected leading to a higher probability of failure at a given stress, hence, greater uncertainty in performance. For these reasons, NASA has been actively engaged in research to develop NDE methods that can be used during post-manufacture qualification, in-service inspection, and in-situ structural health monitoring. Acoustic emission (AE) is one of the more promising NDE techniques for detecting and monitoring, in real-time, the strain energy release and corresponding stress-wave propagation produced by actively growing flaws and defects in composite materials [3,4,5,6,7,8]. To gain further insight into the mechanisms responsible for composite rupture, broadband modal acoustic emission analysis was used. Also, since AE data reduction proved to be very time consuming, specialized data reduction software was written to automate the process.

Waller, Jess↗
NASA NTRS2003

High-Performance Acousto-Ultrasonic Scan System Being Developed

Acousto-ultrasonic (AU) interrogation is a single-sided nondestructive evaluation (NDE) technique employing separated sending and receiving transducers. It is used for assessing the microstructural condition and distributed damage state of the material between the transducers. AU is complementary to more traditional NDE methods, such as ultrasonic cscan, x-ray radiography, and thermographic inspection, which tend to be used primarily for discrete flaw detection. Throughout its history, AU has been used to inspect polymer matrix composites, metal matrix composites, ceramic matrix composites, and even monolithic metallic materials. The development of a high-performance automated AU scan system for characterizing within-sample microstructural and property homogeneity is currently in a prototype stage at NASA. This year, essential AU technology was reviewed. In addition, the basic hardware and software configuration for the scanner was developed, and preliminary results with the system were described. Mechanical and environmental loads applied to composite materials can cause distributed damage (as well as discrete defects) that plays a significant role in the degradation of physical properties. Such damage includes fiber/matrix debonding (interface failure), matrix microcracking, and fiber fracture and buckling. Investigations at the NASA Glenn Research Center have shown that traditional NDE scan inspection methods such as ultrasonic c-scan, x-ray imaging, and thermographic imaging tend to be more suited to discrete defect detection rather than the characterization of accumulated distributed micro-damage in composites. Since AU is focused on assessing the distributed micro-damage state of the material in between the sending and receiving transducers, it has proven to be quite suitable for assessing the relative composite material state. One major success story at Glenn with AU measurements has been the correlation between the ultrasonic decay rate obtained during AU inspection and the mechanical modulus (stiffness) seen during fatigue experiments with silicon carbide/silicon carbide (SiC/SiC) ceramic matrix composite samples. As shown in the figure, ultrasonic decay increased as the modulus decreased for the ceramic matrix composite tensile fatigue samples. The likely microstructural reason for the decrease in modulus (and increase in ultrasonic decay) is the matrix microcracking that commonly occurs during fatigue testing of these materials. Ultrasonic decay has shown the capability to track the pattern of transverse cracking and fiber breakage in these composites.

Roth, Don J.↗
NASA NTRS2003

High-Performance Acousto-Ultrasonic Scan System Being Developed

Acousto-ultrasonic (AU) interrogation is a single-sided nondestructive evaluation (NDE) technique employing separated sending and receiving transducers. It is used for assessing the microstructural condition and distributed damage state of the material between the transducers. AU is complementary to more traditional NDE methods, such as ultrasonic cscan, x-ray radiography, and thermographic inspection, which tend to be used primarily for discrete flaw detection. Throughout its history, AU has been used to inspect polymer matrix composites, metal matrix composites, ceramic matrix composites, and even monolithic metallic materials. The development of a high-performance automated AU scan system for characterizing within-sample microstructural and property homogeneity is currently in a prototype stage at NASA. This year, essential AU technology was reviewed. In addition, the basic hardware and software configuration for the scanner was developed, and preliminary results with the system were described. Mechanical and environmental loads applied to composite materials can cause distributed damage (as well as discrete defects) that plays a significant role in the degradation of physical properties. Such damage includes fiber/matrix debonding (interface failure), matrix microcracking, and fiber fracture and buckling. Investigations at the NASA Glenn Research Center have shown that traditional NDE scan inspection methods such as ultrasonic c-scan, x-ray imaging, and thermographic imaging tend to be more suited to discrete defect detection rather than the characterization of accumulated distributed microdamage in composites. Since AU is focused on assessing the distributed microdamage state of the material in between the sending and receiving transducers, it has proven to be quite suitable for assessing the relative composite material state. One major success story at Glenn with AU measurements has been the correlation between the ultrasonic decay rate obtained during AU inspection and the mechanical modulus (stiffness) seen during fatigue experiments with silicon carbide/silicon carbide (SiC/SiC) ceramic matrix composite samples. As shown in the figure, ultrasonic decay increased as the modulus decreased for the ceramic matrix composite tensile fatigue samples. The likely microstructural reason for the decrease in modulus (and increase in ultrasonic decay) is the matrix microcracking that commonly occurs during fatigue testing of these materials. Ultrasonic decay has shown the capability to track the pattern of transverse cracking and fiber breakage in these composites.

Roth, Don J.↗
DOE OSTI2023

Summary of Methodology for Mitigating Risks Associated with Licensing and Qualifying AM Nuclear Materials

The US Department of Energy’s Advanced Materials and Manufacturing Technologies (AMMT) program focuses on accelerating the development, qualification, demonstration, and deployment of advanced materials and manufacturing technologies to enable reliable and economical nuclear energy. Laser powder bed fusion (LPBF) is one of the most popular additive manufacturing (AM) processes for fabricating components with intrinsically complex geometries. LPBF was extensively explored for nuclear applications under the previous Transformational Challenge Reactor program. Additionally, Oak Ridge National Laboratory developed and licensed the Peregrine software and larger digital platform that couples machine learning and in situ data collection during AM to detect anomalies and any evolved defects. The digital platform will be critical to (1) the qualification of AM components for nuclear applications that link location-specific data to macroscopic properties and (2) predict final component performance. Current in situ process monitoring tools are valuable for observing the formation of stochastic flaws, but additional data are needed to predict the resulting microstructures and associated material performance. Rapid cooling rates and large thermal gradients have caused large heterogeneities in the microstructure, which cause anisotropy in mechanical performance. The AMMT program is evaluating the best approaches for addressing these heterogeneities and their effect on component performance using a combination of multiscale modeling, enhanced in situ process monitoring, and high throughput experimental testing. This report summarizes strategies for mitigating the risks associated with qualifying AM components, including developing new sensing capabilities for in situ process monitoring and characterizing melt pool solidification and residual stresses to inform multiscale modeling efforts.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗