Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “deep learning methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Testing convolutional neural network based deep learning systems: a statistical metamorphic approach

Machine learning technology spans many areas and today plays a significant role in addressing a wide range of problems in critical domains,i.e., healthcare, autonomous driving, finance, manufacturing, cybersecurity,etc. Metamorphic testing (MT) is considered a simple but very powerful approach in testing such computationally complex systems for which either an oracle is not available or is available but difficult to apply. Conventional metamorphic testing techniques have certain limitations in verifying deep learning-based models (i.e., convolutional neural networks (CNNs)) that have a stochastic nature (because of randomly initializing the network weights) in their training. In this article, we attempt to address this problem by using a statistical metamorphic testing (SMT) technique that does not require software testers to worry about fixing the random seeds (to get deterministic results) to verify the metamorphic relations (MRs). We propose seven MRs combined with different statistical methods to statistically verify whether the program under test adheres to the relation(s) specified in the MR(s). We further use mutation testing techniques to show the usefulness of the proposed approach in the healthcare space and test two CNN-based deep learning models (used for pneumonia detection among patients). The empirical results show that our proposed approach uncovers 85.71% of the implementation faults in the classifiers under test (CUT). Furthermore, we also propose an MRs minimization algorithm for the CUT, thus saving computational costs and organizational testing resources.

Computer Science↗
DOE OSTI2025

Quantum AI Based Enhanced Detection of Dementia

Quantum computing has the potential to significantly improve the early detection of Alzheimer's Disease and Related Dementias (ADRD). Quantum-enhanced machine learning can be used to perform an early screening of Alzheimer's disease using brain imaging data based on dataset of MRI scans from both healthy individuals and those diagnosed with Alzheimer's. This study aims to demonstrate the potential of quantum transfer learning to enhance the performance of the classical deep learning model for dementia detection. Using the MRI sagittal images available in the OASIS-2 (64 demented and 72 non-demented subjects between 60 and 96 years), we show how quantum techniques can transform a suboptimal classical model into a more effective solution for dementia detection, highlighting their potential impact on advancing healthcare technology. We begin with a simple classical deep learning model with a significantly smaller number of parameters, which gives suboptimal performance on the problem. Then, we apply different configurations of quantum transfer learning based on the pre-trained weak classifier (Figure 1). We fix the weak classifier's initial convolutional layers at their fixed pre-trained parameters and replace the last set of dense layers with a dressed quantum circuit (DQN), which we train to enhance performance. We performed 4-fold cross-validation for both the classical and the hybrid quantum models and trained them using Pennylane's `default.qubit' simulator and IonQ's Aria-1 simulator (noisy simulation). We showed that with significantly fewer parameters, the quantum transfer learning-based hybrid models showed significant performance enhancement over the base weak classical deep learning model for dementia detection. To classify between a demented and non-demented subject, the accuracy of quantum-based AI methods improved by 6 to 14% compared to classical methods. The sensitivity of the models improved by 4 to 17%. This shows that there are fewer chances of misclassifying demented patients. Figure 2 compares the performance of the hybrid quantum models and their base classical model, and Table 1 summarizes the results. We illustrated that with assistance from quantum machine learning, it is possible to enhance detection for dementia based on brain images. This shows the potential for practical utility of quantum computing in ADRD research.

Bhowmik, Sounak [University of Tennessee, Knoxvill↗
DOE OSTI2025

Deep-learning-based canopy height model generation from sub-meter resolution panchromatic satellite imagery

Canopy height models (CHMs) with sufficient resolution to distinguish individual trees are useful for a variety of applications. However, standard techniques to acquire such data, such as airborne lidar surveying, are often prohibitively expensive. Deep learning techniques for generating CHMs from high-resolution imagery are an attractive option to reduce costs. To date, success with these methods has been demonstrated using multichannel aerial photography and specialized satellite data products derived from multiple sensors, neither of which is commonly available at temporal resolutions finer than one year. Here we demonstrate a method to generate sub-meter resolution CHMs in three forests in California using a more abundant data source: sub-meter resolution, panchromatic satellite imagery from a single sensor. We show that phenology and species composition play important roles in model transferability; when trained using imagery from a single conifer forest in autumn, the model performs well on autumn imagery from a second conifer forest several hundred kilometers distant with no re-training. With modest additions to the training dataset, the same model generates minimally biased estimates of canopy height in both conifer and deciduous forests during multiple seasons. Because the model operates on satellite data with global coverage and a relatively short return interval, we propose its suitability to extrapolate tree-level canopy height data to remote regions and conduct high-temporal resolution monitoring of forest structure. We furthermore demonstrate the workflow’s applicability to fire modeling by conducting simulations in forests populated by trees measured using both this approach and airborne lidar surveying. We find minimal differences in fire behavior relative to a baseline case in which only statistical distributions of tree height and crown area are known. This result underscores the value of forest structural information derived from our workflow for improving the fidelity of wildland fire simulations, among other ecological applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Exploring Classification of Topological Priors With Machine Learning for Feature Extraction

In many scientific endeavors, increasingly abstract representations of data allow for new interpretive methodologies and conceptualization of phenomena. For example, moving from raw imaged pixels to segmented and reconstructed objects allows researchers new insights and means to direct their studies toward relevant areas. Thus, the development of new and improved methods for segmentation remains an active area of research. With advances in machine learning and neural networks, scientists have been focused on employing deep neural networks such as U-Net to obtain pixel-level segmentations, namely, defining associations between pixels and corresponding/referent objects and gathering those objects afterward. Topological analysis, such as the use of the Morse-Smale complex to encode regions of uniform gradient flow behavior, offers an alternative approach: first, create geometric priors, and then apply machine learning to classify. This approach is empirically motivated since phenomena of interest often appear as subsets of topological priors in many applications. Using topological elements not only reduces the learning space but also introduces the ability to use learnable geometries and connectivity to aid the classification of the segmentation target. Here, in this article, we describe an approach to creating learnable topological elements, explore the application of ML techniques to classification tasks in a number of areas, and demonstrate this approach as a viable alternative to pixel-level classification, with similar accuracy, improved execution time, and requiring marginal training data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Non-conformity Scores for High-Quality Uncertainty Quantification from Conformal Prediction

High-quality uncertainty quantification (UQ) is a critical component of enabling trust in deep learning (DL) models and is especially important if DL models are to be deployed in high-consequence applications. Conformal prediction (CP) methods represent an emerging nonparametric approach for producing UQ that is easily interpretable and, under weak assumptions, provides a guarantee regarding UQ quality. This report describes the research outputs of an Exploratory Express Laboratory Directed Research and Development (LDRD) project at Sandia National Laboratories. This project focused on how best to implement CP methods for DL models. This report introduces new methodology for obtaining high-quality UQ from DL models using CP methods, describes a novel system of assessing UQ quality, and provides experimental results that demonstrate the quality of the new methodology and utility of the UQ quality assessment system. Avenues for future research and discussion of potential impacts at Sandia and in the wider research community are also given.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Independent malware detection architecture

A system and method (referred to as the system) detect malware by training a rule-based model, a functional based model, and a deep learning-based model from a memory snapshot of a malware free operating state of a monitored device. The system extracts a feature set from a second memory snapshot captured from an operating state of the monitored device and processes the feature set by the rule-based model, the functional-based model, and the deep learning-based model. The system identifies identifying instances of malware on the monitored device without processing data identifying an operating system of the monitored device, data associated with a prior identification of the malware, data identifying a source of the malware, data identifying a location of the malware on the monitored device, or any operating system specific data contained within the monitored device.

Smith, Jared M.↗
DOE OSTI2022

Estimating cluster masses from SDSS multiband images with transfer learning

ABSTRACT The total masses of galaxy clusters characterize many aspects of astrophysics and the underlying cosmology. It is crucial to obtain reliable and accurate mass estimates for numerous galaxy clusters over a wide range of redshifts and mass scales. We present a transfer-learning approach to estimate cluster masses using the ugriz-band images in the SDSS Data Release 12. The target masses are derived from X-ray or SZ measurements that are only available for a small subset of the clusters. We designed a semisupervised deep learning model consisting of two convolutional neural networks. In the first network, a feature extractor is trained to classify the SDSS photometric bands. The second network takes the previously trained features as inputs to estimate their total masses. The training and testing processes in this work depend purely on real observational data. Our algorithm reaches a mean absolute error (MAE) of 0.232 dex on average and 0.214 dex for the best fold. The performance is comparable to that given by redMaPPer, 0.192 dex. We have further applied a joint integrated gradient and class activation mapping method to interpret such a two-step neural network. The performance of our algorithm is likely to improve as the size of training data set increases. This proof-of-concept experiment demonstrates the potential of deep learning in maximizing the scientific return of the current and future large cluster surveys.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

Hybrid Imitation Learning for Real-Time Service Restoration in Resilient Distribution Systems

Self-healing capability is a critical factor for a resilient distribution system, which requires intelligent agents to automatically perform service restoration online, including network reconfiguration and reactive power dispatch. Here, the article proposes the imitation learning framework for training such an agent, where the agent will interact with an expert built based on the mixed-integer program to learn its optimal policy, and therefore significantly improve the training efficiency compared with exploration-dominant reinforcement learning (RL) methods. This significantly improved training efficiency makes the training problem under N-k scenarios tractable. A hybrid policy network is proposed to handle tie-line operations and reactive power dispatch simultaneously to further improve the restoration performance. The 33-bus and 119-bus systems with N-k disturbances are employed to conduct the training. The results indicate that the proposed method outperforms traditional RL algorithms such as the deep-Q network.

42 ENGINEERING↗
DOE OSTI2025

Accelerating Hamiltonian Monte Carlo for Bayesian inference in neural networks and neural operators

Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network’s parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. Finally, we show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.

Bayesian inference↗
DOE OSTI2020

Designing accurate emulators for scientific processes using calibration-driven deep models

Abstract Predictive models that accurately emulate complex scientific processes can achieve speed-ups over numerical simulators or experiments and at the same time provide surrogates for improving the subsequent analysis. Consequently, there is a recent surge in utilizing modern machine learning methods to build data-driven emulators. In this work, we study an often overlooked, yet important, problem of choosing loss functions while designing such emulators. Popular choices such as the mean squared error or the mean absolute error are based on a symmetric noise assumption and can be unsuitable for heterogeneous data or asymmetric noise distributions. We propose Learn-by-Calibrating, a novel deep learning approach based on interval calibration for designing emulators that can effectively recover the inherent noise structure without any explicit priors. Using a large suite of use-cases, we demonstrate the efficacy of our approach in providing high-quality emulators, when compared to widely-adopted loss function choices, even in small-data regimes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Time-lapse seismic inversion for CO 2 saturation with SeisCO2Net: An application to Frio-II site

Seismic monitoring of geological CO 2 storage (GCS) involves highly nonlinear seismic inversion and petrophysical inversion, making it challenging to estimate CO 2 volume efficiently and detect possible early CO 2 leakages. Deep learning (DL) using convolutional neural networks (CNNs) has shown promise in solving highly nonlinear seismic inversion problems. However, direct estimation of CO 2 plume extent/saturation from time-lapse seismic gathers using DL is still underexplored, with no reported field applications to date. The investigation of field data is primarily hindered by scarcity of field data for neural network training. Other obstacles include highly nonlinear seismic-petrophysics inverse relationship, and presence of noise in field seismic data. We introduce SeisCO2Net, a deep CNN that predicts CO 2 saturation maps directly from time-lapse full waveform shot gathers. For training, we use site-specific geological information, fluid flow physics, rock physics, and seismic modeling to generate synthetic datasets that closely resemble the CO 2 storage site. Synthetic tests show promising results, inspiring us to apply SeisCO2Net's trained weights on field data collected at Frio-II GCS site by leveraging transfer learning principles. As reference, we compare SeisCO2Net's predicted CO 2 saturation maps with results obtained from physics-based inversion. Our analyses show both methods display similar CO 2 plume shapes, reasonable CO 2 plume characteristics, and comparable saturation values. Our results suggest pre-training CNNs on physics-informed synthetic datasets and then applying the learned weights to field data is a viable approach to estimating field CO 2 saturation. This method effectively addresses the scarcity of field training data, thus encouraging the feasibility of long-term GCS monitoring.

58 GEOSCIENCES↗
DOE OSTI2022

A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems

Constructing surrogate models for uncertainty quantification (UQ) on complex partial differential equations (PDEs) having inherently high-dimensional O(10 n ), n ≥ 2, stochastic inputs (e.g., forcing terms, boundary conditions, initial conditions) poses tremendous challenges. The “curse of dimensionality” can be addressed with suitable unsupervised learning techniques used as a pre-processing tool to encode inputs onto lower-dimensional subspaces while retaining its structural information and meaningful properties. In this work, we review and investigate thirteen dimension reduction methods including linear and nonlinear, spectral, blind source separation, convex and non-convex methods and utilize the resulting embeddings to construct a mapping to quantities of interest via polynomial chaos expansions (PCE). Here, we refer to the general proposed approach as manifold PCE (m-PCE), where manifold corresponds to the latent space resulting from any of the studied dimension reduction methods. To investigate the capabilities and limitations of these methods we conduct numerical tests for three physics-based systems (treated as black-boxes) having high-dimensional stochastic inputs of varying complexity modeled as both Gaussian and non-Gaussian random fields to investigate the effect of the intrinsic dimensionality of input data. We demonstrate both the advantages and limitations of the unsupervised learning methods and we conclude that a suitable m-PCE model provides a cost-effective approach compared to alternative algorithms proposed in the literature, including recently proposed expensive deep neural network-based surrogates and can be readily applied for high-dimensional UQ in stochastic PDEs.

42 ENGINEERING↗
DOE OSTI2020

Detection and Diagnosis of Data Integrity Attacks in Solar Farms Based on Multilayer Long Short-Term Memory Network

Photovoltaic (PV) systems are becoming more vulnerable to cyber threats. In response to this emerging concern, developing cyber-secure power electronics converters has received increased attention from the IEEE Power Electronics Society that recently launched a cyber-physical-security initiative. Here this letter proposes a deep sequence learning based diagnosis solution for data integrity attacks on PV systems in smart grids, including dc–dc and dc–ac converters. The multilayer long short-term memory networks are used to leverage time-series electric waveform data from current and voltage sensors in PV systems. The proposed method has been evaluated in a PV smart grid benchmark model with extensive quantitative analysis. As a comparison, we have evaluated classic data-driven methods, including K-nearest neighbor, decision tree, support vector machine, artificial neural network, and convolutional neural network. Comparison results verify performances of the proposed method for detection and diagnosis of various data integrity attacks on PV systems.

42 ENGINEERING↗
DOE OSTI2022

Alpert multi-wavelets for functional inverse problems: direct optimization and deep learning

Computational engineering models often contain unknown entities (e.g. parameters, initial and boundary conditions) that require estimation from other measured observable data. Estimating such unknown entities is challenging when they involve spatio-temporal fields because such functional variables often require an infinite-dimensional representation. Here, we address this problem by transforming an unknown functional field using Alpert wavelet bases and truncating the resulting spectrum. Hence the problem reduces to the estimation of few coefficients that can be performed using common optimization methods. We apply this method on a one-dimensional heat transfer problem where we estimate the heat source field varying in both time and space. The observable data is comprised of temperature measured at several thermocouples in the domain. This latter is composed of either copper or stainless steel. The optimization using our method based on wavelets is able to estimate the heat source with an error between 5% and 7%. We analyze the effect of the domain material and number of thermocouples as well as the sensitivity to the initial guess of the heat source. Finally, we estimate the unknown heat source using a different approach based on deep learning techniques where we consider the input and output of a multi-layer perceptron in wavelet form. We find that this deep learning approach is more accurate than the optimization approach with errors below 4%.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Scalable Deep Learning-Based Microarchitecture Simulation on GPUs

Cycle-accurate microarchitecture simulators are essential tools for designers to architect, estimate, optimize, and manufacture new processors that meet specific design expectations. However, conventional simulators based on discrete-event methods often require an exceedingly long time-to-solution for the simulation of applications and architectures at full complexity and scale. Given the excitement around wielding the machine learning (ML) hammer to tackle various architecture problems, there have been attempts to employ ML to perform architecture simulations, such as Ithemal and SimNet. However, the direct application of existing ML approaches to architecture simulation may be even slower due to overwhelming memory traffic and stringent sequential computation logic. This work proposes the first graphics processing unit (GPU)-based microarchitecture simulator that fully unleashes the potential of GPUs to accelerate state-of-the-art ML-based simulators. First, considering the application traces are loaded from central processing unit (CPU) to GPU for simulation, we introduce various designs to reduce the data movement cost between CPUs and GPUs. Second, we propose a parallel simulation paradigm that partitions the application trace into sub-traces to simulate them in parallel with rigorous error analysis and effective error correction mechanisms. Combined, this scalable GPU-based simulator outperforms by orders of magnitude the traditional CPU-based simulators and the state-of-the-art ML-based simulators, i.e., SimNet and Ithemal.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Deep Learning for Spectroscopic X-ray Nano-Imaging Denoising

Synchrotron transmission X-ray microscopy with absorption near edge structure (TXM-XANES) is a powerful tool for investigating the structure and composition of materials at nano- to meso-scales. It is, however, often challenged by high levels of noise that obscure critical details at the single-pixel level. To address this issue, a deep learning-based algorithm is developed for suppressing the image noise, grounded in self-supervised learning principles. In contrast to traditional image denoising methods, this approach successfully enhances the visibility of fine details while significantly reducing the noise in the X-ray images. Through this advancement, the potential of the approach for improving the accuracy and interpretability of the TXM-XANES data is demonstrated, thereby enabling more precise detection of nanoscale phenomena such as inhomogeneous cation redox and metal segregation in battery cathode materials. This technique offers an effective new avenue for harnessing the full potential of synchrotron TXM-XANES imaging, paving the way for a range of exciting new studies in materials science and beyond.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Atom Identification in Bilayer Moiré Materials with Gomb-Net

Moiré patterns in van der Waals bilayer materials complicate the analysis of atomic-resolution images, hindering the atomic-scale insight typically attainable with scanning transmission electron microscopy. Here, we report a method to detect the positions and identities of atoms in each of the individual layers that compose twisted bilayer heterostructures. We developed a deep learning model, Gomb-Net, which identifies the coordinates and atomic species in each layer, deconvoluting the moiré pattern. This enables layer-specific mapping of atomic positions and dopant distributions, unlike other commonly used segmentation models which struggle with moiré-induced complexity. Using this approach, we explored the Se atom substitutional site distribution in a twisted fractional Janus WS 2 -WS 2(1–x) Se 2x heterostructure and found that layer-specific implantation sites are unaffected by the moiré pattern’s local energetic or electronic modulation. In conclusion, this advancement enables atom identification within material regimes where it was not possible before, opening new insights into previously inaccessible material physics.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗