Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “deep learning methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,045 records · Page 58
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

REV-INR: Regularized Evidential Implicit Neural Representation for Uncertainty-Aware Volume Visualization

Applications of Implicit Neural Representations (INRs) have emerged as a promising deep learning approach for compactly representing large volumetric datasets. These models can act as surrogates for volume data, enabling efficient storage and on-demand reconstruction via model predictions. However, conventional deterministic INRs only provide value predictions without insights into the model’s prediction uncertainty or the impact of inherent noisiness in the data. This limitation can lead to unreliable data interpretation and visualization due to prediction inaccuracies in the reconstructed volume. Identifying erroneous results extracted from model-predicted data may be infeasible, as raw data may be unavailable due to its large size. To address this challenge, we introduce REV-INR, Regularized Evidential Implicit Neural Representation, which learns to predict data values accurately along with the associated coordinate-level data uncertainty and model uncertainty using only a single forward pass of the trained REV-INR during inference. By comprehensively comparing and contrasting REV-INR with existing well-established deep uncertainty estimation methods, we show that REV-INR achieves the best volume reconstruction quality with robust data (aleatoric) and model (epistemic) uncertainty estimates using the fastest inference time. Consequently, we demonstrate that REV-INR facilitates assessment of the reliability and trustworthiness of the extracted isosurfaces and volume visualization results, enabling analyses to be solely driven by model-predicted data.

Saklani, Shanu [Indian Institute of Technology, Ka↗
DOE OSTI2025

Machine learning models for PDE constrained optimization

Partial differential equation (PDE)-constrained optimization problems arise in a variety of scientific and engineering applications, such as topology optimization, electrodynamics, fluid dynamics, and structural dynamics. However, these problems are often challenging and computationally expensive to solve, due to the need to solve the PDEs within the optimization loop. One approach to reducing the computational cost of these methods while providing convergence guarantees is through inexact trust region methods; this method uses lower fidelity solutions of the PDE at early stages of the optimization and adjusts the required accuracy of inexact PDE solvers as the optimization progresses. In this work, we explore the use of machine learning based surrogate models with these inexact trust region methods. We first demonstrate the potential of this approach by using Gaussian processes as the surrogate model and test this on a simple PDE-constrained optimization problem. We then document explorations into improving the computational costs of evolutional deep neural network / neural Galerkin methods, with the eventual goal of using these methods with the inexact trust region algorithms. We are able to speed up these approaches, albeit at the cost of lower accuracy.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Physics-Guided, Physics-Informed, and Physics-Encoded Neural Networks and Operators in Scientific Computing: Fluid and Solid Mechanics

Abstract Advancements in computing power have recently made it possible to utilize machine learning and deep learning to push scientific computing forward in a range of disciplines, such as fluid mechanics, solid mechanics, materials science, etc. The incorporation of neural networks is particularly crucial in this hybridization process. Due to their intrinsic architecture, conventional neural networks cannot be successfully trained and scoped when data are sparse, which is the case in many scientific and engineering domains. Nonetheless, neural networks provide a solid foundation to respect physics-driven or knowledge-based constraints during training. Generally speaking, there are three distinct neural network frameworks to enforce the underlying physics: (i) physics-guided neural networks (PgNNs), (ii) physics-informed neural networks (PiNNs), and (iii) physics-encoded neural networks (PeNNs). These methods provide distinct advantages for accelerating the numerical modeling of complex multiscale multiphysics phenomena. In addition, the recent developments in neural operators (NOs) add another dimension to these new simulation paradigms, especially when the real-time prediction of complex multiphysics systems is required. All these models also come with their own unique drawbacks and limitations that call for further fundamental research. This study aims to present a review of the four neural network frameworks (i.e., PgNNs, PiNNs, PeNNs, and NOs) used in scientific computing research. The state-of-the-art architectures and their applications are reviewed, limitations are discussed, and future research opportunities are presented in terms of improving algorithms, considering causalities, expanding applications, and coupling scientific and deep learning solvers.

Computer Science↗
DOE OSTI2025

DNN-based Signal Processing for Liquid Argon Time Projection Chambers

We investigate a deep learning-based signal processing for liquid argon time projection chambers (LArTPCs), a leading detector technology in neutrino physics. Identifying regions of interest (ROIs) in LArTPCs is challenging due to signal cancellation from bipolar responses and various detector effects observed in real data. We approach ROI identification as an image segmentation task, and employ a U-ResNet architecture. The network is trained on samples that incorporate detector geometry information and include a range of detector variations. Our approach significantly outperforms traditional methods while maintaining robustness across diverse detector conditions. This method has been adopted for signal processing in the Short-Baseline Neutrino program and provides a valuable foundation for future experiments such as the Deep Underground Neutrino Experiment.

Bhat, Avinay [Chicago U.]↗
DOE OSTI2024

Identifying Sample Provenance From SEM/EDS Automated Particle Analysis via Few-Shot Learning Coupled With Similarity Graph Clustering

Automated particle analysis (APA) provides a vast amount of compositional data via energy-dispersive X-ray spectroscopy along with size and shape data via scanning electron microscopy for individual particles in a sample. In many instances, APA data are leveraged to support identification of the source of a sample based on the detection of particles of a specific composition. Often, the particles that provide context make up a minuscule portion of the sample. Additionally, the interpretation of complex samples can be difficult due to the diversity of compositions both in the mixture and within a particle. In this work, we demonstrate a method to compute and cluster similarity graphs that describe inter-particle relationships within a sample using a multi-modal few-shot learning neural network. Here, as a proof-of-concept, we show that samples known to have been exposed to gunshot residue can be distinguished from samples occasionally mistaken for gunshot residue. Our workflow builds upon standard APA techniques and data processing methods to unveil additional information in a readily interpretable and quantitatively comparable format.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Real-Time Fault Tracking and Ground Motion Prediction for Large Earthquakes With HR-GNSS and Deep Learning

Earthquake early warning (EEW) systems aim to forecast the shaking intensity rapidly after an earthquake occurs and send warnings to affected areas before the onset of strong shaking. The system relies on rapid and accurate estimation of earthquake source parameters. However, it is known that source estimation for large ruptures in real-time is challenging, and it often leads to magnitude underestimation. In a previous study, we showed that machine learning, HR-GNSS, and realistic rupture synthetics can be used to reliably predict earthquake magnitude. This model, called Machine-Learning Assessed Rapid Geodetic Earthquake model (M-LARGE), can rapidly forecast large earthquake magnitudes with an accuracy of 99%. Here, we expand M-LARGE to predict centroid location and fault size, enabling the construction of the fault rupture extent for forecasting shaking intensity using existing ground motion models. We test our model in the Chilean Subduction Zone with thousands of simulated and five real large earthquakes. The result achieves an average warning time of 40.5 s for shaking intensity MMI4+, surpassing the 34 s obtained by a similar GNSS EEW model. Our approach addresses a critical gap in existing EEW systems for large earthquakes by demonstrating real-time fault tracking feasibility without saturation issues. This capability leads to timely and accurate ground motion forecasts and can support other methods, enhancing the overall effectiveness of EEW systems. Additionally, the ability to predict source parameters for real Chilean earthquakes implies that synthetic data, governed by our understanding of earthquake scaling, is consistent with the actual rupture processes.

58 GEOSCIENCES↗
DOE OSTI2020

Ensembles of Networks Produced from Neural Architecture Search

Neural architecture search (NAS) is a popular topic at the intersection of deep learning and high performance computing. NAS focuses on optimizing the architecture of neural networks along with their hyperparameters in order to produce networks with superior performance. Much of the focus has been on how to produce a single best network to solve a machine learning problem, but as NAS methods produce many networks that work very well, this affords the opportunity to ensemble these networks to produce an improved result. Additionally, the diversity of network structures produced by NAS drives a natural bias towards diversity of predictions produced by the individual networks. This results in an improved ensemble over simply creating an ensemble that contains duplicates of the best network architecture retrained to have unique weights.

Herron, Emily↗
DOE OSTI2023

Fast and accurate reduced-order modeling of a MOOSE-based additive manufacturing model with operator learning

One predominant challenge in additive manufacturing (AM) is to achieve specific material properties by manipulating manufacturing process parameters during the runtime. Such manipulation tends to increase the computational load imposed on existing simulation tools employed in AM. The goal of the present work is to construct a fast and accurate reduced-order model (ROM) for an AM model developed within the Multiphysics Object-Oriented Simulation Environment (MOOSE) framework, ultimately reducing the time/cost of AM control and optimization processes. Our adoption of the operator learning (OL) approach enabled us to learn a family of differential equations produced by altering process variables in the laser’s Gaussian point heat source. More specifically, we used the Fourier neural operator (FNO) and deep operator network (DeepONet) to develop ROMs for time-dependent responses. Furthermore, we benchmarked the performance of these OL methods against a conventional deep neural network (DNN)-based ROM. Ultimately, we found that OL methods offer comparable performance and, in terms of accuracy and generalizability, even outperform DNN at predicting scalar model responses. The DNN-based ROM afforded the fastest training time. Furthermore, all the ROMs were faster than the original MOOSE model yet still provided accurate predictions. FNO had a smaller mean prediction error than DeepONet, with a larger variance for time-dependent responses. Unlike DNN, both FNO and DeepONet were able to simulate time series data without the need for dimensionality reduction techniques. Finally, the present work can help facilitate the AM optimization process by enabling faster execution of simulation tools while still preserving evaluation accuracy.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Neural network potential for Zr–Rh system by machine learning

Zr–Rh metallic glass has enabled its many applications in vehicle parts, sports equipment and so on due to its outstanding performance in mechanical property, but the knowledge of the microstructure determining the superb mechanical property remains yet insufficient. Here, we develop a deep neural network potential of Zr–Rh system by using machine learning, which breaks the dilemma between the accuracy and efficiency in molecular dynamics simulations, and greatly improves the simulation scale in both space and time. Here, the results show that the structural features obtained from the neural network method are in good agreement with the cases in ab initio molecular dynamics simulations. Furthermore, we build a large model of 5400 atoms to explore the influences of simulated size and cooling rate on the melt-quenching process of Zr 77 Rh 23 . Our study lays a foundation for exploring the complex structures in amorphous Zr 77 Rh 23 , which is of great significance for the design and practical application.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2024

Classical analog of quantum models in synthetic dimensions

We introduce a classical analog of quantum matter in ultracold molecule-synthetic or Rydberg atom-synthetic dimensions, by extending the Potts model to include interactions J 1 between atoms adjacent in both real and synthetic space and studying its finite-temperature properties. For intermediate values of J 1 , the resulting phases and phase diagrams are similar to those of the clock and Villain models, in which three phases emerge. There exists a sheet phase analogous to that found in quantum synthetic dimension models between the high-temperature disordered phase and the low-temperature ferromagnetic phase. Furthermore, we also employ machine learning to uncover nontrivial features of the phase diagram using the learning by confusion approach, which is able to discern several successive phase transitions.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2022

Adaptive Hierarchical Cyber Attack Detection and Localization in Active Distribution Systems

Development of a cyber security strategy for the active distribution systems is challenging due to the inclusion of distributed renewable energy generations. Here this paper proposes an adaptive hierarchical cyber attack detection and localization framework for distributed active distribution systems via analyzing electrical waveforms. Cyber attack detection is based on a sequential deep learning model, via which even minor cyber attacks can be identified. The two-stage cyber attack localization algorithm first estimates the cyber attack sub-region, and then localize the specified cyber attack within the estimated subregion. We propose a modified spectral clustering-based network partitioning method for the hierarchical cyber attack ‘coarse’ localization. Next, to further narrow down the cyber attack location, a normalized impact score based on waveform statistical metrics is proposed to obtain a ‘fine’ cyber attack location by characterizing different waveform properties. Finally, compared with classical and state-of-art methods, a comprehensive quantitative evaluation with two case studies shows promising estimation results of the proposed framework.

42 ENGINEERING↗
DOE OSTI2020

Calorimetry with deep learning: particle simulation and reconstruction for collider physics

Using detailed simulations of calorimeter showers as training data, we investigate the use of deep learning algorithms for the simulation and reconstruction of single isolated particles produced in high-energy physics collisions. We train neural networks on single-particle shower data at the calorimeter-cell level, and show significant improvements for simulation and reconstruction when using these networks compared to methods which rely on currently-used state-of-the-art algorithms. We define two models: an end-to-end reconstruction network which performs simultaneous particle identification and energy regression of particles when given calorimeter shower data, and a generative network which can provide reasonable modeling of calorimeter showers for different particle types at specified angles and energies. We investigate the optimization of our models with hyperparameter scans. Furthermore, we demonstrate the applicability of the reconstruction model to shower inputs from other detector geometries, specifically ATLAS-like and CMS-like geometries. These networks can serve as fast and computationally light methods for particle shower simulation and reconstruction for current and future experiments at particle colliders.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

pnnl/neural_ODE_ICLR2020

We show how to model discrete ordinary differential equations (ODE) with algebraic nonlinearities as deep neural networks with varying degrees of prior knowledge. We derive the stability guarantees of the network layers based on the implicit constraints imposed on the weight's eigenvalues. Moreover, we show how to use barrier methods to generically handle additional inequality constraints. We demonstrate the prediction accuracy of learned neural ODEs evaluated on open-loop simulations compared to ground truth dynamics with bi-linear terms.

Tuor, Aaron↗
DOE OSTI2021

Variational Autoencoders for Learning Nonlinear Dynamics of Physical Systems

We develop data-driven methods for incorporating physical information for priors to learn parsimonious representations of nonlinear systems arising from parameterized PDEs and mechanics. Our approach is based on Variational Autoencoders (VAEs) for learning nonlinear state space models from observations. We develop ways to incorporate geometric and topological priors through general manifold latent space representations. We investigate the performance of our methods for learning low dimensional representations for the nonlinear Burgers equation and constrained mechanical systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Automating the detection of hydrological barriers and fragmentation in wetlands using deep learning and InSAR

The loss of hydrological connectivity and fragmentation of natural wetlands is a widespread driver of wetland degradation. Understanding where and how natural connectivity is impaired is essential for managing, protecting and remediating these ecosystems. Wetland Interferometric Synthetic Aperture Radar (Wetland InSAR) can provide information on surface flow orientation in wetlands at a high spatial resolution, which can be used for barrier detection. However, the broad application of this approach is constrained by the labour-intensive manual delineation of barriers based on mapped water levels. This study presents the first deep learning-based methodology for the automated detection of hydrological barriers. We trained a deep convolutional network to segment edge features of hydrological barriers in 25 image pairs captured by ALOS PALSAR-1 L-Band InSAR between 2006 and 2011. The training dataset consists of manually labelled and delineated barriers showing abrupt changes in water surface elevation and wrapped interferograms with high coherence. We tested this method across three wetland sites: the Everglades and southern Louisiana wetlands (United States) and the Cienaga de Zapata (Cuba). Across these sites, the convolutional network detected hydrological barriers with up to 84% accuracy. The model performed particularly well for linear hydrological barriers such as roads, dikes, and channels. Notably, some barriers impede flow only seasonally, appearing during low water levels and disappearing when water levels rise. Our automated approach to detecting and assessing wetland hydrologic connectivity can be applied more broadly to support the effective management of fragmented wetland ecosystems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

A physics-informed and hierarchically regularized data-driven model for predicting fluid flow through porous media

This paper presents a new deep learning data-driven model for predicting structure dependent pore-fluid velocity fields in rock. The model is based on a Convolutional Auto-Encoder (CAE) artificial neural network capable of learning from image data generated by direct numerical simulations of fluid flow through pore-structures, such as by Lattice Boltzmann or molecular dynamics methods. The main novelty of the model in comparison to previous CAE-based data-driven approaches consists of three parts. The first is a methodology for decomposing the full-domain of the porous media into sub-regions, or “sub-domains”, in order to reduce the overall size of the CAE, batch process the sub-domains in parallel, and enable the CAE to learn local and generalizable nonlinear mappings of pore-fluid velocities. The second consists of embedding the finite difference solutions of the incompressible Navier-Stokes and continuity equations into convolutional layers prior to the CAE in order to provide the CAE with knowledge of fluid dynamics physics (PhyFlow). The third main novelty is that the training of the CAE is regularized with a hierarchical loss function that encourages the learning of fluid flow patterns (in a way similar to ranked modes in principal component analysis), ranking from most to least important. This is shown to increase the stability in learning, reduce over-fitting, and promote interpretability of the CAE neural network layers (HierCAE). The comprehensive new data-driven model, which we call the PhyFlow-HierCAE model, is shown to exhibit improved accuracy and generalizability of flow field predictions over conventional CAE models, attributable to the embedded physical knowledge and the hierarchical regularization, as well as realize orders of magnitude speed-ups in computation times as a surrogate for the direct numerical simulations. Examples of training and forward predictions on unseen pore-structures are provided and evaluated for data from Lattice Boltzmann and molecular dynamics simulations of pore-fluid flow. The model is shown to be a fast and accurate emulator (or “surrogate”) for predicting effective permeability of unseen pore-structures based on learning from relatively small direct numerical simulation datasets.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Finch: Toxicity Dose Response Curve Prediction of Chemical Compounds and Mixtures

A paradigm shift in chemical risk assessment is emphasizing mixture testing over single compound analysis, eliminating animal testing, and adopting advanced modeling approaches to understand mixture activity profiles. However, existing computational models largely focus on single chemicals, with few effective solutions for modeling complex mixtures that account for synergistic or antagonistic effects and multiple Modes of Action (MoA). Conventional methods like concentration addition (CA) and independent action (IA) are insufficient for this task as they are designed for simplistic interactions and struggle to account for the dynamic and multifaceted nature of chemical mixtures, such as overlapping MoA and non-linear interactions. Finch offers a novel approach utilizing deep learning (DL) embeddings and multi-task quantitative structure-activity relationship (QSAR) models to improve chemical exposure prediction. By leveraging molecular descriptors, physiochemical properties, and large language model (LLM) embeddings from SMILES inputs, Finch preserves critical information in a latent space thereby enhancing predictive accuracy. The multi-task learning aspect of Finch is highly advantageous, as it simultaneously optimizes multiple loss functions, leveraging all available data across tasks to develop generalized representations that effectively capture complex ingredient interactions within mixtures.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Measure Utility, Gain Trust: Practical Advice for XAI Researchers

Research into explanation of machine learning models, i.e. explainable AI (XAI), has seen a sympathetic exponential growth alongside deep artificial neural networks throughout the past decade. For historical reasons explanation and trust have been intertwined. However this focus on trust is too narrow, and has led the research community astray from tried and true empirical methods that lead to more defensible scientific knowledge about people and explanations. To address this, we contribute a practical path forward for researchers in the XAI field. We recommend researchers focus on the utility and impact of their explanations instead of trust. We outline five broad use cases where explanations are useful and, for each, we describe pseudo-experiments that rely on objective empirical measurements and falsifiable hypotheses. We believe that this experimental rigor is necessary to contribute to scientific knowledge in the field of XAI.

Davis, Brittany F.↗