Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NEURAL NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,063 records · Page 59
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

NuGraph2: A Graph Neural Network for Neutrino Event Reconstruction

Neutrino experiments are set to probe some of the most important open questions in physics, from CP violation and the nature of dark matter. The technology of choice for many of these experiments is the liquid argon time projection chamber (LArTPC). In current LArTPC experiments, reconstruction performance often represents a limiting factor for the sensitivity. New developments are therefore needed to unlock the full potential of LArTPC experiments. NuGraph2 is a state of the art Graph Neural Network for reconstruction of data in LArTPC experiments. NuGraph2 utilizes a heterogeneous graph structure, with separate subgraphs of 2D nodes (hits in each plane) connected across planes via 3D nodes (space points). The model provides a consistent description of the neutrino interaction across all planes. NuGraph2 is a multi-purpose network, with a common message-passing attention engine connected to multiple decoders with different classification or regression tasks. These include the classification of detector hits according to the particle type that produced them (semantic segmentation) and the separation of hits from the neutrino interaction from hits due to noise or cosmic-ray background. Additional decoders are being developed, performing tasks such as the regression of the neutrino interaction vertex position. Performance results will be presented based on publicly available samples from MicroBooNE. These include both physics performance metrics, achieving 95% accuracy for semantic segmentation and 98% classification of neutrino hits, as well as computational metrics for training and for inference on CPU or GPU. The status of the NuGraph integration in the LArSoft software framework will be presented, as well as initial studies about model interpretability and injection of domain knowledge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

NuGraph2: A Graph Neural Network for Neutrino Event Reconstruction

Neutrino experiments are set to probe some of the most important open questions in physics, from CP violation and the nature of dark matter. The technology of choice for many of these experiments is the liquid argon time projection chamber (LArTPC). In current LArTPC experiments, reconstruction performance often represents a limiting factor for the sensitivity. New developments are therefore needed to unlock the full potential of LArTPC experiments. NuGraph2 is a state of the art Graph Neural Network for reconstruction of data in LArTPC experiments [https://arxiv.org/abs/2403.11872]. NuGraph2 utilizes a heterogeneous graph structure, with separate subgraphs of 2D nodes (hits in each plane) connected across planes via 3D nodes (space points). The model provides a consistent description of the neutrino interaction across all planes. NuGraph2 is a multi-purpose network, with a common message-passing attention engine connected to multiple decoders with different classification or regression tasks. These include the classification of detector hits according to the particle type that produced them (semantic segmentation) and the separation of hits from the neutrino interaction from hits due to noise or cosmic-ray background. Additional decoders are being developed, performing tasks such as the regression of the neutrino interaction vertex position. Performance results will be presented based on publicly available samples from MicroBooNE. These include both physics performance metrics, achieving 95% accuracy for semantic segmentation and 98% classification of neutrino hits, as well as computational metrics for training and for inference on CPU or GPU. The status of the NuGraph integration in the LArSoft software framework will be presented, as well as initial studies about model interpretability and injection of domain knowledge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Neural Network Analysis of Nuclear Magnetic Resonance and Infrared Spectra

Nuclear magnetic resonance (NMR) spectroscopy and infrared (IR) spectroscopy are powerful chemical characterization techniques with broad general usage. However, the manual evaluation of the resulting spectra is time-consuming and requires significant expertise, preventing insights from being used in real-time applications. With recent advances in computation and artificial intelligence (AI), new tools are available for automating spectral interpretation. In this work, machine learning (ML) algorithms using 1-dimensional convolutional neural networks (CNNs) were applied to identify common functional groups from spectral information. Raw spectra were collected virtually from the Human Metabolome Database (HMDB) and National Institute of Standards and Technology (NIST) Chemistry WebBook and processed into a suitable standard. Algorithm design was tailored to best fit the nature of the problem, with built-in flexibility to accommodate relevant parameters beyond the raw spectral input, specifically solvent identity and magnetic frequency for NMR. The predictive capability of the algorithm in identifying functional groups is displayed in several examples. This methodology has been compiled into a code repository and could easily be modified to adapt alternative data sources, including other spectrum types. To mitigate overfitting, a common problem in mathematical modeling where overfamiliarity with training data produces trends that are not representative of the general data, a novel metric was developed, referred to as Accufit. Accufit includes a parameter that penalizes substantial differences in the training accuracy and the accuracy of an independent validation set. Examples are presented showing the effectiveness of Accufit in maintaining the model’s predictive capability while controlling the overfitting when used as a custom metric for hyperparameter tuning.

Sturgill, James↗
DOE OSTI2024

Phase Diagrams of Alloys and Their Hydrides via On-Lattice Graph Neural Networks and Limited Training Data

Efficient prediction of sampling-intensive thermodynamic properties is needed to evaluate material performance and permit high-throughput materials modeling for a diverse array of technology applications. To alleviate the prohibitive computational expense of high-throughput configurational sampling with density functional theory (DFT), surrogate modeling strategies like cluster expansion are many orders of magnitude more efficient but can be difficult to construct in systems with high compositional complexity. We therefore employ minimal-complexity graph neural network models that accurately predict and can even extrapolate to out-of-train distribution formation energies of DFT-relaxed structures from an ideal (unrelaxed) crystallographic representation. This enables the large-scale sampling necessary for various thermodynamic property predictions that may otherwise be intractable and can be achieved with small training data sets. Two exemplars, optimizing the thermodynamic stability of low-density high-entropy alloys and modulating the plateau pressure of hydrogen in metal alloys, demonstrate the power of this approach, which can be extended to a variety of materials discovery and modeling problems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Convolutional Neural Networks Based Remote Sensing Scene Classification under Clear and Cloudy Environments

Remote sensing (RS) scene classification has wide applications in the environmental monitoring and geological survey. In the real-world applications, the RS scene images taken by the satellite might have two scenarios: clear and cloudy environments. However, most of existing methods did not consider these two environments simultaneously. In this paper, we assume that the global and local features are iscriminative in either clear or cloudy environments. Many existing Convolution Neural Networks (CNN) based models have made excellent achievements in the image classification, however they somewhat ignored the global and local features in their network structure. In this paper, we propose a new CNN based network (named GLNet) with the Global Encoder and Local Encoder to extract the discriminative global and local features for the RS scene classification, where the constraints for inter-class dispersion and intra-class compactness are embedded in the GLNet training. The experimental results on two publicized RS scene classification datasets show that the proposed GLNet could achieve better performance based on many existing CNN backbones under both clear and cloudy environments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Predicting Elastic Properties of Materials from Electronic Charge Density Using 3D Deep Convolutional Neural Networks

Materials representation plays a key role in machine learning-based prediction of materials properties and new materials discovery. Currently both graph and three-dimensional (3D) voxel representation methods are based on the heterogeneous elements of the crystal structures. Here, we propose to use electronic charge density (ECD) as a generic unified 3D descriptor for materials property prediction with the advantage of possessing close relation with the physical and chemical properties of materials. We developed an ECD-based 3D convolutional neural networks (CNNs) for predicting the elastic properties of materials, in which CNNs can learn effective hierarchical features with multiple convolving and pooling operations. Extensive benchmark experiments over 2170 $Fm\bar3m$ face-centered-cubic materials show that our ECD-based CNNs can achieve good performance for elasticity prediction. Especially, our CNN models based on the fusion of elemental Materials-Agnostic Platform for Informatics and Exploration features and ECD descriptors achieved the best fivefold cross-validation performance. More importantly, we showed that our ECD-based CNN models can achieve significantly better extrapolation performance when evaluated over nonredundant data sets, where there are few neighbor-training samples around test samples. As an additional validation, we evaluated the predictive performance of our models on 329 materials of space group $Fm\bar3m$ by comparing to density functional theory calculated values, which shows a better prediction power of our model for bulk modulus than shear modulus. Because of the unified representation power of ECD, it is expected that our ECD-based CNN approach can also be applied to predict other physical and chemical properties of crystalline materials.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Magnetic field mapping of inaccessible regions using physics-informed neural networks

A difficult problem concerns the determination of magnetic field components within an experimentally inaccessible region when direct field measurements are not feasible. In this paper, we propose a new method of accessing magnetic field components using non-disruptive magnetic field measurements on a surface enclosing the experimental region. Magnetic field components in the experimental region are predicted by solving a set of partial differential equations (Ampere’s law and Gauss’ law for magnetism) numerically with the aid of physics-informed neural networks (PINNs). Prediction errors due to noisy magnetic field measurements and small number of magnetic field measurements are regularized by the physics information term in the loss function. We benchmark our model by comparing it with an older method. The new method we present will be of broad interest to experiments requiring precise determination of magnetic field components, such as searches for the neutron electric dipole moment.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Z-Target Radiography Postprocessing With A Deep Convolution Neural Network

Analyzing X-ray radiographs is crucial for understanding target behavior in Inertial Confinement Fusion (ICF) and High Energy Density (HED) platforms. However, the density of Magneto Raleigh Taylor (MRT) bands and limitations of target materials often obscure relevant spike growth and density information. To address this issue, machine learning postprocessing techniques can be applied to remove darkened regions in radiography images. In this study, a novel method is presented for removing MRT darkened regions from z-target radiographs using a convolutional neural network (CNN). The CNN, consisting of six layers, treats the darkened regions as noise and employs a mixed loss function and end-to-end frameworks to suppress them while preserving sharpness. The six-layer architecture is designed to effectively learn features when provided with a larger volume of learning space. Each layer is optimized using a mixed loss function that combines a standard loss pixel approach with a multi-scaled structural similarity index loss, which considers luminance, contrast, and structure in local neighborhoods. This approach is particularly beneficial for capturing the stochastic structure of MRT limbs. Due to the limited availability of experimental data, training is conducted using synthetic target radiography from 3D Alegra simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Machine Learning Full NMR Chemical Shift Tensors of Silicon Oxides with Equivariant Graph Neural Networks

The nuclear magnetic resonance (NMR) chemical shift tensor is a highly sensitive probe of the electronic structure of an atom and furthermore its local structure. Recently, machine learning has been applied to NMR in the prediction of isotropic chemical shifts from a structure. Current machine learning models, however, often ignore the full chemical shift tensor for the easier-to-predict isotropic chemical shift, effectively ignoring a multitude of structural information available in the NMR chemical shift tensor. Here we use an equivariant graph neural network (GNN) to predict full 29 Si chemical shift tensors in silicate materials. The equivariant GNN model predicts full tensors to a mean absolute error of 1.05 ppm and is able to accurately determine the magnitude, anisotropy, and tensor orientation in a diverse set of silicon oxide local structures. When compared with other models, the equivariant GNN model outperforms the state-of-the-art machine learning models by 53%. The equivariant GNN model also outperforms historic analytical models by 57% for isotropic chemical shift and 91% for anisotropy. The software is available as a simple-to-use open-source repository, allowing similar models to be created and trained with ease.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Thermal transport in warm dense matter revealed by refraction-enhanced x-ray radiography with a deep-neural-network analysis

Abstract Transport properties of high energy density matter affect the evolution of many systems, ranging from the geodynamo in the Earth’s core, to hydrodynamic instability growth in inertial confinement fusion capsules. Large uncertainties of these properties are present in the warm dense matter regime where both plasma models and condensed matter models become invalid. To overcome this limit, we devise an experimental platform based on x-ray differential heating and time-resolved refraction-enhanced radiography coupled to a deep neural network. We retrieve the first measurement of thermal conductivity of CH and Be in the warm dense matter regime and compare our measurement with the most commonly adopted models. The discrepancies observed are related to the estimation of a correction term from electron-electron collisions. The results necessitate improvement of transport models in the warm dense matter regime and could impact the understanding of the implosion performance for inertial confinement fusion.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2025

Spin-informed universal graph neural networks for simulating magnetic ordering

The screening and discovery of magnetic materials are hindered by the computational cost of first-principles density-functional theory (DFT) calculations required to find the ground state magnetic ordering. Although universal machine-learning interatomic potentials (uMLIPs), also known as atomistic foundation models, offer high-fidelity models of many atomistic systems with significant speedup, they currently lack the inputs required for predicting magnetic ordering. In this work, we present a data-efficient, spin-informed graph neural network framework that incorporates spin degrees of freedom as inputs and preserves physical symmetries, extending the functionality of uMLIPs to simulate magnetic orderings. This framework speeds up DFT calculations through better initial guesses for magnetic moments, determines the ground-state ordering of bulk materials and even generalizes to magnetic ordering in surfaces. Furthermore, we implement a closed-loop anomaly detection approach that effectively addresses the classic "chicken-and-egg" problem of creating a high-quality dataset while developing a uMLIP, unearthing anomalies in large benchmark datasets and boosting model accuracy.

Xu, Wenbin↗
DOE OSTI2021

Noise reduction in X-ray photon correlation spectroscopy with convolutional neural networks encoder–decoder models

Abstract Like other experimental techniques, X-ray photon correlation spectroscopy is subject to various kinds of noise. Random and correlated fluctuations and heterogeneities can be present in a two-time correlation function and obscure the information about the intrinsic dynamics of a sample. Simultaneously addressing the disparate origins of noise in the experimental data is challenging. We propose a computational approach for improving the signal-to-noise ratio in two-time correlation functions that is based on convolutional neural network encoder–decoder (CNN-ED) models. Such models extract features from an image via convolutional layers, project them to a low dimensional space and then reconstruct a clean image from this reduced representation via transposed convolutional layers. Not only are ED models a general tool for random noise removal, but their application to low signal-to-noise data can enhance the data’s quantitative usage since they are able to learn the functional form of the signal. We demonstrate that the CNN-ED models trained on real-world experimental data help to effectively extract equilibrium dynamics’ parameters from two-time correlation functions, containing statistical noise and dynamic heterogeneities. Strategies for optimizing the models’ performance and their applicability limits are discussed.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Calibrating constitutive models with full‐field data via physics informed neural networks

Abstract The calibration of solid constitutive models with full‐field experimental data is a long‐standing challenge, especially in materials that undergo large deformations. In this paper, we propose a physics‐informed deep‐learning framework for the discovery of hyperelastic constitutive model parameterizations given full‐field surface displacement data and global force‐displacement data. Contrary to the majority of recent literature in this field, we work with the weak form of the governing equations rather than the strong form to impose physical constraints upon the neural network predictions. The approach presented in this paper is computationally efficient, suitable for irregular geometric domains, and readily ingests displacement data without the need for interpolation onto a computational grid. A selection of canonical hyperelastic material models suitable for different material classes is considered including the Neo–Hookean, Gent, and Blatz–Ko constitutive models as exemplars for general non‐linear elastic behaviour, elastomer behaviour with finite strain lock‐up, and compressible foam behaviour, respectively. We demonstrate that physics informed machine learning is an enabling technology and may shift the paradigm of how full‐field experimental data are utilized to calibrate constitutive models under finite deformations.

Hamel, Craig M.↗
DOE OSTI2024

Moiré Patterns in Pt Overlayers on Gold: A Graph Neural Network Interatomic Potential Study

Overlayer structures in bimetallic catalysts are relevant to a variety of catalytic reactions, particularly in electrocatalysis for fuel cell applications. Previous computational studies largely consider these overlayer structures to be those of a pseudomorphic overlayer, where there is a 1:1 atomic ratio between the overlayer and the support metal. Our previous work based on density functional theory (DFT) has shown that there exist nonstoichiometric overlayer structures that are more stable than the stoichiometric ones. Here, in this work, we developed a graph neural network interatomic potential (GNN-IP) to analyze structures formed in Pt overlayers on Au(111). The GNN-IP was used to explore properties of nonstoichiometric overlayers at length scales that are prohibitively expensive to pursue with planewave DFT calculations. In particular, we examined large Pt islands on top of a 48 × 48 Au(111) unit cell to explore the influence of the rotational angle (α) between the Pt overlayer and support Au(111) on both the stability of and the preferred atomic density in the overlayer. Island structures with smaller rotational angles between the Pt overlayer and Au(111) tend to be more stable. Further, smaller rotational angles tend to result in lower atomic density in the Pt overlayer.

chemical structure↗
DOE OSTI2025

Testing convolutional neural network based deep learning systems: a statistical metamorphic approach

Machine learning technology spans many areas and today plays a significant role in addressing a wide range of problems in critical domains,i.e., healthcare, autonomous driving, finance, manufacturing, cybersecurity,etc. Metamorphic testing (MT) is considered a simple but very powerful approach in testing such computationally complex systems for which either an oracle is not available or is available but difficult to apply. Conventional metamorphic testing techniques have certain limitations in verifying deep learning-based models (i.e., convolutional neural networks (CNNs)) that have a stochastic nature (because of randomly initializing the network weights) in their training. In this article, we attempt to address this problem by using a statistical metamorphic testing (SMT) technique that does not require software testers to worry about fixing the random seeds (to get deterministic results) to verify the metamorphic relations (MRs). We propose seven MRs combined with different statistical methods to statistically verify whether the program under test adheres to the relation(s) specified in the MR(s). We further use mutation testing techniques to show the usefulness of the proposed approach in the healthcare space and test two CNN-based deep learning models (used for pneumonia detection among patients). The empirical results show that our proposed approach uncovers 85.71% of the implementation faults in the classifiers under test (CUT). Furthermore, we also propose an MRs minimization algorithm for the CUT, thus saving computational costs and organizational testing resources.

Computer Science↗
DOE OSTI2023

Testing Paired Neural Network Models for Aftershock Identification

Aftershock sequences are a burden to real-time seismic monitoring. Cross-correlation can be used because aftershocks exhibit similar waveforms, but the method is computationally expensive. Deep learning may be an alternative, as it is computationally efficient, but great attention to training and testing is required in order to trust that the model can generalize to new aftershock sequences. This is problematic for aftershock sequences, because large-magnitude earthquakes are unpredictable and are globally widespread. Here, we test several paired neural network (PNN) models trained on a augmented (noise-added) earthquake dataset, to determine whether they can be generalized to process real aftershock sequences. Two aftershock datasets that were originally detected by cross-correlation and subsequently validated by an expert analyst were used. We found that current PNN models struggle to generalize to aftershock sequences. However, we identify approaches to improve training future PNN models and believe that improvements may be achieved by transfer learning.

58 GEOSCIENCES↗
DOE OSTI2022

Three‐Dimensional Permeability Inversion Using Convolutional Neural Networks and Positron Emission Tomography

Abstract Quantification of heterogeneous multiscale permeability in geologic porous media is key for understanding and predicting flow and transport processes in the subsurface. Recent utilization of in situ imaging, specifically positron emission tomography (PET), enables the measurement of three‐dimensional (3‐D) time‐lapse radiotracer solute transport in geologic media. However, accurate and computationally efficient characterization of the permeability distribution that controls the solute transport process remains challenging. Leveraging the relationship between local permeability variation and solute advection behavior, an encoder‐decoder based convolutional neural network (CNN) is implemented as a permeability inversion scheme using a single PET scan of a radiotracer pulse injection experiment as input. The CNN can accurately capture the 3‐D spatial correlation between the permeability and the radiotracer solute arrival time difference maps in geologic cores. We first test the inversion accuracy using synthetic test datasets and then test the accuracy on a suite of experimental PET imaging datasets acquired on four different geologic cores. The network‐predicted permeability maps from the geologic cores are used to parameterize forward numerical models that are directly compared with the experimental PET imaging data. The results indicate that a single trained network can generate robust 3‐D permeability inversion maps in seconds. Numerical models parameterized with these permeability maps closely capture the experimentally observed solute arrival time behavior. This work provides an unprecedented approach for efficiently characterizing multiscale permeability heterogeneity in complex geologic samples.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Tetris-inspired detector with neural network for radiation mapping

Abstract Radiation mapping has attracted widespread research attention and increased public concerns on environmental monitoring. Regarding materials and their configurations, radiation detectors have been developed to identify the position and strength of the radioactive sources. However, due to the complex mechanisms of radiation-matter interaction and data limitation, high-performance and low-cost radiation mapping is still challenging. Here, we present a radiation mapping framework using Tetris-inspired detector pixels. Applying inter-pixel padding for enhancing contrast between pixels and neural networks trained with Monte Carlo (MC) simulation data, a detector with as few as four pixels can achieve high-resolution directional prediction. A moving detector with Maximum a Posteriori (MAP) further achieved radiation position localization. Field testing with a simple detector has verified the capability of the MAP method for source localization. Our framework offers an avenue for high-quality radiation mapping with simple detector configurations and is anticipated to be deployed for real-world radiation detection.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗