Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural simulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,099 records · Page 61
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Applying deep learning methods to develop new models of molecular charge transfer, nonadiabatic dynamics, and nonlinear spectroscopy in the condensed phase

Photon- and field-induced charge transfer has central importance in the generation and storage of electricity, the novel properties of materials, photo-induced catalysis, and electro-optic activity (e.g., photovoltaic cells, fuel cells, and organic chromophores for use in optical fibers and light-emission diodes). These non-equilibrium electronic and chemical transformations are probed by ultrafast, nonlinear spectroscopies. Accurate simulations play a crucial role in our ability to understand, optimize, and control these transformations. This project applies modern deep learning and machine learning (ML) methods to dramatically improve models of electronic dynamics, electronic-nuclear dynamics, and spectroscopic measurements for improved simulations of chemistry in complex environments, far from equilibrium phenomena, and processes in extreme environments, such as materials exposed to strong or resonant fields. This project develops accurate neural net models that go beyond predictive capability to also provide new insight into the fundamental physics underlying electron and nuclear dynamics. To achieve its objectives, this project explores and develops customized versions of high-capacity deep learning algorithms/models. These techniques are developed with an emphasis on fundamental chemical insight, not just predictive accuracy, to assist the development of the next generation of quantum simulation methods.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

A Hybrid Deep Learning Approach to Cosmological Constraints from Galaxy Redshift Surveys

We present a deep machine learning (ML)–based technique for accurately determining σ g and Ω m from mock 3D galaxy surveys. The mock surveys are built from the AbacusCosmos suite of N -body simulations, which comprises 40 cosmological volume simulations spanning a range of cosmological parameter values, and we account for uncertainties in galaxy formation scenarios through the use of generalized halo occupation distributions (HODs). We explore a trio of ML models: a 3D convolutional neural network (CNN), a power spectrum–based fully connected network, and a hybrid approach that merges the two to combine physically motivated summary statistics with flexible CNNs. We describe best practices for training a deep model on a suite of matched-phase simulations, and we test our model on a completely independent sample that uses previously unseen initial conditions, cosmological parameters, and HOD parameters. Despite the fact that the mock observations are quite small (~0.07 h -3 Gpc 3 ) and the training data span a large parameter space (six cosmological and six HOD parameters), the CNN and hybrid CNN can constrain estimates of σ g and Ω m to ~3% and ~4%, respectively.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

A co-kurtosis PCA based dimensionality reduction with nonlinear reconstruction using neural networks

For turbulent reacting flow systems, identification of low-dimensional representations of the thermo-chemical state space is vitally important, primarily to significantly reduce the computational cost of device-scale simulations. Principal component analysis (PCA), and its variants, are a widely employed class of methods. Recently, an alternative technique that focuses on higher-order statistical interactions, co-kurtosis PCA (CoK-PCA), has been shown to effectively provide a low-dimensional representation by capturing the stiff chemical dynamics associated with spatiotemporally localized reaction zones. While its effectiveness has only been demonstrated based on a priori analyses with linear reconstruction, in this work, we employ nonlinear techniques to reconstruct the full thermo-chemical state and evaluate the efficacy of CoK-PCA compared to PCA. Specifically, we combine a CoK-PCA-/PCA-based dimensionality reduction (encoding) with an artificial neural network (ANN) based reconstruction (decoding) and examine, a priori, the reconstruction errors of the thermo-chemical state. In addition, we evaluate the errors in species production rates and heat release rates, which are nonlinear functions of the reconstructed state, as a measure of the overall accuracy of the dimensionality reduction technique. We employ four datasets to assess CoK-PCA/PCA coupled with ANN-based reconstruction: zero-dimensional (homogeneous) reactor for autoignition of an ethylene/air mixture that has conventional single-stage ignition kinetics, a dimethyl ether (DME)/air mixture which has two-stage (low and high temperature) ignition kinetics, a one-dimensional freely propagating premixed ethylene/air laminar flame, and a two-dimensional dataset representing turbulent autoignition of ethanol in a homogeneous charge compression ignition (HCCI) engine. Finally, results from the analyses demonstrate the robustness of the CoK-PCA based low-dimensional manifold with ANN reconstruction in accurately capturing the data, specifically from the reaction zones.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

An investigation on machine learning predictive accuracy improvement and uncertainty reduction using VAE-based data augmentation

The confluence of ultrafast computers with large memory, rapid progress in Machine Learning (ML) algorithms, and the availability of large datasets place multiple engineering fields at the threshold of dramatic progress. However, a unique challenge in nuclear engineering is data scarcity because experimentation on nuclear systems is usually more expensive and time-consuming than most other disciplines. One potential way to resolve the data scarcity issue is deep generative learning, which uses certain ML models to learn the underlying distribution of existing data and generate synthetic samples that resemble the real data. In this way, one can significantly expand the dataset to train more accurate predictive ML models. In this study, our objective is to evaluate the effectiveness of data augmentation using variational autoencoder (VAE)-based deep generative models. We investigated whether the data augmentation leads to improved accuracy in the predictions of a deep neural network (DNN) model trained using the augmented data. Additionally, the DNN prediction uncertainties are quantified using Bayesian Neural Networks (BNN) and conformal prediction (CP) to assess the impact on predictive uncertainty reduction. To test the proposed methodology, we used TRACE simulations of steady-state void fraction data based on the NUPEC Boiling Water Reactor Full-size Fine-mesh Bundle Test (BFBT) benchmark. Here, we found that augmenting the training dataset using VAEs has improved the DNN model’s predictive accuracy, improved the prediction confidence intervals, and reduced the prediction uncertainties.

Bayesian neural network↗
DOE OSTI2020

Predicting Small Molecule Transfer Free Energies by Combining Molecular Dynamics Simulations and Deep Learning

Accurately predicting small molecule partitioning and hydrophobicity is critical in the drug discovery process. There are many heterogeneous chemical environments within a cell and entire human body. For example, drugs must be able to cross the hydrophobic cellular membrane to reach their intracellular targets, and hydrophobicity is an important driving force for drug–protein binding. Atomistic molecular dynamics (MD) simulations are routinely used to calculate free energies of small molecules binding to proteins, crossing lipid membranes, and solvation but are computationally expensive. Machine learning (ML) and empirical methods are also used throughout drug discovery but rely on experimental data, limiting the domain of applicability. We present atomistic MD simulations calculating 15,000 small molecule free energies of transfer from water to cyclohexane. This large data set is used to train ML models that predict the free energies of transfer. We show that a spatial graph neural network model achieves the highest accuracy, followed closely by a 3D-convolutional neural network, and shallow learning based on the chemical fingerprint is significantly less accurate. A mean absolute error of ~4 kJ/mol compared to the MD calculations was achieved for our best ML model. We also show that including data from the MD simulation improves the predictions, tests the transferability of each model to a diverse set of molecules, and show multitask learning improves the predictions. This work provides insight into the hydrophobicity of small molecules and ML cheminformatics modeling, and our data set will be useful for designing and testing future ML cheminformatics methods.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Harnessing graph convolutional neural networks for identification of glassy states in metallic glasses

Graph Convolutional Neural Networks (GCNNs) have emerged as powerful tools for analyzing materials. In this study, we employ GCNNs to examine structural characteristics of CuZr metallic glasses (MGs) and identify their states. We use molecular dynamics to simulate the quenching process of CuZr, using cooling rates ranging from 10 9 to 10 15 K/s, to produce six unique glassy states. For each state, we create a dataset comprising 1,800 distinct samples. We evaluate the effectiveness of various GCNNs, including Graph Attention Neural Network (GANN), Graph Sample and AggreGatE (GraphSAGE), Graph Isomorphism Network (GIN), and Relational Graph Convolutional Neural Network (RGCN). GANN and GraphSAGE demonstrate comparable performance, achieving an overall accuracy of 81% in classifying the MG states. Furthermore, these results underscore the potential of GCNNs to detect subtle structural variances in disordered materials and point to broader application of deep learning in the analysis of MGs and other amorphous substances.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Learning model combining convolutional deep neural network with a self-attention mechanism for AC optimal power flow

Alternating current optimal power flow (OPF) analysis is critical for efficient and reliable operation of power systems. For large systems or repetitive computations, the traditional methods such as the direct and gradient methods, or non-traditional methods, such as the genetic algorithm and simulating annealing, are time-consuming and unsuitable for real-time computing. The work in this paper proposes a novel framework to obtain the optimal solution of power flow in real-time using a combination of convolutional neural networks and a self-attention mechanism. All parameters of the power networks are rearranged in an image-like shape of a multi-channel image where each channel is a two-dimensional matrix. The proposed approach is adaptive with every input size of power systems as well as frequent variations of network topologies without intervention to the framework core. The encompassment of all power system contexts in which all parameters of internal elements, generation costs, and topology information are included, contributes to the higher accuracy of inference compared to other current machine-learning-based OPF-solving methods. Besides, the proposed framework established on ubiquitous platforms is effortlessly integrated into current infrastructures of power systems, and the great efficiency along with the computation speed may serve as a critical point for practical implications, such as enabling faster decision-making during real-time operations, predicting system contingencies, and remedial actions based on an offline pre-trained model. Furthermore, this supervised learning process is applied to the dataset of four case studies of meshed power systems: the IEEE 5-bus system (IEEE-5), the IEEE 30-bus system (IEEE-30), the IEEE 39-bus system (IEEE-39), and the IEEE 57-bus system (IEEE-57) to prove the efficacy of the proposed method.

42 ENGINEERING↗
DOE OSTI2021

Deep Learning Approaches to Surrogates for Solving the Diffusion Equation for Mechanistic Real-World Simulations

In many mechanistic medical, biological, physical, and engineered spatiotemporal dynamic models the numerical solution of partial differential equations (PDEs), especially for diffusion, fluid flow and mechanical relaxation, can make simulations impractically slow. Biological models of tissues and organs often require the simultaneous calculation of the spatial variation of concentration of dozens of diffusing chemical species. One clinical example where rapid calculation of a diffusing field is of use is the estimation of oxygen gradients in the retina, based on imaging of the retinal vasculature, to guide surgical interventions in diabetic retinopathy. Furthermore, the ability to predict blood perfusion and oxygenation may one day guide clinical interventions in diverse settings, i.e., from stent placement in treating heart disease to BOLD fMRI interpretation in evaluating cognitive function (Xie et al., 2019; Lee et al., 2020). Since the quasi-steady-state solutions required for fast-diffusing chemical species like oxygen are particularly computationally costly, we consider the use of a neural network to provide an approximate solution to the steady-state diffusion equation. Machine learning surrogates, neural networks trained to provide approximate solutions to such complicated numerical problems, can often provide speed-ups of several orders of magnitude compared to direct calculation. Surrogates of PDEs could enable use of larger and more detailed models than are possible with direct calculation and can make including such simulations in real-time or near-real time workflows practical. Creating a surrogate requires running the direct calculation tens of thousands of times to generate training data and then training the neural network, both of which are computationally expensive. Often the practical applications of such models require thousands to millions of replica simulations, for example for parameter identification and uncertainty quantification, each of which gains speed from surrogate use and rapidly recovers the up-front costs of surrogate generation. We use a Convolutional Neural Network to approximate the stationary solution to the diffusion equation in the case of two equal-diameter, circular, constant-value sources located at random positions in a two-dimensional square domain with absorbing boundary conditions. Such a configuration caricatures the chemical concentration field of a fast-diffusing species like oxygen in a tissue with two parallel blood vessels in a cross section perpendicular to the two blood vessels. To improve convergence during training, we apply a training approach that uses roll-back to reject stochastic changes to the network that increase the loss function. The trained neural network approximation is about 1000 times faster than the direct calculation for individual replicas. Because different applications will have different criteria for acceptable approximation accuracy, we discuss a variety of loss functions and accuracy estimators that can help select the best network for a particular application. We briefly discuss some of the issues we encountered with overfitting, mismapping of the field values and the geometrical conditions that lead to large absolute and relative errors in the approximate solution.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Quarknet Project: Simulating Double-Source Plane Lenses and Einstein Rings for Domain Adaptation

Double-Source Plane Lenses (DSPLs) are a form of strong gravitational lensing that can be used to uniquely constrain cosmological parameters like the dark energy equation of state. These objects are exceedingly rare: to date, only four have been discovered in astronomical surveys, like the Sloan Digital Sky Survey and the HyperSuprimeCam Survey. Additionally, the main features of these objects are typically co-located sets of arcs, which look like Einstein Rings of single-source galaxy-galaxy lenses. Larger astronomical surveys typically have low image resolution, which significantly exacerbates issues when trying to find DSPLs. Future surveys are likely to include many more of these objects and have higher image resolution. Traditional methods for finding strong lenses struggle to identify simple galaxy-galaxy lenses. Recently, neural networks have become the standard for lens identification, but these tools have not yet been used to search for DSPLs. In this work, we use the software deeplenstronomy to simulate varied images of DSPLs and Einstein Rings – with and without noise and artifacts of observational surveys, forming a comprehensive benchmark dataset of 40,000 images. We utilise this dataset to perform experiments with convolutional neural networks to discern between the two phenomena. Sofia Grimm and Jerry Zhou are co-first authors on this work.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

New Machine Learning Techniques for Simulation-Based Inference: InferoStatic Nets, Kernel Score Estimation, and Kernel Likelihood Ratio Estimation

We propose an intuitive, machine-learning approach to multiparameter inference, dubbed the InferoStatic Networks (ISN) method, to model the score and likelihood ratio estimators in cases when the probability density can be sampled but not computed directly. The ISN uses a backend neural network that models a scalar function called the inferostatic potential $\varphi$. In addition, we introduce new strategies, respectively called Kernel Score Estimation (KSE) and Kernel Likelihood Ratio Estimation (KLRE), to learn the score and the likelihood ratio functions from simulated data. We illustrate the new techniques with some toy examples and compare to existing approaches in the literature. We mention en passant some new loss functions that optimally incorporate latent information from simulations into the training procedure.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRSDate not supplied

Machine learning based noise reduction for satellite products: application to solar-induced fluorescence retrievals using simulated and real data

In the past two decades, global satellite measurements of terrestrial chlorophyll solar-induced fluorescence (SIF) have been used widely for a number of different applications related to physiology, phenology, and productivity of plants. However, SIF retrievals are inherently noisy due to the relatively small SIF spectral signature in comparison with observational noise. In this work, we examine how a spectral-based approach that employs principal component analysis along with a relatively shallow artificial neural network can be used to reduce noise and other artifacts in satellite level 2 (L2) products. We first apply the approach in a controlled environment in which radiance spectra are simulated with a full atmospheric and surface radiative transfer model for different scenarios including various SIF values that are known. Various levels of noise can be added to the simulated spectra. Resulting noisy and noise-reduced SIF retrievals are compared with the true values to assess performance. We then apply the noise reduction approach to real SIF derived from instruments flying on meteorological satellites. The results are evaluated by comparing SIF retrievals from different platforms with each other and with other independent data sets, showing enhanced capability to capture seasonal and interannual variability in SIF.

Chlorophyll fluorescence↗
DOE OSTI2021

Estimation of constituent properties of concrete materials with an artificial neural network based method

Multi-scale models are developed for heterogeneous concrete materials to estimate their macroscopic mechanical properties in terms of micro-structural data. One crucial challenge of those models is the identification of local properties of constituent phases. In this paper, we present an efficient method based on Artificial Neural Networks (ANN). Typical concrete materials are taken as example. A macroscopic analytical strength criterion is established from three steps of nonlinear homogenization procedure. The macroscopic strength of materials is determined as a function of the frictional coefficient and cohesion of solid cement particles at nanometer scale, intra-particle pores, inter-particle pores and aggregates (inclusions). The objective is to identify the nanoscopic frictional coefficient and cohesion of cement particle from measured macroscopic values of uniaxial compression and tensile strengths. For this purpose, a numerical method based on the ANN is developed. With the analytical macroscopic strength criterion, sensitivity studies are first realized to identify the most important micro-structural parameters influencing the macroscopic strength of concrete. A simplified analytical macroscopic strength criterion is then proposed. A large dataset is further constructed through the inversion of the analytical strength criterion by using the aggregates volume fraction, porosity, macroscopic uniaxial tensile and compressive strengths as input variables and the frictional coefficient and cohesion of cement particles as output unknowns. An ANN model containing four hidden layers and 100 neurons in each layer is constructed and trained by using this dataset. Various types of validation of the ANN model are performed. It is found that the proposed ANN based model can effectively predict the frictional coefficient and cohesion of porous cement paste at the microscopic scale with a very good accuracy.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Accurate, Uncertainty-Aware Classification of Molecular Chemical Motifs from Multimodal X-ray Absorption Spectroscopy

Accurate classification of molecular chemical motifs from experimental measurement is an important problem in molecular physics, chemistry, and biology. In this work, we present neural network ensemble classifiers for predicting the presence (or lack thereof) of 41 different chemical motifs on small molecules from simulated C, N, and O K-edge X-ray absorption near-edge structure (XANES) spectra. Our classifiers not only achieve class-balanced accuracies of more than 0.95 but also accurately quantify uncertainty. Here, we also show that including multiple XANES modalities improves predictions notably on average, demonstrating a “multimodal advantage” over any single modality. In addition to structure refinement, our approach can be generalized to broad applications with molecular design pipelines.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Deep material network via a quilting strategy: visualization for explainability and recursive training for improved accuracy

Recent developments integrating micromechanics and neural networks offer promising paths for rapid predictions of the response of heterogeneous materials with similar accuracy as direct numerical simulations. The deep material network is one such approaches, featuring a multi-layer network and micromechanics building blocks trained on anisotropic linear elastic properties. Once trained, the network acts as a reduced-order model, which can extrapolate the material’s behavior to more general constitutive laws, including nonlinear behaviors, without the need to be retrained. However, current training methods initialize network parameters randomly, incurring inevitable training and calibration errors. Here, we introduce a way to visualize the network parameters as an analogous unit cell and use this visualization to “quilt” patches of shallower networks to initialize deeper networks for a recursive training strategy. The result is an improvement in the accuracy and calibration performance of the network and an intuitive visual representation of the network for better explainability.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Toward ultra-efficient high-fidelity predictions of wind turbine wakes: Augmenting the accuracy of engineering models with machine learning

This study proposes a novel machine learning (ML) methodology for the efficient and cost-effective prediction of high-fidelity three-dimensional velocity fields in the wake of utility-scale turbines. The model consists of an autoencoder convolutional neural network with U-Net skipped connections, fine-tuned using high-fidelity data from large-eddy simulations (LES). The trained model takes the low-fidelity velocity field cost-effectively generated from the analytical engineering wake model as input and produces the high-fidelity velocity fields. The accuracy of the proposed ML model is demonstrated in a utility-scale wind farm for which datasets of wake flow fields were previously generated using LES under various wind speeds, wind directions, and yaw angles. Comparing the ML model results with those of LES, the ML model was shown to reduce the error in the prediction from 20% obtained from the Gauss Curl hybrid (GCH) model to less than 5%. In addition, the ML model captured the non-symmetric wake deflection observed for opposing yaw angles for wake steering cases, demonstrating a greater accuracy than the GCH model. The computational cost of the ML model is on par with that of the analytical wake model while generating numerical outcomes nearly as accurate as those of the high-fidelity LES.

Mechanics↗
DOE OSTI2023

Quantum Kerr learning

Quantum machine learning is a rapidly evolving field of research that could facilitate important applications for quantum computing and also significantly impact data-driven sciences. In our work, based on various arguments from complexity theory and physics, we demonstrate that a single Kerr mode can provide some 'quantum enhancements' when dealing with kernel-based methods. Using kernel properties, neural tangent kernel theory, first-order perturbation theory of the Kerr non-linearity, and non-perturbative numerical simulations, we show that quantum enhancements could happen in terms of convergence time and generalization error. Furthermore, we make explicit indications on how higher-dimensional input data could be considered. Finally, we propose an experimental protocol, that we call quantum Kerr learning, based on circuit QED.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Arrested Phase Separation in Double-Exchange Models: Large-Scale Simulation Enabled by Machine Learning

In this work, we present large-scale dynamical simulations of electronic phase separation in the single-band double-exchange model based on deep-learning neural-network potentials trained from small-size exact diagonalization solutions. We uncover an intriguing correlation-induced freezing behavior as doped holes are segregated from half filled insulating background during equilibration. While the aggregation of holes is stabilized by the formation of ferromagnetic clusters through Hund’s coupling between charge carriers and local magnetic moments, this stabilization also creates confining potentials for holes when antiferromagnetic spin-spin correlation is well developed in the background. The dramatically reduced mobility of the self-trapped holes prematurely disrupts further growth of the ferromagnetic clusters, leading to an arrested phase separation. Implications of our findings for phase separation dynamics in materials that exhibit colossal magnetoresistance effect are discussed.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2025

Accelerated Over-The-Air Neural Receiver Training Using Self-Contrastive Learning

Self-contrastive learning (SCL), a self-supervised learning method, has been shown to improve image and signal classifier accuracies and reduce the training time for neural communications receivers. In particular, prior work has shown that SCL applied as a pre-training step can improve simulated performance of OFDM in 3GPP TDL channel models by reducing the training time of the downstream classification task (demodulation and demapping). In this work a practical implementation demonstrating SCL pre-training using software defined radios (SDRs) is proposed.

Cooke, Corey [ORNL] (ORCID:0000000234263672)↗