Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Generative Neural Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Deep convolutional neural networks for multi-scale time-series classification and application to disruption prediction in fusion devices

The multi-scale, mutli-physics nature of fusion plasmas makes predicting plasma events challenging. Recent advances in deep convolutional neural network architectures (CNN) utilizing dilated convolutions enable accurate predictions on sequences which have long-range, multi-scale characteristics, such as the time-series generated by diagnostic instruments observing fusion plasmas. Here we apply this neural network architecture to the popular problem of disruption prediction in fusion tokamaks, utilizing raw data from a single diagnostic, the Electron Cyclotron Emission imaging (ECEi) diagnostic from the DIII-D tokamak. ECEi measures a fundamental plasma quantity (electron temperature) with high temporal resolution over the entire plasma discharge, making it sensitive to a number of potential pre-disruptions markers with different temporal and spatial scales. Promising, initial disruption prediction results are obtained training a deep CNN with large receptive field ({$\sim$}30k), achieving an $F_1$-score of {$\sim$}91\% on individual time-slices using only the ECEi data.

convolutional neural networks↗
DOE OSTI2023

Neural Networks-Based Inverter Control: Modeling and Adaptive Optimization for Smart Distribution Networks

The optimal voltage control of inverter-based resources, especially under the high penetration of solar photovoltaics, is critical to the stability of the distribution power system. However, the computational complexity as well as the coordinated operation performance of the voltage control optimization in the distribution power system limits the real-time applications. To mitigate this issue, a model-free based adaptive optimal control scheme for the smart inverter is proposed to maximize the active power generation, minimize the power loss, and maintain the bus voltages in smart distribution networks. An inverter-based optimization model for coordinated operation is first established, considering the uncertainties of renewable power generation. Subsequently, by collecting the data and control strategies, the neural networks (NNs) based algorithm is proposed to efficiently predict the best possible control strategy. The main objective of this scheme is to accurately predict candidate optimal solutions with near-negligible feasibility and optimization gaps, with the advantage of avoiding complicated iteration-based numerical algorithms. Thereafter, the co-simulation among OpenDSS, MATLAB, and Python is set up to fully take advantage of the three individual software. Experiments are conducted based on different control parameter characteristics and structures of NNs. Finally, the results reveal that an average mean squared error of 0.013 and 1 ms response time are achieved, which is lower than some state-of-the-art methods.

42 ENGINEERING↗
DOE OSTI2021

Modeling Liquid Water by Climbing up Jacob’s Ladder in Density Functional Theory Facilitated by Using Deep Neural Network Potentials

Within the framework of Kohn–Sham density functional theory (DFT), the ability to provide good predictions of water properties by employing a strongly constrained and appropriately normed (SCAN) functional has been extensively demonstrated in recent years. Here, we further advance the modeling of water by building a more accurate model on the fourth rung of Jacob’s ladder with the hybrid functional, SCAN0. In particular, we carry out both classical and Feynman path-integral molecular dynamics calculations of water with the SCAN0 functional and the isobaric–isothermal ensemble. To generate the equilibrated structure of water, a deep neural network potential is trained from the atomic potential energy surface based on ab initio data obtained from SCAN0 DFT calculations. For the electronic properties of water, a separate deep neural network potential is trained by using the Deep Wannier method based on the maximally localized Wannier functions of the equilibrated trajectory at the SCAN0 level. The structural, dynamic, and electric properties of water were analyzed. The hydrogen-bond structures, density, infrared spectra, diffusion coefficients, and dielectric constants of water, in the electronic ground state, are computed by using a large simulation box and long simulation time. For the properties involving electronic excitations, we apply the GW approximation within many-body perturbation theory to calculate the quasiparticle density of states and bandgap of water. Compared to the SCAN functional, mixing exact exchange mitigates the self-interaction error in the meta-generalized-gradient approximation and further softens liquid water toward the experimental direction. For most of the water properties, the SCAN0 functional shows a systematic improvement over the SCAN functional. However, some important discrepancies remain. The H-bond network predicted by the SCAN0 functional is still slightly overstructured compared to the experimental results.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

A physics-informed and hierarchically regularized data-driven model for predicting fluid flow through porous media

This paper presents a new deep learning data-driven model for predicting structure dependent pore-fluid velocity fields in rock. The model is based on a Convolutional Auto-Encoder (CAE) artificial neural network capable of learning from image data generated by direct numerical simulations of fluid flow through pore-structures, such as by Lattice Boltzmann or molecular dynamics methods. The main novelty of the model in comparison to previous CAE-based data-driven approaches consists of three parts. The first is a methodology for decomposing the full-domain of the porous media into sub-regions, or “sub-domains”, in order to reduce the overall size of the CAE, batch process the sub-domains in parallel, and enable the CAE to learn local and generalizable nonlinear mappings of pore-fluid velocities. The second consists of embedding the finite difference solutions of the incompressible Navier-Stokes and continuity equations into convolutional layers prior to the CAE in order to provide the CAE with knowledge of fluid dynamics physics (PhyFlow). The third main novelty is that the training of the CAE is regularized with a hierarchical loss function that encourages the learning of fluid flow patterns (in a way similar to ranked modes in principal component analysis), ranking from most to least important. This is shown to increase the stability in learning, reduce over-fitting, and promote interpretability of the CAE neural network layers (HierCAE). The comprehensive new data-driven model, which we call the PhyFlow-HierCAE model, is shown to exhibit improved accuracy and generalizability of flow field predictions over conventional CAE models, attributable to the embedded physical knowledge and the hierarchical regularization, as well as realize orders of magnitude speed-ups in computation times as a surrogate for the direct numerical simulations. Examples of training and forward predictions on unseen pore-structures are provided and evaluated for data from Lattice Boltzmann and molecular dynamics simulations of pore-fluid flow. The model is shown to be a fast and accurate emulator (or “surrogate”) for predicting effective permeability of unseen pore-structures based on learning from relatively small direct numerical simulation datasets.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Inverse mapping of properties to composition through generative modeling for designing molten salts

Generative modeling (GM) has been increasingly used for the inverse design and optimization of materials, yet its application to molten salt mixtures remains unexplored despite how a successful approach to the inverse design of molten salts would contribute to efficiently exploiting their customizability and unlocking their advantages in applications, such as energy production and energy storage. This work presents a workflow for the inverse design of molten salts with targeted density values, addressing the challenge of representing these complex mixtures in GM. A dataset of critically evaluated molten salt densities is used to train a variational autoencoder coupled with a predictive deep neural network, which then can be used to generate new molten salt compositions with desired density values. The effectiveness of the approach is demonstrated by designing mixtures with distinct densities and validating the predicted values using ab initio molecular dynamics simulations.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

Confusion-Driven Machine Learning of Structural Phases of a Flexible, Magnetic Stockmayer Polymer

We use a semisupervised, neural-network-based machine learning technique, the confusion method, to investigate structural transitions in magnetic polymers, which we model as chains of magnetic colloidal nanoparticles characterized by dipole–dipole and Lennard-Jones interactions. As input for the neural network, we use the particle positions and magnetic dipole moments of equilibrium polymer configurations, which we generate via replica-exchange Wang–Landau simulations. We demonstrate that by measuring the classification accuracy of neural networks, we can effectively identify transition points between multiple structural phases without any prior knowledge of their existence or location. We corroborate our findings by investigating relevant conventional order parameters. Our study furthermore examines previously unexplored low-temperature regions of the phase diagram, where we find new structural transitions between highly ordered helicoidal polymer configurations.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Unveiling mesoscopic structures in distorted lamellar phases through deep learning-based small angle neutron scattering analysis

Hypothesis: The formation of distorted lamellar phases, distinguished by their arrangement of crumpled, stacked layers, is frequently accompanied by the disruption of long-range order, leading to the formation of interconnected network structures commonly observed in the sponge phase. Nevertheless, traditional scattering functions grounded in deterministic modeling fall short of fully representing these intricate structural characteristics. Our hypothesis posits that a deep learning method, in conjunction with the generalized leveled wave approach used for describing structural features of distorted lamellar phases, can quantitatively unveil the inherent spatial correlations within these phases. Experiments and Simulations: This report outlines a novel strategy that integrates convolutional neural networks and variational autoencoders, supported by stochastically generated density fluctuations, into a regression analysis framework for extracting structural features of distorted lamellar phases from small angle neutron scattering data. To evaluate the efficacy of our proposed approach, we conducted computational accuracy assessments and applied it to the analysis of experimentally measured small angle neutron scattering spectra of AOT surfactant solutions, a frequently studied lamellar system. Findings: The findings unambiguously demonstrate that deep learning provides a dependable and quantitative approach for investigating the morphology of wide variations of distorted lamellar phases. It is adaptable for deciphering structures from the lamellar to sponge phase including intermediate structures exhibiting fused topological features. In conclusion, this research highlights the effectiveness of deep learning methods in tackling complex issues in the field of soft matter structural analysis and beyond.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Generative $β$-hairpin design using a residue-based physicochemical property landscape

De novo peptide design is a new frontier that has broad application potential in the biological and biomedical fields. Most existing models for de novo peptide design are largely based on sequence homology that can be restricted based on evolutionarily derived protein sequences and lack the physicochemical context essential in protein folding. Generative machine learning for de novo peptide design is a promising way to synthesize theoretical data that are based on, but unique from, the observable universe. In this study, we created and tested a custom peptide generative adversarial network intended to design peptide sequences that can fold into the -hairpin secondary structure. This deep neural network model is designed to establish a preliminary foundation of the generative approach based on physicochemical and conformational properties of 20 canonical amino acids, for example, hydrophobicity and residue volume, using extant structure-specific sequence data from the PDB. The beta generative adversarial network model robustly distinguishes secondary structures of hairpin from α helix and intrinsically disordered peptides with an accuracy of up to 96% and generates artificial -hairpin peptide sequences with minimum sequence identities around 31% and 50% when compared against the current NCBI PDB and nonredundant databases, respectively. These results highlight the potential of generative models specifically anchored by physicochemical and conformational property features of amino acids to expand the sequence-to-structure landscape of proteins beyond evolutionary limits.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

HEPOM: Using Graph Neural Networks for the Accelerated Predictions of Hydrolysis Free Energies in Different pH Conditions

Hydrolysis is a fundamental family of chemical reactions where water facilitates the cleavage of bonds. The process is ubiquitous in biological and chemical systems, owing to water’s remarkable versatility as a solvent. However, accurately predicting the feasibility of hydrolysis through computational techniques is a difficult task, as subtle changes in reactant structure like heteroatom substitutions or neighboring functional groups can influence the reaction outcome. Furthermore, hydrolysis is sensitive to the pH of the aqueous medium, and the same reaction can have different reaction properties at different pH conditions. In this work, we have combined reaction templates and high-throughput ab initio calculations to construct a diverse data set of hydrolysis free energies. The developed framework automatically identifies reaction centers, generates hydrolysis products, and utilizes a trained graph neural network (GNN) model to predict ΔG values for all potential hydrolysis reactions in a given molecule. The long-term goal of the work is to develop a data-driven, computational tool for high-throughput screening of pH-specific hydrolytic stability and the rapid prediction of reaction products, which can then be applied in a wide array of applications including chemical recycling of polymers and ion-conducting membranes for clean energy generation and storage.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Data for Monitoring Fracture Saturation with Internal Transportable Seismic Sources and Twin Neural Networks

Seismic coda-wave analysis is a well-developed method for detecting subtle physical changes in complex media by measuring arrival times in the late-arriving energy from multiply-scattered or reflected waves. However, a challenge arises when multiply-scattered waves are not sufficiently separated in time from the direct arrivals to provide a clear coda wave train. Additional complications for monitoring changes in fracture systems arise when the signals originate from unsynchronized internal sources, such as natural induced seismicity, from acoustic emission, or from transportable intra-fracture sources (chattering dust), that generate uncontrolled signals that vary in arrival time, amplitude and frequency content. Here, we use a twin neural network (also known as a Siamese neural network) for dimensionality reduction to analyze signals from chattering dust to classify the fluid saturation state of a synthetic fracture system. The twin neural network with shared weights generates a low-dimensional representation of the data input by minimizing contrastive loss, serving as the input to a multiclass classifier that accurately classifies whether multiple fractures in a fracture system are fully saturated or partially saturated, or whether a change in saturation has occurred in different fractures in the system. These results show that information buried in unresolved codas from uncontrolled sources can be extracted using machine learning to monitor the evolution of fracture systems caused by physical and chemical processes even when the scattered and direct wave fields overlap.

58 GEOSCIENCES↗
DOE OSTI2025

Structure and Dynamics of CO 2 at the Air–Water Interface from Classical and Neural Network Potentials

The accurate description of the structure and dynamics of CO 2 at the instantaneous air–water interface, along with the effects of surface fluctuations on the CO 2 -transport processes, is essential for the development of negative emission technologies aimed at minimizing climate change. In this study, we performed molecular dynamics simulations of CO 2 at the air–water interface using neural network potentials (NNPs) trained on ab initio data generated through density-functional-theory-based molecular dynamics simulations. We compared these results with classical force fields to assess their performance in modeling interfacial CO 2 behavior. Our findings revealed that the asymmetric interactions, coupled with thermal surface fluctuations at the air–water interface, significantly influence CO 2 transport into the aqueous phase. The simulations demonstrate that classical force fields underestimate both the free energy of CO 2 transport and the strength of its interactions at the interface compared with the neural network potentials. In conclusion, the free energy and the interfacial dynamics of CO 2 are primarily influenced by the distribution of water within the instantaneous interfacial water layer, responsible for creating an asymmetric intermolecular interaction environment within the interfacial region.

Ab initio molecular dynamics↗
DOE OSTI2023

Combining protein sequences and structures with transformers and equivariant graph neural networks to predict protein function

Abstract Motivation Millions of protein sequences have been generated by numerous genome and transcriptome sequencing projects. However, experimentally determining the function of the proteins is still a time consuming, low-throughput, and expensive process, leading to a large protein sequence-function gap. Therefore, it is important to develop computational methods to accurately predict protein function to fill the gap. Even though many methods have been developed to use protein sequences as input to predict function, much fewer methods leverage protein structures in protein function prediction because there was lack of accurate protein structures for most proteins until recently. Results We developed TransFun—a method using a transformer-based protein language model and 3D-equivariant graph neural networks to distill information from both protein sequences and structures to predict protein function. It extracts feature embeddings from protein sequences using a pre-trained protein language model (ESM) via transfer learning and combines them with 3D structures of proteins predicted by AlphaFold2 through equivariant graph neural networks. Benchmarked on the CAFA3 test dataset and a new test dataset, TransFun outperforms several state-of-the-art methods, indicating that the language model and 3D-equivariant graph neural networks are effective methods to leverage protein sequences and structures to improve protein function prediction. Combining TransFun predictions and sequence similarity-based predictions can further increase prediction accuracy. Availability and implementation The source code of TransFun is available at https://github.com/jianlin-cheng/TransFun.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Quantum Neural Networks: Issues, Training, and Applications

Our work in the field aims at explaining the limitations and expressive power of Quantum Machine Learning models, as well as finding feasible training algorithms that could be implemented in near-term Quantum Computers. The promise of Quantum Machine Learning is that by incorporating quantum effects, such as entanglement, into machine learning models researchers can improve model performance and understand more complex datasets. This pledge is particularly pronounced in the design of Quantum neural networks (QNNs), a promising framework for creating quantum algorithms, that promise to outperform classical models by combining the speedups of quantum computation with the widespread successes of deep learning. We show that applying this approach alone to quantum deep learning is problematic given that an excess of entanglement between the hidden and visible layers can destroy the predictive power of our QNN models. We address the barren plateau problem by suggesting the use of a generative, unbounded, nonlinear loss function with simple gradients. The loss function quantifies how much the quantum states generated by the QNNs differ from the data and the goal during training is to minimize it. Finally, we showcase how to use generative training to construct a "classical-quantum" neural network to accurately interpolate between the ground states of a Molecular Hamiltonian, a central question in Quantum Chemistry.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

ESM data downscaling: a comparison of super-resolution deep learning models

Abstract Climate projections at fine spatial resolutions are required to conduct accurate risk assessment for critical infrastructure and design adaptation planning. Generating these projections using advanced Earth system models (ESM) requires significant computational resources. To address this issue, various statistical downscaling techniques have been introduced to generate fine-resolution data from coarse-resolution simulations. In this study, we evaluate and compare five deep learning-based downscaling techniques, namely, super-resolution convolutional neural networks, fast super-resolution convolutional neural network ESM, efficient sub-pixel convolutional neural network, enhanced deep residual network (EDRN), and super-resolution generative adversarial network (SRGAN). These techniques are applied to a dataset generated by the Energy Exascale Earth System Model (E3SM), focusing on key surface variables such as surface temperature, shortwave heat flux, and longwave heat flux. Models are trained and validated using paired fine-resolution (0.25 $$^{\circ }$$ ∘ ) and coarse-resolution (1 $$^{\circ }$$ ∘ ) monthly data obtained from a 9-year simulation. Next, blind testing is performed using monthly data obtained from two different years outside of the training and validation set. To evaluate the efficiency of each technique, different statistical metrics are used, including mean squared error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and learned perceptual image patch similarity (LPIPS). The results show that EDRN outperforms other algorithms in terms of PSNR, SSIM, and MSE, but struggles to capture fine-scale features in the data. In contrast, SRGAN, a generative model that uses perceptual loss, excels in capturing fine details at boundaries and internal structures, resulting in lower LPIPS than other methods.

Pawar, Nikhil M. (ORCID:0000000211613289)↗
DOE OSTI2023

Deep learning workflow for the inverse design of molecules with specific optoelectronic properties

The inverse design of novel molecules with a desirable optoelectronic property requires consideration of the vast chemical spaces associated with varying chemical composition and molecular size. First principles-based property predictions have become increasingly helpful for assisting the selection of promising candidate chemical species for subsequent experimental validation. However, a brute-force computational screening of the entire chemical space is decidedly impossible. To alleviate the computational burden and accelerate rational molecular design, we here present an iterative deep learning workflow that combines (i) the density-functional tight-binding method for dynamic generation of property training data, (ii) a graph convolutional neural network surrogate model for rapid and reliable predictions of chemical and physical properties, and (iii) a masked language model. As proof of principle, we employ our workflow in the iterative generation of novel molecules with a target energy gap between the highest occupied molecular orbital (HOMO) and the lowest unoccupied molecular orbital (LUMO).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A Deep Gravity model for mobility flows generation

The movements of individuals within and among cities influence critical aspects of our society, such as well-being, the spreading of epidemics, and the quality of the environment. When information about mobility flows is not available for a particular region of interest, we must rely on mathematical models to generate them. In this work, we propose Deep Gravity, an effective model to generate flow probabilities that exploits many features (e.g., land use, road network, transport, food, health facilities) extracted from voluntary geographic data, and uses deep neural networks to discover non-linear relationships between those features and mobility flows. Our experiments, conducted on mobility flows in England, Italy, and New York State, show that Deep Gravity achieves a significant increase in performance, especially in densely populated regions of interest, with respect to the classic gravity model and models that do not use deep neural networks or geographic data. Deep Gravity has good generalization capability, generating realistic flows also for geographic areas for which there is no data availability for training. Finally, we show how flows generated by Deep Gravity may be explained in terms of the geographic features and highlight crucial differences among the three considered countries interpreting the model’s prediction with explainable AI techniques.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Application of deep learning to single-shot gas-phase laser-induced breakdown spectroscopy

Single-shot fs laser-induced breakdown spectroscopy (LIBS) has the potential to capture ns-scale electrode desorption phenomena in pulsed power fusion drivers. However, the successful implementation of the diagnostic for this purpose is challenging, as it requires interpreting single-shot measurements collected from low-density gas mixtures. In this work, we demonstrate the efficacy of a Bayesian-optimized convolutional neural network (CNN) to interpret these measurements. We generated 256 distinct measurement conditions at relevant gas pressures ranging from 80–530 mTorr by mixing 100–250 sccm H 2 and 50–200 sccm CH 4 in increments of 10 sccm. Despite the considerable overlap between signals separated by 20 sccm, the CNN is able to predict the H 2 flow rate with a root-mean-square error (RMSE) of 15.9 sccm and the CH 4 flow rate with an RMSE of 12.0 sccm. The average relative prediction error is <9% for each gas and largely remains below or near 10%.

Brown, Nathan Parnell [Sandia National Lab. (SNL-N↗
DOE OSTI2021

BETO 2021 Peer Review - Inverse Bioproduct Design Through Machine Learning and Molecular Simulation

This work aims to identify performance advantaged bioproducts (PABPs) through property prediction, which will guide experimental synthesis. The impact of this work will be faster market adoption of bioproducts with greater performance relative to incumbent products. We have identified >106 bioproduct candidates, but only some will have superior performance to create a market pull. High-throughput property prediction, enabled by machine learning, and elucidation of structure-function relationships, enabled by molecular simulation, provide a hypothesis driven approach for down selection of candidate biomolecules to pursue experimentally. To enable machine learning and molecular simulation for bioproduct discovery, automated structure generation and embedding must capture relevant features for prediction, databases must cover domains applicable to biobased products, and best practices for simulation of polymer systems must be developed. To address these challenges, we have established bioproduct relevant datasets, developed high-throughput polymer structure generation, and built end-to-end neural networks that have predicted 8 properties for >1.4 x 106 biopolymers. A molecular simulation pipeline for building, running, and analyzing polymers and polymer additives is being used to predict performance and develop design principles of biobased products. In collaboration with the PABP synthesis project, these computational tools are guiding synthesis and informing design of PABPs.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗