Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NEURAL NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 649 records · Page 36
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Neural Networks to Find the Optimal Forcing for Offsetting the Anthropogenic Climate Change Effects

Abstract Of great relevance to climate engineering is the systematic relationship between the radiative forcing to the climate system and the response of the system, a relationship often represented by the linear response function (LRF) of the system. However, estimating the LRF often becomes an ill-posed inverse problem due to high-dimensionality and nonunique relationships between the forcing and response. Recent advances in machine learning make it possible to address the ill-posed inverse problem through regularization and sparse system fitting. Here, we develop a convolutional neural network (CNN) for regularized inversion. The CNN is trained using the surface temperature responses from a set of Green’s function perturbation experiments as imagery input data together with data sample densification. The resulting CNN model can infer the forcing pattern responsible for the temperature response from out-of-sample forcing scenarios. This promising proof of concept suggests a possible strategy for estimating the optimal forcing to negate certain undesirable effects of climate change. The limited success of this effort underscores the challenges of solving an inverse problem for a climate system with inherent nonlinearity. Significance Statement Predicting the climate response for a given climate forcing is a direct problem, while inferring the forcing for a given desired climate response is often an inverse, ill-posed, problem, posing a new challenge to the climate community. This study makes the first attempt to infer the radiative forcing for a given target pattern of global surface temperature response using a deep learning approach. The resulting deeply trained convolutional neural network inversion model shows promise in capturing the forcing pattern corresponding to a given surface temperature response, with a significant implication on the design of an optimal solar radiation management strategy for curbing global warming. This study also highlights the technical challenges that future research should prioritize in seeking feasible solutions to the inverse climate problem.

Ren, Huiying↗
DOE OSTI2021

Identifying Genomic Islands with Deep Neural Networks

Background Horizontal gene transfer is the main source of adaptability for bacteria, through which genes are obtained from different sources including bacteria, archaea, viruses, and eukaryotes. This process promotes the rapid spread of genetic information across lineages, typically in the form of clusters of genes referred to as genomic islands (GIs). Different types of GIs exist, and are often classified by the content of their cargo genes or their means of integration and mobility. While various computational methods have been devised to detect different types of GIs, no single method is capable of detecting all types. Results We propose a method, which we call Shutter Island, that uses a deep learning model (Inception V3, widely used in computer vision) to detect genomic islands. The intrinsic value of deep learning methods lies in their ability to generalize. Via a technique called transfer learning, the model is pre-trained on a large generic dataset and then re-trained on images that we generate to represent genomic fragments. We demonstrate that this image-based approach generalizes better than the existing tools. Conclusions We used a deep neural network and an image-based approach to detect the most out of the correct GI predictions made by other tools, in addition to making novel GI predictions. The fact that the deep neural network was re-trained on only a limited number of GI datasets and then successfully generalized indicates that this approach could be applied to other problems in the field where data is still lacking or hard to curate.

Computer Vision↗
DOE OSTI2022

Discovering Ca II absorption lines with a neural network

Quasar absorption line analysis is critical for studying gas and dust components and their physical and chemical properties as well as the evolution and formation of galaxies in the early universe. Calcium II (Ca II ) absorbers, which are one of the dustiest absorbers and are located at lower redshifts than most other absorbers, are especially valuable when studying physical processes and conditions in recent galaxies. However, the number of known quasar Ca II absorbers is relatively low due to the difficulty of detecting them with traditional methods. In this work, we developed an accurate and quick approach to search for Ca II absorption lines using deep learning. In our deep learning model, a convolutional neural network, tuned using simulated data, is used for the classification task. The simulated training data are generated by inserting artificial Ca II absorption lines into original quasar spectra from the Sloan Digital Sky Survey (SDSS), while an existing Ca II catalogue is adopted as the test set. The resulting model achieves an accuracy of 96 per cent on the real data in the test set. Our solution runs thousands of times faster than traditional methods, taking a fraction of a second to analyse thousands of quasars, while traditional methods may take days to weeks. The trained neural network is applied to quasar spectra from SDSS’s DR7 and DR12 and discovered 399 new quasar Ca II absorbers. In addition, we confirmed 409 known quasar Ca II absorbers identified previously by other research groups through traditional methods.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

EFIT-Prime: Probabilistic and physics-constrained reduced-order neural network model for equilibrium reconstruction in DIII-D

We introduce EFIT-Prime, a novel machine learning surrogate model for EFIT (Equilibrium FIT) that integrates probabilistic and physics-informed methodologies to overcome typical limitations associated with deterministic and ad hoc neural network architectures. EFIT-Prime utilizes a neural architecture search-based deep ensemble for robust uncertainty quantification, providing scalable and efficient neural architectures that comprehensively quantify both data and model uncertainties. Physically informed by the Grad–Shafranov equation, EFIT-Prime applies a constraint on the current density J tor and a smoothness constraint on the first derivative of the poloidal flux, ensuring physically plausible solutions. Furthermore, the spatial location of the diagnostics is explicitly incorporated in the inputs to account for their spatial correlation. Extensive evaluations demonstrate EFIT-Prime's accuracy and robustness across diverse scenarios, most notably showing good generalization on negative-triangularity discharges that were excluded from training. Timing studies indicate an ensemble inference time of 15 ms for predicting a new equilibrium, offering the possibility of plasma control in real-time, if the model is optimized for speed.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Neural network based prediction of no-wall β N limits due to ideal external kink instabilities

Multi-layer perceptron based neural networks (NNs) are trained to predict the Troyon no-wall beta limits due to the onset of low-n (n = 1 – 3, n is the toroidal mode number) ideal external kink instabilities in tokamak plasmas. It is revealed that a well-trained NN can predict the n = 1 no-wall beta limit within 10% relative error. The NN performance is somewhat worse for the n = 2 and 3 no-wall beta limits, but still a relative error within 20% is achievable. The trained NNs well reproduce the known dependences of the no-wall beta limits on the plasma pressure (pressure peaking factor) and current (plasma internal inductance) profiles. Other scalings are further easily established with NNs, for parametric dependences such as on the aspect ratio, the elongation and triangularity of the plasma boundary shape. These NN-based Troyon beta limit predictors can be incorporated into integrated modeling platforms, or directly implemented as a real time stability estimator for the purpose of disruption avoidance or mitigation during experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Multi-task graph neural networks for simultaneous prediction of global and atomic properties in ferromagnetic systems *

Abstract We introduce a multi-tasking graph convolutional neural network, HydraGNN, to simultaneously predict both global and atomic physical properties and demonstrate with ferromagnetic materials. We train HydraGNN on an open-source ab initio density functional theory (DFT) dataset for iron-platinum with a fixed body centered tetragonal lattice structure and fixed volume to simultaneously predict the mixing enthalpy (a global feature of the system), the atomic charge transfer, and the atomic magnetic moment across configurations that span the entire compositional range. By taking advantage of underlying physical correlations between material properties, multi-task learning (MTL) with HydraGNN provides effective training even with modest amounts of data. Moreover, this is achieved with just one architecture instead of three, as required by single-task learning (STL). The first convolutional layers of the HydraGNN architecture are shared by all learning tasks and extract features common to all material properties. The following layers discriminate the features of the different properties, the results of which are fed to the separate heads of the final layer to produce predictions. Numerical results show that HydraGNN effectively captures the relation between the configurational entropy and the material properties over the entire compositional range. Overall, the accuracy of simultaneous MTL predictions is comparable to the accuracy of the STL predictions. In addition, the computational cost of training HydraGNN for MTL is much lower than the original DFT calculations and also lower than training separate STL models for each property.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2021

Convolutional Neural Networks for the CHIPS Neutrino Detector R&D Project.

The CHerenkov detectors In mine PitS (Chips) neutrino detector R&D project aims to develop novel strategies and technologies for very large yet ‘cheap as chips’ water Cherenkov neutrino detectors. Via deployment in a body of water, use of commercially available components, and instrumentation coverage optimisation for the study of exclusively accelerator beam neutrinos, Chips will enable megaton scale detectors to become a reality at the cost of $200k-$300k per kt of sensitive mass. During the summer of 2019 a prototype Chips detector, Chips-5, was deployed into the Wentworth 2W disused mine pit in northern Minnesota, 7 mrad off the NuMI beam axis. A novel data acquisition system was introduced using cheap single-board computers and open-source software. This work presents a novel approach to water Cherenkov neutrino detector event reconstruction and classification. Three forms of a Convolutional Neural Network, a type of deep learning algorithm, have been trained to reject cosmic muon events, classify beam events, and estimate neutrino energies, all using only the raw detector event as input. When evaluated on the expected distribution of Chips-5 events, this new approach is shown to be robust and explainable as well as providing a significant performance increase over the standard likelihood-based reconstruction and simple neural network classification. Promisingly, the performance presented here is comparable to the more complex (and expensive) neutrino oscillation experiments within the field.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

AP-Net: An atomic-pairwise neural network for smooth and transferable interaction potentials

Intermolecular interactions are critical to many chemical phenomena, but their accurate computation using ab initio methods is often limited by computational cost. The recent emergence of machine learning (ML) potentials may be a promising alternative. Useful ML models should not only estimate accurate interaction energies but also predict smooth and asymptotically correct potential energy surfaces. However, existing ML models are not guaranteed to obey these constraints. Indeed, systemic deficiencies are apparent in the predictions of our previous hydrogen-bond model as well as the popular ANI-1X model, which we attribute to the use of an atomic energy partition. As a solution, we propose an alternative atomic-pairwise framework specifically for intermolecular ML potentials, and we introduce AP-Net—a neural network model for interaction energies. The AP-Net model is developed using this physically motivated atomic-pairwise paradigm and also exploits the interpretability of symmetry adapted perturbation theory (SAPT). We show that in contrast to other models, AP-Net produces smooth, physically meaningful intermolecular potentials exhibiting correct asymptotic behavior. Initially trained on only a limited number of mostly hydrogen-bonded dimers, AP-Net makes accurate predictions across the chemically diverse S66x8 dataset, demonstrating significant transferability. On a test set including experimental hydrogen-bonded dimers, AP-Net predicts total interaction energies with a mean absolute error of 0.37 kcal mol−1, reducing errors by a factor of 2–5 across SAPT components from previous neural network potentials. The pairwise interaction energies of the model are physically interpretable, and an investigation of predicted electrostatic energies suggests that the model “learns” the physics of hydrogen-bonded interactions.

Glick, Zachary L. (ORCID:0000000309002849)↗
DOE OSTI2023

Lower and upper threshold limit for artificial neural network based chilled and condenser water temperatures set-point control in a chilled water system

In this study, an ANN (artificial neural network) based real-time optimized control algorithm for a chilled water cooling system was developed and applied in an actual building to analyze its cooling energy saving effects through in-situ application and actual measurements. For this purpose, the cooling tower’s CndWT (condenser water temperature) and the chiller’s ChWT (chilled water temperature) were set as system control variables. To evaluate algorithm performance, the electric consumption and the COP (coefficient of performance) were compared and analyzed when ChWT and CndWTs were controlled conventionally and controlled based on the ANN. During the analysis, unexpected abnormal data was observed due to insufficient training data and limited consideration of OWBT (outdoor air wet-bulb temperature) when determining the CndWT set-point. Therefore, it is necessary to further build training data from a wider range of conditions and to set the lower limit of CndWT set-point to at least +3.6 °C above OWBT when the OWBT is higher than 23 °C, so that further energy savings can be achieved.

42 ENGINEERING↗
DOE OSTI2024

Learning model combining convolutional deep neural network with a self-attention mechanism for AC optimal power flow

Alternating current optimal power flow (OPF) analysis is critical for efficient and reliable operation of power systems. For large systems or repetitive computations, the traditional methods such as the direct and gradient methods, or non-traditional methods, such as the genetic algorithm and simulating annealing, are time-consuming and unsuitable for real-time computing. The work in this paper proposes a novel framework to obtain the optimal solution of power flow in real-time using a combination of convolutional neural networks and a self-attention mechanism. All parameters of the power networks are rearranged in an image-like shape of a multi-channel image where each channel is a two-dimensional matrix. The proposed approach is adaptive with every input size of power systems as well as frequent variations of network topologies without intervention to the framework core. The encompassment of all power system contexts in which all parameters of internal elements, generation costs, and topology information are included, contributes to the higher accuracy of inference compared to other current machine-learning-based OPF-solving methods. Besides, the proposed framework established on ubiquitous platforms is effortlessly integrated into current infrastructures of power systems, and the great efficiency along with the computation speed may serve as a critical point for practical implications, such as enabling faster decision-making during real-time operations, predicting system contingencies, and remedial actions based on an offline pre-trained model. Furthermore, this supervised learning process is applied to the dataset of four case studies of meshed power systems: the IEEE 5-bus system (IEEE-5), the IEEE 30-bus system (IEEE-30), the IEEE 39-bus system (IEEE-39), and the IEEE 57-bus system (IEEE-57) to prove the efficacy of the proposed method.

42 ENGINEERING↗
DOE OSTI2023

Dilated causal convolutional neural networks for forecasting zone airflow to estimate short-term energy consumption

Here this paper investigates the use of dilated causal convolutional neural networks for fine- grained temporal forecasting of building zone states. Specifically, we build and evaluate models using a small set of exogenous features (e.g., external temperature) to autoregressively predict zone airflow setpoints every minute for a 24-hour prediction window. We carefully explore the trade-off between generality and specificity in these models, training and evaluating them based on zone, zone type, month, season, and combinations thereof. When evaluated for a commercial office building in Eastern Washington with 16 zones served by variable air volume air handling units, we find that the highest performance comes from a zone-specific, season-agnostic approach; with it, we obtain an R 2 of 0.704 (averaged over zones) and an average normalized root mean square error (nRMSE) of 0.111. In contrast, the most general model (trained across all zones and seasons) yields an R 2 of only 0.416 and a nRMSE of 0.168, while a baseline zone-specific reduced order model obtains 0.443 R 2 and 0.159 nRMSE. We also report on factors affecting airflow forecasting performance, on the ability of models trained on a specific zone to generalize to other zones, and on the capability of those models trained on a specific month to generalize to other months.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Active learning of neural network potentials for rare events

Developing an automated active learning framework for Neural Network Potentials, focusing on accurately simulating bond-breaking in hexane chains through steered molecular dynamics sampling and assessing model transferability.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Cartesian message passing neural networks for directional properties: Fast and transferable atomic multipoles

The message passing neural network (MPNN) framework is a promising tool for modeling atomic properties but is, until recently, incompatible with directional properties, such as Cartesian tensors. We propose a modified Cartesian MPNN (CMPNN) suitable for predicting atom-centered multipoles, an essential component of ab initio force fields. The efficacy of this model is demonstrated on a newly developed dataset consisting of 46 623 chemical structures and corresponding high-quality atomic multipoles, which was deposited into the publicly available Molecular Sciences Software Institute QCArchive server. We show that the CMPNN accurately predicts atom-centered charges, dipoles, and quadrupoles and that errors in the predicted atomic multipoles have a negligible effect on multipole–multipole electrostatic energies. The CMPNN is accurate enough to model conformational dependencies of a molecule’s electronic structure. This opens up the possibility of recomputing atomic multipoles on the fly throughout a simulation in which they might exhibit strong conformational dependence.

Glick, Zachary L. (ORCID:0000000309002849)↗
DOE OSTI2022

WPGNN and PLayGen (Wind Plant Graph Neural Network and Plant Layout Generator) [SWR-21-90]

WPGNN is the graph neural network machine learning based surrogate model and software that provides a streamlined approximation of wind plant wake models. It can rapidly estimate the energy production of the plant and turbines for any arbitrarily sized wind plant and layout under any inflow condition. Associated tools include graphing and visualization capabilities as well as a wind plant generator capable of creating randomized realizations of canonical wind plant layouts. The WPGNN architecture and application is extendable to multiple wake models, turbine technologies and features, and supports downstream optimizations of wind plant layouts and control strategies. In addition to the WPGNN, we include the code for the plant layout generator (PLayGen) playgen.py. This generator can produce random realizations of realistic wind plant layouts from one of the four canonical styles: cluster, single string, multiple string, or parallel string. The PLayGen_demo.ipynb notebook provides a demonstration of how to use the generator tool.

Harrison-Atlas, Dylan↗
DOE OSTI2023

User Manual - HydraGNN: Distributed PyTorch Implementation of Multi-Headed Graph Convolutional Neural Networks

This document serves as user manual for HydraGNN, a scalable graph neural network (GNN) architecture that allows for a simultaneous prediction of multiple target properties using multi-task learning (MTL). The HydraGNN architecture is constructed by successive superposition of three different sets of layers. The first set is made of message-passing layers to exchange information across nodes in the graph and use this to update the nodal features. The second set is made of global pooling layers that aggregate information from all the nodes in the graph and map it into a scalar, and is needed only for global target properties that are related to the entire graph. The third set of layers is dedicated to the implementation of MTL, which is enabled by forking of the architecture into separate heads, each one of them dedicated to the predictive task of one specific target property. Through an object-oriented programming paradigm, HydraGNN is templated over different message-passing policies, which allows for a user-friendly hyperparameter study to assess the sensitivity of the predictive performance of the HydraGNN architecture on a specific dataset with respect to the choice of the message-passing policy. The object-oriented paradigm used by HydraGNN also allows for a user-friendly inclusion of newly developed message passing policies within the existing framework. HydraGNN supports distributed computing capabilities for scalable data reading and scalable training on leadership-class supercomputers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Invertible neural networks for E3SM land model calibration and simulation

We apply an invertible neural network (INN) for E3SM land model calibration and simulation with eight parameters at the Missouri Ozark AmeriFlux forest site. INN provides bijective (two-way) mappings between inputs and outputs, thus it can solve probabilistic inverse problems and forward approximations simultaneously. We demonstrate INN's inverse and forward capability in both synthetic and real-data applications. Results indicate that INN produces accurate parameter posterior distributions similar to Markov Chain Monte Carlo sampling and it generates model outputs close to the forward model simulations. Additionally, both the inverse and forward evaluations in INN are computationally efficient which allows for rapid integration of observations for parameter estimation and fast model predictions.

Lu, Dan↗