Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Generative Neural Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Machine learning–assisted prediction of heat fluxes through thermally anisotropic building envelopes

Thermally anisotropic building envelope (TABE) is a novel active building envelope that can save energy use to maintain thermal comfort in buildings by redirecting heat and coolness from building envelopes to thermal loops. Finite element models (FEMs) can be used to compute the heat fluxes through TABEs, but the high computational cost of finite element simulations has prevented parametric studies and design optimizations. This paper proposes a domain knowledge–informed, finite element–based machine learning framework to reduce the computation cost for the energy management of buildings installed with TABE that uses a ground thermal loop. First, the training heat flux data set was generated by FEM simulations with different thermal loop schedules. Then, both shallow learning models (i.e., multivariate linear regression and eXtreme Gradient Boost, or XGBoost) and a deep learning model (i.e., deep neural network, or DNN) were trained to predict the heat fluxes. Domain knowledge was used for data preprocessing and feature selection. Finally, the suitability of the selected machine learning model was tested under different thermal loop schedules. Herein, the case study results showed that: (1) XGBoost can be as accurate as DNN (coefficient of determination equal to 0.81) with much less training time; (2) the annual energy cost savings for different thermal loop schedules obtained by the XGBoost-predicted and FEM-calculated heat fluxes are consistent, having a difference of only 4%; and (3) XGBoost can reduce the computation time for the annual energy analysis of the case study building with a given thermal loop schedule from around 12 h by using FEM to less than 1 min.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

A hybrid deep neural operator/finite element method for ice-sheet modeling

One of the most challenging and consequential problems in climate modeling is to provide probabilistic projections of sea level rise. A large part of the uncertainty of sea level projections is due to uncertainty in ice sheet dynamics. At the moment, accurate quantification of the uncertainty is hindered by the cost of ice sheet computational models. In this work we develop a hybrid approach to approximate existing ice sheet models at a fraction of their cost. Our approach consists of replacing the finite element model for the momentum equations for the ice velocity, the most expensive part of an ice sheet model, with a Deep Operator Network, while we retain a classic finite element discretization for the evolution of the ice thickness. We show that the resulting hybrid model is very accurate and it is an order of magnitude faster than the traditional finite element model. Further, a distinctive feature of the proposed model, compared to other neural network approaches, is that it can handle high-dimensional parameter spaces (parameter fields) such as the basal friction at the bed of the glacier and can therefore be used for generating samples for uncertainty quantification. Further, we study the impact of hyper-parameters, number of unknowns and correlation length of the parameter distribution on the training and accuracy of the Deep Operator Network on a synthetic ice sheet model. We then target the evolution of the Humboldt glacier in Greenland and show that our hybrid model can provide accurate statistics of the glacier mass loss and can be effectively used to accelerate the quantification of uncertainty.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Photometric redshifts from SDSS images with an interpretable deep capsule network

ABSTRACT Studies of cosmology, galaxy evolution, and astronomical transients with current and next-generation wide-field imaging surveys like the Rubin Observatory Legacy Survey of Space and Time are all critically dependent on estimates of photometric redshifts. Capsule networks are a new type of neural network architecture that is better suited for identifying morphological features of the input images than traditional convolutional neural networks. We use a deep capsule network trained on ugriz images, spectroscopic redshifts, and Galaxy Zoo spiral/elliptical classifications of ∼400 000 Sloan Digital Sky Survey galaxies to do photometric redshift estimation. We achieve a photometric redshift prediction accuracy and a fraction of catastrophic outliers that are comparable to or better than current methods for SDSS main galaxy sample-like data sets (r ≤ 17.8 and zspec ≤ 0.4) while requiring less data and fewer trainable parameters. Furthermore, the decision-making of our capsule network is much more easily interpretable as capsules act as a low-dimensional encoding of the image. When the capsules are projected on a two-dimensional manifold, they form a single redshift sequence with the fraction of spirals in a region exhibiting a gradient roughly perpendicular to the redshift sequence. We perturb encodings of real galaxy images in this low-dimensional space to create synthetic galaxy images that demonstrate the image properties (e.g. size, orientation, and surface brightness) encoded by each dimension. We also measure correlations between galaxy properties (e.g. magnitudes, colours, and stellar mass) and each capsule dimension. We publicly release our code, estimated redshifts, and additional catalogues at https://biprateep.github.io/encapZulate-1.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Synchrophasor Measurements-based Events Detection Using Deep Learning

Deep learning algorithms have been developed for phasor measurement units (PMUs) analysis aiming at providing grid operators to observe and react to significant real-time changes in the grid associated with multiple factors (e.g., power generation and load variations, different type of faults, and equipment mailfunction), or for offline post-event system diagnostics. In this study, a Long Short-Term Memory (LSTM)-based deep neural network (DNN) is adopted and evaluated to identify the most appropriate model configurations for event detection and longer-term anomalous pattern extraction. The proposed DNN model shows the potential on long-term predictions with the ability to capture nonlinear and nonstationary mixture complex patterns in PMU datasets. Real-world PMU in the WECC system were used for model development and validation.

deep learning↗
DOE OSTI2021

Deep Learning Systems for Increased Safeguards Surveillance Review Productivity

Nuclear safeguards inspectors expend significant time and maintain intense focus in reviewing video surveillance for safeguards relevant events. To increase efficiency and reduce the time burden of safeguards inspectors performing surveillance data review, this paper presents a novel deep learning (DL) systems concept to integrate generalized DL models into the safeguards surveillance review workflow. The Agency is investigating several DL algorithms for object recognition, localization, tracking, and flagging relevant activities. The project team is working closely with nuclear safeguards inspectors to identify review use cases (based on specific safeguards objectives) and collect their associated requirements. We focused on CANDU and LWR Nuclear Power Plants (NPPs) and their associated dry storage areas as these present a particularly heavy burden on the inspector surveillance review process due to the number of these facilities under safeguards worldwide. Initial DL algorithm results on safeguards data are promising. Using a convolutional neural network, the team attained a mean average precision (mAP) of 92.9% identifying and localizing spent fuel (SF) casks from a 475 surveillance image dataset. Further, the team had initial success in training a recurrent neural network to identify reactor area activities in video clips, successfully indicating when SF casks enter or exit a pool. We discuss how such DL algorithms would be integrated into the Next Generation Surveillance Review (NGSR) software application. Another issue impacting review productivity is the long time inspectors may have to wait when running these algorithms in NGSR. We present a concept to pre-process remotely collected surveillance data with DL models as the data arrives to IAEA headquarters so that results are already available when starting a new review in NGSR. The proposed DL system concept shows a pathway and workflow for increasing an inspector’s surveillance review productivity by quickly and accurately identifying declared and undeclared safeguards relevant objects and activities in large quantities of surveillance imagery data.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2021

Multimode Metastructures: Novel Hybrid 3D Lattice Topologies

With the rapid proliferation of additive manufacturing and 3D printing technologies, architected cellular solids including truss-like 3D lattice topologies offer the opportunity to program the effective material response through topological design at the mesoscale. The present report summarizes several of the key findings from a 3-year Laboratory Directed Research and Development Program. The program set out to explore novel lattice topologies that can be designed to control, redirect, or dissipate energy from one or multiple insult environments relevant to Sandia missions, including crush, shock/impact, vibration, thermal, etc. In the first 4 sections, we document four novel lattice topologies stemming from this study: coulombic lattices, multi-morphology lattices, interpenetrating lattices, and pore-modified gyroid cellular solids, each with unique properties that had not been achieved by existing cellular/lattice metamaterials. The fifth section explores how unintentional lattice imperfections stemming from the manufacturing process, primarily sur face roughness in the case of laser powder bed fusion, serve to cause stochastic response but that in some cases such as elastic response the stochastic behavior is homogenized through the adoption of lattices. In the sixth section we explore a novel neural network screening process that allows such stocastic variability to be predicted. In the last three sections, we explore considerations of computational design of lattices. Specifically, in section 7 using a novel generative optimization scheme to design novel pareto-optimal lattices for multi-objective environments. In section 8, we use computational design to optimize a metallic lattice structure to absorb impact energy for a 1000 ft/s impact. And in section 9, we develop a modified micromorphic continuum model to solve wave propagation problems in lattices efficiently.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Neural networks for modeling and control of particle accelerators

Charged particle accelerators support a wide variety of scientific, industrial, and medical applications. They range in scale and complexity from systems with just a few components for beam acceleration and manipulation, to large scientific user facilities that span many kilometers and have hundreds-to-thousands of individually-controllable components. Specific operational requirements must be met by adjusting the many controllable variables of the accelerator. Meeting these requirements can be challenging, both in terms of the ability to achieve specific beam quality metrics in a reliable fashion and in terms of the time needed to set up and maintain the optimal operating conditions. One avenue toward addressing this challenge is to incorporate techniques from the fields of machine learning (ML) and artificial intelligence (AI) into the way particle accelerators are modeled and controlled. While many promising approaches within AI/ML could be used for particle accelerators, this dissertation focuses on approaches based on neural networks. Neural networks are particularly well-suited to modeling, control, and diagnostic analysis of nonlinear systems, as well as systems with large parameter spaces. They are also very appealing for their ability to process high-dimensional data types, such as images and time series (both of which are ubiquitous in particle accelerators). In this work, key studies that demonstrated the potential utility of modern neural network-based approaches to modeling and control of particle accelerators are presented. The context for this work is important: at the start of this work in 2012, there was little interest in AI/ML in the particle accelerator community, and many of the advances in neural networks and deep learning that enabled its present success had not yet been made at that time. As such, this work was both an exploration of possible application areas and a generator of proof-of-concept demonstrations in these areas.

43 PARTICLE ACCELERATORS↗
DOE OSTI2025

Surrogate-driven Variance-based Sensitivity Analysis of Thermal Storage Tanks in Integrated Energy Systems

Sensitivity analysis and uncertainty quantification are essential steps for enhancing the accuracy of computational models by identifying and mitigating uncertainties. This study focuses on these steps for the Thermal Energy Delivery System at Idaho National Laboratory, specifically targeting the thermocline tank. Using a Modelica/Dymola simulation model, the study perturbed various design parameters and boundary conditions, including shape factor, porosity, outlet temperature, inlet mass flow rate, and system pressure, to predict and quantify uncertainty in the tank’s ax- ial temperature. A dataset of over 1,000 simulations was generated, and surrogate models were developed using the pyMAISE (Michigan Artificial Intelligence Standard Environment) library, which is an Automatic Machine Learning library for nuclear engineering applications. The optimal model, a feedforward neural network with two hidden layers, achieved an R2 score above 0.99 and a mean absolute error below 1 Kelvin. Sensitivity analyses using Sobol indices and Fourier amplitude sensitivity testing methods on this surrogate model revealed that the inlet mass flow rate at initial timestamps and porosity significantly impacts predicted temperatures across all sensors and time steps.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

AI-driven Neutrino Beam Diagnostics for Next-Generation Neutrino Experiments

The accelerator-driven beam uncertainty limits oscillation measurements in long-baseline neutrino experiments. Spill-resolved beam diagnostics and real-time inference are necessary to address these neutrino flux systematics. As such, we present a machine-learning-based beam monitoring framework developed and validated using data from the T2K experiment. Our approach uses downstream, spill-by-spill muon monitor observables to predict upstream parameters such as proton beam position and width. We achieve high predictive accuracy on nominal runs, demonstrating robust baseline performance whether the model is trained on stable runs or systematically varied conditions. The framework is designed to be robust against domain shifts, allowing the neural network architectures and inference strategies developed with T2K data to be retrained and validated using LBNF simulations, with the goal of eventual deployment under real LBNF/DUNE operating conditions. This scalable approach to real-time beam inference offers a pathway toward reducing flux systematics for next-generation neutrino experiments such as DUNE.

Aney, Noah [Fermilab; U. Chicago (main)]↗
DOE OSTI2025

Building molecular model series from heterogeneous CryoEM structures using Gaussian mixture models and deep neural networks

Cryogenic electron microscopy (CryoEM) produces structures of macromolecules at near-atomic resolution. However, building molecular models with good stereochemical geometry from those structures can be challenging and time-consuming, especially when many structures are obtained from datasets with conformational heterogeneity. Here we present a model refinement protocol that automatically generates series of molecular models from CryoEM datasets, which describe the dynamics of the macromolecular system and have near-perfect geometry scores. This method makes it easier to interpret the movement of the protein complex from heterogeneity analysis and to compare the structural dynamics observed from CryoEM data with results from other experimental and simulation techniques.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Efficient data acquisition and training of collisional-radiative model artificial neural network surrogates through adaptive parameter space sampling

Abstract Effective plasma transport modeling of magnetically confined fusion devices relies on having an accurate understanding of the ion composition and radiative power losses of the plasma. Generally, these quantities can be obtained from solutions of a collisional-radiative (CR) model at each time step within a plasma transport simulation. However, even compact, approximate CR models can be computationally onerous to evaluate, and in-situ evaluation of these models within a larger plasma transport code can lead to a rigid bottleneck. As a way to bypass this bottleneck, we propose deploying artificial neural network (ANN) surrogates to allow rapid evaluation of the necessary plasma quantities. However, one issue with training an accurate ANN surrogate is the reliance on a sufficiently large and representative training and validation data set, which can be time-consuming to generate. In this work we explore a data-driven active learning and training routine to allow autonomous adaptive sampling of the problem parameter space to ensure a sufficiently large and meaningful set of training data is assembled for the network training. As a result, we can demonstrate approximately order-of-magnitude savings in required training data samples to produce an accurate surrogate.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Explicit physics-informed neural networks for nonlinear closure: The case of transport in tissues

In upscaling methods, closures for nonlinear problems present a well-known challenge. While a number of theoretical methods have been proposed for handling such closures, nonlinearities still remain a significant obstacle for many problems. In this work, we use a combination of formal upscaling and data-driven machine learning for explicitly closing a nonlinear transport and reaction process in multiscale tissues. The classical effectiveness factor model is used to formulate the macroscale reaction kinetics. We train a multilayer perceptron network using training data generated by direct numerical simulations over microscale examples. Once trained, the network is used in an algorithm for numerically solving the upscaled (coarse-grained) differential equation describing mass transport and reaction in two example tissues. The network is described as being explicit in the sense that the network is trained using macroscale concentrations and gradients of concentration as components of the feature space rather than incorporating them as part of a constraint in the optimization process. Network training and solutions to the macroscale transport equations were computed for two different tissues. The two tissue types (brain and liver) exhibit markedly different geometrical complexity and spatial scale (cell size and sample size). The upscaled solutions for the average concentration are compared with numerical solutions derived from the microscale concentration fields by a posteriori averaging. There are three outcomes of this work of particular note. 1) Our overall approach results in an upscaled nonlinear PDE. The PDE is closed using a neural network, and our approach results in the definition of the classical effectiveness factor for effecting closure. 2) We identify particular source terms for the closure problem that are important for representing the structure of the closure. These source terms involve macroscale concentrations and their gradients. We adopt these source terms to use as explicit features in the learning algorithm. We find the trained networks that include the macroscale source terms generate models that are able to predict the correction factor with increased fidelity over those that do not. 3) We find that the trained network exhibits good generalizability, and it is able to predict the effectiveness factor with high fidelity for realistically-structured tissues despite the significantly different scale and geometrical complexity of the two example tissue types. This latter result emphasizes our purposeful connection between conventional averaging methods with the use of machine learning for closure; this contrasts with some machine learning methods for upscaling where the exact form of the macroscale equation remains unknown.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Structural differences between human and mouse neurons and their implementation in generative AIs

Mouse and human brains have different functions that depend on their neuronal networks. We analyzed nanometer-scale three-dimensional structures of brain tissues of the mouse medial prefrontal cortex and compared them with structures of the human anterior cingulate cortex. The obtained results indicated that mouse neuronal somata are smaller and neurites are thinner than those of human neurons. We implemented these characteristics of mouse neurons in convolutional layers of a generative adversarial network (GAN) and a denoising diffusion implicit model (DDIM), which were then subjected to image generation tasks using photo datasets of cat faces, cheese, human faces, birds, and automobiles. The mouse-mimetic GAN outperformed a standard GAN in the image generation task using the cat faces and cheese photo datasets, but underperformed for human faces and birds. The mouse-mimetic DDIM gave similar results, suggesting that the nature of the datasets affected the results. Analyses of the five datasets indicated differences in their image entropy, which should influence the number of parameters required for image generation. The preferences of the mouse-mimetic AIs coincided with the impressions commonly associated with mice. The relationship between the neuronal network and brain function should be investigated by implementing other biological findings in artificial neural networks.

generative AI↗
DOE OSTI2021

Machine-learning based prediction of injection rate and solenoid voltage characteristics in GDI injectors

We report that current state-of-the-art gasoline direct-injection (GDI) engines use multiple injections as one of the key technologies to improve exhaust emissions and fuel efficiency. For this technology to be successful, secured adequate control of fuel quantity for each injection is mandatory. However, nonlinearity and variations in the injection quantity can deteriorate the accuracy of fuel control, especially with small fuel injections. Therefore, it is necessary to understand the complex injection behavior and to develop a predictive model to be utilized in the development process. This study presents a methodology for rate of injection (ROI) and solenoid voltage modeling using artificial neural networks (ANNs) constructed from a set of Zeuch-style hydraulic experimental measurements conducted over a wide range of conditions. A quantitative comparison between the ANN model and the experimental data shows that the model is capable of predicting not only general features of the ROI trend, but also transient and non-linear behaviors at particular conditions. In addition, the end of injection (EOI) could be detected precisely with a virtually generated solenoid voltage signal and the signal processing method, which applies to an actual engine control unit. A correlation between the detected EOI timings calculated from the modeled signal and the measurement results showed a high coefficient of determination.

42 ENGINEERING↗
DOE OSTI2023

Integration of Ag-CBRAM crossbars and Mott ReLU neurons for efficient implementation of deep neural networks in hardware

In-memory computing with emerging non-volatile memory devices (eNVMs) has shown promising results in accelerating matrix-vector multiplications. However, activation function calculations are still being implemented with general processors or large and complex neuron peripheral circuits. Here, we present the integration of Ag-based conductive bridge random access memory (Ag-CBRAM) crossbar arrays with Mott rectified linear unit (ReLU) activation neurons for scalable, energy and area-efficient hardware (HW) implementation of deep neural networks. We develop Ag-CBRAM devices that can achieve a high ON/OFF ratio and multi-level programmability. Compact and energy-efficient Mott ReLU neuron devices implementing ReLU activation function are directly connected to the columns of Ag-CBRAM crossbars to compute the output from the weighted sum current. We implement convolution filters and activations for VGG-16 using our integrated HW and demonstrate the successful generation of feature maps for CIFAR-10 images in HW. Our approach paves a new way toward building a highly compact and energy-efficient eNVMs-based in-memory computing system.

Mott insulators↗
DOE OSTI2023

Denoising diffusion algorithm for inverse design of microstructures with fine-tuned nonlinear material properties

Here we introduce a denoising diffusion algorithm to discover microstructures with nonlinear fine-tuned properties. Denoising diffusion probabilistic models are generative models that use diffusion-based dynamics to gradually denoise images and generate realistic synthetic samples. By learning the reverse of a Markov diffusion process, we design an artificial intelligence to efficiently manipulate the topology of microstructures to generate a massive number of prototypes that exhibit constitutive responses sufficiently close to designated nonlinear constitutive behaviors. To identify the subset of microcstructures with sufficiently precise fine-tuned properties, a convolutional neural network surrogate is trained to replace high-fidelity finite element simulations to filter out prototypes outside the admissible range. Results of this study indicate that the denoising diffusion process is capable of creating microstructures of fine-tuned nonlinear material properties within the latent space of the training data. More importantly, this denoising diffusion algorithm can be easily extended to incorporate additional topological and geometric modifications by introducing high-dimensional structures embedded in the latent space. Numerical experiments are conducted on the open-source mechanical MNIST data set (Lejeune, 2020). Consequently, this algorithm is not only capable of performing inverse design of nonlinear effective media, but also learns the nonlinear structure–property map to quantitatively understand the multiscale interplay among the geometry, topology, and their effective macroscopic properties.

42 ENGINEERING↗
DOE OSTI2023

Feature-adjacent multi-fidelity physics-informed machine learning for partial differential equations

Physics-informed neural networks have emerged as an alternative method for solving partial differential equations. However, for complex problems, the training of such networks can still require high-fidelity data which can be expensive to generate. To reduce or even eliminate the dependency on high-fidelity data, we propose a novel multi-fidelity architecture which is based on a feature space shared by the low- and high-fidelity solutions. In the feature space, the projections of the low-fidelity and high-fidelity solutions are adjacent by constraining their relative distance. The feature space is represented with an encoder and its mapping to the original solution space is effected through a decoder. In conclusion, the proposed multi-fidelity approach is validated on forward and inverse problems for steady and unsteady problems described by partial differential equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Integrated analysis of X-ray diffraction patterns and pair distribution functions for machine-learned phase identification

Abstract To bolster the accuracy of existing methods for automated phase identification from X-ray diffraction (XRD) patterns, we introduce a machine learning approach that uses a dual representation whereby XRD patterns are augmented with simulated pair distribution functions (PDFs). A convolutional neural network is trained directly on XRD patterns calculated using physics-informed data augmentation, which accounts for experimental artifacts such as lattice strain and crystallographic texture. A second network is trained on PDFs generated via Fourier transform of the augmented XRD patterns. At inference, these networks classify unknown samples by aggregating their predictions in a confidence-weighted sum. We show that such an integrated approach to phase identification provides enhanced accuracy by leveraging the benefits of each model’s input representation. Whereas networks trained on XRD patterns provide a reciprocal space representation and can effectively distinguish large diffraction peaks in multi-phase samples, networks trained on PDFs provide a real space representation and perform better when peaks with low intensity become important. These findings underscore the importance of using diverse input representations for machine learning models in materials science and point to new avenues for automating multi-modal characterization.

36 MATERIALS SCIENCE↗