Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “layered networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Bayesian tensorized neural networks with automatic rank selection

Tensor decomposition is an effective approach to compress over-parameterized neural networks and to enable their deployment on resource-constrained hardware platforms. However, directly applying tensor compression in the training process is a challenging task due to the difficulty of choosing a proper tensor rank. In order to address this challenge, this paper proposes a low-rank Bayesian tensorized neural network. Our Bayesian method performs automatic model compression via an adaptive tensor rank determination. We also present approaches for posterior density calculation and maximum a posteriori (MAP) estimation for the end-to-end training of our tensorized neural network. Here, we provide experimental validation on a two-layer fully connected neural network, a 6-layer CNN and a 110-layer residual neural network where our work produces 7.4x to 137x more compact neural networks directly from the training while achieving high prediction accuracy.

Low-rank tensor↗
DOE OSTI2024

Surrogate construction via weight parameterization of residual neural networks

Surrogate model development is a critical step for uncertainty quantification or other sample-intensive tasks for complex computational models. Here, in this work, we develop a multi-output surrogate form using a class of neural networks (NNs) that employ shortcut connections, namely Residual NNs (ResNets). ResNets are known to regularize the surrogate learning problem and improve the efficiency and accuracy of the resulting surrogate. Inspired by the continuous, Neural ODE analogy, we augment ResNets with weight parameterization strategy with respect to ResNet depth. Weight-parameterized ResNets regularize the NN surrogate learning problem and allow better generalization with a drastically reduced number of learnable parameters. We demonstrate that weight-parameterized ResNets are more accurate and efficient than conventional feed-forward multi-layer perceptron networks. We also compare various options for parameterization of the weights as functions of ResNet depth. We demonstrate the results on both synthetic examples and a large scale earth system model of interest.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Tensor network investigation of the double layer Kagome compound Ca 10 Cr 7 O 28

Quantum spin liquids are exotic quantum phases of matter that do not order even at zero temperature. While there are several toy models and simple Hamiltonians that could host a quantum spin liquid as their ground state, it is very rare to find actual, realistic materials that exhibit their properties. At the same time, the classical simulation of such instances of strongly correlated systems is intricate and reliable methods are scarce. Here, in this work, we investigate the quantum magnet Ca 10 Cr 7 O 28 that has recently been discovered to exhibit properties of a quantum spin liquid in inelastic neutron scattering experiments. This compound has a distorted bilayer Kagome lattice crystal structure consisting of Cr 5+ ions with spin- moments. Coincidentally, the lattice structure renders a tensor network algorithm in 2D applicable that can be seen as a new variant of a projected entangled simplex state algorithm in the thermodynamic limit. In this first numerical investigation of this material that takes into account genuine quantum correlations, good agreement with the experimental findings is found. Our study contributes to uplifting tensor networks from conceptual tools to methods to describe real two-dimensional quantum materials.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

Hierarchical, Grid-Aware, and Economically Optimal Coordination of Distributed Energy Resources in Realistic Distribution Systems

Renewable portfolio standards are targeting high levels of variable solar photovoltaics (PV) in electric distribution systems, which makes reliability more challenging to maintain for distribution system operators (DSOs). Distributed energy resources (DERs), including smart, connected appliances and PV inverters, represent responsive grid resources that can provide flexibility to support the DSO in actively managing their networks to facilitate reliability under extreme levels of solar PV. This flexibility can also be used to optimize system operations with respect to economic signals from wholesale energy and ancillary service markets. Here, we present a novel hierarchical scheme that actively controls behind-the-meter DERs to reliably manage each unbalanced distribution feeder and exploits the available flexibility to ensure reliable operation and economically optimizes the entire distribution network. Each layer of the scheme employs advanced optimization methods at different timescales to ensure that the system operates within both grid and device limits. The hierarchy is validated in a large-scale realistic simulation based on data from the industry. Simulation results show that coordination of flexibility improves both system reliability and economics, and enables greater penetration of solar PV. Discussion is also provided on the practical viability of the required communications and controls to implement the presented scheme within a large DSO.

distribution network↗
DOE OSTI2026

When does global attention help: a unified empirical study on atomistic graph learning

Graph neural networks (GNNs) are widely used as surrogates for costly experiments and first-principles simulations to study the behavior of compounds at atomistic scale, and their architectural complexity is constantly increasing to enable the modeling of complex physics. While most recent GNNs combine more traditional message passing neural networks (MPNNs) layers to model short-range interactions with more advanced graph transformers (GTs) with global attention mechanisms to model long-range interactions, it is still unclear when global attention mechanisms provide real benefits over well-tuned MPNN layers due to inconsistent implementations, features, or hyperparameter tuning. We introduce the first unified, reproducible benchmarking framework–built on HydraGNN–that enables seamless switching among four controlled model classes: MPNN, MPNN with chemistry/topology encoders, GPS-style hybrids of MPNN with global attention, and fully fused localglobal models with encoders. Using seven diverse open-source datasets for benchmarking across regression and classification tasks, we systematically isolate the contributions of message passing, global attention, and encoder-based feature augmentation. Our study shows that encoder-augmented MPNNs form a robust baseline, while fused localglobal models yield the clearest benefits for properties governed by long-range interaction effects. We further quantify the accuracycompute trade-offs of attention, reporting its overhead in memory. Together, these results establish the first controlled evaluation of global attention in atomistic graph learning and provide a reproducible testbed for future model development.

Equivariant graph neural networks↗
DOE OSTI2024

Applying Machine‐Learning Methods to Laser Acceleration of Protons: Lessons Learned From Synthetic Data

ABSTRACT In this study, we consider three different machine‐learning methods—a three‐hidden‐layer neural network, support vector regression, and Gaussian process regression—and compare how well they can learn from a synthetic data set for proton acceleration in the Target Normal Sheath Acceleration regime. The synthetic data set was generated from a previously published theoretical model by Fuchs et al. 2005 that we modified. Once trained, these machine‐learning methods can assist with efforts to maximize the peak proton energy, or with the more general problem of configuring the laser system to produce a proton energy spectrum with desired characteristics. In our study, we focus on both the accuracy of the machine‐learning methods and the performance on one GPU including memory consumption. Although it is arguably the least sophisticated machine‐learning model we considered, support vector regression performed very well in our tests.

Desai, Ronak↗
DOE OSTI2020

Co-optimizing water-alternating-carbon dioxide injection projects using a machine learning assisted computational framework

In this report, a robust machine-learning-based computational framework that couples multi-layer neural network (MLNN) proxies and a multi-objective particle swarm optimizer (MOPSO) to design water-alternating-carbon dioxide injection (CO 2 -WAG) projects is presented. The proposed optimization protocol considers various objectives, including oil recovery and CO 2 storage volume. Expert MLNN systems are trained and employed as surrogate models of the high-fidelity compositional simulator in the optimization workflow. When multiple objective functions are considered, two approaches are employed to treat the objectives: the weighted sum method and the Pareto-front-based scheme. A field-scale implementation focusing on tertiary recovery in the Morrow B formation at Farnsworth Unit (FWU) is presented. The developed Pareto-optimal solutions indicate the maximal available oil production can be 1.64 × 10 7 barrels and maximal carbon storage can achieve 2.35 × 10 7 tons. Trade-offs factor is defined to divide the constructed Pareto front into 4 sections with the trade-off factors’ value ranges from 0.35 to 49.9. This work also compares the optimum solution found by the aggregative objective function and the solution repository covered by the Pareto front that considers the physical and operational constraints and reduces uncertainties involved by the multi-objective optimization process. Our comparison indicates multiple solutions exist to satisfy the objective criteria of the WAG design, and these results cannot be found using the traditional weighted sum method. The Pareto front solution can provide more options for project designers, but decisions regarding necessary trade-offs must be made using the solution repository to balance the project economics and CO 2 storage amount.

10 SYNTHETIC FUELS↗
DOE OSTI2023

Application of machine learning for optical emission spectroscopy data in NAGDIS-II

In this study, we applied machine learning to optical emission spectroscopy (OES) data and device parameters from the linear plasma device NAGDIS-II to explore the potential application of machine learning for predicting electron density, $n$ e , and temperature, $T$ e . The covered ranges of $n$ e and $T$ e , which were measured by an electrostatic probe, are 3.6 × 10 17 –2.4 × 10 19 m -3 and 0.3–7.1 eV, respectively. A three hidden layer neural network (NN) is introduced to model the relationship between $n$ e /$T$ e and the combination of line intensities, radial position, and device parameters. It is shown that the errors in $n$ e and $T$ e become 18.0 and 18.8%, respectively, which were almost the same level for the electrostatic probe, using all available data. Lasso regression and greedy algorithm are used to select the necessary line emissions. In conclusion, it is shown that four- or five-line intensities are sufficient to obtain almost the same quality as the one with all the other lines.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

UQpy Version 4.2: Uncertainty quantification with Python

We introduce a new module for the UQpy software package which extends its capabilities into the field of Scientific Machine Learning. This module builds on PyTorch to create a flexible and robust platform for uncertainty quantification in machine learning. The scientific machine learning module of UQpy introduces custom layers, neural networks, and neural network trainers that are compatible with torch version 2.2.2 and allow for “plug and play” integration into existing torch code.

Neural networks↗
DOE OSTI2022

A Machine‐Learning‐Assisted Stochastic Cloud Population Model as a Parameterization of Cumulus Convection

Abstract A machine‐learning‐assisted stochastic cloud population model is coupled with the Advanced Research Weather Research and Forecasting (WRF) model to represent fluctuations in the cloud‐base mass flux associated with the life cycles and interactions among cumulus convection cells. In this cloud population model, the size distribution and the associated cloud‐base mass flux of the convective cells are related to their previous state and to the change in the total convective area via a transition function. The convective area tendency in turn is assumed to depend on the cloud‐base mass flux that is resolved by the host WRF model. The transition function is represented by a single hidden‐layer neural network trained by the evolution of convective cell size distributions in a 1‐km grid‐spacing WRF simulation run over the Australian Monsoon region. At every grid point of the host model, the cloud population model predicts the cell size and cloud‐base mass flux distributions from which a random sample of cells is fed to an entraining parcel model that calculates precipitation as well as the associated liquid water potential temperature and total moisture tendencies. These tendencies are averaged over the cells and provided to the host model. Several regional simulations are performed over tropical and midlatitude domains to test this as a potential approach to scale‐aware parameterization. It is shown that such an approach could be a new promising path to simulating realistic precipitation statistics and propagation of precipitation associated with the Madden‐Julian Oscillation while maintaining realistic depictions of the diurnal cycle over both land and ocean.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Effect of Acid Etching Time in Ti 3 C 2 MXene’s Interlayer Spacing and Conductivity

Materials with sheet-like morphologies often form interconnected networks of layers or flakes, offering continuous channels for electron and ion transport in electrochemical energy storage applications. One such material is the recently discovered class of 2-D transition metal carbides/nitrides, called MXene, whose general formula is M n+1 X n T x (where M = transition metal; n = 1, 2, or 3; X = carbon or nitrogen; and T x = termination group such as –F, –OH, and/or =O). Ti 3 C 2 , one of the most studied MXene, can be synthesized by selectively etching the aluminum layer in Ti 3 AlC 2 (also called MAX phase). The most straightforward technique to exfoliate this layer is by wet-chemical etching with high-concentration hydrofluoric acid (HF). In this study, the effect of etching time on the morphology, interlayer spacing, and electrical conductivity of the resultant MXene was studied.

25 ENERGY STORAGE↗
DOE OSTI2024

Invariant Features for Accurate Predictions of Quantum Chemical UV-vis Spectra of Organic Molecules

Including invariance of global properties of a phys-ical system as an intrinsic feature in graph neural networks (GNNs) enhances the model's robustness and generalizability and reduces the amount of training data required to obtain a desired accuracy for predictions of these properties. Existing open source GNN libraries construct invariant features only for specific GNN architectures. This precludes the generalization of invariant features to arbitrary message passing neural network (MPNN) layers which, in turn, precludes the use of these libraries for new, user-specified predictive tasks. To address this limitation, we implement invariant MPNNs into the flexible and scalable HydraGNN architecture. HydraGNN enables a seamless switch between various MPNNs in a unified layer sequence and allows for a fair comparison between the predictive performance of different MPNNs. We trained this enhanced HydraGNN archi-tecture on the ultraviolet-visible (UV-vis) spectrum of GDB-9 molecules, a feature that describes the molecule's electronic exci-tation modes, computed with time-dependent density functional tight binding (TD-DFTB) and available open source through the GDB-9-Ex dataset. We assess the robustness (i.e., accuracy and generalizability) of the predictions obtained using different invariant MPNNs with respect to different values of the full width at half maximum (FWHM) for the Gaussian smearing of the theoretical peaks. Our numerical results show that incorporating invariance in the HydraGNN architecture significantly enhances both accuracy and generalizability in predicting UV-vis spectra of organic molecules.

Baker, Justin↗
DOE OSTI2020

Robust and resilient coordination of feeders with uncertain distributed energy resources: from real-time control to long-term planning

To enable flexible demand and ensure reliable and economical operation of distribution systems, we present a novel hierarchical scheme that actively coordinates responsive behind-the-meter grid resources, including distributed energy resources (DERs), to reliably manage each unbalanced distribution feeder and exploits the available flexibility to economically optimize the entire network. Each layer of the scheme employs advanced optimization methods at different timescales to ensure that the system operates within both grid and device limits. The hierarchy is validated in a large-scale realistic simulation based on data from the industry. Simulation results show that coordination of flexibility improves both system reliability and economics, and enables greater penetration of solar PV. Discussion is also provided on the practical viability of the required communications and controls to implement the presented scheme within a large DSO.

14 SOLAR ENERGY↗
DOE OSTI2022

Dimensionally Reduced Model for Rapid and Accurate Prediction of Gas Saturation, Pressure, and Brine Production in a CO 2 Storage Application: Case Study Using the SACROC Field as Part of SMART Task 5

This technical report presents work conducted by the sub surface analysis team of the Strategic Systems Analysis & Engineering group at NETL for Task 5 of SMART Phase 1. This study involved the development of deep learning models for CO 2 geologic storage that are capable of accurate prediction of spatio-temporal outputs of CO 2 saturation, pressure, and brine production in three dimensional space over a storage operation's injection and post-injection timeframes. The model framework involves ensembling multi-layer encoder networks that provide dimesionality reduction of geologic inputs with fully connected long short-term memory (LSTM) neural networks that generate time-series prediction This approach offers a means to maximize training time efficiency, reduce computational memory burden, and minimize prediction turnaround.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Smart Pixels: In-pixel AI for on-sensor data filtering

We present a smart pixel prototype readout integrated circuit (ROIC) designed in CMOS 28 nm bulk process, with in-pixel implementation of an artificial intelligence (AI) / machine learning (ML) based data filtering algorithm designed as proof-of-principle for a Phase III upgrade at the Large Hadron Collider (LHC) pixel detector. The first version of the ROIC consists of two matrices of 256 smart pixels, each 25$\times$25 µm\textsuperscript{2} in size. Each pixel consists of a charge-sensitive preamplifier with leakage current compensation and three auto-zero comparators for a 2-bit flash-type ADC. The frontend is capable of synchronously digitizing the sensor charge within 25 ns. Measurement results show an equivalent noise charge (ENC) of $\sim$30e\textsuperscript{-} and a total dispersion of $\sim$100e\textsuperscript{-} The second version of the ROIC uses a fully connected two-layer neural network (NN) to process information from a cluster of 256 pixels to determine if the pattern corresponds to highly desirable high-momentum particle tracks for selection and readout. The digital NN is embedded in-between analog signal processing regions of the 256 pixels without increasing the pixel size and is implemented as fully combinatorial digital logic to minimize power consumption and eliminate clock distribution, and is active only in the presence of an input signal. The total power consumption of the neural network is $\sim$ 300 $\mu$W. The NN performs momentum classification based on the generated cluster patterns and even with a modest momentum threshold, it is capable of 54.4\% – 75.4\% total data rejection, opening the possibility of using the pixel information at 40MHz for the trigger. The total power consumption of analog and digital functions per pixel is $\sim$ 6 $\mu$W per pixel, which corresponds to $\sim$ 1 W/cm\textsuperscript{2} staying within the experimental constraints.

Parpillon, Benjamin↗