Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Artificial Neural Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Modeling Combustion Reaction ODEs with Neural Networks

The chemistry of combustion reactions is complex as it involves many species and reactions. In practice, such a system is often modeled computationally using an empirically derived chemical mechanism. Given the large range of reaction rates, the system is then time evolved using a stiff ODE solver. However, even when reduced chemical mechanisms are employed, the system can become computationally expensive for two-dimensional or three-dimensional systems. Such systems can also face problems with instability. As such, it is desirable to find a cheaper, stable alternative to solving the reaction system. Neural networks offer the potential to learn these reaction ODEs and time evolve a combustion reaction in a more cost-efficient manner than stiff ODE solvers. In this study, a variety of neural networks are trained on zero-dimensional Cantera simulations of methane combustion with varying initial conditions. Several predictive approaches as well as several neural network architectures (artificial neural network with dropout, ResNet, and Neural ODE) are compared in their ability to predict combustion trajectories. Promising models are then identified.

combustion kinetics↗
DOE OSTI2021

A machine learning-based interaction force model for non-spherical and irregular particles in low Reynolds number incompressible flows

In this study, interaction force of non-spherical particles in low Reynolds number gas-solid flow is investigated by neural network approaches. An artificial neural network (ANN) model is developed to correlate the non-spherical particle shape and the flow conditions with the interaction force. To define the particle shape, spherical harmonic expansion is applied. Furthermore, variational autoencoder model is then used to extract latent geometric features. The latent vector is utilized as an input with the Reynolds number for the ANN. The interaction force data, which is used as output data of the ANN, is obtained by particle resolved direct numerical simulation for 5200 non-spherical particles. The proposed model enables unsupervised extraction for non-spherical particle shapes and accurate predictions on the interaction force without heavy computation. This study provides the model that can explain complicated shapes of particles and be applied to a large scale, computational fluid dynamics simulation.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2025

Spatiotemporal forecasting of the edge localized modes in tokamak plasmas using neural networks

Artificial intelligence techniques have been increasingly adopted by the plasma and fusion science to address problems like plasma reconstruction, surrogate modeling, and tokamak/stellarator optimization. A key focus in sustained fusion research is the prediction and mitigation of edge-localized-modes (ELMs), instabilities that occur in short, periodic bursts and can cause erosion to the tokamak vessel wall. Recent research has demonstrated the power of neural networks in approximating continuous functions. In this work, we build spatiotemporal forecasting models that can predict the onset of ELMs and their evolution at early stages. We leverage recent advances in generative modeling, sequence-to-sequence modeling, and Fourier neural operators to propose architectures and training strategies that can learn to forecast short to long term dynamics of the noisy signals due to ELMs. We benchmark the developed model against a state-of-the-art foundation model using the beam emission spectroscopy (BES) data that captures the plasma fluctuations due to ELMs over a 8 x 8 spatial grid. Our models demonstrate high accuracy, outperforming the baselines, in predicting the evolution of BES signals during ELM events. Furthermore, the developed models exhibit high accuracy in predicting the rapid rise and relaxation of the signals due to ELMs within 30–80 µs.

edge localized modes↗
DOE OSTI2024

Artificial Neural Network-based State Estimation for Low Observable, Unbalanced Microgrids for Microgrid Building Blocks

The microgrid building blocks (MBB) were proposed as microgrid components with combined sub-components with power conversion, communication, and microgrid control capability, or a subset of such sub-components. This work addresses the microgrid controller, present in an MBB, which requires accurate state estimation to perform its tasks, including for monitoring, power flow (dispatch), fault detection, etc. In this paper, an artificial neural network (ANN)-based framework for state estimation is proposed for an MBB, especially for unbalanced and low observable microgrids. To overcome the challenge of low observability in unbalanced systems, a concept of extended adjacent matrix is introduced to reduce the required number of measurements for state estimation. Addressing the challenges, a feed forward neural network (FNN) is utilized to enhance estimation accuracy and reliability with the reduced number of measurements. The proposed state estimation is validated through extensive simulations on a microgrid, which was achieved from the modified IEEE 34-bus distribution test feeder with multiple distributed energy resources (DERs) and demonstrated superior performance in estimation accuracy and low observability.

Choi, Jongchan↗
DOE OSTI2020

Fifth-Generation District Heating and Cooling Substations: Demand Response with Artificial Neural Network-Based Model Predictive Control

District heating and cooling (DHC) is considered one of the most sustainable technologies to meet the heating and cooling demands of buildings in urban areas. The fifth-generation district heating and cooling (5GDHC) concept, often referred to as ambient loops, is a novel solution emerging in Europe and has become a widely discussed topic in current energy system research. 5GDHC systems operate at a temperature close to the ground and include electrically driven heat pumps and associated thermal energy storage in a building-sited energy transfer station (ETS) to satisfy user comfort. This work presents new strategies for improving the operation of these energy transfer stations by means of a model predictive control (MPC) method based on recurrent artificial neural networks. The results show that, under simple time-of-use utility rates, the advanced controller outperforms a rule-based controller for smart charging of the domestic hot water (DHW) thermal energy storage under specific boundary conditions. By exploiting the available thermal energy storage capacity, the MPC controller is capable of shifting up to 14% of the electricity consumption of the ETS from on-peak to off-peak hours. Therefore, the advanced control implemented in 5GDHC networks promotes coupling between the thermal and the electric sector, producing flexibility on the electric grid.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Machine-learning-based inversion of nuclear responses

A microscopic description of the interaction of atomic nuclei with external electroweak probes is required for elucidating aspects of short-range nuclear dynamics and for the correct interpretation of neutrino oscillation experiments. Nuclear quantum Monte Carlo methods infer the nuclear electroweak response functions from their Laplace transforms. Inverting the Laplace transform is a notoriously ill-posed problem; and Bayesian techniques, such as maximum entropy, are typically used to reconstruct the original response functions in the quasielastic region. In this work, we present a physics-informed artificial neural network architecture suitable for approximating the inverse of the Laplace transform. Utilizing simulated, albeit realistic, electromagnetic response functions, we show that this physics-informed artificial neural network outperforms maximum entropy in both the low-energy transfer and the quasielastic regions, thereby allowing for robust calculations of electron scattering and neutrino scattering on nuclei and inclusive muon capture rates.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Neural network emulation of flow in heavy-ion collisions at intermediate energies

Applications of new techniques in machine learning are speeding up progress in research in various fields. In this work, we construct and evaluate a deep neural network (DNN) to be used within a Bayesian statistical framework as a faster and more reliable alternative to the Gaussian process (GP) emulator of an isospin-dependent Boltzmann-Uehling-Uhlenbeck (IBUU) transport model simulator of heavy-ion reactions at intermediate beam energies. We found strong evidence of the DNN being able to emulate the IBUU simulator's prediction on the strengths of protons' directed and elliptical flow very efficiently even with small training datasets and with accuracy about ten times higher than the GP. Here, limitations of our present work and future improvements are also discussed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Towards a more general understanding of the algorithmic utility of recurrent connections

Lateral and recurrent connections are ubiquitous in biological neural circuits. Yet while the strong computational abilities of feedforward networks have been extensively studied, our understanding of the role and advantages of recurrent computations that might explain their prevalence remains an important open challenge. Foundational studies by Minsky and Roelfsema argued that computations that require propagation of global information for local computation to take place would particularly benefit from the sequential, parallel nature of processing in recurrent networks. Such “tag propagation” algorithms perform repeated, local propagation of information and were originally introduced in the context of detecting connectedness, a task that is challenging for feedforward networks. Here, we advance the understanding of the utility of lateral and recurrent computation by first performing a large-scale empirical study of neural architectures for the computation of connectedness to explore feedforward solutions more fully and establish robustly the importance of recurrent architectures. In addition, we highlight a tradeoff between computation time and performance and construct hybrid feedforward/recurrent models that perform well even in the presence of varying computational time limitations. We then generalize tag propagation architectures to propagating multiple interacting tags and demonstrate that these are efficient computational substrates for more general computations of connectedness by introducing and solving an abstracted biologically inspired decision-making task. Our work thus clarifies and expands the set of computational tasks that can be solved efficiently by recurrent computation, yielding hypotheses for structure in population activity that may be present in such tasks.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Combining Spike Time Dependent Plasticity (STDP) and Backpropagation (BP) for Robust and Data Efficient Spiking Neural Networks (SNN)

National security applications require artificial neural networks (ANNs) that consume less power, are fast and dynamic online learners, are fault tolerant, and can learn from unlabeled and imbalanced data. We explore whether two fundamentally different, traditional learning algorithms from artificial intelligence and the biological brain can be merged. We tackle this problem from two directions. First, we start from a theoretical point of view and show that the spike time dependent plasticity (STDP) learning curve observed in biological networks can be derived using the mathematical framework of backpropagation through time. Second, we show that transmission delays, as observed in biological networks, improve the ability of spiking networks to perform classification when trained using a backpropagation of error (BP) method. These results provide evidence that STDP could be compatible with a BP learning rule. Combining these learning algorithms will likely lead to networks more capable of meeting our national security missions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Backpropagation-based learning with local derivative approximation and memory replay in biologically plausible neural systems

When learning, the brain modifies individual synaptic connections to reach a desired behavior. Animal and human brains have been shown to be incredibly capable of learning complex and varied functions across a wide variety of tasks. In recent years, artificial neural networks, inspired by human and animal brains, have shown great capabilities in learning a wide variety of difficult tasks. However, artificial neural networks primarily teach themselves through the use of backpropagation, a learning method which has no clear analogue within the brain. Additionally, Artificial Neural Networks primarily use continuous activation functions, which differ significantly from the spiking neuronal behavior present in the brain. In this paper, we discuss and demonstrate a biologically plausible learning method that approximates backpropagation through two techniques on Spiking Neural Networks. First, we show that the local temporal derivatives that are necessary for backpropagation can be approximately recovered through reconstruction using spike timings. Second, we show that through learning during a sleep phase, inspired by neuroscience research into memory replay, the localized parallel feedback path can learn to approximate the derivative through the forward path weight matrix, thus solving the weight transport problem. Lastly, we demonstrate that the combination of these two methods can approach or exceed the accuracy of backpropagation-based methods for a variety of neuromorphic vision tasks while maintaining biological plausibility.

42 ENGINEERING↗
DOE OSTI2021

Atomistic Mechanism Underlying the Si ( 111 ) - ( 7 × 7 ) Surface Reconstruction Revealed by Artificial Neural-Network Potential

The 7×7 reconstruction of the Si(111) surface represents arguably the most fascinating surface reconstruction so far observed in nature. Yet, the atomistic mechanism underpinning its formation remains unclear after it was discovered sixty years ago. Experimentally, it is observed post priori so that analysis of its formation mechanism can only be carried out in analogy with archaeology. Theoretically, density-functional theory (DFT) correctly predicts the Si(111)-(7×7) ground state but is impractical to simulate its formation process; while empirical potentials failed to produce it as the ground state. Developing an artificial neural-network potential of DFT quality, we carried out accurate large-scale simulations to unravel the formation of the Si(111)-(7×7) surface. We reveal a possible step-mediated atom-pop rate-limiting process that triggers massive nonconserved atomic rearrangements, most remarkably, a critical process of collective vacancy diffusion that mediates a sequence of selective dimer, corner-hole, stacking-fault, and dimer-line pattern formation, to fulfill the 7×7 reconstruction. Our findings may not only solve the long-standing mystery of this famous surface reconstruction but they also illustrate the power of machine learning in studying complex structures.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Criticality analysis of nuclear binding energy neural networks

Machine learning methods, in particular deep learning methods such as artificial neural networks (ANNs) with many layers, have become widespread and useful tools in nuclear physics. However, these ANNs are typically treated as ‘black boxes’, with their architecture (width, depth, and weight/bias initialization) and the training algorithm and parameters chosen empirically by optimizing learning based on limited exploration. We test a non-empirical approach to understanding and optimizing nuclear physics ANNs by adapting a criticality analysis based on renormalization group flows in terms of the hyperparameters for weight/bias initialization, training rates, and the ratio of depth to width. This treatment utilizes the statistical properties of neural network initialization to find a generating functional for network outputs at any layer, allowing for a path integral formulation of the ANN outputs as a Euclidean statistical field theory. We use a prototypical example to test the applicability of this approach: a simple ANN for nuclear binding energies. We find that with training using a stochastic gradient descent optimizer, the predicted criticality behavior is realized, and optimal performance is found with critical tuning. However, the use of an adaptive learning algorithm leads to somewhat superior results without concern for tuning and thus obscures the analysis. Nevertheless, the criticality analysis offers a way to look within the black box of ANNs, which is a first step towards potential improvements in network performance beyond using adaptive optimizers.

artificial neural network↗
DOE OSTI2024

GAINN: The Galaxy Assembly and Interaction Neural Networks for High-redshift JWST Observations

We present the Galaxy Assembly and Interaction Neural Networks (Gainn), a series of artificial neural networks for predicting the redshift, stellar mass, halo mass, and mass-weighted age of simulated galaxies based on James Webb Space Telescope (JWST) photometry. Our goal is to determine the best neural network for predicting these variables at 11 < z < 15. The parameters of the optimal neural network can then be used to estimate these variables for real, observed galaxies. The inputs of the neural networks are JWST filter magnitudes of a subset of five broadband filters (F150W, F200W, F277W, F356W, and F444W) and two medium-band filters (F162M and F182M). We compare the performance of the neural networks using different combinations of these filters, as well as different activation functions and numbers of layers. The best neural network predicted redshift with a normalized rms error of $0.010^{+0.003}_{-0.001}$, stellar mass with rms = $0.089^{+0.044}_{-0.022}$, halo mass with a mean-squared error of $0.022^{+0.014}_{-0.008}$, and mass-weighted age with rms = $12.466^{+5.065}_{-2.408}$. We also test the performance of Gainn on real data from MACS0647JD, an object observed by JWST. Predictions from Gainn for the first projection of the object (JD1) have normalized bias $\langle$Δz$\rangle$ < 0.00228, which is significantly smaller than found with template-fitting methods. We find that the optimal filter combination is F277W, F356W, F162M, and F200W when considering both theoretical accuracy and observational resources from JWST.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Fully‐Printed Ion Sensor Arrays for Measuring Agricultural Nitrogen and Potassium Concentrations Using Nernstian and AI Models

Abstract The chemical composition of growing media is a key factor for plant growth, impacting agricultural yield and sustainability. However, there is a lack of affordable chemical sensors for ubiquitous nutrient ion monitoring in agricultural applications. This work investigates using fully printed ion‐sensor arrays to measure the concentrations of nitrate, ammonium, and potassium in mixed‐electrolyte media. Ion sensor arrays composed of nitrate, ammonium, and potassium ion‐selective electrodes and a printed silver‐silver chloride (Ag/AgCl) reference electrode are fabricated and characterized in aqueous solutions in a range of concentrations that encompass what is typical for agricultural growing media (0.01 m m –1 m ). The sensors are also tested in mixed‐electrolyte solutions of NaNO 3 , NH 4 Cl, and KCl of varying concentrations, and the recorded potentials are input into Nernstian and artificial neural network models to compare the prediction accuracy of the models against ground truth. The artificial neural network models demonstrated higher accuracy over the Nernstian model, and the model using only ion‐sensor inputs is 7.5% more accurate than the Nernstian model under the same conditions. By enabling more precise and efficient fertilizer application, these sensor arrays coupled to computational models can help increase crop yields, optimize resource use, and reduce environmental impact.

Goodrich, Payton [University of California Berkele↗
DOE OSTI2021

In situ Parallel Training of Analog Neural Network Using Electrochemical Random-Access Memory

In-memory computing based on non-volatile resistive memory can significantly improve the energy efficiency of artificial neural networks. However, accurate in situ training has been challenging due to the nonlinear and stochastic switching of the resistive memory elements. One promising analog memory is the electrochemical random-access memory (ECRAM), also known as the redox transistor. Its low write currents and linear switching properties across hundreds of analog states enable accurate and massively parallel updates of a full crossbar array, which yield rapid and energy-efficient training. While simulations predict that ECRAM based neural networks achieve high training accuracy at significantly higher energy efficiency than digital implementations, these predictions have not been experimentally achieved. In this work, we train a 3 × 3 array of ECRAM devices that learns to discriminate several elementary logic gates (AND, OR, NAND). We record the evolution of the network’s synaptic weights during parallel in situ (on-line) training, with outer product updates. Due to linear and reproducible device switching characteristics, our crossbar simulations not only accurately simulate the epochs to convergence, but also quantitatively capture the evolution of weights in individual devices. The implementation of the first in situ parallel training together with strong agreement with simulation results provides a significant advance toward developing ECRAM into larger crossbar arrays for artificial neural network accelerators, which could enable orders of magnitude improvements in energy efficiency of deep neural networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A comparison of computational models for predicting yield sooting index

Sooting propensity, a measurement of how much particulate matter is produced when a fuel is burned, is a property of significant interest among researchers who are striving to discover the next generation of cleaner, more efficient fuels and fuel additives. Many compounds are not viable as fuels and/or fuel additives, and as a result, designing cleaner-burning biofuels using only experimental techniques is inefficient. Predictive models have been instrumental in reducing this inherent difficulty, providing researchers with a tool to preemptively screen compounds before production and testing. The present work compares the accuracies and interpretabilities of existing models used to predict a particular measure of sooting propensity, Yield Sooting Index (YSI). These models include artificial neural networks, graph neural networks, and multivariate equations. A novel equation for predicting YSI based on atom path count and bond order is proposed, which can highlight key structural components that contribute to YSI. It was found that artificial neural networks slightly outperform graph neural networks and greatly outperform multivariate equations in blind (test set) prediction accuracy; however, graph neural networks and multivariate equations provide significantly more interpretability as to how compound structure relates to YSI. Predictions of YSI are compared to experimental measurements for previously un-tested compounds with cetane numbers comparable to diesel fuel (50-60) (butyl decanoate, ethyl decanoate, 1,4-bis(ethenoxymethyl)cyclohexane, and 5-heptyloxolan-2-one), and it was found that these compounds produce significantly less soot compared to diesel fuel.

09 BIOMASS FUELS↗
DOE OSTI2023

Identification and localization of rotational spectra using recurrent neural networks

A method of identifying molecular parameters in a complex mixture may include receiving a set of combined transition frequencies and analyzing the set of combined transition frequencies using a first trained artificial neural network to generate a plurality of separated transition frequency sets. Each of the plurality of separated frequency sets may be analyzed using a second trained artificial neural network to generate a respective set of estimated spectral parameters. The method may include identifying a set of molecular parameters corresponding to the set of separated transition frequencies.

Prozuments, Kirills↗
DOE OSTI2022

Reduced Order Models Generation for HTGRs Pebble Shuffling Procedure Optimization Studies

This report provides an initial study for producing reduced-order models (ROMs) of pebble-bed high temperature gas reactor (HTGR) models for the purposes of design optimization. As an initial study, this work is meant to be exploratory---identifying useful workflows and methods for ROM generation---and not meant to be a catch-all analysis of HTGR ROM generation and usage for optimization. This report summarizes three tasks performed in Fiscal Year 2022: 1) the creation of HTGR model, 2) the sensitivity analysis of model design parameters, and 3) an introduction to ROM generation techniques. The representative HTGR model created in this work is a multiphysics equilibrium-core using the BlueCRAB (comprehensive reactor analysis bundle) reactor analysis application, coupling four physical phenomena: neutronics, streamline depletion, porous flow thermal hydraulics, and pebble heat conduction. Part of the model creation was identifying some design parameters and quantities of interest that are relevant in an optimization analysis and adjustable in the model. The sensitivity analysis utilized a polynomial chaos expansion methodology to compute global sensitivity metrics. This analysis showed that thermal hydraulics parameters and quantities of interest had a relatively small impact on simulation results. Finally, the ROM generation work involved exploring three different ROM methodologies: polynomial regression, a Gaussian process, and artificial neural networks. Using a cross-validation technique to characterize ROM performance, the Gaussian process and single-layer artificial neural networks showed the most promising results. Overall, this study was insightful and the lessons learned will be invaluable for the eventual development of an HTGR design optimization workflow.

97 MATHEMATICS AND COMPUTING↗