Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural Networks (Computer)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46

Chaotic neural dynamics facilitate probabilistic computations through sampling

Cortical neurons exhibit highly variable responses over trials and time. Theoretical works posit that this variability arises potentially from chaotic network dynamics of recurrently connected neurons. Here, we demonstrate that chaotic neural dynamics, formed through synaptic learning, allow networks to perform sensory cue integration in a sampling-based implementation. We show that the emergent chaotic dynamics provide neural substrates for generating samples not only of a static variable but also of a dynamical trajectory, where generic recurrent networks acquire these abilities with a biologically plausible learning rule through trial and error. Furthermore, the networks generalize their experience in the stimulus-evoked samples to the inference without partial or all sensory information, which suggests a computational role of spontaneous activity as a representation of the priors as well as a tractable biological computation for marginal distributions. These findings suggest that chaotic neural dynamics may serve for the brain function as a Bayesian generative model.

60 APPLIED LIFE SCIENCES↗

Scalable deep learning for watershed model calibration

Watershed models such as the Soil and Water Assessment Tool (SWAT) consist of high-dimensional physical and empirical parameters. These parameters often need to be estimated/calibrated through inverse modeling to produce reliable predictions on hydrological fluxes and states. Existing parameter estimation methods can be time consuming, inefficient, and computationally expensive for high-dimensional problems. In this paper, we present an accurate and robust method to calibrate the SWAT model (i.e., 20 parameters) using scalable deep learning (DL). We developed inverse models based on convolutional neural networks (CNN) to assimilate observed streamflow data and estimate the SWAT model parameters. Scalable hyperparameter tuning is performed using high-performance computing resources to identify the top 50 optimal neural network architectures. We used ensemble SWAT simulations to train, validate, and test the CNN models. We estimated the parameters of the SWAT model using observed streamflow data and assessed the impact of measurement errors on SWAT model calibration. We tested and validated the proposed scalable DL methodology on the American River Watershed, located in the Pacific Northwest-based Yakima River basin. Our results show that the CNN-based calibration is better than two popular parameter estimation methods (i.e., the generalized likelihood uncertainty estimation [GLUE] and the dynamically dimensioned search [DDS], which is a global optimization algorithm). For the set of parameters that are sensitive to the observations, our proposed method yields narrower ranges than the GLUE method but broader ranges than values produced using the DDS method within the sampling range even under high relative observational errors. The SWAT model calibration performance using the CNNs, GLUE, and DDS methods are compared using R 2 and a set of efficiency metrics, including Nash-Sutcliffe, logarithmic Nash-Sutcliffe, Kling-Gupta, modified Kling-Gupta, and non-parametric Kling-Gupta scores, computed on the observed and simulated watershed responses. The best CNN-based calibrated set has scores of 0.71, 0.75, 0.85, 0.85, 0.86, and 0.91. The best DDS-based calibrated set has scores of 0.62, 0.69, 0.8, 0.77, 0.79, and 0.82. The best GLUE-based calibrated set has scores of 0.56, 0.58, 0.71, 0.7, 0.71, and 0.8. The scores above show that the CNN-based calibration leads to more accurate low and high streamflow predictions than the GLUE and DDS sets. Our research demonstrates that the proposed method has high potential to improve our current practice in calibrating large-scale integrated hydrologic models.

54 ENVIRONMENTAL SCIENCES↗

Infrared-safe energy weighting does not guarantee small nonperturbative effects

Infrared and collinear (IRC) safety has long been used a proxy for robustness when developing new jet substructure observables. This guiding philosophy has been carried into the deep learning era, where IRC-safe neural networks have been used for many jet studies. For graph-based neural networks, the most straightforward way to achieve IRC safety is to weight particle inputs by their energies. However, energy-weighting by itself does not guarantee that perturbative calculations of machine-learned observables will enjoy small nonperturbative corrections. Here, in this paper, we demonstrate the sensitivity of IRC-safe networks to nonperturbative effects, by training an energy flow network (EFN) to maximize its sensitivity to hadronization. We then show how to construct Lipschitz energy flow networks (L-EFNs), which are both IRC safe and relatively insensitive to nonperturbative corrections. We demonstrate the performance of L-EFNs on generated samples of quark and gluon jets, and showcase fascinating differences between the learned latent representations of EFNs and L-EFNs.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Neural Network Approach to Locating Cryptography in Object Code

Finding and identifying cryptography is a growing concern in the malware analysis community. In this paper, artificial neural networks are used to classify functional blocks from a disassembled program as being either cryptography related or not. The resulting system, referred to as NNLC (Neural Net for Locating Cryptography) is presented and results of applying this system to various libraries are described.

97 MATHEMATICS AND COMPUTING↗

Physics informed deep neural network embedded in a chemical transport model for the Amazon rainforest

Secondary organic aerosols (SOA) are fine particles in the atmosphere, which interact with clouds, radiation and affect the Earth’s energy budget. SOA formation involves chemistry in gas phase, aqueous aerosols, and clouds. Simulating these chemical processes involve solving a stiff set of differential equations, which are computationally expensive steps for three-dimensional chemical transport models. Deep neural networks (DNNs) are universal function approximators that could be used to represent the complex nonlinear changes in aerosol physical and chemical processes; however, key challenges such as generalizability to extended time periods, preservation of mass balance, simulating sparse model outputs, and maintaining physical constraints have limited their use in atmospheric chemistry. Here, we develop an approach of using a physics-informed DNN that overcomes previous such challenges and demonstrates its applicability for the chemical formation processes of isoprene epoxydiol SOA (IEPOX-SOA) over the Amazon rainforest. The DNN is trained with data generated by simulating IEPOX-SOA over the entire atmospheric column, using the Weather Research and Forecasting Model coupled with Chemistry (WRF-Chem). The trained DNN is then embedded within WRF-Chem to replace the computationally expensive default solver of IEPOX-SOA formation. The trained DNN predictions generalizes well with the default model simulation of the IEPOX-SOA mass concentrations and its size distribution (20 size bins) over several days of simulations in both dry and wet seasons. The embedded DNN reduces the computational expense of WRF-Chem by a factor of 2. Our approach shows promise in terms of application to other computationally expensive chemistry solvers in climate models.

54 ENVIRONMENTAL SCIENCES↗

Easy-plane spin Hall nano-oscillators as spiking neurons for neuromorphic computing

Here we show analytically using a macrospin approximation that easy-plane spin Hall nano-oscillators excited by a spin current polarized perpendicularly to the easy plane have phase dynamics analogous to that of Josephson junctions. Similarly to Josephson junctions, they can reproduce the spiking behavior of biological neurons that is appropriate for neuromorphic computing. To take advantage of typical spin-orbit torques, we use a nanoconstriction geometry, in which the magnetostatic interaction and magnetocrystalline anisotropy are tuned to create an easy plane that includes the interface normal direction. We perform micromagnetic simulations of such oscillators realized in this geometry and show that the easy-plane spiking dynamics is preserved in this experimentally feasible architecture. Finally we simulate two elementary neural network blocks that implement operations essential for neuromorphic computing. First, we show that output spikes energies from two neurons can be summed and injected into a following layer neuron and second, we demonstrate that outputs can be multiplied by synaptic weights implemented by locally modifying the anisotropy.

36 MATERIALS SCIENCE↗

Learning stochastic dynamics with statistics-informed neural network

We introduce a machine-learning framework named statistics-informed neural network (SINN) for learning stochastic dynamics from data. This new architecture was theoretically inspired by a universal approximation theorem for stochastic systems, which we introduce in this paper, and the projection-operator formalism for stochastic modeling. Here, we devise mechanisms for training the neural network model to reproduce the correct statistical behavior of a target stochastic process. Numerical simulation results demonstrate that a well-trained SINN can reliably approximate both Markovian and non-Markovian stochastic dynamics. We demonstrate the applicability of SINN to coarse-graining problems and the modeling of transition dynamics. Furthermore, we show that the obtained reduced-order model can be trained on temporally coarse-grained data and hence is well suited for rare-event simulations.

97 MATHEMATICS AND COMPUTING↗

Application of Machine Learning to Multigroup Microscopic Cross Sections

Presentation discussing the research and development of deep neural network models for modeling microscopic neutron cross-section data in the Griffin reactor physics application for pebble-bed reactors. This work details advancements made between the last review meeting in July 2024 until July 2025.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

Graph Neural Networks for Surrogate Modeling of Offshore Floating Platforms

Floating offshore wind turbines (FOWTs) present an significant opportunity to increase renewable energy generation. However, significant challenges remain before FOWTs can be widely commercialized and deployed. In particular, hydrodynamic loading on the platforms can stress the overall structure, damage the mooring systems, and impact power generation. Studying these loads is difficult and often relies on computationally expensive models or experiments. In this work, we explore the use of graph neural networks (GNNs) to construct flexible, data-driven surrogates for hydrodynamic loads on platforms. We leverage the natural graph-like structure of offshore wind platform designs to enable the GNN model to learn to approximate the loads for different wave conditions and structural designs. We demonstrate potential uses for the surrogate by performing parameter sweeps and ridge analysis on the trained model to identify the impacts of different wave and structural features on the loads.

floating offshore wind turbines↗

Training neural networks using Metropolis Monte Carlo and an adaptive variant

Abstract We examine the zero-temperature Metropolis Monte Carlo (MC) algorithm as a tool for training a neural network by minimizing a loss function. We find that, as expected on theoretical grounds and shown empirically by other authors, Metropolis MC can train a neural net with an accuracy comparable to that of gradient descent (GD), if not necessarily as quickly. The Metropolis algorithm does not fail automatically when the number of parameters of a neural network is large. It can fail when a neural network’s structure or neuron activations are strongly heterogenous, and we introduce an adaptive Monte Carlo algorithm (aMC) to overcome these limitations. The intrinsic stochasticity and numerical stability of the MC method allow aMC to train deep neural networks and recurrent neural networks in which the gradient is too small or too large to allow training by GD. MC methods offer a complement to gradient-based methods for training neural networks, allowing access to a distinct set of network architectures and principles.

97 MATHEMATICS AND COMPUTING↗

Programmable Synaptic Arrays For Electronic Neural Networks

High resistances prevent hotspots in parallel input and output operation. Nonvolatile computer memory combines ultrahigh storage density with extremely-low power dissipation. Accommodates about 1 billion bits in square centimeter of surface area. Bit written with expenditure of less than 1 nanojoule of energy and read with even lower energy. Developed for parallel input and output operation.

Thakoor, Anilkumar P.↗

Leveraging a Neural Network-Enhanced Reproducing Kernel Particle Method for Multiphysics Degradation Modeling of Energy Storage Materials

Energy storage materials exhibit strong electro-chemo-mechanical coupling and highly anisotropic material properties, contributing to the formation and propagation of micro-cracking during charge/discharge cycling and resulting in reduced performance and service life. A coupled electro-chemo-mechanical reproducing kernel particle method (RKPM) formulation has been developed to analyze this system. With microstructural images supplied by the National Renewable Energy Laboratory (NREL), pixel-based model construction by RKPM is used to represent the complex material microstructures that dictate the coupled physics of these systems. Traditional electro-chemo-mechanical models rely on mesh-based finite element methods, which can lead to difficulties in meshing such complex geometries and capturing crack propagation due to mesh dependency. Here, a neural network-enhanced reproducing kernel particle method (NN-RKPM) [1, 2] is introduced to effectively model damage and crack propagation in the material microstructures; the location, orientation, and solution transition near a localization are automatically captured by superimposed block-level NN optimizations. This NN enrichment approach allows for effective modeling of localizations via a fixed background discretization, relieving tedious efforts for adaptive refinement in traditional mesh-based methods. Applications to the heterogeneous microstructures of Li-ion battery cathodes will be presented to demonstrate the effectiveness of the proposed methods. NN-RKPM is additionally used to inform how crack opening and closure in turn affect the coupled chemical equations and material microstructure. Reference: [1] Baek, J., Chen, J. S., Susuki, K., "Neural Network enhanced Reproducing Kernel Particle Method for Modeling Localizations," International Journal for Numerical Methods in Engineering, Vol. 123, pp 4422-4454, https://doi.org/10.1002/nme.7040, 2022. [2] Baek, J., Chen, J. S., "A Neural Network-Based Enrichment of Reproducing Kernel Approximation for Modeling Brittle Fracture", Computer Methods in Applied Mechanics and Engineering Vol. 410, 116590, 2024.

degradation↗

Bi-fidelity modeling of uncertain and partially unknown systems using DeepONets

Recent advances in modeling large-scale, complex physical systems have shifted research focuses towards data-driven techniques. However, generating datasets by simulating complex systems can require significant computational resources. Similarly, acquiring experimental datasets can prove difficult. For these systems, often computationally inexpensive, but in general inaccurate models, known as the low-fidelity models, are available. Here in this paper, we propose a bi-fidelity modeling approach for complex physical systems, where we model the discrepancy between the true system's response and a low-fidelity response in the presence of a small training dataset from the true system's response using a deep operator network, a neural network architecture suitable for approximating nonlinear operators. We apply the approach to systems that have parametric uncertainty and are partially unknown. Three numerical examples are used to show the efficacy of the proposed approach to model uncertain and partially unknown physical systems.

MATHEMATICS AND COMPUTING↗

Efficient Neural Network Approaches for Conditional Optimal Transport with Applications in Bayesian Inference

In this work, we present two neural network approaches that approximate the solutions of static and dynamic conditional optimal transport (COT) problems. Both approaches enable conditional sampling and conditional density estimation, which are core tasks in Bayesian inference—particularly in the simulation-based (“likelihood-free”) setting. Our methods represent the target conditional distribution as a transformation of a tractable reference distribution. Obtaining such a transformation, chosen here to be an approximation of the COT map, is computationally challenging even in moderate dimensions. To improve scalability, our numerical algorithms use neural networks to parameterize candidate maps and further exploit the structure of the COT problem. Our static approach approximates the map as the gradient of a partially input convex neural network. It uses a novel numerical implementation to increase computational efficiency compared to state-of-the-art alternatives. Our dynamic approach approximates the conditional optimal transport via the flow map of a regularized neural ODE; compared to the static approach, it is slower to train but offers more modeling choices and can lead to faster sampling. We demonstrate both algorithms numerically, comparing them with competing state-of-the-art approaches, using benchmark datasets and simulation-based Bayesian inverse problems.

97 MATHEMATICS AND COMPUTING↗

Online evolutionary neural architecture search for multivariate non-stationary time series forecasting

Time series forecasting (TSF) is one of the most important tasks in data science. TSF models are usually pre-trained with historical data and then applied on future unseen datapoints. However, real-world time series data is usually non-stationary and models trained offline usually face problems from data drift. Models trained and designed in an offline fashion can not quickly adapt to changes quickly or be deployed in real-time. To address these issues, this work presents the Online NeuroEvolution-based Neural Architecture Search (ONE-NAS) algorithm, which is a novel neural architecture search method capable of automatically designing and dynamically training recurrent neural networks (RNNs) for online forecasting tasks. Without any pre-training, ONE-NAS utilizes populations of RNNs that are continuously updated with new network structures and weights in response to new multivariate input data. ONE-NAS is tested on real-world, large-scale multivariate wind turbine data as well as the univariate Dow Jones Industrial Average (DJIA) dataset. These results demonstrate that ONE-NAS outperforms traditional statistical time series forecasting methods, including online linear regression, fixed long short-term memory (LSTM) and gated recurrent unit (GRU) models trained online, as well as state-of-the-art, online ARIMA strategies. Additionally, results show that utilizing multiple populations of RNNs which are periodically repopulated provide significant performance improvements, allowing this online neural network architecture design and training to be successful.

97 MATHEMATICS AND COMPUTING↗

Mapping Stochastic Devices to Probabilistic Algorithms

Probabilistic and Bayesian neural networks have long been proposed as a method to incorporate uncertainty about the world (both in training data and operation) into artificial intelligence applications. One approach to making a neural network probabilistic is to leverage a Monte Carlo sampling approach that samples a trained network while incorporating noise. Such sampling approaches for neural networks have not been extensively studied due to the prohibitive requirement of many computationally expensive samples. While the development of future microelectronics platforms that make this sampling more efficient is an attractive option, it has not been immediately clear how to sample a neural network and what the quality of random number generation should be. This research aimed to start addressing these two fundamental questions by examining basic “off the shelf” neural networks can be sampled through a few different mechanisms (including synapse “dropout” and neuron “dropout”) and examine how these sampling approaches can be evaluated both in terms of evaluating algorithm effectiveness and the required quality of random numbers.

97 MATHEMATICS AND COMPUTING↗

Modern deep neural networks for Direct Normal Irradiance forecasting: A classification approach

The escalating energy demand and the adverse environmental impacts of fossil-fuel use necessitate a shift towards cleaner and renewable alternatives. Concentrated Solar Power (CSP) technology emerges as a promising solution, offering a carbon-free alternative for power generation. The efficiency and profitability of CSP depend on the Direct Normal Irradiance (DNI) component of solar radiation; hence, accurate DNI forecasting can help optimize CSP plants’ operations and performance. The unpredictable nature of weather phenomena, particularly cloud cover, introduces uncertainty into DNI projections. Existing DNI forecasting models use meteorological factors, which are both challenging to estimate numerically over short prediction windows and expensive to model through data at a sufficiently high spatial and temporal resolution. This research addresses the challenge by presenting a novel approach that formulates DNI prediction as a multi-class classification problem, departing from conventional regression-based methods. The primary objective of this classification framework is to identify optimal periods aligning with specific operational thresholds for CSP plants, contributing to enhanced dispatch optimization strategies. We model the DNI classification problem using four advanced deep neural networks – rectified linear unit (ReLU) networks, 1D residual networks (ResNets), bidirectional long short-term memory (BiLSTM) networks, and transformers – achieving accuracies up to 93.5% without requiring meteorological parameters.

14 SOLAR ENERGY↗

Resolving the Solvation Structure and Transport Properties of Aqueous Zinc Electrolytes from Salt-in-Water to Water-in-Salt Using Neural Network Potential

Zn Cl 2 solutions are promising electrolytes for aqueous zinc-ion batteries. Here, we report a joint computational and experimental study of the structural and dynamic properties of aqueous Zn Cl 2 electrolytes with concentrations ranging from salt-in-water to water-in-salt (WIS). By developing a neural network potential (NNP) model, we perform molecular dynamics (MD) simulations with accuracy but at much larger lengths and longer timescales. The NNP predicted structures are validated by the structure factors measured by X-ray total scattering experiments. The MD trajectories provide a comprehensive and quantitative picture of the Zn 2 + solvation shell structures. Additionally, we find that the O − H covalent bonds in water are strengthened with increasing salt concentration, thus expanding the electrochemical stability window of aqueous electrolytes. In terms of dynamic properties, the calculated and experimentally measured conductivities are in good agreement. Through the analysis of the calculated cation transference number, we propose a three-stage charge carrier transport mechanism with increasing concentration: independent ion transport, strongly correlated ion transport, and small positive charge carrier diffusion through negatively charged polymeric clusters. Our study provides fundamental atomic scale insights into the structure and transport properties of the Zn Cl 2 electrolyte that can aid the optimization and development of WIS electrolytes. Published by the American Physical Society 2025

25 ENERGY STORAGE↗