Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “ANN”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Machine Learning Self-Diffusion Prediction for Lennard-Jones Fluids in Pores

Predicting the diffusion coefficient of fluids under nanoconfinement is important for many applications including the extraction of shale gas from kerogen and product turnover in porous catalysts. Due to the large number of important variables, including pore shape and size, fluid temperature and density, and the fluid–wall interaction strength, simulating diffusion coefficients using molecular dynamics (MD) in a systematic study could prove to be prohibitively expensive. Here, we use machine learning models trained on a subset of MD data to predict the self-diffusion coefficients of Lennard-Jones fluids in pores. Our MD data set contains 2280 simulations of ideal slit pore, cylindrical pore, and hexagonal pore geometries. We use the forward feature selection method to determine the most useful features (i.e., descriptors) for developing an artificial neutral network (ANN) model with an emphasis on easily acquired features. Our model shows good predictive ability with a coefficient of determination (i.e., R 2 ) of ~0.99 and a mean squared error of ~2.9 × 10 –5 . Finally, we propose an alteration to our feature set that will allow the ANN model to be applied to nonideal pore geometries.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Machine Learning-Based Upscaling of Finite-Size Molecular Dynamics Diffusion Simulations for Binary Fluids

Molecular diffusion coefficients calculated using molecular dynamics (MD) simulations suffer from finite-size (i.e., finite box size and finite particle number) effects. Results from finite-sized MD simulations can be upscaled to infinite simulation size by applying a correction factor. For self-diffusion of single-component fluids, this correction has been well-studied by many researchers including Yeh and Hummer (YH); for binary fluid mixtures, a modified YH correction was recently proposed for correcting MD-predicted Maxwell–Stephan (MS) diffusion rates. In this study we use both empirical and machine learning methods to identify improvements to the finite-size correction factors for both self-diffusion and MS diffusion of binary Lennard-Jones (LJ) fluid mixtures. Using artificial neural networks (ANNs), the error in the corrected LJ fluid diffusion is reduced by an order of magnitude versus existing YH corrections, and the ANN models perform well for mixtures with large dissimilarities in size and interaction energies where the YH correction proves insufficient.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Modeled Surface-Atmosphere Fluxes From Paired Sites in the Upper Great Lakes Region Using Neural Networks

The eddy covariance (EC) method is one of the most widely used approaches to quantify surface-atmosphere fluxes. However, scaling up from a single EC tower to the landscape remains an open challenge. To address this, we used 63 site years of data to examine simulated annual and growing season sums of carbon fluxes from three paired land-cover type sites of corn, restored-prairie, and switchgrass ecosystems. This was also done across the landscape by modeling fluxes using different land-cover type input data. An artificial neural network (ANN) approach was used to model net ecosystem exchange (NEE), ecosystem respiration (R eco ), and gross primary production (GPP) at one paired site using environmental observations from the second site only. With a mean spatial separation of 11 km between paired sites, we were able to model annual sums of NEE, R eco , and GPP with uncertainties of 20%, 22%, and 8%, respectively, relative to observation sums. When considering the growing season only, model uncertainties were 17%, 22%, and 9%, respectively for the three flux terms. We also show that ANN models can estimate sums of R eco and GPP fluxes without needing the constraint of similar land-cover-type, with annual uncertainties of 12% and 10%. These results provide new insights to scaling up observations from one EC site beyond the footprint of the EC tower to multiple land-cover types across the landscape.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Short–Term Forecasting of Wind Gusts at Airports Across CONUS Using Machine Learning

Short–term forecasting of wind gusts, particularly those of higher intensity, is of great societal importance but is challenging due to the presence of multiple gust generation mechanisms. Wind gust observations from eight high–passenger–volume airports across the continental United States (CONUS) are summarized and used to develop predictive models of wind gust occurrence and magnitude. These short–term (same hour) forecast models are built using multiple logistic and linear regression, as well as artificial neural networks (ANNs) of varying complexity. A suite of 19 upper–air predictors drawn from the ERA5 reanalysis and an autoregressive (AR) term are used. Stepwise procedures instruct predictor selection, and resampling is used to quantify model stability. All models are developed separately for the warm (April–September) and cold (October–March) seasons. Results show that ANNs of 3–5 hidden layers (HLs) generally exhibit higher hit rates than logistic regression models and also improve skill with respect to wind gust magnitudes. However, deeper networks with more HLs increase false alarm rates in occurrence models and mean absolute error in magnitude models due to model overfitting. For model skill, inclusion of the AR term is critical while the majority of the remaining skill derives from wind speeds and lapse rates. A predictive ceiling is also clearly demonstrated, particularly for the strong and damaging gust magnitudes, which appears to be partially due to ERA5 predictor characteristics and the presence of mixed wind climates.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Modeling Ring Current Proton Fluxes Using Artificial Neural Network and Van Allen Probe Measurements

Abstract Terrestrial ring current dynamics are a critical part of the near‐space environment, in that they directly drive geomagnetic field variations that control particle drifts, and define geomagnetic storms. The present study aims to specify a global and time‐varying distribution of ring current proton using geomagnetic indices and solar wind parameters with their history as input. We train an artificial neural network (ANN) model to reproduce proton fluxes measured by the Radiation Belt Storm Probes Ion Composition Experiment instrument onboard Van Allen Probes. By choosing optimal feature parameters and their history length, the model results show a high correlation and a small error between model specifications and satellite measurements. The modeled results well capture energy‐dependent proton dynamics in association with geomagnetic storms, including inward radial diffusion, acceleration and decay. Our ANN model produces proton fluxes with their corresponding 3D spatiotemporal variations, capturing the latitudinal distribution and local time asymmetry that are consistent with observations and that can further inform theory.

Li, Jinxing↗
DOE OSTI2023

High–Resolution Maps of Near–Surface Permafrost for Three Watersheds on the Seward Peninsula, Alaska Derived From Machine Learning

Permafrost soils are a critical component of the global carbon cycle and are locally important because they regulate the hydrologic flux from uplands to rivers. Furthermore, degradation of permafrost soils causes land surface subsidence, damaging infrastructure that is crucial for local communities. Regional and hemispherical maps of permafrost are too coarse to resolve distributions at a scale relevant to assessments of infrastructure stability or to illuminate geomorphic impacts of permafrost thaw. Here we train machine learning models to generate meter–scale maps of near–surface permafrost for three watersheds in the discontinuous permafrost region. The models were trained using ground truth determinations of near–surface permafrost presence from measurements of soil temperature and electrical resistivity. We trained three classifiers: extremely randomized trees (ERTr), support vector machines (SVM), and an artificial neural network (ANN). Model uncertainty was determined using k–fold cross validation, and the modeled extents of near–surface permafrost were compared to the observed extents at each site. At–a–site near–surface permafrost distributions predicted by the ERTr produced the highest accuracy (70%–90%). However, the transferability of the ERTr to the sites outside of the training data set was poor, with accuracies ranging from 50% to 77%. The SVM and ANN models had lower accuracies for at–a–site prediction (70%–83%), yet they had greater accuracy when transferred to the non–training site (62%–78%). These models demonstrate the potential for integrating high–resolution spatial data and machine learning models to develop maps of near–surface permafrost extent at resolutions fine enough to assess infrastructure vulnerability and landscape morphology influenced by permafrost thaw.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Identification of high-dielectric constant compounds from statistical design

Abstract The discovery of high-dielectric materials is crucial to increasing the efficiency of electronic devices and batteries. Here, we report three previously unexplored materials with very high dielectric constants (69 < ϵ < 101) and large band gaps (2.9 < E g (eV) < 5.5) obtained by screening materials databases using statistical optimization algorithms aided by artificial neural networks (ANN). Two of these new dielectrics are mixed-anion compounds (Eu 5 SiCl 6 O 4 and HoClO) and are shown to be thermodynamically stable against common semiconductors via phase diagram analysis. We also uncovered four other materials with relatively large dielectric constants (20 < ϵ < 40) and band gaps (2.3 < E g (eV) < 2.7). While the ANN training-data are obtained from the Materials Project, the search-space consists of materials from the Open Quantum Materials Database (OQMD)—demonstrating a successful implementation of cross-database materials design. Overall, we report the dielectric properties of 17 materials calculated using ab initio calculations, that were selected in our design workflow. The dielectric materials with high-dielectric properties predicted in this work open up further experimental research opportunities.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Comparison of machine learning techniques to optimize the analysis of plutonium surrogate material via a portable LIBS device

The utilization of machine learning techniques has become commonplace in the analysis of optical emission spectra. These methods are often limited to variants of principal components analysis (PCA), partial-least squares (PLS), and artificial neural networks (ANNs). A plethora of other techniques exist and are well established in the world of data science, yet are seldom investigated for their use in spectroscopic problems. In this study, machine learning techniques were used to analyze optical emission spectra of laser-induced plasma from ceria pellets doped with silicon in order to predict silicon content. Additionally, a boosted regression ensemble model was created, and its predictive accuracy was compared to that of traditional PCA, PLS, and ANN regression models. Boosted regression tree ensembles yielded fits with R-squared (R2) values as high as 0.964 and mean-squared errors of prediction (MSEPs) as low as 0.074, providing the most accurate predictive model. Neural networks performed with slightly lower R2 values and higher MSEPs compared to the ensemble methods, thus indicating susceptibility to overfitting.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Machine learning prediction of self-diffusion in Lennard-Jones fluids

In this work, different machine learning (ML) methods were explored for the prediction of self-diffusion in Lennard-Jones (LJ) fluids. Using a database of diffusion constants obtained from the molecular dynamics simulation literature, multiple Random Forest (RF) and Artificial Neural Net (ANN) regression models were developed and characterized. The role and improved performance of feature engineering coupled to the RF model development was also addressed. The performance of these different ML models was evaluated by comparing the prediction error to an existing empirical relationship used to describe LJ fluid diffusion. It was found that the ANN regression models provided superior prediction of diffusion in comparison to the existing empirical relationships.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2022

Symbolic regression development of empirical equations for diffusion in Lennard-Jones fluids

Symbolic regression (SR) with a multi-gene genetic program has been used to elucidate new empirical equations describing diffusion in Lennard-Jones (LJ) fluids. Some examples include equations to predict self-diffusion in pure LJ fluids and equations describing the finite-size correction for self-diffusion in binary LJ fluids. The performance of the SR-obtained equations was compared to that of both the existing empirical equations in the literature and to the results from artificial neural net (ANN) models recently reported. It is found that the SR equations have improved predictive performance in comparison to the existing empirical equations, even though employing a smaller number of adjustable parameters, but show an overall reduced performance in comparison to more extensive ANNs.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Efficient data acquisition and training of collisional-radiative model artificial neural network surrogates through adaptive parameter space sampling

Abstract Effective plasma transport modeling of magnetically confined fusion devices relies on having an accurate understanding of the ion composition and radiative power losses of the plasma. Generally, these quantities can be obtained from solutions of a collisional-radiative (CR) model at each time step within a plasma transport simulation. However, even compact, approximate CR models can be computationally onerous to evaluate, and in-situ evaluation of these models within a larger plasma transport code can lead to a rigid bottleneck. As a way to bypass this bottleneck, we propose deploying artificial neural network (ANN) surrogates to allow rapid evaluation of the necessary plasma quantities. However, one issue with training an accurate ANN surrogate is the reliance on a sufficiently large and representative training and validation data set, which can be time-consuming to generate. In this work we explore a data-driven active learning and training routine to allow autonomous adaptive sampling of the problem parameter space to ensure a sufficiently large and meaningful set of training data is assembled for the network training. As a result, we can demonstrate approximately order-of-magnitude savings in required training data samples to produce an accurate surrogate.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Benchmarking machine learning interatomic potentials via phonon anharmonicity

Abstract Machine learning approaches have recently emerged as powerful tools to probe structure-property relationships in crystals and molecules. Specifically, machine learning interatomic potentials (MLIPs) can accurately reproduce first-principles data at a cost similar to that of conventional interatomic potential approaches. While MLIPs have been extensively tested across various classes of materials and molecules, a clear characterization of the anharmonic terms encoded in the MLIPs is lacking. Here, we benchmark popular MLIPs using the anharmonic vibrational Hamiltonian of ThO 2 in the fluorite crystal structure, which was constructed from density functional theory (DFT) using our highly accurate and efficient irreducible derivative methods. The anharmonic Hamiltonian was used to generate molecular dynamics (MD) trajectories, which were used to train three classes of MLIPs: Gaussian approximation potentials, artificial neural networks (ANN), and graph neural networks (GNN). The results were assessed by directly comparing phonons and their interactions, as well as phonon linewidths, phonon lineshifts, and thermal conductivity. The models were also trained on a DFT MD dataset, demonstrating good agreement up to fifth-order for the ANN and GNN. Our analysis demonstrates that MLIPs have great potential for accurately characterizing anharmonicity in materials systems at a fraction of the cost of conventional first principles-based approaches.

interatomic potentials↗
DOE OSTI2025

Leveraging dendritic complexity for neuromorphic computing

Abstract Beyond-von Neumann computing approaches are necessary to sustain the growth of microelectronics and the increasing appetite for artificial intelligence/machine learning algorithms. Neuromorphic computing is an emerging paradigm that takes inspiration from the brain to provide a path forward to improve the computational efficiency and computational density of next-generation computing architectures. In nature, we observe brains performing complex computations with a much smaller energy footprint than conventional computing approaches. Current neuromorphic systems are focused primarily on scalability, namely, increasing the number of computational units (neurons) and connections between units (synapses). However, for brain-like cognition and efficiency in next-generation computing hardware, we need increased complexity in function, as well as improved connection density for scalability. Here, we present our work that aims to incorporate dendrites for ‘compute-on-wire’ in neuromorphic architectures to increase the computational complexity (e.g. number of programmable parameters, nonlinear dynamics) as well as computational efficiency (energy/compute) of artificial neural networks (ANNs). We do this by showcasing neuromorphic dendrite elements that can be leveraged for various applications. We will present examples of neuroscience-inspired direction-selective circuits and an ANN with active dendrites leveraging shunting inhibition. We also demonstrate the benefits of using dendrites in deep neural networks. To conclude, we discuss how we can utilize emerging hardware devices in these systems and design next-generation neuromorphic architectures with dendrites.

Cardwell, Suma G. (ORCID:0000000226575545)↗
DOE OSTI2025

A Continuous Galactic Line Source of Axions: The Remarkable Case of 23 Na

Here, we argue that $^{23}$Na is a potentially significant source of galactic axions. For temperatures $\gtrsim 7 \times 10^8$K -- characteristic of carbon burning in the massive progenitors of supernovae and ONeMg white dwarfs -- the 440 keV first excited state of $^{23}$Na is thermally populated, with its repeated decays pumping stellar energy into escaping axions. Odd-A nuclear abundances are typically very low in high-temperature stellar environments (or absent entirely due to burn-up). $^{23}$Na is an exception: $\approx 0.1 M_\odot$ of the isotope is synthesized during carbon burning then maintained at $\approx 10^9$K for times ranging up to $6 \times 10^4$y. Using MESA simulations, a galactic model, and sampling over progenitor masses, locations, and evolutionary stages, we find a continuous flux at earth of $\langle ϕ_a \rangle \approx 22$/cm$^2$s for $g^\mathrm{eff}_{aNN} = 10^{-9}$. Some fraction of these axions convert to photons as they propagate through the galactic magnetic field, producing a distinctive 440 keV line $γ$ ray detectable by all-sky detectors like the Compton Spectrometer and Imager (COSI). Assuming a 1$μ$G galactic magnetic field and a sufficiently light axion mass, we find that COSI will be able to probe $| g_{aNN}^\mathrm{eff} g_{a γγ} | \gtrsim1.8 \times 10^{-22}$ GeV$^{-1}$ at $3σ$ after two years of surveying.

Dark matter↗
DOE OSTI2025

Optimizing temperature distributions for training neural quantum states using parallel tempering

Parametrized artificial neural networks (ANNs) can be very expressive ansatzes for variational algorithms, reaching state-of-the-art energies on many quantum many-body Hamiltonians. Nevertheless, the training of the ANN can be slow and stymied by the presence of local minima in the parameter landscape. One approach to mitigate this issue is to use parallel tempering methods, and in this work, we focus on the role played by the temperature distribution of the parallel tempering replicas. Using an adaptive method that adjusts the temperatures in order to equate the exchange probability between neighboring replicas, we show that this temperature optimization can significantly increase the success rate of the variational algorithm with negligible computational cost by eliminating bottlenecks in the replicas' random walk. Furthermore, we demonstrate this using two different neural networks, a restricted Boltzmann machine and a feedforward network, which we use to study a toy problem based on a permutation invariant Hamiltonian with a pernicious local minimum and the 𝐽 1 −𝐽 2 model on a rectangular lattice.

Neural network simulations↗
DOE OSTI2023

Effective permeability of an immiscible fluid in porous media determined from its geometric state

Based on the phenomenological extension of Darcy's law, two-fluid flow is dependent on a relative permeability function of saturation only that is process or path dependent with an underlying dependency on pore structure and wettability. For applications (fuel cells to underground CO 2 storage), it is imperative to determine the effective phase permeability relationships where the traditional approach is based on the inverse modeling of time-consuming experiments. The underlying reason is that the fundamental upscaling step from pore to Darcy scale, which links the pore structure of the porous medium to the continuum hydraulic conductivities, is not solved. Herein, we develop an artificial neural network (ANN) that relies on fundamental geometrical relationships to determine the mechanical energy dissipation during creeping immiscible two-fluid flow. Finally, the developed ANN is based on a prescribed set of state variables based on physical insights that predict the effective permeability of 4500 unseen pore-scale geometrical states with R 2 = 0.98.

42 ENGINEERING↗
DOE OSTI2021

Mitigating Catastrophic Forgetting in Deep Learning in a Streaming Setting Using Historical Summary

Recent advancements in scientific equipment and the adaptation of electronics and the Internet of Things (IoT) in our everyday lives resulted in large and complex data production at a high rate. Making meaningful and timely knowledge discovery at a modest cost from this big data is difficult for computing power and storage limitations. Training deep learning models incrementally in a streaming setting can help us with overcoming these limitations. However, in a well-known phenomenon named catastrophic forgetting, incrementally trained models increasingly perform poorly on the past data. To mitigate catastrophic forgetting in training in a streaming setting, we propose constructing a historical summary over time and use the summary with newly arrived data during incremental training. We propose various data summarization techniques such as random sampling, micro clustering, coreset computation, and Auto Encoders to counteract catastrophic forgetting. We built a pipeline for incremental training with a historical summary for training deep learning models for streaming data. We demonstrate the effectiveness of historical summary in mitigating catastrophic forgetting using three case studies involving three different deep learning applications: an Artificial Neural Network (ANN) for classification task on MNIST dataset, a language model (RNN-LM) on the WikiText2 dataset, and a Convolutional Neural Network (CNN), ResNet50 to classify the ImageNet dataset. Through the training of the models, we observe that catastrophic forgetting is evident in ANN and CNN but not in an RNN. For the first task, our method recovers up to 47.9% lost accuracy due to catastrophic forgetting. For the third task, the historical summary recovers classification accuracy by up to 25%. For the second task, though there is not proof of catastrophic forgetting, the training performance (PPL) improves by up to 26% with historical summary.

Dash, Sajal↗
DOE OSTI2026

Accelerating Traction Motor Optimization Design with AI Surrogate Models

The advancement of artificial intelligence systems enables the use of data-driven physics-based surrogate models to explore design spaces rapidly and deeply for engineering projects. This work presents a surrogate model workflow that accelerates electric traction motor design optimization by replacing finite element analysis (FEA) with an artificial neural network (ANN) and using this model in a genetic algorithm for design optimization. A baseline interior permanent-magnet motor is parameterized and sampled to generate FEA-labeled training data, after which a feed-forward ANN predicts key outputs (e.g., loss components and weight). The validated surrogate enables genetic-algorithm optimization and deep search over the design space without new FEA runs, producing Pareto-optimal trade-offs between weight and losses and set of optimized designs for rapid downselection of manufacturable motor designs.

Ribeiro, Pedro [ORNL] (ORCID:0009000921026641)↗