SEARCH · Engineering Papers

Results for “ANN”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Benchmarking machine learning interatomic potentials via phonon anharmonicity

Abstract Machine learning approaches have recently emerged as powerful tools to probe structure-property relationships in crystals and molecules. Specifically, machine learning interatomic potentials (MLIPs) can accurately reproduce first-principles data at a cost similar to that of conventional interatomic potential approaches. While MLIPs have been extensively tested across various classes of materials and molecules, a clear characterization of the anharmonic terms encoded in the MLIPs is lacking. Here, we benchmark popular MLIPs using the anharmonic vibrational Hamiltonian of ThO 2 in the fluorite crystal structure, which was constructed from density functional theory (DFT) using our highly accurate and efficient irreducible derivative methods. The anharmonic Hamiltonian was used to generate molecular dynamics (MD) trajectories, which were used to train three classes of MLIPs: Gaussian approximation potentials, artificial neural networks (ANN), and graph neural networks (GNN). The results were assessed by directly comparing phonons and their interactions, as well as phonon linewidths, phonon lineshifts, and thermal conductivity. The models were also trained on a DFT MD dataset, demonstrating good agreement up to fifth-order for the ANN and GNN. Our analysis demonstrates that MLIPs have great potential for accurately characterizing anharmonicity in materials systems at a fraction of the cost of conventional first principles-based approaches.

interatomic potentials↗

DOE OSTI2025

Leveraging dendritic complexity for neuromorphic computing

Abstract Beyond-von Neumann computing approaches are necessary to sustain the growth of microelectronics and the increasing appetite for artificial intelligence/machine learning algorithms. Neuromorphic computing is an emerging paradigm that takes inspiration from the brain to provide a path forward to improve the computational efficiency and computational density of next-generation computing architectures. In nature, we observe brains performing complex computations with a much smaller energy footprint than conventional computing approaches. Current neuromorphic systems are focused primarily on scalability, namely, increasing the number of computational units (neurons) and connections between units (synapses). However, for brain-like cognition and efficiency in next-generation computing hardware, we need increased complexity in function, as well as improved connection density for scalability. Here, we present our work that aims to incorporate dendrites for ‘compute-on-wire’ in neuromorphic architectures to increase the computational complexity (e.g. number of programmable parameters, nonlinear dynamics) as well as computational efficiency (energy/compute) of artificial neural networks (ANNs). We do this by showcasing neuromorphic dendrite elements that can be leveraged for various applications. We will present examples of neuroscience-inspired direction-selective circuits and an ANN with active dendrites leveraging shunting inhibition. We also demonstrate the benefits of using dendrites in deep neural networks. To conclude, we discuss how we can utilize emerging hardware devices in these systems and design next-generation neuromorphic architectures with dendrites.

Cardwell, Suma G. (ORCID:0000000226575545)↗

DOE OSTI2025

A Continuous Galactic Line Source of Axions: The Remarkable Case of 23 Na

Here, we argue that $^{23}$Na is a potentially significant source of galactic axions. For temperatures $\gtrsim 7 \times 10^8$K -- characteristic of carbon burning in the massive progenitors of supernovae and ONeMg white dwarfs -- the 440 keV first excited state of $^{23}$Na is thermally populated, with its repeated decays pumping stellar energy into escaping axions. Odd-A nuclear abundances are typically very low in high-temperature stellar environments (or absent entirely due to burn-up). $^{23}$Na is an exception: $\approx 0.1 M_\odot$ of the isotope is synthesized during carbon burning then maintained at $\approx 10^9$K for times ranging up to $6 \times 10^4$y. Using MESA simulations, a galactic model, and sampling over progenitor masses, locations, and evolutionary stages, we find a continuous flux at earth of $\langle ϕ_a \rangle \approx 22$/cm$^2$s for $g^\mathrm{eff}_{aNN} = 10^{-9}$. Some fraction of these axions convert to photons as they propagate through the galactic magnetic field, producing a distinctive 440 keV line $γ$ ray detectable by all-sky detectors like the Compton Spectrometer and Imager (COSI). Assuming a 1$μ$G galactic magnetic field and a sufficiently light axion mass, we find that COSI will be able to probe $| g_{aNN}^\mathrm{eff} g_{a γγ} | \gtrsim1.8 \times 10^{-22}$ GeV$^{-1}$ at $3σ$ after two years of surveying.

Dark matter↗

DOE OSTI2025

Optimizing temperature distributions for training neural quantum states using parallel tempering

Parametrized artificial neural networks (ANNs) can be very expressive ansatzes for variational algorithms, reaching state-of-the-art energies on many quantum many-body Hamiltonians. Nevertheless, the training of the ANN can be slow and stymied by the presence of local minima in the parameter landscape. One approach to mitigate this issue is to use parallel tempering methods, and in this work, we focus on the role played by the temperature distribution of the parallel tempering replicas. Using an adaptive method that adjusts the temperatures in order to equate the exchange probability between neighboring replicas, we show that this temperature optimization can significantly increase the success rate of the variational algorithm with negligible computational cost by eliminating bottlenecks in the replicas' random walk. Furthermore, we demonstrate this using two different neural networks, a restricted Boltzmann machine and a feedforward network, which we use to study a toy problem based on a permutation invariant Hamiltonian with a pernicious local minimum and the 𝐽 1 −𝐽 2 model on a rectangular lattice.

Neural network simulations↗

DOE OSTI2023

Effective permeability of an immiscible fluid in porous media determined from its geometric state

Based on the phenomenological extension of Darcy's law, two-fluid flow is dependent on a relative permeability function of saturation only that is process or path dependent with an underlying dependency on pore structure and wettability. For applications (fuel cells to underground CO 2 storage), it is imperative to determine the effective phase permeability relationships where the traditional approach is based on the inverse modeling of time-consuming experiments. The underlying reason is that the fundamental upscaling step from pore to Darcy scale, which links the pore structure of the porous medium to the continuum hydraulic conductivities, is not solved. Herein, we develop an artificial neural network (ANN) that relies on fundamental geometrical relationships to determine the mechanical energy dissipation during creeping immiscible two-fluid flow. Finally, the developed ANN is based on a prescribed set of state variables based on physical insights that predict the effective permeability of 4500 unseen pore-scale geometrical states with R 2 = 0.98.

42 ENGINEERING↗

DOE OSTI2021

Mitigating Catastrophic Forgetting in Deep Learning in a Streaming Setting Using Historical Summary

Recent advancements in scientific equipment and the adaptation of electronics and the Internet of Things (IoT) in our everyday lives resulted in large and complex data production at a high rate. Making meaningful and timely knowledge discovery at a modest cost from this big data is difficult for computing power and storage limitations. Training deep learning models incrementally in a streaming setting can help us with overcoming these limitations. However, in a well-known phenomenon named catastrophic forgetting, incrementally trained models increasingly perform poorly on the past data. To mitigate catastrophic forgetting in training in a streaming setting, we propose constructing a historical summary over time and use the summary with newly arrived data during incremental training. We propose various data summarization techniques such as random sampling, micro clustering, coreset computation, and Auto Encoders to counteract catastrophic forgetting. We built a pipeline for incremental training with a historical summary for training deep learning models for streaming data. We demonstrate the effectiveness of historical summary in mitigating catastrophic forgetting using three case studies involving three different deep learning applications: an Artificial Neural Network (ANN) for classification task on MNIST dataset, a language model (RNN-LM) on the WikiText2 dataset, and a Convolutional Neural Network (CNN), ResNet50 to classify the ImageNet dataset. Through the training of the models, we observe that catastrophic forgetting is evident in ANN and CNN but not in an RNN. For the first task, our method recovers up to 47.9% lost accuracy due to catastrophic forgetting. For the third task, the historical summary recovers classification accuracy by up to 25%. For the second task, though there is not proof of catastrophic forgetting, the training performance (PPL) improves by up to 26% with historical summary.

Dash, Sajal↗

DOE OSTI2026

Accelerating Traction Motor Optimization Design with AI Surrogate Models

The advancement of artificial intelligence systems enables the use of data-driven physics-based surrogate models to explore design spaces rapidly and deeply for engineering projects. This work presents a surrogate model workflow that accelerates electric traction motor design optimization by replacing finite element analysis (FEA) with an artificial neural network (ANN) and using this model in a genetic algorithm for design optimization. A baseline interior permanent-magnet motor is parameterized and sampled to generate FEA-labeled training data, after which a feed-forward ANN predicts key outputs (e.g., loss components and weight). The validated surrogate enables genetic-algorithm optimization and deep search over the design space without new FEA runs, producing Pareto-optimal trade-offs between weight and losses and set of optimized designs for rapid downselection of manufacturable motor designs.

Ribeiro, Pedro [ORNL] (ORCID:0009000921026641)↗

DOE OSTI2024

Machine Learning-Driven Reliability Estimation of PV Inverters Considering Alert-Ambient Variability

Weather-induced spatio-temporal degradation limits outdoor PV inverter lifetime and reliability, necessitating advanced data analysis. This study employs a top-down, data-driven approach utilizing multiple machine learning (ML) algorithms to estimate inverter reliability in a 1.4 MW PV power plant, considering factors such as irradiance, humidity, temperature, time of day, and weather conditions. An extensive alert dataset from 17 identical inverters, including alert types, propagation, and frequency, reveals significant correlations with environmental factors and inverter output power, enabling the construction of a performance reliability model. Dual-stage supervised-ML models are evaluated for accuracy, with the ‘classification-regression’ model by an artificial neural network (ANN) tested on the averaged “Alert-Ambient” dataset, which is outperformed by ‘clustering-regression’ models using random forest (RF) and K-Nearest Neighbors (KNN) on individual inverter datasets. K-means clustering applies principal component analysis to reduce dimensions, achieving improved accuracy beyond the 80% achieved by ANN on the averaged dataset. Second-stage regression estimates inverter reliability with a mean square error of 0.0195 on the averaged dataset and as low as 0.002 on individual inverter datasets using RF. Furthermore, these findings highlight the method's suitability for estimating PV inverter output reliability under ambient conditions, essential for digital twin development and related applications.

14 SOLAR ENERGY↗

DOE OSTI2021

Accurate and Accelerated Neuromorphic Network Design Leveraging A Bayesian Hyperparameter Pareto Optimization Approach

Neuromorphic systems allow for extremely efficient hardware implementations for neural networks (NNs). In recent years, several algorithms have been presented to train spiking NNs (SNNs) for neuromorphic hardware. However, SNNs often provide lower accuracy than their artificial NNs (ANNs) counterparts or require computationally expensive and slow training/inference methods. To close this gap, designers typically rely on reconfiguring SNNs through adjustments in the neuron/synapse model or training algorithm itself. Nevertheless, these steps incur significant design time, while still lacking the desired improvement in terms of training/inference times (latency). Designing SNNs that can mimic the accuracy of ANNs with reasonable training times is an exigent challenge in neuromorphic computing. In this work, we present an alternative approach that looks at such designs as an optimization problem rather than algorithm or architecture redesign. We develop a versatile multiobjective hyperparameter optimization (HPO) for automatically tuning HPs of two state-of-the-art SNN training algorithms, SLAYER and HYBRID. We emphasize that, to the best of our knowledge, this is the first work trying to improve SNNs’ computational efficiency, accuracy, and training time using an efficient HPO. We demonstrate significant performance improvements for SNNs on several datasets without the need to redesign or invent new training algorithms/architectures. Our approach results in more accurate networks with lower latency and, in turn, higher energy efficiency than previous implementations. In particular, we demonstrate improvement in accuracy and more than 5× reduction in the training/inference time for the SLAYER algorithm on the DVS Gesture dataset. In the case of HYBRID, we demonstrate 30% reduction in timesteps while surpassing the accuracy of the state-of-the-art networks on CIFAR10. Further, our analysis suggests that even a seemingly minor change in HPs could change the accuracy by 5 - 6×.

Parsa, Maryam↗

DOE OSTI2021

Combined Mixed Potential Electrochemical Sensors and Artificial Neural Networks for the Quantificationand Identification of Methane in Natural Gas Emissions Monitoring

Sensors capable of quantifying methane concentration and discriminating between possible sources are needed for natural gas leak detection where multiple spatially overlapping sources including wetlands and agriculture may be present. We report on the fabrication by an additive manufacturing process of a four electrode La 0.87 Sr 0.13 CrO 3 , Indium Tin Oxide (In 2 O 3 90 wt%, SnO 2 10 wt%), Au, Pt mixed potential electrochemical sensor using yttria-stabilized zirconia (YSZ) as a solid electrolyte to natural gas detection. Artificial neural networks (ANNs) are used to automatically decode the possible source and concentration of methane. The ANNs trained on sensor data are capable of correctly discriminating between three sources of methane emissions from simulated mixtures of emissions from cattle, wetlands, or natural gas with >98% accuracy. Quantification error for methane in mixtures of CH 4 in air, CH 4 + NH3 in air, and simulated natural gas is less than 1.5% ppm when a two-temperature dataset is employed.

03 NATURAL GAS↗

DOE OSTI2020

Framework For Spatial Agricultural Crop Yield Prediction Model Development

This framework was developed to provide data preprocessing for spatiotemporal agricultural yield data and remote sensing data for modelling using artificial neural networks (ANNs) to predict subfield crop yield estimates. The software includes methods to train, validate, and test ANN models. It also include methods to infer on new remote sensing data.

Griffel, LloydM.↗

DOE OSTI2020

Nested active learning for efficient model contextualization and parameterization: pathway to generating simulated populations using multi-scale computational models

There is increasing interest in the use of mechanism-based multi-scale computational models (such as agent-based models (ABMs)) to generate simulated clinical populations in order to discover and evaluate potential diagnostic and therapeutic modalities. The description of the environment in which a biomedical simulation operates (model context) and parameterization of internal model rules (model content) requires the optimization of a large number of free parameters. In this work, we utilize a nested active learning (AL) workflow to efficiently parameterize and contextualize an ABM of systemic inflammation used to examine sepsis. Contextual parameter space was examined using four parameters external to the model’s rule set. The model’s internal parameterization, which represents gene expression and associated cellular behaviors, was explored through the augmentation or inhibition of signaling pathways for 12 signaling mediators associated with inflammation and wound healing. We have implemented a nested AL approach in which the clinically relevant (CR) model environment space for a given internal model parameterization is mapped using a small Artificial Neural Network (ANN). The outer AL level workflow is a larger ANN that uses AL to efficiently regress the volume and centroid location of the CR space given by a single internal parameterization. We have reduced the number of simulations required to efficiently map the CR parameter space of this model by approximately 99%. In addition, we have shown that more complex models with a larger number of variables may expect further improvements in efficiency.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Machine Learning Application to Assess Occurrence and Saturations of Methane Hydrate in Marine Deposits Offshore India

Artificial Neural Networks (ANN) were used to assess methane hydrate occurrence and saturation in marine sediments offshore India. The ANN analysis classifies the gas hydrate occurrence into three types: methane hydrate in pore space, methane hydrate in fractures, or no methane hydrate. Further, predicted saturation characterizes the volume of gas hydrate with respect to the available void volume. Log data collected at six wells, which were drilled during the India National Gas Hydrate Program Expedition 02 (NGHP-02), provided a combination of well log measurements that were used as input for machine learning (ML) models. Well log measurements included density, porosity, electrical resistivity, natural gamma radiation, and acoustic wave velocity. Combinations of well logs used in the ML models provide good overall balanced accuracy (0.79 to 0.86) for the prediction of the gas hydrate occurrence and good accuracy (0.68 to 0.92) for methane hydrate saturation prediction in the marine accumulations against reference data. The accuracy scores indicate that the ML models can successfully predict reservoir characteristics for marine methane hydrate deposits. In conclusion, the results indicate that the ML models can either augment physics-driven methods for assessing the occurrence and saturation of methane hydrate deposits or serve as an independent predictive tool for those characteristics.

58 GEOSCIENCES↗

DOE OSTI2023

Capturing Travel Mode Adoption in Designing On-Demand Multimodal Transit Systems

This paper studies how to integrate rider mode preferences into the design of on-demand multimodal transit systems (ODMTSs). It is motivated by a common worry in transit agencies that an ODMTS may be poorly designed if the latent demand, that is, new riders adopting the system, is not captured. This paper proposes a bilevel optimization model to address this challenge, in which the leader problem determines the ODMTS design, and the follower problems identify the most cost efficient and convenient route for riders under the chosen design. The leader model contains a choice model for every potential rider that determines whether the rider adopts the ODMTS given her proposed route. To solve the bilevel optimization model, the paper proposes an exact decomposition method that includes Benders optimal cuts and no-good cuts to ensure the consistency of the rider choices in the leader and follower problems. Moreover, to improve computational efficiency, the paper proposes upper and lower bounds on trip durations for the follower problems, valid inequalities that strengthen the no-good cuts, and approaches to reduce the problem size with problem-specific preprocessing techniques. The proposed method is validated using an extensive computational study on a real data set from the Ann Arbor Area Transportation Authority, the transit agency for the broader Ann Arbor and Ypsilanti region in Michigan. The study considers the impact of a number of factors, including the price of on-demand shuttles, the number of hubs, and access to transit systems criteria. The designed ODMTSs feature high adoption rates and significantly shorter trip durations compared with the existing transit system and highlight the benefits of ensuring access for low-income riders. Finally, the computational study demonstrates the efficiency of the decomposition method for the case study and the benefits of computational enhancements that improve the baseline method by several orders of magnitude. Funding: This research was partly supported by National Science Foundation [Leap HI Proposal NSF-1854684] and the Department of Energy [Research Award 7F-30154].

Operations Research & Management Science↗

DOE OSTI2022

Quantification of neural networks uncertainties with applications to SAFARI-1 axial neutron flux profiles

Deep Neural Networks (DNNs) have been widely used as a data-driven modelling tool in nuclear engineering. However, as a Machine Learning model, Artificial Neural Network (ANN) predictions are subjected to uncertainties originating from the noise in training data, incomplete coverage of the domain, and imperfect neural network architectures. In this work, we target at quantifying the prediction/approximation uncertainties of ANNs using Monte Carlo Dropout (MCD), as well as Bayesian Neural Networks (BNNs) which are solved by variational inference. With a demonstration problem in which neural networks are used to predict the assembly axial neutron flux profiles, the results have shown that the three different neural network models (regular DNNs, DNNs solved with MCD and BNNs) can produce results that agree very well among each other and with the measurement data, on cycles that are not used in the training process. Besides the excellent generalization capability, the uncertainty bands produced by MCD and BNN agree very well, and in general, they can fully envelop the noisy measurement data points. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2023

High Energy Theory and Cosmology Workshops at the Leinweber Center for Theoretical Physics at the University of Michigan

This award partially supported a workshop and a summer school held at the Leinweber Center for Theoretical Physics at the University of Michigan in Ann Arbor. Funds were primarily used to support the travel of external participants. The programs were substantially delayed owing to the pandemic. The symposium was initially slated for Spring 2020. The 2023 Michigan Cosmology Summer School was held June 5-9 2023 in Ann Arbor Michigan. There were about 150 in-person international participants (of whom about 35Nine external lecturers listed below delivered a series of lectures ranging on topics from “Measuring and Mapping Cosmic Shear” to “Dark Dark Energy and Modified Gravity.” Many archived version of the slides and videos of the lectures may be found at https://johannesulf.github.io/cosmology-school-2023/program.html

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2025

Physics-Informed Neural Network (PINN) Prediction of Mixed Mass-Heat-Crystallization Limited Methane Hydrate Formation and Dissociation in Micro-Confinement

The creation and use of Physics-Informed Neural Networks (PINNs) for simulating the dynamics of methane hydrate formation and dissociation will be presented. The PINN framework's main benefit is its capacity to impose physical consistency with only a partial comprehension of the governing equations. This makes the algorithm especially useful for systems with little experimental evidence or a lack of theoretical knowledge. A strong basis for forecasting methane hydrate behavior over the verified operating ranges of 30.0-80.9 bar pressure and 1.0-4.0 K sub-cooling conditions is provided by the combination of conductive heat transfer equations and mixed mass-transfer–crystallization kinetics. PINNs were more accurate at predicting the mixed mass-heat-crystallization limited kinetics than conventional Artificial Neural Networks (ANNs), demonstrating remarkable predictive accuracy for methane hydrate production over the ANN model. The efficiency of incorporating physical limitations from first principles into machine learning frameworks for methane hydrate crystallizations is reinforced by these findings. For hydrate-related applications in energy generation, carbon sequestration, and climate modelling, our study establishes PINNs as a computational tool that is both scalable and efficient. The proven capacity to close the gap between conventional physics-based simulations and solely data-driven models creates new opportunities for expedited hydrate research and practical applications.

Hartman, Ryan L [NYU Tandon School of Engineering]↗

DOE OSTI2023

Efficient human activity recognition with spatio-temporal spiking neural networks

In this study, we explore Human Activity Recognition (HAR), a task that aims to predict individuals' daily activities utilizing time series data obtained from wearable sensors for health-related applications. Although recent research has predominantly employed end-to-end Artificial Neural Networks (ANNs) for feature extraction and classification in HAR, these approaches impose a substantial computational load on wearable devices and exhibit limitations in temporal feature extraction due to their activation functions. To address these challenges, we propose the application of Spiking Neural Networks (SNNs), an architecture inspired by the characteristics of biological neurons, to HAR tasks. SNNs accumulate input activation as presynaptic potential charges and generate a binary spike upon surpassing a predetermined threshold. This unique property facilitates spatio-temporal feature extraction and confers the advantage of low-power computation attributable to binary spikes. We conduct rigorous experiments on three distinct HAR datasets using SNNs, demonstrating that our approach attains competitive or superior performance relative to ANNs, while concurrently reducing energy consumption by up to 94%.

60 APPLIED LIFE SCIENCES↗