Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NEURAL NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 991 records · Page 55
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Million-scale data integrated deep neural network for phonon properties of heuslers spanning the periodic table

Existing machine learning potentials for predicting phonon properties of crystals are typically limited on a material-to-material basis, primarily due to the exponential scaling of model complexity with the number of atomic species. We address this bottleneck with the developed Elemental Spatial Density Neural Network Force Field, namely Elemental-SDNNFF. The effectiveness and precision of our Elemental-SDNNFF approach are demonstrated on 11,866 full, half, and quaternary Heusler structures spanning 55 elements in the periodic table by prediction of complete phonon properties. Self-improvement schemes including active learning and data augmentation techniques provide an abundant 9.4 million atomic data for training. Deep insight into predicted ultralow lattice thermal conductivity (<1 Wm –1 K –1 ) of 774 Heusler structures is gained by p–d orbital hybridization analysis. Additionally, a class of two-band charge-2 Weyl points, referred to as “double Weyl points”, are found in 68% and 87% of 1662 half and 1550 quaternary Heuslers, respectively.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Identification and denoising of radio signals from cosmic-ray air showers using convolutional neural networks

Radio pulses generated by cosmic-ray air showers can be used to reconstruct key properties like the energy and depth of the electromagnetic component of cosmic-ray air showers. Radio detection threshold, influenced by natural and anthropogenic radio background, can be reduced through various techniques. In this work, we demonstrate that convolutional neural networks (CNNs) are an effective way to lower the threshold. We developed two CNNs: a classifier to distinguish radio signal waveforms from background noise and a denoiser to clean contaminated radio signals. Following the training and testing phases, we applied the networks to air-shower data triggered by scintillation detectors of the prototype station for the enhancement of IceTop, IceCube’s surface array at the South Pole. Over a four-month period, we identified 554 cosmic-ray events in coincidence with IceTop, approximately five times more compared to a reference method based on a cut on the signal-to-noise ratio. Comparisons with IceTop measurements of the same air showers confirmed that the CNNs reliably identified cosmic-ray radio pulses and outperformed the reference method. Additionally, we find that CNNs reduce the false-positive rate of air-shower candidates and effectively denoise radio waveforms, thereby improving the accuracy of the power and arrival time reconstruction of radio pulses.

Abbasi, R↗
DOE OSTI2023

Uncertainty-aware photovoltaic generation estimation through fusion of physics with harmonics information using Bayesian neural networks

We develop an aggregate photovoltaic generation estimation methodology that uses diverse inputs and can reason on its current input-dependent predictive uncertainty. Named PVPHEst, for PhotoVoltaic Physics- & Harmonics-driven Estimator, the resulting tool is intelligently weighing and fusing information carried by the output of physics models, harmonics, and line sensors, using Bayesian neural networks and related techniques aimed at solving machine learning problems with intrinsic uncertainty quantification. As each of the three input classes carries heterogeneous information that only sheds light on one facet of the estimation problem but its value can diminish in the face of diverse grid phenomena, PV-PHEst with its estimation and uncertainty reasoning capabilities perform a nontrivial and potentially mission-critical task of value to grid operators.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Signal-to-noise improvement through neural network contour deformations for 3D $SU(2)$ lattice gauge theory

Complex contour deformations of the path integral have been demonstrated to significantly improve the signal-to-noise ratio of observables in previous studies of two-dimensional gauge theories with open boundary conditions. In this work, new developments based on gauge fixing and a neural network definition of the deformation are introduced, which enable an effective application to theories in higher dimensions and with generic boundary conditions. Improvements of the signal-to-noise ratio by up to three orders of magnitude for Wilson loop measurements are shown in $SU(2)$ lattice gauge theory in three spacetime dimensions.

Detmold, William↗
DOE OSTI2023

Recurrent and convolutional neural networks for sequential multispectral optoacoustic tomography ( MSOT ) imaging

Abstract Multispectral optoacoustic tomography (MSOT) is a beneficial technique for diagnosing and analyzing biological samples since it provides meticulous details in anatomy and physiology. However, acquiring high through‐plane resolution volumetric MSOT is time‐consuming. Here, we propose a deep learning model based on hybrid recurrent and convolutional neural networks to generate sequential cross‐sectional images for an MSOT system. This system provides three modalities (MSOT, ultrasound, and optoacoustic imaging of a specific exogenous contrast agent) in a single scan. This study used ICG‐conjugated nanoworms particles (NWs‐ICG) as the contrast agent. Instead of acquiring seven images with a step size of 0.1 mm, we can receive two images with a step size of 0.6 mm as input for the proposed deep learning model. The deep learning model can generate five other images with a step size of 0.1 mm between these two input images meaning we can reduce acquisition time by approximately 71%.

Juhong, Aniwat↗
DOE OSTI2021

Nuclei with Up to $$\varvec{A=6}$$ Nucleons with Artificial Neural Network Wave Functions

The ground-breaking works of Weinberg have opened the way to calculations of atomic nuclei that are based on systematically improvable Hamiltonians. Solving the associated many-body Schrödinger equation involves non-trivial difficulties, due to the non-perturbative nature and strong spin-isospin dependence of nuclear interactions. Artificial neural networks have proven to be able to compactly represent the wave functions of nuclei with up to $A=4$ nucleons. In this work, we extend this approach to $^6$Li and $^6$He nuclei, using as input a leading-order pionless effective field theory Hamiltonian. We successfully benchmark their binding energies, point-nucleon densities, and radii with the highly-accurate hyperspherical harmonics method.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Complementing Dynamical Downscaling With Super‐Resolution Convolutional Neural Networks

Despite advancements in Artificial Intelligence (AI) methods for climate downscaling, significant challenges remain for their practicality in climate research. Current AI-methods exhibit notable limitations, such as limited application in downscaling Global Climate Models (GCMs), and accurately representing extremes. To address these challenges, we implement an AI-based methodology using super-resolution convolutional neural networks (SRCNN), trained and evaluated on 40 years of daily precipitation data from a reanalysis and a high-resolution dynamically downscaled counterpart. The dynamical downscaled simulations, constrained using spectral nudging, enable the replication of historical events at a higher resolution. This allows the SRCNN to emulate dynamical downscaling effectively. Modifications, such as incorporating elevation data and data pre-processing enhances overall model performance, while using exponential and quantile loss functions improve the simulation of extremes. Our findings show SRCNN models efficiently and skillfully downscale precipitation from GCMs. Future work will expand this methodology to downscale additional variables for future climate projections.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Lie–Poisson Neural Networks (LPNets): Data-based computing of Hamiltonian systems with symmetries

An accurate data-based prediction of the long-term evolution of Hamiltonian systems requires a network that preserves the appropriate structure under each time step. Every Hamiltonian system contains two essential ingredients: the Poisson bracket and the Hamiltonian. Hamiltonian systems with symmetries, whose paradigm examples are the Lie–Poisson systems, have been shown to describe a broad category of physical phenomena, from satellite motion to underwater vehicles, fluids, geophysical applications, complex fluids, and plasma physics. The Poisson bracket in these systems comes from the symmetries, while the Hamiltonian comes from the underlying physics. We view the symmetry of the system as primary, hence the Lie–Poisson bracket is known exactly, whereas the Hamiltonian is regarded as coming from physics and is considered not known, or known approximately. Using this approach, we develop a network based on transformations that exactly preserve the Poisson bracket and the special functions of the Lie–Poisson systems (Casimirs) to machine precision. We present two flavors of such systems: one, where the parameters of transformations are computed from data using a dense neural network (LPNets), and another, where the composition of transformations is used as building blocks (G-LPNets). We also show how to adapt these methods to a larger class of Poisson brackets. We apply the resulting methods to several examples, such as rigid body (satellite) motion, underwater vehicles, a particle in a magnetic field, and others. The methods developed in this paper are important for the construction of accurate data-based methods for simulating the long-term dynamics of physical systems.

Data-based modeling↗
DOE OSTI2023

Emulating aerosol optics with randomly generated neural networks

Abstract. Atmospheric aerosols have a substantial impact on climate and remain one of the largest sources of uncertainty in climate prediction. Accurate representation of their direct radiative effects is a crucial component of modern climate models. However, direct computation of the radiative properties of aerosol populations is far too computationally expensive to perform in a climate model, so optical properties are typically approximated using a parameterization. This work develops artificial neural networks (ANNs) capable of replacing the current aerosol optics parameterization used in the Energy Exascale Earth System Model (E3SM). A large training dataset is generated by using Mie code to directly compute the optical properties of a range of atmospheric aerosol populations given a large variety of particle sizes, wavelengths, and refractive indices. Optimal neural architectures for shortwave and longwave bands are identified by evaluating ANNs with randomly generated wirings. Randomly generated deep ANNs are able to outperform conventional multilayer-perceptron-style architectures with comparable parameter counts. Finally, the ANN-based parameterization produces significantly more accurate bulk aerosol optical properties than the current parameterization when compared with direct Mie calculations using mean absolute error. The success of this approach makes possible the future inclusion of much more sophisticated representations of aerosol optics in climate models that cannot be captured by extension of the existing parameterization scheme and also demonstrates the potential of random-wiring-based neural architecture search in future applications in the Earth sciences.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

A Convolutional Neural Network Model for Battery Capacity Fade Curve Prediction using Early Life Data

Herein, early prediction of battery performance degradation trends can facilitate research of new materials and cell designs, rapid deployment of batteries in real-world applications, timely replacement of batteries in critical applications, and even the secondary use market. In this study, we design a convolutional neural network model to predict the entire battery capacity fade curve - a critical indicator of battery performance degradation - using first 100 cycles of data (~ three weeks of testing). We use the discharge voltage-capacity curves as input to the model and automate the feature extraction process through the convolutional layers of the network. Our approach can predict the per cycle capacity fade rate and rollover cycle (knee point) in the capacity fade curve, which indicate the onset of rapid capacity decay. On the publicly available graphite/LiFePO 4 battery dataset, optimized networks predict the capacity fade curves, rollover cycle, and end of life with 3.7% (worst-case), 19%, and 17% mean absolute percentage errors, respectively.

25 ENERGY STORAGE↗
DOE OSTI2022

Inferring protein expression changes from mRNA in Alzheimer’s dementia using deep neural networks

Identifying the molecular systems and proteins that modify the progression of Alzheimer’s disease and related dementias (ADRD) is central to drug target selection. However, discordance between mRNA and protein abundance, and the scarcity of proteomic data, has limited our ability to advance candidate targets that are mainly based on gene expression. Therefore, by using a deep neural network that predicts protein abundance from mRNA expression, here we attempt to track the early protein drivers of ADRD. Specifically, by applying the clei2block deep learning model to 1192 brain RNA-seq samples, we identify protein modules and disease-associated expression changes that were not directly observed at the mRNA level. Moreover, pseudo-temporal trajectory inference based on the predicted proteome became more closely correlated with cognitive decline and hippocampal atrophy compared to RNA-based trajectories. This suggests that the predicted changes in protein expression could provide a better molecular representation of ADRD progression. Furthermore, overlaying clinical traits on protein pseudotime trajectory identifies protein modules altered before cognitive impairment. These results demonstrate how our method can be used to identify potential early protein drivers and possible drug targets for treating and/or preventing ADRD.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Defect Recognition for Eddy Current Testing of Spent Nuclear Fuel Canister using Convolutional Neural Network

This paper proposes an accurate and robust defect detection solution for 304L and 306L stainless steel (SS) weld. In the proposed solution, Eddy current testing (ECT) is employed to generate 2-dimensional (2D) data for samples under test with defects. The 2D data can be treated as images for deep learning-based defect detection. Since convolutional neural networks (CNNs) are powerful in processing images, CNN is employed in this study for defect detection. Experiments are conducted on a submerged arc welding (SAW) 304L SS weld sample with an artificial crack generated by waterjet cutting. The ECT data on this seeded fault sample is utilized to verify the proposed solution. For this purpose, the ECT measurement are separated as from Fault area and Normal area, which are used for CNN training. After training, the testing data is used for verification. Experimental results demonstrate the feasibility and effectiveness of the proposed solution.

Niu, Guangxing↗
DOE OSTI2024

Statistical and Neural Network for Real Sensor-Data-Driven Anomaly Detection in Nuclear Applications

Anomaly detection (AD) in sensor data is critical to ensure uninterrupted functionality of nuclear power plants (NPPs). Consequently, AD model validation through real-world sensor data is important for applications in nuclear facilities. In this paper, we propose an Autoencoder (AE)—a multi-layered neural network, for AD in sensor data from an operational NPP testbed. Since the dataset lacks labels for irregularities, we introduce random noise and label them to effectively train our model. The proposed AE model assigns a higher reconstruction error to the abnormal samples that deviate from those encountered during the training phase and uses the reconstruction loss to detect anomalies in a representative imbalanced dataset. We also introduce an analytical solution—seasonal trend decomposition (STD)—as another AD scheme for identifying irregularities withinthe same time-series dataset. In contrast to the AE model which relies on reconstruction loss, the STD scheme decomposes the entire dataset into its trend, seasonality, and residual components to pinpoint irregularities. Our findings indicate that the proposed AE and STD models individually achieve recall scores of 97% and 92%, respectively. We validate the performance of the two models on both balanced and imbalanced data. We further solidify the results by picking the combined selected anomalies of the two solutions with an "AND" operator for more reliable predictions.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Formation and retrieval of cell assemblies in a biologically realistic spiking neural network model of area CA3 in the mouse hippocampus

The hippocampal formation is critical for episodic memory, with area Cornu Ammonis 3 (CA3) a necessary substrate for auto-associative pattern completion. Recent theoretical and experimental evidence suggests that the formation and retrieval of cell assemblies enable these functions. Yet, how cell assemblies are formed and retrieved in a full-scale spiking neural network (SNN) of CA3 that incorporates the observed diversity of neurons and connections within this circuit is not well understood. Here, we demonstrate that a data-driven SNN model quantitatively reflecting the neuron type-specific population sizes, intrinsic electrophysiology, connectivity statistics, synaptic signaling, and long-term plasticity of the mouse CA3 is capable of robust auto-association and pattern completion via cell assemblies. Our results show that a broad range of assembly sizes could successfully and systematically retrieve patterns from heavily incomplete or corrupted cues after a limited number of presentations. Furthermore, performance was robust with respect to partial overlap of assemblies through shared cells, substantially enhancing memory capacity. These novel findings provide computational evidence that the specific biological properties of the CA3 circuit produce an effective neural substrate for associative learning in the mammalian brain.

hetero-association↗
DOE OSTI2025

Design and simulation of a SiPM-on-tile ZDC for the future EIC, and its performance with graph neural networks

We present a design for a high-granularity zero-degree calorimeter (ZDC) for the upcoming Electron-Ion Collider (EIC). The design uses SiPM-on-tile technology and features a novel staggered-layer arrangement that improves spatial resolution. To fully leverage the design’s high granularity and non-trivial geometry, we employ graph neural networks (GNNs) for energy and angle regression as well as signal classification. The GNN-boosted performance metrics meet, and in some cases, significantly surpass the requirements set in the report on science requirements and detector requirements for the EIC (Yellow Report), laying the groundwork for enhanced measurements that will facilitate a wide physics program. Our studies show that GNNs can significantly enhance the performance of high-granularity CALICE-style calorimeters by automating and optimizing the software compensation algorithms required for these systems. This improvement holds true even in the case of complicated geometries that pose challenges for image-based AI/ML methods.

Calorimeter↗
DOE OSTI2023

Regional Heatwave Prediction Using Graph Neural Network and Weather Station Data

Abstract Heatwaves lead to catastrophic consequences on public health and the economy. Accurate and timely predictions of regional heatwaves can improve climate preparedness and foster decision‐making to alleviate the burdens due to climate change. In this paper, we propose a heatwave prediction algorithm based on a novel deep learning model, that is, Graph Neural Network (GNN). This new GNN framework can provide real time warnings of the sudden occurrence of regional heatwaves with high accuracy at lower costs of computation and data collection. In addition, its interpretable structure unravels the spatiotemporal patterns of regional heatwaves and helps to enrich our understanding of the general climate dynamics and the causal influences between locations. The proposed GNN framework can be applied for the detection and prediction of other extreme or compound climate events, which calls for future studies.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Exploring far-from-equilibrium ultrafast polarization control in ferroelectric oxides with excited-state neural network quantum molecular dynamics

Ferroelectric materials exhibit a rich range of complex polar topologies, but their study under far-from-equilibrium optical excitation has been largely unexplored because of the difficulty in modeling the multiple spatiotemporal scales involved quantum-mechanically. To study optical excitation at spatiotemporal scales where these topologies emerge, we have performed multiscale excited-state neural network quantum molecular dynamics simulations that integrate quantum-mechanical description of electronic excitation and billion-atom machine learning molecular dynamics to describe ultrafast polarization control in an archetypal ferroelectric oxide, lead titanate. Far-from-equilibrium quantum simulations reveal a marked photo-induced change in the electronic energy landscape and resulting cross-over from ferroelectric to octahedral tilting topological dynamics within picoseconds. The coupling and frustration of these dynamics, in turn, create topological defects in the form of polar strings. The demonstrated nexus of multiscale quantum simulation and machine learning will boost not only the emerging field of ferroelectric topotronics but also broader optoelectronic applications.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Statistical and Neural Network for Real Sensor-Data-Driven Anomaly Detection in Nuclear Applications

Anomaly detection (AD) in sensor data is critical to ensure uninterrupted functionality of nuclear power plants (NPPs). Consequently, validation of AD models through real-world sensor data is important for their application in nuclear facilities. In this paper, we propose an Autoencoder (AE)— a multi-layered neural network, for AD in sensor data from an operational NPP testbed. Since the dataset lacks labels for irregularities, we introduce random noise and label them to effectively train our model. The proposed AE model assigns a higher reconstruction error to the abnormal samples that deviate from those encountered during the training phase and uses the reconstruction loss to detect anomalies in a representative imbalanced dataset. We also introduce an analytical solution—seasonal trend decomposition (STD) — as another AD scheme for identifying irregularities within the same time-series dataset. In contrast to the AE model which relies on reconstruction loss, the STD scheme decomposes the entire dataset into its trend, seasonality, and residual components to pinpoint irregularities. Our findings indicate that the proposed AE and STD models individually achieve recall scores of 97% and 92%, respectively. We also validate the performance of the two models on both balanced and imbalanced data. We further solidify the results by picking the combined selected anomalies of the two solutions with an "AND" operator for more reliable predictions.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗