Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural network architecture”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Verification of Triso Fuel Burnup Using Machine Learning Algorithms

Pebble Bed Reactors are fueled with fuel pebbles that are circulated multiple times through the reactor vessel before discharge. During the normal operation of a PBR, ejected pebbles are returned to the reactor or discharged depending on the fuel burnup and physical condition of the pebbles. The burnup measurement is usually based on detected radiation signatures of fission products accumulated in the pebble fuel over burnup. Previous research has shown that height of photopeaks of fission products, such as 134 Cs, 137 Cs, 154 Eu, etc., can be used independently or in combination to infer or predict the level of burnup in the fuel. However, it remains challenging to measure such complex sources due to self-shielding effects, strong radiation background and intervening materials. Another operational challenge is the required high throughput of burnup measurement, which necessitates limited measurement time and thus impacts quality of measured gamma-ray spectra. Hence, advanced spectral analysis methods are needed to analyze the noisy gamma spectra and predict the burnup values. We propose to use machine learning (ML) method to interpret gamma-ray spectra and predict the burnup values of the pebbles. ML has achieved widespread success and adoption across a few domains that require pattern recognition and analysis in varied data types. In this work, we apply three proven ML approaches - multilayer perceptrons, convolutional neural networks, and transformers - to the task of predicting fuel burnup from measured gamma spectra, and compile a dataset of simulated spectra for training and validation of the ML models. In this paper, we will discuss the network architecture of these three ML approaches and compare the performance of the simplest of these (MLP) to a standard linear regression.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2024

Z-Target Radiography Postprocessing With A Deep Convolution Neural Network

Analyzing X-ray radiographs is crucial for understanding target behavior in Inertial Confinement Fusion (ICF) and High Energy Density (HED) platforms. However, the density of Magneto Raleigh Taylor (MRT) bands and limitations of target materials often obscure relevant spike growth and density information. To address this issue, machine learning postprocessing techniques can be applied to remove darkened regions in radiography images. In this study, a novel method is presented for removing MRT darkened regions from z-target radiographs using a convolutional neural network (CNN). The CNN, consisting of six layers, treats the darkened regions as noise and employs a mixed loss function and end-to-end frameworks to suppress them while preserving sharpness. The six-layer architecture is designed to effectively learn features when provided with a larger volume of learning space. Each layer is optimized using a mixed loss function that combines a standard loss pixel approach with a multi-scaled structural similarity index loss, which considers luminance, contrast, and structure in local neighborhoods. This approach is particularly beneficial for capturing the stochastic structure of MRT limbs. Due to the limited availability of experimental data, training is conducted using synthetic target radiography from 3D Alegra simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

VERIFICATION OF TRISO FUEL BURNUP USING MACHINE LEARNING ALGORITHMS

Pebble Bed Reactors are fueled with fuel pebbles that are circulated multiple times through the reactor vessel before discharge. During the normal operation of a PBR, ejected pebbles are returned to the reactor or discharged depending on the fuel burnup and physical condition of the pebbles. The burnup measurement is usually based on detected radiation signatures of fission products accumulated in the pebble fuel over burnup. Previous research has shown that height of photopeaks of fission products, such as 134Cs, 137Cs, 154Eu, etc., can be used independently or in combination to infer or predict the level of burnup in the fuel. However, it remains challenging to measure such complex sources due to self-shielding effects, strong radiation background and intervening materials. Another operational challenge is the required high throughput of burnup measurement, which necessitates limited measurement time and thus impacts quality of measured gamma-ray spectra. Hence, advanced spectral analysis methods are needed to analyze the noisy gamma spectra and predict the burnup values. We propose to use machine learning (ML) method to interpret gamma-ray spectra and predict the burnup values of the pebbles. ML has achieved widespread success and adoption across a few domains that require pattern recognition and analysis in varied data types. In this work, we apply three proven ML approaches - multilayer perceptrons, convolutional neural networks, and transformers - to the task of predicting fuel burnup from measured gamma spectra, and compile a dataset of simulated spectra for training and validation of the ML models. In this paper, we will discuss the network architecture of these three ML approaches and compare the performance of the simplest of these (MLP) to a standard linear regression.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2025

Improving the Transportability of a Deep Learning Denoising Model Using Transfer Learning Techniques

The adoption of machine learning techniques in the seismology community has led to great performance improvements in several areas, including signal processing. Specifically, the development of deep learning–based seismic waveform denoising models has the potential to yield improvements in signal detection capabilities for networks operating in particularly noisy environments. Recent advancements in the design of these deep learning denoising models have included the incorporation of continuous and discrete wavelet transform functions into the network architecture to improve the learning capabilities and efficiency of said models. These wavelet transform–based seismic denoising models have shown improved denoising capabilities in regions where there is good agreement between the data features present in the training and evaluation datasets. However, questions remain about the overall transportability of these models to other monitoring regions. Here, in this study, we will determine the baseline transportability of a newly developed multilevel wavelet‐transform convolutional neural network (MWCNN) seismic denoising model. We accomplish this by taking a version of the MWCNN denoising model trained on data collected from the Utah region and evaluating its denoising performance on datasets collected from the neighboring Nevada region, which differ with regard to monitoring sensor types and event histories. We find that there is a notable variability in denoising performance related to the degree of similarity between the initial and new target datasets. The most notable difference in denoising performance is the ability of the denoising model to preserve accurate amplitude information associated with the signal energy present in the waveform data. Finally, we evaluate the ability of transfer learning techniques to improve the transportability of the MWCNN denoising model. We find that although there is still a performance gap present in the denoising results of the MWCNN model, transfer learning did yield improved results.

Quinones, Louis [Sandia National Laboratories (SNL↗
DOE OSTI2022

Inferring safety critical events from vehicle kinematics in naturalistic driving environment: Application of deep learning Algorithms

Advances in sensing technology has enabled the collection of countless terabytes of second-by-second kinematics data. Such data provides opportunities for real-time monitoring of driving behavior and identification of safety critical events (SCEs) including crashes and near crashes. The concept of volatility is relevant in this context, which identifies instability and erratic variations in driving behavior prior to involvement in SCEs. This study utilized vehicle kinematics from a large-scale naturalistic driving data to develop a deep learning approach based on 1D convolutional neural networks (CNN) for inferring SCEs. The data are unique in the sense that such accurate pre-crash data at high fidelity are not available in traditional crash repositories. This study contributes to the literature by providing a first attempt at predicting responses to SCEs by developing deep learning-based CNN architectures using novel driving volatility based kinematic thresholds for a sample of 9553 events. The key contribution lies in developing a volatility-based CNN input layout that is acceptable to CNN schemes and represents the motion kinematics such as speed, acceleration and volatility measures. Several 1D-CNN architectures were developed using layers, numbers of convolutions, layer patterns, and kernels. Shallow and deep architectures were tested, revealing higher accuracy of shallow architectures in detecting SCEs. The optimal number of epochs were identified using an early stopping method while the CNN performance was improved by increasing the number of epochs. The ensemble CNN had the highest predictive accuracy of 95.6% for detection of crashes and near crashes, which was 2.5% higher than the optimal CNN using 20% hold out test data. The ensemble CNN also outperformed classical machine learning models and model performance reported in past studies on detection of SCEs. Finally, these results have implications for identification of safety hotspots and providing real-time alerts and warnings in connected and highly automated vehicle environment including society of automotive engineers levels 3–5.

42 ENGINEERING↗
DOE OSTI2022

Identifying Transient Candidates in the Dark Energy Survey Using Convolutional Neural Networks

The ability to discover new transient candidates via image differencing without direct human intervention is an important task in observational astronomy. For these kind of image classification problems, machine learning techniques such as Convolutional Neural Networks (CNNs) have shown remarkable success. In this work, we present the results of an automated transient candidate identification on images with CNNs for an extant data set from the Dark Energy Survey Supernova program, whose main focus was on using Type Ia supernovae for cosmology. By performing an architecture search of CNNs, we identify networks that efficiently select non-artifacts (e.g., supernovae, variable stars, AGN, etc.) from artifacts (image defects, mis-subtractions, etc.), achieving the efficiency of previous work performed with random Forests, without the need to expend any effort in feature identification. The CNNs also help us identify a subset of mislabeled images. Performing a relabeling of the images in this subset, the resulting classification with CNNs is significantly better than previous results, lowering the false positive rate by 27% at a fixed missed detection rate of 0.05.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Probabilistic neural networks for fluid flow surrogate modeling and data recovery

We consider the use of probabilistic neural networks for fluid flow surrogate modeling and data recovery. This framework is constructed by assuming that the target variables are sampled from a Gaussian distribution conditioned on the inputs. Consequently, the overall formulation sets up a procedure to predict the hyperparameters of this distribution which are then used to compute an objective function given training data. We demonstrate that this framework has the ability to provide for prediction confidence intervals based on the assumption of a probabilistic posterior, given an appropriate model architecture and adequate training data. The applicability of the present framework to cases with noisy measurements and limited observations is also assessed. To demonstrate the capabilities of this framework, we consider canonical regression problems of fluid dynamics from the viewpoint of reduced-order modeling and spatial data recovery for four canonical data sets. The examples considered in this study arise from (i) the shallow-water equations, (ii) a two-dimensional cylinder flow, (iii) the wake of a NACA0012 airfoil with a Gurney flap, and (iv) the NOAA sea surface temperature data set. Furthermore, the present results indicate that the probabilistic neural network not only produces a machine-learning-based fluid flow surrogate model but also systematically quantifies the uncertainty therein to assist with model interpretability.

42 ENGINEERING↗
NASA NTRS1993

Application of fuzzy logic-neural network based reinforcement learning to proximity and docking operations: Special approach/docking testcase results

As part of the RICIS project, the reinforcement learning techniques developed at Ames Research Center are being applied to proximity and docking operations using the Shuttle and Solar Maximum Mission (SMM) satellite simulation. In utilizing these fuzzy learning techniques, we use the Approximate Reasoning based Intelligent Control (ARIC) architecture, and so we use these two terms interchangeably to imply the same. This activity is carried out in the Software Technology Laboratory utilizing the Orbital Operations Simulator (OOS) and programming/testing support from other contractor personnel. This report is the final deliverable D4 in our milestones and project activity. It provides the test results for the special testcase of approach/docking scenario for the shuttle and SMM satellite. Based on our experience and analysis with the attitude and translational controllers, we have modified the basic configuration of the reinforcement learning algorithm in ARIC. The shuttle translational controller and its implementation in ARIC is described in our deliverable D3. In order to simulate the final approach and docking operations, we have set-up this special testcase as described in section 2. The ARIC performance results for these operations are discussed in section 3 and conclusions are provided in section 4 along with the summary for the project.

Jani, Yashvant↗
DOE OSTI2023

Determining the twist angle of stacked MoS 2 layers using machine learning-assisted low-frequency interlayer Raman fingerprints

The investigation of twisted stacked few-layer MoS 2 has revealed novel electronic, optical, and vibrational properties over an extended period. For the successful integration of twisted stacked few-layer MoS 2 into a wide range of applications, it is crucial to employ a noninvasive, versatile technique for characterizing the layered architecture of these complex structures. Here, in this work, we introduce a machine learning-assisted low-frequency Raman spectroscopy method to characterize the twist angle of few-layer stacked MoS 2 samples. A feedforward neural network (FNN) is utilized to analyze the low-frequency breathing mode as a function of the twist angle. Moreover, using finite difference method (FDM) and density functional theory (DFT) calculations, we show that the low-frequency Raman spectra of MoS 2 are mainly influenced by the effect of the nearest and second nearest layers. A new improved linear chain model (TA-LCM) with taking the twist angle into the consideration is developed to understand the interlayer breathing modes of stacked few-layer MoS 2 . This approach can be extended to other 2D materials systems and provides an intelligent way to investigate naturally stacked and twisted interlayer interactions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

On the Relationship Between Shallow Cumulus Cloud Field Properties and Surface Solar Irradiance

Abstract Shallow cumulus clouds exhibit highly three‐dimensional (3‐D) spatial structure leading to complex variability in the surface solar irradiance (SSI) beneath. This variability is captured by the typically bimodal shape of the SSI probability density function (PDF). Using large eddy simulation to generate well‐resolved cloud fields and Monte Carlo 3‐D radiative transfer to reproduce realistic associated SSI PDFs, we seek direct relationships between the cloud field properties and the SSI PDF shape. Applying both random forest and artificial neural network algorithms, we find variations in the two modes of the SSI PDF are well predicted by just a handful of cloud field properties. The two algorithms utilize cloud properties similarly, with indistinguishable performance despite their different architectures. These results offer a marked improvement in realism relative to one‐dimensional radiative transfer while bypassing computationally expensive 3‐D radiative transfer, with immediate application to renewable energy assessments, and potential for several other geophysical applications.

58 GEOSCIENCES↗
NASA NTRS1995

Neural Network Burst Pressure Prediction in Graphite/Epoxy Pressure Vessels from Acoustic Emission Amplitude Data

Acoustic emission (AE) data were taken during hydroproof for three sets of ASTM standard 5.75 inch diameter filament wound graphite/epoxy bottles. All three sets of bottles had the same design and were wound from the same graphite fiber; the only difference was in the epoxies used. Two of the epoxies had similar mechanical properties, and because the acoustic properties of materials are a function of their stiffnesses, it was thought that the AE data from the two sets might also be similar; however, this was not the case. Therefore, the three resin types were categorized using dummy variables, which allowed the prediction of burst pressures all three sets of bottles using a single neural network. Three bottles from each set were used to train the network. The resin category, the AE amplitude distribution data taken up to 25 % of the expected burst pressure, and the actual burst pressures were used as inputs. Architecturally, the network consisted of a forty-three neuron input layer (a single categorical variable defining the resin type plus forty-two continuous variables for the AE amplitude frequencies), a fifteen neuron hidden layer for mapping, and a single output neuron for burst pressure prediction. The network trained on all three bottle sets was able to predict burst pressures in the remaining bottles with a worst case error of + 6.59%, slightly greater than the desired goal of + 5%. This larger than desired error was due to poor resolution in the amplitude data for the third bottle set. When the third set of bottles was eliminated from consideration, only four hidden layer neurons were necessary to generate a worst case prediction error of - 3.43%, well within the desired goal.

Hill, Eric v. K.↗
NASA NTRS2026

PERSIANN-Unet: A Global Deep Learning Framework for Near-Real-Time Precipitation Estimation Using Infrared Data

Access to high-quality, high-resolution, near-real-time precipitation data is essential for hydrological and meteorological research and disaster mitigation. Traditional tools such as rain gauges and radar networks, though effective, have limitations, including sparse coverage in remote areas and high operational costs. Satellite data, with its global coverage and high spatial and temporal resolutions, mitigates limitations in coverage. Satellite precipitation products like Hydro Estimator (HE), Integrated Multi-satellitE Retrievals for Global Precipitation Measurement (IMERG), and Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) utilize both geosynchronous thermal infrared (IR) and passive microwave (PMW) data in their operation. PMW sensors offer detailed atmospheric profiles but suffer from higher latency, whereas IR sensors provide lower latency but only capture cloud-top information. Despite this constraint, IR data remains attractive for low-latency precipitation estimation. Recent advances in deep learning, particularly convolutional neural networks (CNNs), have further improved satellite precipitation retrievals. This study introduces PERSIANN-Unet (PUnet or PERSIANN V3), a quasi-global algorithm covering 60°N–60°S that combines IR data, monthly climatology, and the UNet architecture to produce half-hourly precipitation estimates at 0.04° resolution. The product is evaluated against HE, IMERG, and PDIR-Now for 2022–2023. Results show that PUnet closely matches its training target, IMERG V07 Final, at the global scale, and performance is further evaluated against Stage IV as a reference over CONUS. Training PUnet on IMERG (2016–2021) leverages a high-quality, integrated PMW IR-gauge precipitation product while developing an IR-based framework not reliant on PMW availability. By operating on a single global image, PUnet avoids tile partitioning and blending steps, reducing edge discontinuities, and produces more spatially consistent precipitation fields across hemispheres.

Phu Nguyen↗
DOE OSTI2024

Augmenting machine learning of Grad–Shafranov equilibrium reconstruction with Green's functions

This work presents a method for predicting plasma equilibria in tokamak fusion experiments and reactors. The approach involves representing the plasma current as a linear combination of basis functions using principal component analysis of plasma toroidal current densities (J t ) from the EFIT-AI equilibrium database. Then utilizing EFIT's Green's function tables, basis functions are created for the poloidal flux (ψ) and diagnostics generated from the toroidal current (J t ). Similar to the idea of a physics-informed neural network (NN), this physically enforces consistency between ψ, J t , and the synthetic diagnostics. First, the predictive capability of a least squares technique to minimize the error on the synthetic diagnostics is employed. The results show that the method achieves high accuracy in predicting ψ and moderate accuracy in predicting J t with median R 2 = 0.9993 and R 2 = 0.978, respectively. A comprehensive NN using a network architecture search is also employed to predict the coefficients of the basis functions. The NN demonstrates significantly better performance compared to the least squares method with median R 2 = 0.9997 and 0.9916 for J t and ψ, respectively. The robustness of the method is evaluated by handling missing or incorrect data through the least squares filling of missing data, which shows that the NN prediction remains strong even with a reduced number of diagnostics. Additionally, the method is tested on plasmas outside of the training range showing reasonable results.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS1994

Three-D Artificial Neural Network (3DANN) technology. Blueprint for the future

Irvine Sensors Corporation (ISC), working closely with JPL under BMDO/ONR sponsorship, is developing a radically new neural computing technology. Primarily aimed at discrimination and target recognition for BMDO missile interceptor applications, it appears to have near term commercial applicability to such problems as handwriting and face recognition, just to name two. In its earliest form it will be able to perform inner product computation using 262 thousand 64x64 templates (weighted synapse arrays) where the 64(exp 5) weights can all be changed every millisecond. Internal switching provides an inherent capability to zoom, translate, or rotate the templates. The 3D silicon architecture is manufactured on a commercial, high volume DRAM production line at very low cost, enabling its commercialization. Two technology thrusts are beginning: in the first, the 64 layer capability of 3DANN-I will be extended to 1024 layers and beyond. In the second layer size will be shrunk to 2-3 millimeters to reduce layer costs. Our workshop goal is to expose this technology to the neural network community as an emerging tool for their use and to obtain their desire for its future development.

Carson, John↗
DOE OSTI2022

Latent-Space Dynamics for Prediction and Fault Detection in Geothermal Power Plant Operations

This paper presents a latent-space dynamic neural network (LSDNN) model for the multi-step-ahead prediction and fault detection of a geothermal power plant’s operation. The model was trained to learn the dynamics of the power generation process from multivariate time-series data and the effects of exogenous variables, such as control adjustment and ambient temperature. In the LSDNN model, an encoder–decoder architecture was designed to capture cross-correlation among different measured variables. In addition, a latent space dynamic structure was proposed to propagate the dynamics in the latent space to enable prediction. The prediction power of the LSDNN was utilized for monitoring a geothermal power plant and detecting abnormal events. The model was integrated with principal component analysis (PCA)-based process monitoring techniques to develop a fault-detection procedure. The performance of the proposed LSDNN model and fault detection approach was demonstrated using field data collected from a geothermal power plant.

15 GEOTHERMAL ENERGY↗
DOE OSTI2024

Phase Identification in Synchrotron X-ray Diffraction Patterns of Ti–6Al–4V Using Computer Vision and Deep Learning

X-ray diffraction patterns contain information about the atomistic structure and microstructure (defect population) of materials, extracting detailed information from diffraction patterns is complex, demanding and relies on prior knowledge. Here, we hypothesize that deep-learning techniques can help to perform an effective and accurate analysis with high throughput rates. To demonstrate this concept, we applied a novel deep learning framework to determine the evolution of the β-phase volume fraction in a Ti–6Al–4V alloy during heat-treatment from video sequences of 2D diffraction patterns recorded in transmission and with highly monochromatic radiation in a synchrotron beamline. In particular, we studied the impact of network design on prediction reliability and computational performance. Networks of different architectures were trained using 3008 experimental 2D patterns. A well-tuned model was found to reproduce the phase fractions of another experimental data set, consisting of 1100 diffraction patterns, with a mean-square error as small as 2.6 x 10 -4 . The average prediction error of β-phase volume fraction was within 1.6 x 10 -2 (in each diffraction pattern) of the values obtained by conventional methods. Our work demonstrates that convolutional neural networks can evaluate high energy X-ray diffraction patterns with a remarkable level of reliability. Furthermore, it demonstrates the significance of network design on the reliability of predictions and computational performance. The most complex models do not necessarily result in highest accuracy and may even fail to learn from the data.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Freely scalable and reconfigurable optical hardware for deep learning

Abstract As deep neural network (DNN) models grow ever-larger, they can achieve higher accuracy and solve more complex problems. This trend has been enabled by an increase in available compute power; however, efforts to continue to scale electronic processors are impeded by the costs of communication, thermal management, power delivery and clocking. To improve scalability, we propose a digital optical neural network (DONN) with intralayer optical interconnects and reconfigurable input values. The path-length-independence of optical energy consumption enables information locality between a transmitter and a large number of arbitrarily arranged receivers, which allows greater flexibility in architecture design to circumvent scaling limitations. In a proof-of-concept experiment, we demonstrate optical multicast in the classification of 500 MNIST images with a 3-layer, fully-connected network. We also analyze the energy consumption of the DONN and find that digital optical data transfer is beneficial over electronics when the spacing of computational units is on the order of $$>10\,\upmu $$ > 10 μ m.

42 ENGINEERING↗
DOE OSTI2025

Exact block encoding of imaginary time evolution with universal quantum neural networks

We develop a constructive approach to generate quantum neural networks capable of representing the exact thermal states of all many-body qubit Hamiltonians. The Trotter expansion of the imaginary time propagator is implemented through an exact block encoding by means of a unitary, restricted Boltzmann machine architecture. Marginalization over the hidden-layer neurons (auxiliary qubits) creates the nonunitary action on the visible layer. Then, we introduce a unitary deep Boltzmann machine architecture in which the hidden-layer qubits are allowed to couple laterally to other hidden qubits. We prove that this wave-function is closed under the action of the imaginary time propagator and, more generally, can represent the action of a universal set of quantum gate operations. We provide analytic expressions for the coefficients for both architectures, thus enabling exact network representations of thermal states without stochastic optimization of the network parameters. In the limit of large imaginary time, the yields the ground state of the system. The number of qubits grows linearly with the number of interactions and total imaginary time for a fixed interaction order. Both networks can be readily implemented on quantum hardware via midcircuit measurements of auxiliary qubits. If only one auxiliary qubit is measured and reset, the circuit depth scales linearly with imaginary time and number of interactions, while the width is constant. Alternatively, one can employ a number of auxiliary qubits linearly proportional to the number of interactions, and circuit depth grows linearly with imaginary time only. Every midcircuit measurement has a postselection success probability, and the overall success probability is equal to the product of the probabilities of the midcircuit measurements.

97 MATHEMATICS AND COMPUTING↗