Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “backpropagation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Approximate Bayesian Model Inversion for PDEs with Heterogeneous and State-Dependent Coefficients

We present two approximate Bayesian inference methods for parameter estimation in partial differential equation (PDE) models with space-dependent and state-dependent parameters. We demonstrate that these methods provide accurate and cost-effective alternatives to Markov Chain Monte Carlo simulation. We assume a parameterized Gaussian prior on the unknown functions, and approximate the posterior density by a parameterized multivariate Gaussian density. The parameters of the prior and posterior are estimated from sparse observations of the PDE model's states and the unknown functions themselves by maximizing the evidence lower bound (ELBO), a lower bound on the log marginal likelihood of the observations. The first method, Laplace-EM, employs the expectation maximization algorithm to maximize the ELBO, with a Laplace approximation of the posterior on the E-step, and minimization of a Kullback-Leibler divergence on the M-step. The second method, DSVI-EB, employs the doubly stochastic variational inference (DSVI) algorithm, in which the ELBO is maximized via gradient-based stochastic optimization, with nosiy gradients computed via simple Monte Carlo sampling and Gaussian backpropagation. We apply these methods to identifying diffusion coefficients in linear and nonlinear diffusion equations, and we find that both methods provide accurate estimates of posterior densities and the hyperparameters of Gaussian priors. While the Laplace-EM method is more accurate, it requires computing Hessians of the physics model. The DSVI-EB method is found to be less accurate but only requires gradients of the physics model.

approximate Bayesian inference, model inversion, v↗

On the importance of FIB-SEM specific segmentation algorithms for porous media

A new algorithmic approach to segmentation of highly porous three dimensional image data gained by focused ion beam tomography is described which extends the key-principle of local threshold backpropagation described in Salzer et al. (2012). The technique of focused ion beam tomography has shown to be capable of imaging the microstructure of functional materials. In order to perform a quantitative analysis on the corresponding microstructure a segmentation task needs to be performed. However, algorithmic segmentation of images obtained with focused ion beam tomography is a challenging problem for highly porous materials if filling the pore phase, e.g. with epoxy resin, is difficult. The gray intensities of individual voxels are not sufficient to determine the phase represented by them and usual thresholding methods are not applicable. We thus propose a new approach to segmentation that pays respect to the specifics of the imaging process of focused ion beam tomography. As an application of our approach, the segmentation of three dimensional images for a cathode material used in polymer electrolyte membrane fuel cells is discussed. We show that our approach preserves significantly more of the original nanostructure than a thresholding approach. - Highlights: • We describe a new approach to the segmentation of FIB-SEM images of porous media. • The first and last occurrences of structures are detected by analysing the z-profiles. • The algorithm is validated by comparing it to a manual segmentation. • The new approach shows significantly less artifacts than a thresholding approach. • A structural analysis also shows improved results for the obtained microstructure.

36 MATERIALS SCIENCE↗

A differentiable approach to the maximum independent set problem using dataless neural networks

The success of machine learning solutions for reasoning about discrete structures has brought attention to its adoption within combinatorial optimization algorithms. Such approaches generally rely on supervised learning by leveraging datasets of the combinatorial structures of interest drawn from some distribution of problem instances. Reinforcement learning has also been employed to find such structures. Here, in this paper, we propose a different approach in that no data is required for training the neural networks that produce the solution. In this sense, what we present is not a machine learning solution, but rather one that is dependent on neural networks and where backpropagation is applied to a loss function defined by the structure of the neural network architecture as opposed to a training dataset. In particular, we reduce the popular combinatorial optimization problem of finding a maximum independent set to a neural network and employ a dataless training scheme to refine the parameters of the network such that those parameters yield the structure of interest. Additionally, we propose a universal graph reduction procedure to handle large-scale graphs. The reduction exploits community detection for graph partitioning and is applicable to any graph type and/or density. Experimental results on both real and synthetic graphs demonstrate that our proposed method performs on par or outperforms state-of-the-art learning-based methods in terms of the size of the found set without requiring any training data.

97 MATHEMATICS AND COMPUTING↗

Regularization via f -Divergence: An Application to Multi-Oxide Spectroscopic Analysis

In this paper, we explore the application of convolutional neural networks (CNNs) for predicting the chemical composition of complex geologic samples in a simulated Martian atmospheric environment. Specifically, we aim to characterize oxide weight percentages (wt.%) of rock samples analyzed by remote Laser-Induced Breakdown Spectroscopy (LIBS), framing the problem as a multi-target regression task . Neural networks trained on LIBS spectra are prone to overfitting due to high spectral complexity, limited labeled data, and measurement noise. While regularization is critical for improving generalization, common methods (e.g., ℓ 2 regularization) impose constraints not directly tied to data distribution properties. We propose a novel regularization method based on a specific ƒ-divergence induced by a graph-based estimator, designed to constrain the distributional discrepancy between predictions and targets. This regularizer serves a dual purpose: (a) mitigating overfitting by enforcing a constraint on the distributional difference between predictions and noisy targets, and (b) acting as an auxiliary loss that penalizes large divergences. To enable backpropagation, we develop a differentiable approximation of this particular ƒ-divergence, making the method feasible for neural networks. Experiments on ChemCam and SuperCam LIBS calibration spectra show that mathematical equation-divergence regularization outperforms or matches standard regularization methods (ℓ 1 , ℓ 2 , dropout) and the classical baseline, partial least squares (PLS). Combining ƒ-divergence regularization with standard regularization yields further performance gains, indicating that distributional regularization is useful in this context giving a promising direction for robust model training in planetary science applications. Source code is publicly available at Klein and Li (2025), https://doi.org/10.11578/dc.20250530.7.

58 GEOSCIENCES↗

Dynamics of episodic supershear in the 2023 M7.8 Kahramanmaraş/Pazarcik earthquake, revealed by near-field records and computational modeling

Abstract The 2023 M7.8 Kahramanmaraş/Pazarcik earthquake was larger and more destructive than what had been expected. Here we analyzed nearfield seismic records and developed a dynamic rupture model that reconciles different currently conflicting inversion results and reveals spatially non-uniform propagation speeds in this earthquake, with predominantly supershear speeds observed along the Narli fault and at the southwest (SW) end of the East Anatolian Fault (EAF). The model highlights the critical role of geometric complexity and heterogeneous frictional conditions in facilitating continued propagation and influencing rupture speed. We also constrained the conditions that allowed for the rupture to jump from the Narli fault to EAF and to generate the delayed backpropagating rupture towards the SW. Our findings have important implications for understanding earthquake hazards and guiding future response efforts and demonstrate the value of physics based dynamic modeling fused with near-field data in enhancing our understanding of earthquake mechanisms and improving risk assessment.

Environmental Sciences & Ecology↗

AUTOMATED DEFECT CLASSIFICATION USING AN ARTIFICIAL NEURAL NETWORK

The automated defect classification algorithm based on artificial neural network with multilayer backpropagation structure was utilized. The selected features of flaws were used as input data. In order to train the neural network it is necessary to prepare learning data which is representative database of defects. Database preparation requires the following steps: image acquisition and pre-processing, image enhancement, defect detection and feature extraction. The real digital radiographs of welded parts of a ship were used for this purpose.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Neural Network Modeling of Degradation of Solar Cells

Neural network modeling has been used to predict the degradation in conversion efficiency of solar cells in this work. The model takes intensity of light, temperature and exposure time as inputs and predicts the conversion efficiency of the solar cell. Backpropagation algorithm has been used to train the network. It is found that the neural network model satisfactorily predicts the degradation in efficiency of the solar cell with exposure time. The error in the computed results, after comparison with experimental results, lies in the range of 0.005-0.01, which is quite low.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Prediction of U-Mo dispersion nuclear fuels with Al-Si alloy using artificial neural network

Dispersion nuclear fuels, consisting of U-Mo particles dispersed in an Al-Si matrix, are being developed as fuel for research reactors. The equilibrium relationship for a mixture component can be expressed in the phase diagram. It is important to analyze whether a mixture component is in equilibrium phase or another phase. The purpose of this research it is needed to built the model of the phase diagram, so the mixture component is in the stable or melting condition. Artificial neural network (ANN) is a modeling tool for processes involving multivariable non-linear relationships. The objective of the present work is to develop code based on artificial neural network models of system equilibrium relationship of U-Mo in Al-Si matrix. This model can be used for prediction of type of resulting mixture, and whether the point is on the equilibrium phase or in another phase region. The equilibrium model data for prediction and modeling generated from experimentally data. The artificial neural network with resilient backpropagation method was chosen to predict the dispersion of nuclear fuels U-Mo in Al-Si matrix. This developed code was built with some function in MATLAB. For simulations using ANN, the Levenberg-Marquardt method was also used for optimization. The artificial neural network is able to predict the equilibrium phase or in the phase region. The develop code based on artificial neural network models was built, for analyze equilibrium relationship of U-Mo in Al-Si matrix.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Helical path separation for guided wave tomography

The pipe wall loss caused by corrosion can be quantified across an area by transmitting guided Lamb waves through the region and measuring the resulting signals. Typically the dispersive relationship for these waves, resulting in the wave velocity being a function of thickness, is exploited which enables the wall thickness to be determined from a velocity reconstruction. The accuracy and quality of this reconstruction is commonly limited by the angle of view available from the transducer arrays. These arrays are often attached as a pair of ring arrays either side of the inspected region, and due to the cyclic nature of the pipe, waves are able to travel in an inifinite number of helical paths between any two transducers. The first arrivals can be separated relatively easily by time gating, but by using just these components the angle of view is strongly restricted. To improve the viewing angle, it is necessary to separate the wavepackets. This paper provides an outline of a separation approach: initially the waves are backpropagated to their source to align the different signals, then a filtering technique is applied to select the desired components. The technique is applied to experimental data and demonstrated to robustly separate the signals.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A domain wall-magnetic tunnel junction artificial synapse with notched geometry for accurate and efficient training of deep neural networks

Inspired by the parallelism and efficiency of the brain, several candidates for artificial synapse devices have been developed for neuromorphic computing, yet a nonlinear and asymmetric synaptic response curve precludes their use for backpropagation, the foundation of modern supervised learning. Spintronic devices—which benefit from high endurance, low power consumption, low latency, and CMOS compatibility—are a promising technology for memory, and domain-wall magnetic tunnel junction (DW-MTJ) devices have been shown to implement synaptic functions such as long-term potentiation and spike-timing dependent plasticity. In this work, we propose a notched DW-MTJ synapse as a candidate for supervised learning. Using micromagnetic simulations at room temperature, we show that notched synapses ensure the non-volatility of the synaptic weight and allow for highly linear, symmetric, and reproducible weight updates using either spin transfer torque (STT) or spin–orbit torque (SOT) mechanisms of DW propagation. We use lookup tables constructed from micromagnetics simulations to model the training of neural networks built with DW-MTJ synapses on both the MNIST and Fashion-MNIST image classification tasks. Accounting for thermal noise and realistic process variations, the DW-MTJ devices achieve classification accuracy close to ideal floating-point updates using both STT and SOT devices at room temperature and at 400 K. Our work establishes the basis for a magnetic artificial synapse that can eventually lead to hardware neural networks with fully spintronic matrix operations implementing machine learning.

42 ENGINEERING↗

Coded-mask-based wavefront sensing technique for APS nanofocusing beamline diagnostics

Here, we extend our recently developed coded-mask wavefront sensing technique to enable single-shot measurements of nanofocused x-ray beams. This method accurately reconstructs the focal beam profile by backpropagating the wavefront measured downstream of the beam focus. To validate its performance, we benchmarked it against the conventional fluorescence wire scan method, successfully measuring ∼120 nm focal spots at the 28-ID-B beamline of the Advanced Photon Source using a polymeric compound refractive lens. The results highlight the effectiveness of coded-mask wavefront sensing for high-precision beam profiling and its application as a real-time wavefront monitoring tool.

Shi, Xianbo [Argonne National Laboratory (ANL), Ar↗

Optimized pulsed write schemes improve linearity and write speed for low-power organic neuromorphic devices

Neuromorphic devices are becoming increasingly appealing as efficient emulators of neural networks used to model real world problems. However, no hardware to date has demonstrated the necessary high accuracy and energy efficiency gain over CMOS in both training via backpropagation and in read via vector matrix multiplication. Such shortcomings are due to device non-idealities, particularly asymmetric conductance tuning in response to uniform voltage pulse inputs. Here, by formulating a general circuit model for capacitive ion-exchange neuromorphic devices, we show that asymmetric nonlinearity in organic electrochemical neuromorphic devices (ENODes) can be suppressed by an appropriately chosen write scheme. Simulations based upon our model suggest that a nonlinear write-selector could reduce the switching voltage and energy, enabling analog tuning via a continuous set of resistance states (100 states) with extremely low switching energy (~170 fJ • µm –2 ). Furthermore, this work clarifies the pathway to neural algorithm accelerators capable of parallelism during both read and write operations.

97 MATHEMATICS AND COMPUTING↗

Identification of tau leptons using a convolutional neural network with domain adaptation

A tau lepton identification algorithm,DeepTau, based on convolutional neural network techniques, has been developed in the CMS experiment to discriminate reconstructed hadronic decays of tau leptons (τ h ) from quark or gluon jets and electrons and muons that are misreconstructed as τ h candidates. The latest version of this algorithm, v2.5, includes domain adaptation by backpropagation, a technique that reduces discrepancies between collision data and simulation in the region with the highest purity of genuine τh candidates. Additionally, a refined training workflow improves classification performance with respect to the previous version of the algorithm, with a reduction of 30–50% in the probability for quark and gluon jets to be misidentified as τ h candidates for given reconstruction and identification efficiencies. This paper presents the novel improvements introduced in theDeepTau algorithm and evaluates its performance in LHC proton-proton collision data at √(s) = 13 and 13.6 TeV collected in 2018 and 2022 with integrated luminosities of 60 and 35 fb -1 , respectively. Techniques to calibrate the performance of the τ h identification algorithm in simulation with respect to its measured performance in real data are presented, together with a subset of results among those measured for use in CMS physics analyses.

Large detector-systems performance↗

Differentiable programming for online training of a neural artificial viscosity function within a staggered grid Lagrangian hydrodynamics scheme

Lagrangian methods to solve the inviscid Euler equations produce numerical oscillations near shock waves. A common approach to reducing these oscillations is to add artificial viscosity (AV) to the discrete equations. The AV term acts as a dissipative mechanism that attenuates oscillations by smearing the shock across a finite number of computational cells. However, AV introduces several control parameters that are not determined by the underlying physical model, and hence, in practice are tuned to the characteristics of a given problem. We seek to improve the standard quadratic-linear AV form by replacing it with a learned neural function that reduces oscillations relative to exact solutions of the Euler equations, resulting in a hybrid numerical-neural hydrodynamic solver. Because AV is an artificial construct that exists solely to improve the numerical properties of a hydrodynamic code, there is no offline ‘viscosity data’ against which a neural network can be trained before inserting into a numerical simulation, thus requiring online training. We achieve this via differentiable programming, i.e. end-to-end backpropagation or adjoint solution through both the neural and differential equation code, using automatic differentiation of the hybrid code in the Julia programming language to calculate the necessary loss function gradients. A novel offline pre-training step accelerates training by initializing the neural network to the default numerical AV scheme, which can be learned rapidly by space-filling sampling over the AV input space. We find that online training over early time steps of simulation is sufficient to learn a neural AV function that reduces numerical oscillations in long-term hydrodynamic shock simulations. These results offer an early proof-of-principle that online differentiable training of hybrid numerical schemes with novel neural network components can improve certain performance aspects existing in purely numerical schemes.

97 MATHEMATICS AND COMPUTING↗

High-resolution source imaging and moment tensor estimation of acoustic emissions during brittle creep of basalt undergoing carbonation

SUMMARY As the high-frequency analogue to field-scale earthquakes, acoustic emissions (AEs) provide a valuable complement to study rock deformation mechanisms. During the load-stepping creep experiments with CO2-saturated water injection into a basaltic sample from Carbfix site in Iceland, 8791 AE events are detected by at least one of the seven piezoelectric sensors. Here, we apply a cross-correlation-based source imaging method, called geometric-mean reverse-time migration (GmRTM) to locate those AE events. Besides the attractive picking-free feature shared with other waveform-based methods (e.g. time-reversal imaging), GmRTM is advantageous in generating high-resolution source images with reduced imaging artefacts, especially for experiments with relatively sparse receivers. In general, the imaged AE locations are found to be scattered across the sample, suggesting a complicated fracture network rather than a well-defined major shear fracture plane, in agreement with X-ray computed tomography imaging results after retrieval of samples from the deformation apparatus. Clustering the events in space and time using the nearest-neighbour approach revealed a group of ‘repeaters’, which are spatially co-located over an elongated period of time and likely indicate crack, or shear band growth. Furthermore, we select 2196 AE events with high signal-to-noise-ratio (SNR) and conduct moment tensor estimation using the adjoint (backpropagated) strain tensor fields at the locations of AE sources. The resulting AE locations and focal mechanisms support our previously assertion that creep of basalt at the experimental conditions is accommodated dominantly by distributed microcracking.

58 GEOSCIENCES↗

Inverse Design of Plasma Metamaterial Devices for Optical Computing

We apply inverse-design methods to produce two-dimensional plasma metamaterial (PMM) devices. Backpropagated finite-difference frequency-domain (FDFD) simulations are used to design waveguides and demultiplexers operating under both TE and TM modes. Demultiplexing and waveguiding are demonstrated for devices composed of plasma elements with reasonable plasma densities of approximately 7 GHz, allowing for future in situ training and experimental realization of these designs. We also explore the possible applicability of PMMs to nonlinear boolean operations for use in optical computing. Functionally complete logical connectives (or and and) are achieved in the TM mode.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Impact of Linearity and Write Noise of Analog Resistive Memory Devices in a Neural Algorithm Accelerator

Resistive memory (ReRAM) shows promise for use as an analog synapse element in energy-efficient neural network algorithm accelerators. A particularly important application is the training of neural networks, as this is the most computationally-intensive procedure in using a neural algorithm. However, training a network with analog ReRAM synapses can significantly reduce the accuracy at the algorithm level. In order to assess this degradation, analog properties of ReRAM devices were measured and hand-written digit recognition accuracy was modeled for the training using backpropagation. Bipolar filamentary devices utilizing three material systems were measured and compared: one oxygen vacancy system, Ta-TaO x , and two conducting metallization systems, Cu-SiO 2 , and Ag/chalcogenide. Analog properties and conductance ranges of the devices are optimized by measuring the response to varying voltage pulse characteristics. Key analog device properties which degrade the accuracy are update linearity and write noise. Write noise may improve as a function of device manufacturing maturity, but write nonlinearity appears relatively consistent among the different device material systems and is found to be the most significant factor affecting accuracy. As a result, this suggests that new materials and/or fundamentally different resistive switching mechanisms may be required to improve device linearity and achieve higher algorithm training accuracy.

97 MATHEMATICS AND COMPUTING↗

Task-Based Neuromodulation Architecture for Lifelong Learning

The conventional supervised deep learning frameworks for image classification focus on finding the minimizer of an objective function. However, for a lifelong learning system, a network has to be able to dynamically learn new tasks from very few examples. In biological neural networks, it is observed that neuromodulation acts as a key factor in continual learning. In this work, we propose a dynamic learning system, ModNet, wherein a modulatory unit regulates the learning depending on the context and the internal state of the system. The modulatory unit, which consists of a group of neurons, exercises the use of neuromodulation and Hebbian plasticity mechanisms over a population of neurons instead of relying on backpropagation of error gradients. An efficient digital architecture with on-chip training and resource sharing is proposed to facilitate the efficacy of task-based processing.

brain-inspired↗