Automated detection of corrosion in used nuclear fuel dry storage canisters using residual neural networks
Not Available
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
SEARCH · Engineering Papers
Results for “residual network”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Deep learning of parameterized equations with applications to uncertainty quantification
We propose a learning algorithm for discovering unknown parameterized dynamical systems by using observational data of the state variables. Our method is built upon and extends the recent work of discovering unknown dynamical systems, in particular those using deep neural network (DNN). We propose a DNN structure, largely based upon the residual network (ResNet), to not only learn the unknown form of the governing equation but also take into account the random effect embedded in the system, which is generated by the random parameters. Once the DNN model is successfully constructed, it is able to produce system prediction over longer term and for arbitrary parameter values. For uncertainty quantification, it allows us to conduct uncertainty analysis by evaluating solution statistics over the parameter space.
97 MATHEMATICS AND COMPUTING↗
ESM data downscaling: a comparison of super-resolution deep learning models
Abstract Climate projections at fine spatial resolutions are required to conduct accurate risk assessment for critical infrastructure and design adaptation planning. Generating these projections using advanced Earth system models (ESM) requires significant computational resources. To address this issue, various statistical downscaling techniques have been introduced to generate fine-resolution data from coarse-resolution simulations. In this study, we evaluate and compare five deep learning-based downscaling techniques, namely, super-resolution convolutional neural networks, fast super-resolution convolutional neural network ESM, efficient sub-pixel convolutional neural network, enhanced deep residual network (EDRN), and super-resolution generative adversarial network (SRGAN). These techniques are applied to a dataset generated by the Energy Exascale Earth System Model (E3SM), focusing on key surface variables such as surface temperature, shortwave heat flux, and longwave heat flux. Models are trained and validated using paired fine-resolution (0.25 $$^{\circ }$$ ∘ ) and coarse-resolution (1 $$^{\circ }$$ ∘ ) monthly data obtained from a 9-year simulation. Next, blind testing is performed using monthly data obtained from two different years outside of the training and validation set. To evaluate the efficiency of each technique, different statistical metrics are used, including mean squared error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and learned perceptual image patch similarity (LPIPS). The results show that EDRN outperforms other algorithms in terms of PSNR, SSIM, and MSE, but struggles to capture fine-scale features in the data. In contrast, SRGAN, a generative model that uses perceptual loss, excels in capturing fine details at boundaries and internal structures, resulting in lower LPIPS than other methods.
Pawar, Nikhil M. (ORCID:0000000211613289)↗
A deep learning-accelerated data assimilation and forecasting workflow for commercial-scale geologic carbon storage
Fast assimilation of monitoring data to update forecasts of pressure buildup and carbon dioxide (CO 2 ) plume migration under geologic uncertainties is a challenging problem in geologic carbon storage. The high computational cost of data assimilation with a high-dimensional parameter space impedes fast decision-making for commercial-scale reservoir management. We propose to leverage physical understandings of porous medium flow behavior with deep learning techniques to develop a fast data assimilation-reservoir response forecasting workflow. Applying an Ensemble Smoother Multiple Data Assimilation (ES-MDA) framework, the workflow updates geologic properties and predicts reservoir performance with quantified uncertainty from pressure history and CO 2 plumes interpreted through seismic inversion. As the most computationally expensive component in such a workflow is reservoir simulation, we developed surrogate models to predict dynamic pressure and CO 2 plume extents under multi-well injection. The surrogate models employ deep convolutional neural networks, specifically, a wide residual network and a residual U-Net. The workflow is validated against a flat threedimensional reservoir model representative of a clastic shelf depositional environment. Intelligent treatments are applied to bridge between quantities in a true-3D reservoir model and those in a single-layer reservoir model. The workflow can complete history matching and reservoir forecasting with uncertainty quantification in less than one hour on a mainstream personal workstation.
25 ENERGY STORAGE↗
Training and Generalization of Residual Neural Networks as Discrete Analogues of Neural ODEs.
Abstract not provided.
Sargsyan, Khachik↗
Quantifying Uncertainties in Weight-Parameterized Residual Neural Networks
Explore the source record for details and available documents.
Sargsyan, Khachik↗
Monitoring leaf phenology in moist tropical forests by applying a superpixel-based deep learning method to time-series images of tree canopies
Tropical leaf phenology-particularly its variability at the tree-crown scale-dominates the seasonality of carbon and water fluxes. However, given enormous species diversity, accurate means of monitoring leaf phenology in tropical forests is still lacking. Time series of the Green Chromatic Coordinate (GCC) metric derived from tower-based red-green-blue (RGB) phenocams have been widely used to monitor leaf phenology in temperate forests, but its application in the tropics remains problematic. To improve monitoring of tropical phenology, we explored the use of a deep learning model (i.e. superpixel-based Residual Networks 50, SP-ResNet50) to automatically differentiate leaves from non-leaves in phenocam images and to derive leaf fraction at the tree-crown scale. To evaluate our model, we used a year of data from six phenocams in two contrasting forests in Panama. Here, we first built a comprehensive library of leaf and non-leaf pixels across various acquisition times, exposure conditions and specific phenocams. We then divided this library into training and testing components. We evaluated the model at three levels: 1) superpixel level with a testing set, 2) crown level by comparing the model-derived leaf fractions with those derived using image-specific supervised classification, and 3) temporally using all daily images to assess the diurnal stability of the model-derived leaf fraction. Finally, we compared the model-derived leaf fraction phenology with leaf phenology derived from GCC. Our results show that: 1) the SP-ResNet50 model accurately differentiates leaves from non-leaves (overall accuracy of 93%) and is robust across all three levels of evaluations; 2) the model accurately quantifies leaf fraction phenology across tree-crowns and forest ecosystems; and 3) the combined use of leaf fraction and GCC helps infer the timing of leaf emergence, maturation and senescence, critical information for modeling photosynthetic seasonality of tropical forests. Collectively, this study offers an improved means for automated tropical phenology monitoring using phenocams.
54 ENVIRONMENTAL SCIENCES↗
Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics
Traditional data-driven deep learning models often struggle with high training costs, error accumulation, and poor generalizability in complex physical processes. Physics-informed deep learning (PiDL) addresses these challenges by incorporating physical principles into the model. Most PiDL approaches regularize training by embedding governing equations into the loss function, yet this depends heavily on extensive hyperparameter tuning to weigh each loss term. To this end, we propose to leverage physics prior knowledge by “baking” the discretized governing equations into the neural network architecture via the connection between the partial differential equations (PDE) operators and network structures, resulting in a PDE-preserved neural network (PPNN). This method, embedding discretized PDEs through convolutional residual networks in a multi-resolution setting, largely improves the generalizability and long-term prediction accuracy, outperforming conventional black-box models. The effectiveness and merit of the proposed methods have been demonstrated across various spatiotemporal dynamical systems governed by spatiotemporal PDEs, including reaction-diffusion, Burgers’, and Navier-Stokes equations.
97 MATHEMATICS AND COMPUTING↗
Probabilistic Nanomagnetic Memories for Uncertain and Robust Machine Learning
This project evaluated the use of emerging spintronic memory devices for robust and efficient variational inference schemes. Variational inference (VI) schemes, which constrain the distribution for each weight to be a Gaussian distribution with a mean and standard deviation, are a tractable method for calculating posterior distributions of weights in a Bayesian neural network such that this neural network can also be trained using the powerful backpropagation algorithm. Our project focuses on domain-wall magnetic tunnel junctions (DW-MTJs), a powerful multi-functional spintronic synapse design that can achieve low power switching while also opening the pathway towards repeatable, analog operation using fabricated notches. Our initial efforts to employ DW-MTJs as an all-in-one stochastic synapse with both a mean and standard deviation didn’t end up meeting the quality metrics for hardware-friendly VI. In the future, new device stacks and methods for expressive anisotropy modification may make this idea still possible. However, as a fall back that immediately satisfies our requirements, we invented and detailed how the combination of a DW-MTJ synapse encoding the mean and a probabilistic Bayes-MTJ device, programmed via a ferroelectric or ionically modifiable layer, can robustly and expressively implement VI. This design includes a physics-informed small circuit model, that was scaled up to perform and demonstrate rigorous uncertainty quantification applications, up to and including small convolutional networks on a grayscale image classification task, and larger (Residual) networks implementing multi-channel image classification. Lastly, as these results and ideas all depend upon the idea of an inference application where weights (spintronic memory states) remain non-volatile, the retention of these synapses for the notched case was further interrogated. These investigations revealed and emphasized the importance of both notch geometry and anisotropy modification in order to further enhance the endurance of written spintronic states. In the near future, these results will be mapped to effective predictions for room temperature and elevated operation DW-MTJ memory retention, and experimentally verified when devices become available.
77 NANOSCIENCE AND NANOTECHNOLOGY↗
Multiobjective and categorical global optimization of photonic structures based on ResNet generative neural networks
Abstract We show that deep generative neural networks, based on global optimization networks (GLOnets), can be configured to perform the multiobjective and categorical global optimization of photonic devices. A residual network scheme enables GLOnets to evolve from a deep architecture, which is required to properly search the full design space early in the optimization process, to a shallow network that generates a narrow distribution of globally optimal devices. As a proof-of-concept demonstration, we adapt our method to design thin-film stacks consisting of multiple material types. Benchmarks with known globally optimized antireflection structures indicate that GLOnets can find the global optimum with orders of magnitude faster speeds compared to conventional algorithms. We also demonstrate the utility of our method in complex design tasks with its application to incandescent light filters. These results indicate that advanced concepts in deep learning can push the capabilities of inverse design algorithms for photonics.
Jiang, Jiaqi↗
Deep Learning without Global Optimization by Random Fourier Neural Networks
Here we introduce a new training algorithm for deep neural networks that utilize random complex exponential activation functions. Our approach employs a Markov chain Monte Carlo sampling procedure to iteratively train network layers, avoiding global and gradient-based optimization while maintaining error control. It consistently attains the theoretical approximation rate for residual networks with complex exponential activation functions, determined by network complexity. Additionally, it enables efficient learning of multiscale and high-frequency features, producing interpretable parameter distributions. Despite using sinusoidal basis functions, we do not observe Gibbs phenomena in approximating discontinuous target functions.
97 MATHEMATICS AND COMPUTING↗
Solving the Orszag–Tang vortex magnetohydrodynamics problem with physics-constrained convolutional neural networks
We study the 2D Orszag–Tang vortex magnetohydrodynamics (MHD) problem through the use of physics-constrained convolutional neural networks (PCNNs) for forecasting the density, ρ, and the magnetic field, B, as well as the prediction of B given the velocity field v of the fluid. In addition to translation equivariance from the convolutional architecture, other physics constraints were embedded: absence of magnetic monopoles, non-negativity of ρ, use of only relevant variables, and the periodic boundary conditions of the problem. The use of only relevant variables and the hard constraint of non-negative ρ were found to facilitate learning greatly. The divergenceless condition ∇·B=0 was implemented as a hard constraint up to machine precision through the use of a magnetic potential to define B=∇×A. Residual networks and data augmentation were also used to improve performance. This allowed for some of the residual models to function as surrogate models and provide reasonably accurate simulations. For the prediction task, the PCNNs were evaluated against a physics-informed neural network, which had the ideal MHD induction equation as a soft constraint. Several models were able to generate highly accurate fields, which are visually almost indistinguishable and have low mean squared error. Only methods with built-in hard constraints produced physical fields with ∇·B=0. The use of PCNNs for MHD has the potential to produce physically consistent real-time simulations to serve as virtual diagnostics in cases where inferences must be made with limited observables.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Optimizing transmit field inhomogeneity of parallel RF transmit design in 7T MRI using deep learning
Ultrahigh field (UHF) Magnetic Resonance Imaging (MRI) provides a higher signal-to-noise ratio and, thereby, higher spatial resolution. However, UHF MRI introduces challenges such as transmit radiofrequency (RF) field (B+1) inhomogeneities, leading to uneven flip angles and image intensity anomalies. These issues can significantly degrade imaging quality and its medical applications. This study addresses B+1 field homogeneity through a novel deep learning-based strategy. Traditional methods like Magnitude Least Squares (MLS) optimization have been effective but are time-consuming and dependent on the patient’s presence. Recent machine learning approaches, such as RF Shim Prediction by Iteratively Projected Ridge Regression and deep learning frameworks, have shown promise but face limitations like extensive training times and oversimplified architectures. We propose a two-step deep learning strategy. First, we obtain the desired reference RF shimming weights from multi-channel B+1 fields using random-initialized Adaptive Moment Estimation. Then, we employ Residual Networks (ResNets) to train a model that maps B+1 fields to target RF shimming outputs. Our approach does not rely on pre-calculated reference optimizations for the testing process and efficiently learns residual functions. Comparative studies with traditional MLS optimization demonstrate our method’s advantages in terms of speed and accuracy. The proposed strategy achieves a faster and more efficient RF shimming design, significantly improving imaging quality at UHF. This advancement holds potential for broader applications in medical imaging and diagnostics.
Lu, Zhengyi [Vanderbilt University]↗
ResNet and CycleGAN for pulse shape discrimination of He-4 detector pulses: Recovering pulses conventional algorithms fail to label unanimously
Pulse shape discrimination (PSD) capable detectors, such as He-4, that respond to neutron and gamma-ray 7 interactions have a threshold deposited energy value below which n/γ discrimination vanishes when using 8 conventional PSD algorithms. Recent attempts in applying supervised learning based artificial neural 9 networks for PSD use the pulses in the separated regions to train the networks so they can be used to classify 10 another set of separated pulses. In doing so, pulses previously indistinguishable are not recovered for 11 classification, which would have increased the number of neutron and gamma-ray pulses that could be used 12 for further analysis. Assuming the reason why conventional PSD algorithms have unseparated regions is 13 because the parameter space of the algorithms fail to capture the intrinsic (but subtle) distinguishing 14 behavior of some of the neutron and gamma-ray pulses, a cycle-consistent generative adversarial network 15 (CycleGAN) was trained to amplify those differences and extract well separated neutron and gamma-ray 16 clusters. Results show that, once the network is trained with pulses from separated and unseparated regions, 17 it was able to transform the pulses in the unseparated region to improve the PSD. Subsequent n/γ 18 classification was performed using deep residual network (ResNet) that takes pulses with 512 data points 19 as an input. Two different ResNets were explored – simple ResNet and modified ResNet which takes 20 segmented pulse inputs in the first layer and the corresponding time axis values in the last hidden layer. 21 The later approach enables the network to extract time correlated pulse features to enhance its ability to 22 capture the pulse behaviors relevant for PSD. Although it achieves slightly lower accuracy, 99.41% versus 23 99.89%, based on simply counting the number of correct n/γ labels assigned, compared to the simple 24 ResNet, the modified ResNets architecture was able to decreases the cross-entropy loss function by half, 25 which implies that the correct n/γ labels assigned are less likely to be accidental. PSD parameter 26 distributions based on n/γ classification by ResNet before and after transforming unseparated pulses using 27 CycleGAN show that by enhancing the separation between neutrons and gamma-rays, the transformation 28 helps improve the performance of classifier networks that are trained using labeled dataset. The 29 enhancement of neutron and gamma-ray separation by the CycleGAN increased the PSD figure of merit 30 (FOM) by up to 70% in some regions. Here, the results show that, if a given detector achieves clear separation 31 between neutron and gamma-ray pulses in any energy region, such neural network approaches can help 32 lower the energy threshold for the separation and increasing the number of neutron and gamma-ray pulses 33 that can be used for further analysis.
4He↗
Galaxy deblending using residual dense neural networks
We present a new neural network approach for deblending galaxy images in astronomical data using Residual Dense Neural network (RDN) architecture. We train the network on synthetic galaxy images similar to the typical arrangements of field galaxies with a finite point spread function (PSF) and realistic noise levels. The main novelty of our approach is the usage of two distinct neural networks: i) a deblending network which isolates a single galaxy postage stamp from the composite and, ii) a classifier network which counts the remaining number of galaxies. The deblending proceeds by iteratively peeling one galaxy at a time from the composite until the image contains no further objects as determined by the classifier, or by other stopping criteria. By looking at the consistency in the outputs of the two networks, we can assess the quality of the deblending. Further, we characterize the flux and shape reconstructions in different quality bins and compare our deblender with the industry standard, SExtractor. We also discuss possible future extensions for the project with variable PSFs and noise levels.
79 ASTRONOMY AND ASTROPHYSICS↗
Fast-RF-Shimming: Accelerate RF shimming in 7T MRI using deep learning
Ultrahigh field (UHF) Magnetic Resonance Imaging (MRI) offers an elevated signal-to-noise ratio (SNR), enabling exceptionally high spatial resolution that benefits both clinical diagnostics and advanced research. However, the jump to higher fields introduces complications, particularly transmit radiofrequency (RF) field ($B^{+}_{1}$) inhomogeneities, manifesting as uneven flip angles and image intensity irregularities. These artifacts can degrade image quality and impede broader clinical adoption. Traditional RF shimming methods, such as Magnitude Least Squares (MLS) optimization, effectively mitigate $B^{+}_{1}$ inhomogeneity, but remain time-consuming. Recent machine learning approaches, including RF Shim Prediction by Iteratively Projected Ridge Regression and other deep learning architectures, suggest alternative pathways. Although these approaches show promise, challenges such as extensive training periods, limited network complexity, and practical data requirements persist. In this paper, we introduce a holistic learning-based framework called Fast-RF-Shimming, which achieves a 5000 × speed-up compared to the traditional MLS method. In the initial phase, we employ random-initialized Adaptive Moment Estimation (Adam) to derive the desired reference shimming weights from multi-channel $B^{+}_{1}$ fields. Next, we train a Residual Network (ResNet) to map $B^{+}_{1}$ fields directly to the ultimate RF shimming outputs, incorporating the confidence parameter into its loss function. Finally, we design Non-uniformity Field Detector (NFD), an optional post-processing step, to ensure the extreme non-uniform outcomes are identified. Comparative evaluations with standard MLS optimization underscore notable gains in both processing speed and predictive accuracy, which indicates that our technique shows a promising solution for addressing persistent inhomogeneity challenges.
Deep learning↗
Conformational spread drives the evolution of the calcium–calmodulin protein kinase II
The calcium calmodulin (Ca 2+ /CaM) dependent protein kinase II (CaMKII) decodes Ca 2+ frequency oscillations. The CaMKIIα isoform is predominantly expressed in the brain and has a central role in learning. I matched residue and organismal evolution with collective motions deduced from the atomic structure of the human CaMKIIα holoenzyme to learn how its ring architecture abets function. Protein dynamic simulations showed its peripheral kinase domains (KDs) are conformationally coupled via lateral spread along the central hub. The underlying β-sheet motions in the hub or association domain (AD) were deconvolved into dynamic couplings based on mutual information. They mapped onto a coevolved residue network to partition the AD into two distinct sectors. A second, energetically stressed sector was added to ancient bacterial enzyme dimers for assembly of the ringed hub. The continued evolution of the holoenzyme after AD–KD fusion targeted the sector’s ring contacts coupled to the KD. Among isoforms, the α isoform emerged last and, it alone, mutated rapidly after the poikilotherm–homeotherm jump to match the evolution of memory. The correlation between dynamics and evolution of the CaMKII AD argues single residue substitutions fine-tune hub conformational spread. The fine-tuning could increase CaMKIIα Ca 2+ frequency response range for complex learning functions.
59 BASIC BIOLOGICAL SCIENCES↗
RU Net for Automatic Characterization of TRISO Fuel Cross Sections
TRistructural ISOtropic (TRISO) particle fuel is a type of nuclear fuel known for its high-temperature and high-burnup performance. Each sub-millimeter diameter TRISO particle consists of uranium-oxycarbide (UCO) or UO2 fuel kernel, coated with buffer, inner pyrolytic carbon (IPyC), silicon carbide (SiC), and outer pyrolytic carbon (OPyC) layers. The SiC layer acts as the main containment barrier for the TRISO particle to retain the fission products, while the IPyC and OPyC layers provide additional barriers to the release of fission products, especially fission gases. During irradiation, phenomena like kernel swelling, buffer densification, and IPyC fracture may impact fuel performance. Post-irradiation microscopy on entire compact cross sections or samples of individual particles deconsolidated from compacts is often used to identify these irradiation-induced changes in morphology. However, each fuel compact generally contains thousands of TRISO particles. To get statistical information on these phenomena, it is cumbersome work if done manually. For example, to get information about swelling/densification behaviors of different layers or kernels after irradiation, researchers previously manually measured the perimeter of each TRISO layer in hundreds of particles after four rounds of iterative grinding and polishing encompassing more than 2000 cross-section images for a total of four fuel compacts. To attempt to reduce the subjectivity inherent in that process and accelerate data analysis, we conducted a study on the automatic TRISO layer segmentation on cross-sectional microscopic images using Convolutional Neural Networks (CNNs). CNNs are a class of machine learning algorithms specifically designed for processing structured grid data that have gained popularity in recent years due to their remarkable performance in various computer vision tasks, including image classification, object detection, and image segmentation. In this research, we have generated the large irradiated TRISO layer dataset with more than 2000 cross-section TRISO microscopic images and the corresponding annotated images. Based on these annotated images, we have employed different CNNs for automatic segmentation of different TRISO layers. These include RU-Net (developed in this study), as well as three existing architectures: U-Net, Residual Network (ResNet), and Attention U-Net. The preliminary results show that the model based on RU-Net has the best performance in terms of intersection-over-union (IoU). Through the aid of these CNN models, we can expedite the analysis of TRISO particle cross-sections, significantly reducing the manual labor involved and improving the objectivity of the segmentation results.
Convolutional Neural Networks↗