Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “residual network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

RU Net for Automatic Characterization of TRISO Fuel Cross Sections

TRistructural ISOtropic (TRISO) particle fuel is a type of nuclear fuel known for its high-temperature and high-burnup performance. Each sub-millimeter diameter TRISO particle consists of uranium-oxycarbide (UCO) or UO2 fuel kernel, coated with buffer, inner pyrolytic carbon (IPyC), silicon carbide (SiC), and outer pyrolytic carbon (OPyC) layers. The SiC layer acts as the main containment barrier for the TRISO particle to retain the fission products, while the IPyC and OPyC layers provide additional barriers to the release of fission products, especially fission gases. During irradiation, phenomena like kernel swelling, buffer densification, and IPyC fracture may impact fuel performance. Post-irradiation microscopy on entire compact cross sections or samples of individual particles deconsolidated from compacts is often used to identify these irradiation-induced changes in morphology. However, each fuel compact generally contains thousands of TRISO particles. To get statistical information on these phenomena, it is cumbersome work if done manually. For example, to get information about swelling/densification behaviors of different layers or kernels after irradiation, researchers previously manually measured the perimeter of each TRISO layer in hundreds of particles after four rounds of iterative grinding and polishing encompassing more than 2000 cross-section images for a total of four fuel compacts. To attempt to reduce the subjectivity inherent in that process and accelerate data analysis, we conducted a study on the automatic TRISO layer segmentation on cross-sectional microscopic images using Convolutional Neural Networks (CNNs). CNNs are a class of machine learning algorithms specifically designed for processing structured grid data that have gained popularity in recent years due to their remarkable performance in various computer vision tasks, including image classification, object detection, and image segmentation. In this research, we have generated the large irradiated TRISO layer dataset with more than 2000 cross-section TRISO microscopic images and the corresponding annotated images. Based on these annotated images, we have employed different CNNs for automatic segmentation of different TRISO layers. These include RU-Net (developed in this study), as well as three existing architectures: U-Net, Residual Network (ResNet), and Attention U-Net. The preliminary results show that the model based on RU-Net has the best performance in terms of intersection-over-union (IoU). Through the aid of these CNN models, we can expedite the analysis of TRISO particle cross-sections, significantly reducing the manual labor involved and improving the objectivity of the segmentation results.

Convolutional Neural Networks↗
DOE OSTI2026

RU-net for automatic characterization of TRISO fuel cross sections

During irradiation, phenomena such as kernel swelling and buffer densification may impact the performance of tristructural isotropic (TRISO) particle fuel. Post-irradiation microscopy is often used to identify these irradiation-induced morphologic changes. However, each fuel compact generally contains thousands of TRISO particles. Manually performing the work to get statistical information on these phenomena is cumbersome and subjective. Here, to reduce the subjectivity inherent in that process and to accelerate data analysis, we used convolutional neural networks (CNNs) to automatically segment cross-sectional images of microscopic TRISO layers. CNNs are a class of machine-learning algorithms specifically designed for processing structured grid data. They have gained popularity in recent years due to their remarkable performance in various computer vision tasks, including image classification, object detection, and image segmentation. In this research, we generated a large irradiated TRISO layer dataset with more than 2,000 microscopic images of cross-sectional TRISO particles and the corresponding annotated images. Based on these annotated images, we used different CNNs to automatically segment different TRISO layers. These CNNs include RU-Net (developed in this study), as well as three existing architectures: U-Net, Residual Network (ResNet), and Attention U-Net. The preliminary results show that the model based on RU-Net performs best in terms of Intersection over Union (IoU). Using CNN models, we can expedite the analysis of TRISO particle cross sections, significantly reducing the manual labor involved and improving the objectivity of the segmentation results.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

Moltensaltpropnet

MoltenSaltPropnet is a physics-informed machine learning framework that aims to predict the thermophysical properties of molten fluoride and chloride salt mixtures, which are crucial for the design and safety of Generation IV molten salt reactors. The code processes data from the Molten-Salt Thermal Properties Database (MSTDB-TP) and the Janz compendium, converting critically evaluated correlations into fast, differentiable surrogate models for density, viscosity, thermal conductivity, and heat capacity across 448 distinct salt systems. The implementation consists of several key components: 1. Data Curation: The code parses and cleans the raw data, normalizing elemental mole fractions and extracting relevant regression coefficients for various thermophysical properties. 2. Feature Engineering: It generates fixed-length numerical descriptors that encapsulate the composition and temperature, incorporating polynomial interaction terms and dimensionality-reduction techniques to optimize model performance. 3. Coefficient Learning: Four different machine learning architectures are employed: a deep residual network (ResNet), a Kolmogorov–Arnold network (KAN), a sparsity-inducing neural network (SNN), and classical regression models. Each model learns to predict coefficients that define the temperature-dependent correlations for the thermophysical properties. 4. Property Reconstruction: The predicted coefficients are used to compute temperature-dependent property values, ensuring positivity and monotonic trends through a composite loss function that enforces physical constraints. 5. User Interface: An open-source web application enables users to filter the database, train task-specific models, and visualize the results, allowing for rapid exploration of candidate salt mixtures. MoltenSaltPropnet bridges the gap between limited experimental data and high-fidelity reactor simulations, providing a powerful tool for researchers in the field of molten salt reactors and advanced nuclear energy systems.

Retamales, Mauricio Eduardo Tano [Idaho National L↗
DOE OSTI2025

Comprehensive Neural Posterior Estimation for Galaxy-Galaxy Strong Lensing

We present a deep learning model based on neural posterior estimation (NPE) for comprehensive extraction of astrophysical parameters from galaxy-scale strong gravitational lenses. The unprecedentedly large amount of galaxy-scale strong lenses expected in future cosmological surveys (${\cal O}(10^5)$) promises to enable valuable statistical constraints in various studies ranging from galaxy formation to the nature of dark matter, but it also poses a significant challenge for traditional modelling pipelines. To this end, our automated model includes several new, state-of-the-art features and approaches leveraging the framework of simulation-based inference (SBI). We infer a total of 20 parameters describing the mass and light profiles of both lens and source galaxies, using simulated raw multi-band data modelled under noise and observing conditions expected by the Legacy Survey of Space and Time (LSST), with its summary statistics generated by a residual network. We examine the efficacy of multi-band data in extracting nearly 20 model parameters simultaneous from strong lensing images including lens light. Finally, We perform a comprehensive set of diagnostics for SBI models, evaluating the model's prediction accuracy, stability, and uncertainty quantification.

Zhao, Roy J. [Chicago U., KICP]↗
DOE OSTI2026

Synthetic Atmospheric River Ensembles Generated by Deep-AR

This dataset contains 35,850 synthetic landfalling atmospheric river (AR) realizations generated by the Deep-AR two-stage deep-learning framework over the Northeast Pacific and U.S. West Coast. The archive contains 25 stochastic ensemble members for each of 1,434 held-out observed seed events. Each synthetic realization is initialized from conditions 48 hours before the corresponding observed AR landfall and is generated autoregressively at 6-hour intervals over a 144-hour period. Deep-AR combines a deterministic residual network (ResNet) that advances the large-scale atmospheric state with a Wasserstein generative adversarial network (WGAN) that produces stochastic, high-resolution fields. Each HDF5 file contains 0.25° gridded synthetic integrated vapor transport components (qu, qv), 10 m wind components (u10, v10), and 6-hour accumulated precipitation on a common 200 × 480 grid. The files also include coordinate and datetime arrays. This dataset supports AR hazard analysis, ensemble-based uncertainty characterization, precipitation-extremes research, and regional stress testing. Synthetic files follow the naming convention deepar.model.YYYYMMDD.HHMMSS.vNN.h5. YYYYMMDD.HHMMSS identifies the UTC initial-condition timestamp, which occurs 48 hours before the diagnosed observed landfall, and vNN identifies the zero-padded ensemble member, ranging from v01 through v25. Each synthetic file can be paired with its corresponding observed file by matching the initial-condition timestamp. The paired observed file follows the naming convention deepar.obs.YYYYMMDD.HHMMSS.h5 and is available in the separately registered oracle/deepar.obs dataset at https://wdh.energy.gov/ds/oracle/deepar.obs (DOI: https://doi.org/10.21947/3377671).

17 WIND ENERGY↗
DOE OSTI2023

L-dwarf Detection from SDSS Images using Improved Faster R-CNN

We present a data-driven approach to automatically detect L dwarfs from Sloan Digital Sky Survey (SDSS) images using an improved Faster R-CNN framework based on deep learning. The established L-dwarf automatic detection (LDAD) model distinguishes L dwarfs from other celestial objects and backgrounds in SDSS field images by learning the features of 387 SDSS images containing L dwarfs. Applying the LDAD model to the SDSS images containing 93 labeled L dwarfs in the test set, we successfully detected 83 known L dwarfs with a recall rate of 89.25% for known L dwarfs. Several techniques are implemented in the LDAD model to improve its detection performance for L dwarfs, including the deep residual network and the feature pyramid network. As a result, the LDAD model outperforms the model of the original Faster R-CNN, whose recall rate of known L dwarfs is 80.65% for the same test set. The LDAD model was applied to detect L dwarfs from a larger validation set including 843 labeled L dwarfs, resulting in a recall rate of 94.42% for known L dwarfs. The newly identified candidates include L dwarfs, late M and T dwarfs, which were estimated from color (i – z) and spectral type relation. The contamination rates for the test candidates and validation candidates are 8.60% and 9.27%, respectively. The detection results indicate that our model is effective to search for L dwarfs from astronomical images.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Improving Trustworthiness of Data-Driven Power Grid Contingency Analysis With Bayesian Residual Graph Neural Networks

The evolving energy landscape requires novel tools to efficiently perform contingency analysis and reliability assessment of power grids, potentially in real-time. The high computational cost of traditional power flow solvers limits their applicability in practice. Machine learning (ML) surrogates such as deep neural networks (NNs) accelerate power flow solvers computations, enabling high-order contingency analysis and real-time decision-making by learning highly nonlinear functions and integrating grid topology via graph architectures. However, (graph) NNs lack predictive power away from training data and do not provide predictive confidence estimates. Here, we present a Bayesian residual graph NN that integrates knowledge from low-fidelity data via residual training and embeds granular quantification of uncertainties, improving trustworthiness critical for high-consequence decision-making. Applying Bayesian concepts to NNs is challenging due to the high-dimensionality of both the parameter space, complicating derivation of a meaningful prior, and the output space in large grid systems, requiring enhanced techniques to assess the predicted high-dimensional uncertainties. Our contributions include: (1) Deriving a prior for fully connected and graph NNs that leverages low-fidelity data to guide mean predictions and appropriately control prior predictive uncertainty. (2) Integrating this prior within an ensembling with anchoring scheme for efficient approximate posterior inference. (3) Deriving enhanced metrics to assess accuracy of both the mean and uncertainty predictions in high dimensions, appropriately accounting for correlations propagated through graph layers. The resulting Bayesian residual graph NN is tested on a contingency analysis task for 14-bus and 118-bus grids.

24 - POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

The role of stiffness in training and generalization of ResNets

Neural ordinary differential equations (NODEs) have recently regained popularity as large-depth limits of a large class of neural networks. In particular, residual neural networks (ResNets) are equivalent to an explicit Euler discretization of an underlying NODE, where the transition from one layer to the next is one time step of the discretization. The relationship between continuous and discrete neural networks has been of particular interest. Notably, analysis from the ordinary differential equation viewpoint can potentially lead to new insights for understanding the behavior of neural networks in general. In this work, we take inspiration from differential equations to define the concept of stiffness for a ResNet via the interpretation of a ResNet as the discretization of a NODE. Here, we then examine the effects of stiffness on the ability of a ResNet to generalize, via computational studies on example problems coming from climate and chemistry models. We find that penalizing stiffness does have a unique regularizing effect, but we see no benefit to penalizing stiffness over L 2 regularization (penalization of network parameter norms) in terms of predictive performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Error Estimates of Residual Minimization Using Neural Networks for Linear PDES

We propose an abstract framework for analyzing the convergence of least-squares methods based on residual minimization when feasible solutions are neural networks. With the norm relations and compactness arguments, we derive error estimates for both continuous and discrete formulations of residual minimization in strong and weak forms. The formulations cover recently developed physicsinformed neural networks based on strong and variational formulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Bilevel optimization, deep learning and fractional Laplacian regularization with applications in tomography

Here we consider a generalized bilevel optimization framework for solving inverse problems. We introduce fractional Laplacian as a regularizer to improve the reconstruction quality, and compare it with the total variation regularization. We emphasize that the key advantage of using fractional Laplacian as a regularizer is that it leads to a linear operator, as opposed to the total variation regularization which results in a nonlinear degenerate operator. Inspired by residual neural networks, to learn the optimal strength of regularization and the exponent of fractional Laplacian, we develop a dedicated bilevel optimization neural network with a variable depth for a general regularized inverse problem. We illustrate how to incorporate various regularizer choices into our proposed network. As an example, we consider tomographic reconstruction as a model problem and show an improvement in reconstruction quality, especially for limited data, via fractional Laplacian regularization. We successfully learn the regularization strength and the fractional exponent via our proposed bilevel optimization neural network. We observe that the fractional Laplacian regularization outperforms total variation regularization. This is specially encouraging, and important, in the case of limited and noisy data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

A hybrid Penman-Monteith and machine learning model for simulating evapotranspiration and its components

Integrating physical processes with machine learning has advanced evapotranspiration (ET) simulation, yet most hybrid models fail to partition total ET into its components: soil evaporation (E) and vegetation transpiration (T). This study introduces Residual Neural Network–Penman–Monteith (RNN-PM), a novel hybrid dual-source ET model designed to overcome this limitation. The model synergizes the physically-based Penman–Monteith framework with three specialized residual neural networks trained to estimate key conductance parameters (canopy conductance, soil surface conductance, and aerodynamic conductance). Furthermore this explicit parameterization allows for the direct partitioning of total ET. Validation at National Ecological Observatory Network (NEON) flux sites using high-frequency partitioned E and T shows that RNN-PM reliably reproduces ET and the transpiration fraction (T/ET). For ET, the model achieves an average Kling–Gupta efficiency (KGE) of 0.89 and a root-mean-square error (RMSE) of 0.55 mm/day; for T/ET, the KGE is 0.87 with an RMSE of 0.06. Furthermore, RNN-PM demonstrates robust generalization, accurately simulating ET and its components well beyond the initial training dataset, even under extreme climatic conditions. This study extended the analysis by comparing the RNN-PM model with seven established dual-source ET models. The results indicate that RNN-PM outperforms both conventional machine learning models and purely physical process-based models in simulating ET components in most cases. Among the purely physical process-based dual-source models, those based on surface temperature decomposition showed improved performance as the leaf area index (LAI) decreased when evaluated against high-frequency ET component datasets. In contrast, the performance of conductance-based dual-source models declined with decreasing LAI. Although purely machine learning-based models can produce relatively accurate simulations of ET components, they often exhibit limited generalization capability, an issue that the RNN-PM model effectively overcomes. Ultimately, the RNN-PM model represents a significant advance in simulating ET components, offering a novel and scalable approach for improving the representation of land–atmosphere interactions in Earth system models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Using Direct Numerical Simulation of Pore-Level Events to Improve Pore-Network Models for Prediction of Residual Trapping of CO2

Direct numerical simulation and pore-network modeling are common approaches to study the physics of two-phase flow through natural rocks. For assessment of the long-term performance of geological sequestration of CO 2 , it is important to model the full drainage-imbibition cycle to provide an accurate estimate of the trapped CO 2 . While direct numerical simulation using pore geometry from micro-CT rock images accurately models two-phase flow physics, it is computationally prohibitive for large rock volumes. On the other hand, pore-network modeling on networks extracted from micro-CT rock images is computationally efficient but utilizes simplified physics in idealized geometric pore elements. This study uses the lattice-Boltzmann method for direct numerical simulation of CO 2 -brine flow in idealized pore elements to develop a new set of pore-level flow models for the pore-body filling and snap-off events in pore-network modeling of imbibition. Lattice-Boltzmann simulations are conducted on typical idealized pore-network configurations, and the interface evolution and local capillary pressure are evaluated to develop modified equations of local threshold capillary pressure of pore elements as a function of shape factor and other geometrical parameters. The modified equations are then incorporated into a quasi-static pore-network flow solver. The modified model is applied on extracted pore-network of sandstone samples, and saturation of residual trapped CO 2 is computed for a drainage-imbibition cycle. The modified model yields different statistics of pore-level events compared with the original model; in particular, the occurrence of snap-off in pore-throats is reduced resulting in a more frontal displacement pattern along the main injection direction. Compared to the original model, the modified model is in closer agreement with the residual trapped CO 2 obtained from core flow experiments and direct numerical simulation.

Kohanpur, Amir H.↗
DOE OSTI2021

Bayesian tensorized neural networks with automatic rank selection

Tensor decomposition is an effective approach to compress over-parameterized neural networks and to enable their deployment on resource-constrained hardware platforms. However, directly applying tensor compression in the training process is a challenging task due to the difficulty of choosing a proper tensor rank. In order to address this challenge, this paper proposes a low-rank Bayesian tensorized neural network. Our Bayesian method performs automatic model compression via an adaptive tensor rank determination. We also present approaches for posterior density calculation and maximum a posteriori (MAP) estimation for the end-to-end training of our tensorized neural network. Here, we provide experimental validation on a two-layer fully connected neural network, a 6-layer CNN and a 110-layer residual neural network where our work produces 7.4x to 137x more compact neural networks directly from the training while achieving high prediction accuracy.

Low-rank tensor↗
DOE OSTI2023

HOLISMOKES IX. Neural network inference of strong-lens parameters and uncertainties from ground-based images

Modeling of strong gravitational lenses is a necessity for further applications in astrophysics and cosmology. With the large number of detections in current and upcoming surveys, such as the Rubin Legacy Survey of Space and Time (LSST), it is pertinent to investigate automated and fast analysis techniques beyond the traditional and time-consuming Markov chain Monte Carlo sampling methods. Building upon our (simple) convolutional neural network (CNN), we present here another CNN, specifically a residual neural network (ResNet), that predicts the five mass parameters of a singular isothermal ellipsoid (SIE) profile (lens center x and y, ellipticity e x and e y , Einstein radius θ E ) and the external shear (γ ext, 1 , γ ext, 2 ) from ground-based imaging data. In contrast to our previous CNN, this ResNet further predicts the 1σ uncertainty for each parameter. To train our network, we use our improved pipeline to simulate lens images using real images of galaxies from the Hyper Suprime-Cam Survey (HSC) and from the Hubble Ultra Deep Field as lens galaxies and background sources, respectively. We find very good recoveries overall for the SIE parameters, especially for the lens center in comparison to our previous CNN, while significant differences remain in predicting the external shear. From our multiple tests, it appears that most likely the low ground-based image resolution is the limiting factor in predicting the external shear. Given the run time of milli-seconds per system, our network is perfectly suited to quickly predict the next appearing image and time delays of lensed transients. Therefore, we use the network-predicted mass model to estimate these quantities and compare to those values obtained from our simulations. Unfortunately, the achieved precision allows only a first-order estimate of time delays on real lens systems and requires further refinement through follow-up modeling. Nonetheless, our ResNet is able to predict the SIE and shear parameter values in fractions of a second on a single CPU, meaning that we are able to efficiently process the huge amount of galaxy-scale lenses expected in the near future.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Data-Driven Learning of Nonautonomous Systems

In this work, we present a numerical framework for recovering unknown nonautonomous dynamical systems with time-dependent inputs. To circumvent the difficulty presented by the nonautonomous nature of the system, our method transforms the solution state into piecewise integration of the system over a discrete set of time instances. The time-dependent inputs are then locally parameterized by using a proper model, for example, polynomial regression, in the pieces determined by the time instances. This transforms the original system into a piecewise parametric system that is locally time invariant. We then design a deep neural network structure to learn the local models. Once the network model is constructed, it can be iteratively used over time to conduct global system prediction. We provide theoretical analysis of our algorithm and present a number of numerical examples to demonstrate the effectiveness of the method.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Data-Driven Learning of Non-Autonomous Systems

We present a numerical framework for recovering unknown non-autonomous dynamical systems with time-dependent inputs. To circumvent the difficulty presented by the non-autonomous nature of the system, our method transforms the solution state into piecewise integration of the system over a discrete set of time instances. The time-dependent inputs are then locally parameterized by using a proper model, for example, polynomial regression, in the pieces determined by the time instances. This transforms the original system into a piecewise parametric system that is locally time invariant. We then design a deep neural network structure to learn the local models. Once the network model is constructed, it can be iteratively used over time to conduct global system prediction. We provide theoretical analysis of our algorithm and present a number of numerical examples to demonstrate the effectiveness of the method.

97 MATHEMATICS AND COMPUTING↗