Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “residual network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Adaptive Data-Driven Deep-Learning Surrogate Model for Frontal Polymerization in Dicyclopentadiene

Frontal polymerization (FP) is a self-sustaining curing process that enables rapid and energy-efficient manufacturing of thermoset polymers and composites. Computational methods conventionally used to simulate the FP process are time-consuming, and repeating simulations are required for sensitivity analysis, uncertainty quantification, or optimization of the manufacturing process. Here, in this work, we develop an adaptive surrogate deep-learning model for FP of dicyclopentadiene (DCPD), which predicts the evolution of temperature and degree of cure orders of magnitude faster than the finite-element method (FEM). The adaptive algorithm provides a strategy to select training samples efficiently and save computational costs by reducing the redundancy of FEM-based training samples. The adaptive algorithm calculates the residual error of the FP governing equations using automatic differentiation of the deep neural network. A probability density function expressed in terms of the residual error is used to select training samples from the Sobol sequence space. The temperature and degree of cure evolution of each training sample are obtained by a 2D FEM simulation. The adaptive method is more efficient and has a better prediction accuracy than the random sampling method. With the well-trained surrogate neural network, the FP characteristics (front speed, shape, and temperature) can be extracted quickly from the predicted temperature and degree-of-cure fields.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Parallel molecular mechanisms for enzyme temperature adaptation

The mechanisms that underly the adaptation of enzyme activities and stabilities to temperature are fundamental to our understanding of molecular evolution and how enzymes work. Here, we investigate the molecular and evolutionary mechanisms of enzyme temperature adaption, combining deep mechanistic studies with comprehensive sequence analyses of thousands of enzymes. We show that temperature adaptation in ketosteroid isomerase (KSI) arises primarily from one residue change with limited, local epistasis, and we establish the underlying physical mechanisms. This residue change occurs in diverse KSI backgrounds, suggesting parallel adaptation to temperature. Furthermore, we identify residues associated with organismal growth temperature across 1005 diverse bacterial enzyme families, suggesting widespread parallel adaptation to temperature. We assess the residue properties, molecular interactions, and interaction networks that appear to underly temperature adaptation.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Transient Unfolding and Long-Range Interactions in Viral BCL2 M11 Enable Binding to the BECN1 BH3 Domain

Viral BCL2 proteins (vBCL2s) help to sustain chronic infection of host proteins to inhibit apoptosis and autophagy. However, details of conformational changes in vBCL2s that enable binding to BH3Ds remain unknown. Using all-atom, multiple microsecond-long molecular dynamic simulations (totaling 17 μs) of the murine γ-herpesvirus 68 vBCL2 (M11), and statistical inference techniques, we show that regions of M11 transiently unfold and refold upon binding of the BH3D. Further, we show that this partial unfolding/refolding within M11 is mediated by a network of hydrophobic interactions, which includes residues that are 10 Å away from the BH3D binding cleft. We experimentally validate the role of these hydrophobic interactions by quantifying the impact of mutating these residues on binding to the Beclin1/BECN1 BH3D, demonstrating that these mutations adversely affect both protein stability and binding. To our knowledge, this is the first study detailing the binding-associated conformational changes and presence of long-range interactions within vBCL2s.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Chemical Reactor Network Modeling of Ammonia Rich-Quench-Lean Combustion Using a Partially Stirred Reactor Approach

Ammonia is a promising alternative fuel, but its use is challenging due to low flammability and high nitrogen oxide (NOx) emissions. Two-stage rich-quench-lean (RQL) combustion strategies have shown promise in reducing NOx emissions. This approach involves two stages: a rich stage that oxidizes part of the fuel and decomposes ammonia into hydrogen, and a lean stage that burns out the hydrogen and residual ammonia. Researchers used a chemical reactor network model to study the effects of heat loss and mixing on emissions performance. They found that heat loss and reduced mixing rates can lead to increased NOx emissions and N2O formation. The results will inform the development of optimized two-stage RQL combustors for ammonia, with a focus on minimizing NOx emissions and improving overall efficiency.

ammonia combustion↗
DOE OSTI2023

Neural Network-Based Variational Methods for Solving Quadratic Porous Medium Equations in High Dimensions

Here, in this paper, we propose and study neural network-based methods for solutions of high-dimensional quadratic porous medium equation (QPME). Three variational formulations of this nonlinear PDE are presented: a strong formulation and two weak formulations. For the strong formulation, the solution is directly parameterized with a neural network and optimized by minimizing the PDE residual. It can be proved that the convergence of the optimization problem guarantees the convergence of the approximate solution in the $L^1$ sense. The weak formulations are derived following (Brenier in Examples of hidden convexity in nonlinear PDEs, 2020) which characterizes the very weak solutions of QPME. Specifically speaking, the solutions are represented with intermediate functions who are parameterized with neural networks and are trained to optimize the weak formulations. Extensive numerical tests are further carried out to investigate the pros and cons of each formulation in low and high dimensions. This is an initial exploration made along the line of solving high-dimensional nonlinear PDEs with neural network-based methods, which we hope can provide some useful experience for future investigations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Knowledge-Informed Uncertainty-Aware Machine Learning for Time Series Forecasting of Dynamical Engineered Systems

The high complexity and multiscale nature of many engineered systems—such as those in nuclear power plants—make representing and forecasting their dynamic behavior challenging. Physics-based models can be overly complex and computationally intractable, whereas machine learning (ML) tools are often data-hungry and prone to unphysical solutions. This study proposes a knowledge-informed ML-aided hybrid residual modeling approach that offers accurate and efficient time series forecasting for the operation of dynamical engineered systems. Hybrid residual modeling entails a baseline solution from domain knowledge and known physics expressions about the system dynamics integrated with an ML model to capture undiscovered information from the mismatch (i.e., residuals) between true states from measurements and baseline-predicted outputs. This study further quantifies the ML model uncertainty to provide trustworthy solutions. Real-time operational data from thermal-hydraulic flow loops of the cryogenic moderator system in Oak Ridge National Laboratory’s Spallation Neutron Source facility were used to demonstrate the potential of knowledge-informed uncertainty-aware ML in real-world applications. The state variables of the cryogenic helium loop were modeled with (1) first principles–based system identification (sysID), (2) long short-term memory (LSTM) neural network, and (3) hybrid sysID (baseline) + LSTM (residual). The superior predictive capability of the sysID+LSTM model versus stand-alone sysID and LSTM is confirmed by average performance metrics and individual data points across different prediction horizons. By creating a robust representation of the underlying physical system, the widely applicable hybrid residual modeling approach will enable the future development of digital twins for performance prediction, prognostics, and operation control.

Zhao, Xingang↗
DOE OSTI2026

Improving ideal MHD equilibrium accuracy with physics-informed neural networks

We present a novel approach to compute three-dimensional magnetohydrodynamic equilibria with isotropic pressure profiles and nested surfaces by parametrizing Fourier modes with artificial neural networks (NNs). The full nonlinear global force residual of single equilibria across the volume in real space is then minimized with first order optimizers and compared to equilibria computed by conventional solvers. Already, we observe competitive computational cost to arrive at the same minimum residuals computable with existing codes. With increased computational cost, lower minima of the residual are computable with the NNs than with any other tested solver, establishing a new lower bound for the force residual. We use minimally complex NNs, and we expect significant improvements for solving not only single equilibria with NNs, but also for creating NN models valid over continuous distributions of equilibria.

ideal magnetohydrodynamics↗
DOE OSTI2025

An adaptive and stability-promoting layerwise training approach for sparse deep neural network architecture

This work presents a two-stage adaptive framework for progressively developing deep neural network (DNN) architectures that generalize well for a given training data set. In the first stage, a layerwise training approach is adopted where a new layer is added each time and trained independently by freezing parameters in the previous layers. We impose desirable structures on the DNN by employing manifold regularization, sparsity regularization, and physics-informed terms. We introduce a ε – δ – stability-promoting concept as a desirable property for a learning algorithm and show that employing manifold regularization yields a ε – δ stability-promoting algorithm. Further, we also derive the necessary conditions for the trainability of a newly added layer and investigate the training saturation problem. In the second stage of the algorithm (post-processing), a sequence of shallow networks is employed to extract information from the residual produced in the first stage, thereby improving the prediction accuracy. Numerical investigations on prototype regression and classification problems demonstrate that the proposed approach can outperform fully connected DNNs of the same size. Moreover, by equipping the physics-informed neural network (PINN) with the proposed adaptive architecture strategy to solve partial differential equations, we numerically show that adaptive PINNs not only are superior to standard PINNs but also produce interpretable hidden layers with provable stability. As a result, we also apply our architecture design strategy to solve inverse problems governed by elliptic partial differential equations.

42 ENGINEERING↗
DOE OSTI2021

Time–connectivity superposition and the gel/glass duality of weak colloidal gels

Colloidal gels result from the aggregation of Brownian particles suspended in a solvent. Gelation is induced by attractive interactions between individual particles that drive the formation of clusters, which in turn aggregate to form a space-spanning structure. We study this process in aluminosilicate colloidal gels through time-resolved structural and mechanical spectroscopy. Using the time–connectivity superposition principle a series of rapidly acquired linear viscoelastic spectra, measured throughout the gelation process by applying an exponential chirp protocol, are rescaled onto a universal master curve that spans over eight orders of magnitude in reduced frequency. This analysis reveals that the underlying relaxation time spectrum of the colloidal gel is symmetric in time with power-law tails characterized by a single exponent that is set at the gel point. The microstructural mechanical network has a dual character; at short length scales and fast times it appears glassy, whereas at longer times and larger scales it is gel-like. These results can be captured by a simple three-parameter constitutive model and demonstrate that the microstructure of a mature colloidal gel bears the residual skeleton of the original sample-spanning network that is created at the gel point. Our conclusions are confirmed by applying the same technique to another well-known colloidal gel system composed of attractive silica nanoparticles. The results illustrate the power of the time–connectivity superposition principle for this class of soft glassy materials and provide a compact description for the dichotomous viscoelastic nature of weak colloidal gels.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Hydrogen bond arrangements in (H 2 O) 20, 24, 28 clathrate hydrate cages: Optimization and many-body analysis

Here we provide a detailed study of hydrogen bonding arrangements, relative stability, residual entropy, and an analysis of the many-body effects in the (H 2 O) 20 (D-cage), (H 2 O) 24 (T-cage), and (H 2 O) 28 (H-cage) hollow cages making up structures I (sI) and II (sII) of clathrate hydrate lattices. Based on the enumeration of the possible hydrogen bonding networks for a fixed oxygen atom scaffold, the residual entropy (S 0 ) of these three gas phase cages was estimated at 0.754 82, 0.754 44, and 0.754 17 · Nk b , where N is the number of molecules and k b is Boltzmann’s constant. A previously identified descriptor of enhanced stability based on the relative arrangement and connectivity of nearest-neighbor fragments on the polyhedral water cluster [strong-weak-effective-bond model] also applies to the larger hollow cages. The three cages contain a maximum of 7, 9, and 11 such preferable arrangements of trans nearest dimer pairs with one “free” OH bond on the donor molecule (t1d dimers). The Many-Body Expansion (MBE) up to the 4-body suggests that the many-body terms vary nearly linearly with the cluster binding energy. Using a hierarchical approach of screening the relative stability of networks starting from optimizations with the TIP4P, TTM2.1-F, and MB-pol classical potentials, subsequently refining at more accurate levels of electronic structure theory (DFT and MP2), and finally correcting for zero-point energy, we were able to identify a group of four low-lying isomers of the (H 2 O) 24 T-cage, two of which are antisymmetric and the other two form a pair of antipode configurations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Enabling deeper learning on big data for materials informatics applications

Abstract The application of machine learning (ML) techniques in materials science has attracted significant attention in recent years, due to their impressive ability to efficiently extract data-driven linkages from various input materials representations to their output properties. While the application of traditional ML techniques has become quite ubiquitous, there have been limited applications of more advanced deep learning (DL) techniques, primarily because big materials datasets are relatively rare. Given the demonstrated potential and advantages of DL and the increasing availability of big materials datasets, it is attractive to go for deeper neural networks in a bid to boost model performance, but in reality, it leads to performance degradation due to the vanishing gradient problem. In this paper, we address the question of how to enable deeper learning for cases where big materials data is available. Here, we present a general deep learning framework based on Individual Residual learning (IRNet) composed of very deep neural networks that can work with any vector-based materials representation as input to build accurate property prediction models. We find that the proposed IRNet models can not only successfully alleviate the vanishing gradient problem and enable deeper learning, but also lead to significantly (up to 47%) better model accuracy as compared to plain deep neural networks and traditional ML techniques for a given input materials representation in the presence of big data.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Unusual zwitterionic catalytic site of SARS–CoV-2 main protease revealed by neutron crystallography

The main protease (3CL M pro ) from SARS–CoV-2, the etiological agent of COVID-19, is an essential enzyme for viral replication. 3CL M pro possesses an unusual catalytic dyad composed of Cys 145 and His 41 residues. A critical question in the field has been what the protonation states of the ionizable residues in the substrate-binding active-site cavity are; resolving this point would help understand the catalytic details of the enzyme and inform rational drug development against this pernicious virus. Here, we present the room-temperature neutron structure of 3CL M pro , which allowed direct determination of hydrogen atom positions and, hence, protonation states in the protease. We observe that the catalytic site natively adopts a zwitterionic reactive form in which Cys 145 is in the negatively charged thiolate state and His 41 is doubly protonated and positively charged, instead of the neutral unreactive state usually envisaged. The neutron structure also identified the protonation states, and thus electrical charges, of all other amino acid residues and revealed intricate hydrogen-bonding networks in the active-site cavity and at the dimer interface. The fine atomic details present in this structure were made possible by the unique scattering properties of the neutron, which is an ideal probe for locating hydrogen positions and experimentally determining protonation states at near-physiological temperature. Our observations provide critical information for structure-assisted and computational drug design, allowing precise tailoring of inhibitors to the enzyme's electrostatic environment.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Simultaneous energy and mass calibration of large-radius jets with the ATLAS detector using a deep neural network

The energy and mass measurements of jets are crucial tasks for the Large Hadron Collider experiments. This paper presents a new calibration method to simultaneously calibrate these quantities for large-radius jets measured with the ATLAS detector using a deep neural network (DNN). To address the specificities of the calibration problem, special loss functions and training procedures are employed, and a complex network architecture, which includes feature annotation and residual connection layers, is used. The DNN-based calibration is compared to the standard numerical approach in an extensive series of tests. The DNN approach is found to perform significantly better in almost all of the tests and over most of the relevant kinematic phase space. In particular, it consistently improves the energy and mass resolutions, with a 30% better energy resolution obtained for transverse momenta $p$ T > $500$ GeV.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

A Multi-Objective Approach for Optimizing Edge-Based Resource Allocation Using TOPSIS

Existing approaches for allocating resources on edge environments are inefficient and lack the support of heterogeneous edge devices, which in turn fail to optimize the dependency on cloud infrastructures or datacenters. To this extent, we propose in this paper OpERA, a multi-layered edge-based resource allocation optimization framework that supports heterogeneous and seamless execution of offloadable tasks across edge, fog, and cloud computing layers and architectures. By capturing offloadable task requirements, OpERA is capable of identifying suitable resources within nearby edge or fog layers, thus optimizing the execution process. Throughout the paper, we present results which show the effectiveness of our proposed optimization strategy in terms of reducing costs, minimizing energy consumption, and promoting other residual gains in terms of processing computations, network bandwidth, and task execution time. We also demonstrate that by optimizing resource allocation in computation offloading, it is then possible to increase the likelihood of successful task offloading, particularly for computationally intensive tasks that are becoming integral as part of many IoT applications such robotic surgery, autonomous driving, smart city monitoring device grids, and deep learning tasks. The evaluation of our OpERA optimization algorithm reveals that the TOPSIS MCDM technique effectively identifies optimal compute resources for processing offloadable tasks, with a 96% success rate. Moreover, the results from our experiments with a diverse range of use cases show that our OpERA optimization strategy can effectively reduce energy consumption by up to 88%, and operational costs by 76%, by identifying relevant compute resources.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Ensemble models for circuit topology estimation, fault detection and classification in distribution systems

This paper presents a methodology for simultaneous fault detection, classification, and topology estimation for adaptive protection of distribution systems. The methodology estimates the probability of the occurrence of each one of these events by using a hybrid structure that combines three sub-systems, a convolutional neural network for topology estimation, a fault detection based on predictive residual analysis, and a standard support vector machine with probabilistic output for fault classification. The input to all these sub-systems is the local voltage and current measurements. A convolutional neural network uses these local measurements in the form of sequential data to extract features and estimate the topology conditions. The fault detector is constructed with a Bayesian stage (a multitask Gaussian process) that computes a predictive distribution (assumed to be Gaussian) of the residuals using the input. Since the distribution is known, these residuals can be transformed into a Standard distribution, whose values are then introduced into a one-class support vector machine. The structure allows using a one-class support vector machine without parameter cross-validation, so the fault detector is fully unsupervised. Finally, a support vector machine uses the input to perform the classification of the fault types. All three sub-systems can work in a parallel setup for both performance and computation efficiency. In conclusion, we test all three sub-systems included in the structure on a modified IEEE123 bus system, and we compare and evaluate the results with standard approaches.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Separable physics-informed DeepONet: Breaking the curse of dimensionality in physics-informed machine learning

The deep operator network (DeepONet) has shown remarkable potential in solving partial differential equations (PDEs) by mapping between infinite-dimensional function spaces using labeled datasets. However, in scenarios lacking labeled data, the physics-informed DeepONet (PI-DeepONet) approach, which utilizes the residual loss of the governing PDE to optimize the network parameters, faces significant computational challenges, particularly due to the curse of dimensionality. This limitation has hindered its application to high-dimensional problems, making even standard 3D spatial with 1D temporal problems computationally prohibitive. Additionally, the computational requirement increases exponentially with the discretization density of the domain. Here, to address these challenges and enhance scalability for high-dimensional PDEs, we introduce the Separable physics-informed DeepONet (Sep-PI-DeepONet). This framework employs a factorization technique, utilizing sub-networks for individual one-dimensional coordinates, thereby reducing the number of forward passes and the size of the Jacobian matrix required for gradient computations. By incorporating forward-mode automatic differentiation (AD), we further optimize computational efficiency, achieving linear scaling of computational cost with discretization density and dimensionality, making our approach highly suitable for high-dimensional PDEs. We demonstrate the effectiveness of Sep-PI-DeepONet through three benchmark PDE models: the viscous Burgers’ equation, Biot’s consolidation theory, and a parameterized heat equation. Our framework maintains accuracy comparable to the conventional PI-DeepONet while reducing training time by two orders of magnitude. Notably, for the heat equation solved as a 4D problem, the conventional PI-DeepONet was computationally infeasible (estimated 289.35 h), while the Sep-PI-DeepONet completed training in just 2.5 h. These results underscore the potential of Sep-PI-DeepONet in efficiently solving complex, high-dimensional PDEs, marking a significant advancement in physics-informed machine learning.

Neural operator↗
DOE OSTI2025

Contributions of vegetation heterogeneity within tower footprint to CO 2 flux estimations through graph neural network modeling

Net ecosystem exchange of CO 2 (Fc) measured directly by eddy covariance towers is based on various assumptions, including large, flat and homogenous land cover type. In reality, often a tower site is not large enough for flux measurements, and landscapes consist of patches of different land cover types within the flux footprint. In addition, some portions of fluxes are contributed by different cover types when a footprint exceeds the size of the target ecosystem. The contributions of non-dominant patches to Fc are often ignored. Here, in this study, we propose a novel integrated modeling framework that combines random forest (RF) and XGBoost with a residual correction module based on a deep graph convolutional network (DeeperGCN) to simulate Fc for seven flux measurement sites in southwest Michigan. High-resolution remote sensing vegetation indices, soil properties, meteorological variables, and footprint-weighted spatial features were used as model inputs at three spatial resolutions (10 m, 20 m, 30 m), and their importance in predicting Fc with DeeperGCN was assessed. We found that residual correction using DeeperGCN significantly improved prediction accuracy, with the R 2 increasing from 0.9098 to 0.9479 for RF and from 0.9235 to 0.9433 for XGBoost. At site level, the maximum improvement in R 2 reached 0.1617. Paired t-tests confirmed that these improvements were statistically significant (p < 0.05). Among all predictors, leaf area index and incoming shortwave radiation emerged as the dominant drivers of spatial residual variation, followed by precipitation, relative humidity, and selected vegetation indices. The 20 m resolution yielded the best balance between model performance and computational efficiency. In conclusion, our modeling framework effectively captures both spatial heterogeneity and nonlinear interactions, offering a robust solution for spatially explicit flux modeling in structurally diverse ecosystems beyond the study sites.

footprint model↗