Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Residual learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Gearbox bearing crack growth prognostics and uncertainty quantification with physics-informed machine learning

This paper introduces the extreme theory of functional connections (X-TFC), a physics-informed machine learning algorithm, and tailors it to estimate the remaining useful life (RUL) of wind turbine gearbox bearings experiencing fatigue crack growth. Unlike purely data-driven methods, X-TFC embeds a physics model, based on Head's theory in this work, into its training objective. The core of X-TFC is a random-projection single-layer neural network trained via an extreme learning machine, which requires only limited damage progression data and solves for output weights with a least-squares optimization algorithm. A composite loss function balances the network's fit to observed degradation data against the residuals of the governing crack growth differential equation, ensuring the learned damage trajectory remains physically plausible. When applied to a vibration-based health-index (HI) dataset measured during the growth of a crack on the inner ring of a high-speed bearing in a wind turbine gearbox (Bechhoefer and Dubé, 2020), X-TFC achieves near-zero prediction bias. Even when trained on only the first 10 %–20 % of the damage progression data, with sufficient physics weighting its predictions remain monotonic and smooth, delivering high prognosability and trendability. To quantify the epistemic uncertainty, we employ a Monte Carlo ensemble of independently initialized X-TFC models trained on noise-perturbed data, which yields confidence intervals around each RUL estimate and captures both model-parameter and epistemic uncertainty. In addition to a vibration-based HI, we demonstrate that the proposed framework can be directly applied to a supervisory control and data acquisition (SCADA) data-based HI (Eftekhari Milani et al., 2026) measured during similar wind turbine gearbox bearing crack faults, preserving its accuracy and interpretability. This extension shows the versatility of our approach, which is applicable to bearings of multiple gearbox manufacturers, models, and ratings using only SCADA data. By integrating domain knowledge with machine learning, X-TFC offers a rapid, reliable tool for crack prognostics. Its adaptability to other bearing failure modes, such as pitch bearing ring cracks, positions X-TFC as a powerful enabler of data-driven, physics-informed asset management in the wind energy sector and beyond.

17 WIND ENERGY↗
DOE OSTI2021

Machine learning reveals sequence-function relationships in family 7 glycoside hydrolases

Family 7 glycoside hydrolases (GH7) are among the principal enzymes for cellulose degradation in nature and industrially. These important enzymes are often bimodular, including a catalytic domain and carbohydrate binding module (CBM) attached via a flexible linker, and exhibit an active site that binds cello-oligomers of up to ten glucosyl moieties. GH7 cellulases consist of two major subtypes: cellobiohydrolases (CBH) and endoglucanases (EG). Despite the critical importance of GH7 enzymes, there remain gaps in our understanding of how GH7 sequence and structure relate to function. Here, we employed machine learning to gain data-driven insights into relationships between sequence, structure, and function across the GH7 family. Machine-learning models, trained only on the number of residues in the active-site loops as features, were able discriminate GH7 CBHs and EGs with up to 99% accuracy, demonstrating that the lengths of loops A4, B2, B3, and B4 strongly correlate with functional subtype across the GH7 family. Classification rules were derived such that specific residues at 42 different sequence positions each predicted the functional subtype with accuracies surpassing 87%. A random forest model trained on residues at 19 positions in the catalytic domain predicted the presence of a CBM with 89.5% accuracy. Our ML results recapitulate, as top-performing features, a substantial number of the sequence positions determined by previous experimental studies to play vital roles in GH7 activity. We surmise that the yet-to-be-explored sequence positions among the top-performing features also contribute to GH7 functional variation and may be exploited to understand and manipulate function.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Nuclear Canister Corrosion Detection

The software titled "nccd" is used for running residual neural networks (ResNets) on images of nuclear canisters. The software provides only the code for the implementation (based on the fastai library), but it does not share the image data. The compute code allows the use of residual nets (and more generally of other deep learning models) for classifying images from nuclear canisters as corroded. or intact. An image is considered to be corroded if it contains pitting or stress corrosion cracks. The code trains ResNets on image tiles extracted from original images from nuclear canisters, and then implements a classification rule, which can be applied to a validation set to decide if each original image is corroded or intact. The software automates the process of using images taken from nuclear canisters to detect corrosion. It provides also scope for future research directions on the basis of the existing research and code.

Papamarkou, Theodore↗
DOE OSTI2023

Experimental study of the S 38 excited level scheme

Information on the 38 S level scheme was expanded through experimental work utilizing a fusion-evaporation reaction and in-beam γ-ray spectroscopy. Prompt γ-ray transitions were detected by the Gamma-Ray Energy Tracking Array (GRETINA) and recoiling 38 S residues were selected by the Fragment Mass Analyzer (FMA). Tools based on machine-learning techniques were developed and deployed for the first time in order to enhance the unique selection of 38S residues and identify any associated γ-ray transitions. The new level information, including the extension of the even-spin yrast sequence through J π =8( + ), was interpreted in terms of a basic single-particle picture as well shell-model calculations which incorporated the empirically derived FSU interaction. A comparison between the properties of the yrast states in the even-Z, N = 22 isotones from Z = 14 to 20, and for 36 Si– 38 S in particular, was also presented with an emphasis on the role and influence of the neutron 1 p 3/2 orbital on the structure in the region.

20 ≤ A ≤ 38↗
DOE OSTI2020

Layer-Parallel Training of Deep Residual Neural Networks

Residual neural networks (ResNets) are a promising class of deep neural networks that have shown excellent performance for a number of learning tasks, e.g., image classification and recognition. Mathematically, ResNet architectures can be interpreted as forward Euler discretizations of a nonlinear initial value problem whose time-dependent control variables represent the weights of the neural network. Hence, training a ResNet can be cast as an optimal control problem of the associated dynamical system. For similar time-dependent optimal control problems arising in engineering applications, parallel-in-time methods have shown notable improvements in scalability. This paper demonstrates the use of those techniques for efficient and effective training of ResNets. The proposed algorithms replace the classical (sequential) forward and backward propagation through the network layers with a parallel nonlinear multigrid iteration applied to the layer domain. This adds a new dimension of parallelism across layers that is attractive when training very deep networks. From this basic idea, we derive multiple layer-parallel methods. The most efficient version employs a simultaneous optimization approach where updates to the network parameters are based on inexact gradient information in order to speed up the training process. Finally, using numerical examples from supervised classification, we demonstrate that the new approach achieves a training performance similar to that of traditional methods, but enables layer-parallelism and thus provides speedup over layer-serial methods through greater concurrency.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Fault Detection on Seismic Structural Images Using a Nested Residual U-Net

Automatic identification of faults on seismic structural images is a challenging yet crucial task in quantitative seismic interpretation. Human picking or attribute-based fault detection methods may misidentify faults on noisy, complex seismic images. In this work, we develop a new automatic fault detection method using a nested residual U-shaped convolutional neural network. Each of the encoders and decoders in this neural network is a residual U-Net, leading to a nested architecture. The final fault map results from the fusion of three fault maps with low, medium, and high fault resolutions. We demonstrate the excellent fault-detection capability of our nested neural network using a series of synthetic and field seismic images. We find that our approach produces clearer and more interpretable fault maps than the current state-of-the-art U-Net fault detection method, particularly on noisy seismic images. Our new automatic fault detection method can facilitate reliable quantitative seismic interpretation on field seismic images.

58 GEOSCIENCES↗
DOE OSTI2026

Data for A Hybrid Biophysical-Machine Learning Framework for Diurnal Surface Energy Flux Estimation Using Proximal Sensing

Thermal infrared-based remote sensing of surface energy fluxes has traditionally relied on high spatial resolution satellite data with revisit frequencies on the order of weeks. In this study, we evaluate a biophysics-based analytical surface energy balance model for predicting latent energy (LE) and sensible heat (H) fluxes using proximal sensing observations. The Surface Temperature Initiated Closure (STIC1.2) model has been extensively validated across a wide range of spatial and temporal scales using various satellite-derived thermal infrared data sets. Here we extend this validation by applying STIC at sub-hourly temporal resolution over multiple growing seasons for four distinct agricultural systems. We further develop and evaluate novel STIC variants that incorporate machine learning (ML) techniques to eliminate the need for surface energy balance observations, specifically net radiation and soil heat flux, thereby enhancing model applicability in data-sparse settings. The integration of a ML component to estimate surface available energy is shown to have strong predictive performance for both LE (R2 = 0.81–0.94) and H (R2 = 0.46–0.72) across all agricultural systems examined here, demonstrating the potential of hybrid biophysical-machine learning approaches for surface energy balance modeling with minimal data requirements. This study concludes with a novel application of explainable machine learning (exML) to diagnose sources of model error. This exML framework attributes residual prediction errors to both model input variables and environmental drivers not explicitly included in the simulation experiments. This approach provides a new pathway for improving model design and integrating previously overlooked yet influential variables into future model iterations.

AI/ML↗
DOE OSTI2024

Monitoring covariance in multivariate time series: Comparing machine learning and statistical approaches

Abstract In complex systems with multiple variables monitored at high‐frequency, variables are not only temporally autocorrelated, but they may also be nonlinearly related or exhibit nonstationarity as the inputs or operation changes. One approach to handling such variables is to detrend them prior to monitoring and then apply control charts that assume independence and stationarity to the residuals. Monitoring controlled systems is even more challenging because the control strategy seeks to maintain variables at prespecified mean levels, and to compensate, correlations among variables may change, making monitoring the covariance essential. In this paper, a vector autoregressive model (VAR) is compared with a multivariate random forest (MRF) and a neural network (NN) for detrending multivariate time series prior to monitoring the covariance of the residuals using a multivariate exponentially weighted moving average (MEWMA) control chart. Machine learning models have an advantage when the data's structure is unknown or may change. We design a novel simulation study with nonlinear, nonstationary, and autocorrelated data to compare the different detrending models and subsequent covariance monitoring. The machine learning models have superior performance for nonlinear and strongly autocorrelated data and similar performance for linear data. An illustration with data from a reverse osmosis process is given.

Weix, Derek↗
DOE OSTI2022

Frhodo: A program for simulating chemical kinetic measurements and optimizing kinetic mechanisms

Chemical kinetic simulations are frequently used to extract kinetic and mechanistic information from experimental data by fitting simulations to the data. For simple systems of consecutive single channel reactions this is a relatively straightforward task. However, as the complexity of the system increases and particularly for systems involving competitive multi-channel reactions manually optimizing a mechanism to obtain the best fits to a range of experimental data quickly becomes a time-intensive and challenging process, especially when rate coefficients have pressure and temperature dependencies. There is considerable potential for exploiting automated optimization methods to rapidly screen reaction mechanisms and optimize the fit by adjusting rate coefficients within well-defined constraints. A new chemical kinetics simulation program, Frhodo, has been developed for this purpose. The program allows either manual simulation of individual experiments or fully automated optimization against a range of experimental data by adjusting user selected reactions within user defined constraints. Frhodo incorporates machine learning-based optimization routines that work by either minimizing standardized residuals or through Bayesian parameter estimation. Frhodo's optimization capabilities are demonstrated by reexamining two previously studied systems of relevance to combustion. Dissociation of diacetyl followed by recombination of methyl radicals is an example of a sequence of single channel, consecutive reactions. Pyrolysis of 2-methyl furan exemplifies multi-channel, unimolecular reactions that are often encountered and have temperature and pressure dependencies in the rate coefficients of the channels and branching ratios between them. For both systems the optimization routines resulted in solutions similar to the original studies. Finally, comments are made on the strengths and limitations of the approach.

0D simulation↗
DOE OSTI2025

Self-adaptive weights based on balanced residual decay rate for physics-informed neural networks and deep operator networks

Physics-informed deep learning has emerged as a promising alternative for solving partial differential equations. However, for complex problems, training these networks can still be challenging, often resulting in unsatisfactory accuracy and efficiency. In this work, we demonstrate that the failure of plain physics-informed neural networks arises from the significant discrepancy in the convergence rate of residuals at different training points, where the slowest convergence rate dominates the overall solution convergence. Based on these observations, we propose a pointwise adaptive weighting method that balances the residual decay rate across different training points. The performance of our proposed adaptive weighting method is compared with current state-of-the-art adaptive weighting methods on benchmark problems for both physics-informed neural networks and physics-informed deep operator networks. In conclusion, through extensive numerical results we demonstrate that our proposed approach of balanced residual decay rates offers several advantages, including bounded weights, high prediction accuracy, fast convergence rate, low training uncertainty, low computational cost, and ease of hyperparameter tuning.

Balanced convergence rate↗
DOE OSTI2024

Surrogate construction via weight parameterization of residual neural networks

Surrogate model development is a critical step for uncertainty quantification or other sample-intensive tasks for complex computational models. Here, in this work, we develop a multi-output surrogate form using a class of neural networks (NNs) that employ shortcut connections, namely Residual NNs (ResNets). ResNets are known to regularize the surrogate learning problem and improve the efficiency and accuracy of the resulting surrogate. Inspired by the continuous, Neural ODE analogy, we augment ResNets with weight parameterization strategy with respect to ResNet depth. Weight-parameterized ResNets regularize the NN surrogate learning problem and allow better generalization with a drastically reduced number of learnable parameters. We demonstrate that weight-parameterized ResNets are more accurate and efficient than conventional feed-forward multi-layer perceptron networks. We also compare various options for parameterization of the weights as functions of ResNet depth. We demonstrate the results on both synthetic examples and a large scale earth system model of interest.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

NEAR: Neural Embeddings for Amino acid Relationships

Protein language models (PLMs) have recently demonstrated potential to supplant classical protein database search methods based on sequence alignment, but are slower than common alignment-based tools and appear to be prone to a high rate of false labeling. Here, we present NEAR, a method based on neural representation learning that is designed to improve both speed and accuracy of search for likely homologs in a large protein sequence database. NEAR’s ResNet embedding model is trained using contrastive learning guided by trusted sequence alignments. It computes per-residue embeddings for target and query protein sequences, and identifies alignment candidates with a pipeline consisting of residue-level k-NN search and a simple neighbor aggregation scheme. Tests on a benchmark consisting of trusted remote homologs and randomly shuffled decoy sequences reveal that NEAR substantially improves accuracy relative to state-of-the-art PLMs, with lower memory requirements and faster embedding and search speed. While these results suggest that the NEAR model may be useful for standalone homology detection with increased sensitivity over standard alignment-based methods, in this manuscript we focus on a more straightforward analysis of the model’s value as a high-speed pre-filter for sensitive annotation. In that context, NEAR is at least 5x faster than the pre-filter currently used in the widely-used profile hidden Markov model (pHMM) search tool HMMER3, and also outperforms the pre-filter used in our fast pHMM tool, nail.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Machine Learning Reveals the Critical Interactions for SARS-CoV-2 Spike Protein Binding to ACE2

SARS-CoV and SARS-CoV-2 bind to the human ACE2 receptor in practically identical conformations, although several residues of the receptor-binding domain (RBD) differ between them. Herein, we have used molecular dynamics (MD) simulations, machine learning (ML), and free-energy perturbation (FEP) calculations to elucidate the differences in binding by the two viruses. Although only subtle differences were observed from the initial MD simulations of the two RBD–ACE2 complexes, ML identified the individual residues with the most distinctive ACE2 interactions, many of which have been highlighted in previous experimental studies. FEP calculations quantified the corresponding differences in binding free energies to ACE2, and examination of MD trajectories provided structural explanations for these differences. Lastly, the energetics of emerging SARS-CoV-2 mutations were studied, showing that the affinity of the RBD for ACE2 is increased by N501Y and E484K mutations but is slightly decreased by K417N.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Development of a plug-and-play anti-noise module for fault diagnosis of rotating machines in nuclear power plants

The health of rotating machines is crucial to the stable operation and safety of nuclear power plants. However, research on machine learning-based fault diagnosis of rotating machines in the nuclear industry is still in its infancy. The signal noise generated in the plant may negatively affect the effectiveness of the analysis. In this paper, a plug-and-play anti-noise machine learning module is proposed to fill the knowledge and capability gap. The modules are loaded into a convolutional neural network called deep residual network (ResNet) to obtain a new model with noise reduction capability. The basic idea is that the module is able to identify noise features and include them in the subsequent analysis, effectively filtering noise at the feature level of the network. Nine variants of the new model are compared with the original ResNet as well as four classical machine learning models to test the effectiveness of the module and to examine the impact of the module's loading modes on the performance of the new model. Finally, this research helps facilitate the application of machine learning in the plant noise environment.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2020

Challenges and Solutions in Operation and Lay-Up of the Modular Caustic- Side Solvent Extraction Unit - 20078

The Modular Caustic-Side Solvent Extraction (CSSX) Unit (MCU) was designed and constructed to provide interim salt waste processing so that the Liquid Waste Disposition Program (LWDP) could continue waste removal and tank closure until start-up of the Salt Waste Processing Facility (SWPF). MCU processes Clarified Salt Solution (CSS), or salt solution (SS) that has undergone actinide removal/filtration, to produce two waste streams: the cesium laden Strip Effluent (SE), which is incorporated into glass at the Defense Waste Processing Facility (DWPF), and the cesium depleted Decontaminated SS (DSS), which is incorporated into grout at the Saltstone Production Facility (SPF). MCU completed start-up in 2008 with an initial operating life of 3 years and design life of 5 years. Prior to reaching the end of the design life, multiple critical components were upgraded and/or replaced to mitigate risks associated with any delays in SWPF start-up. During the remainder of operation (until June 2019), MCU was challenged to maximize processing with minimal additional modifications to repair/improve aging infrastructure, while maintaining low levels of risk to workers and the environment. Notable challenges during the extended operating life of the facility involved both mechanical and operational issues. Two of the most significant issues were the increased frequency of Process Vessel Ventilation (PVV) High Efficiency Particulate Air (HEPA) filter change-outs and the biofouling of the SE coalescer (SEC). The major impacts of both issues were that replacement of these components 1) resulted in high exposure to workers, and 2) required substantial downtime, thus hindering achievement of processing goals. PVV HEPA filter replacements are required based on dose rate and differential pressure (dP) limits. After introducing a higher-curie feed to MCU, the operating time between change outs was reduced by approximately half. SEC replacements are required based on dP limits. After restarting from an extended outage, the operating life of the SEC was significantly reduced by more than half due to biofouling. As a result of extensive troubleshooting and process improvements, the facility was able to recover performance and extend the operating life of the PVV HEPA filters and the SEC media. Through troubleshooting both of these issues, the facility was also able to learn from prior experience and adapt in order to minimize down time and maintain throughput. Although the facility took advantage of short periods of downtime to perform nonintrusive troubleshooting and minor corrective/preventative maintenance activities, longer outages were periodically required for corrective maintenance involving process cell entry. During these outages, remote tools and specialized shield plates were employed to minimize dose to workers. Mock-ups of non-routine maintenance activities were critical to early identification of potential issues and improvements, which shortened outage duration. Consistent with the facility's operational strategy, evolutions needed for MCU layup were identified and sequenced to ensure safe conditions within the facility, meet pre-defined criteria for lay-up configuration, and protect workers, while minimizing the impact to SWPF integration. De-inventory and flushing of process areas was prioritized so that the total residual inventory was reduced as efficiently as possible. This paper discusses lessons learned and best practices from troubleshooting unique issues, performing complex maintenance activities, and implementing a layup strategy that best supports the overall system mission. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2025

Deep Learning without Global Optimization by Random Fourier Neural Networks

Here we introduce a new training algorithm for deep neural networks that utilize random complex exponential activation functions. Our approach employs a Markov chain Monte Carlo sampling procedure to iteratively train network layers, avoiding global and gradient-based optimization while maintaining error control. It consistently attains the theoretical approximation rate for residual networks with complex exponential activation functions, determined by network complexity. Additionally, it enables efficient learning of multiscale and high-frequency features, producing interpretable parameter distributions. Despite using sinusoidal basis functions, we do not observe Gibbs phenomena in approximating discontinuous target functions.

97 MATHEMATICS AND COMPUTING↗