Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “supervised learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

A machine learning pipeline for identifying infiltration managed aquifer recharge locations from satellite imagery in the San Joaquin Valley, California

This study focuses on an agricultural region in California’s Central Valley, USA, where Managed Aquifer Recharge (MAR) is widely implemented to mitigate groundwater depletion under increasing water demand and climate variability. A deep learning and machine learning framework was developed to identify infiltration-MAR locations using satellite imagery and environmental data. The framework integrates surface water detection from Sentinel-2 imagery, geospatial delineation of water bodies, spatiotemporal tracking of water body dynamics, and supervised classification using meteorological, environmental, and topographic variables. The framework was applied to a 2379 km² study area southwest of Fresno, where 765 water bodies were detected, including 139 identified MAR sites based on publicly available datasets and expert knowledge. The classification model achieved an accuracy of 0.94 and an F1 score of 0.85. Feature importance analysis indicates that cropland, normalized difference vegetation index (NDVI), and evaporation are among the most influential predictors for infiltration-MAR. Notably, the framework suggests that engineered water management in infiltration-MAR systems can disrupt or even reverse the expected positive correlation between surface water extent and precipitation. These findings provide physically interpretable insights into the characteristics of existing infiltration-MAR facilities and demonstrate the potential of the proposed framework as a reproducible, interpretable, and potentially transferable tool for data-driven infiltration-MAR identification and inventory development under growing climatic and hydrological uncertainty.

Classification↗
DOE OSTI2025

CRCNS22 Learning Rules in the Hippocampus and their Mapping to Neuromorphic Systems (Final Technical Report)

Large scale biologically-realistic computational models are key to investigating the interplay between structure and function in nervous systems, thus paving the way to new clinical methods and neuro-inspired computing solutions. This project focuses on the hippocampus, in particular the CA3-CA1 regions, due to their role in associative learning and memory, pattern separation and completion, and spatial navigation. Investigations into the neuronal organization and learning rule(s) of this circuit can shed light into how declarative memories are formed, stored, recalled and forgotten and inform computational, experimental and clinical neuroscience work. Our project aims at developing a novel data-driven methodology supported by a broad heterogeneous base of neuroscience experimental knowledge and inspired from advances in computer science and engineering. Specifically, this work will benchmark existing and new learning rules within a full-scale spiking neural network simulation of the CA3-CA1 region. The model will be based on an open-source repository, called the Hippocampome, which contains neuronal morphologies, firing patterns, synapse probabilities, and most other required parameters for all known neuron types in the rodent hippocampal formation. The model will be first trained in a supervised fashion for associative memory tasks using backpropagation through time traditionally used in computer science, enhanced with a new technique called the surrogate gradient method. This optimization method will be used to obtain a global loss minimization, but it is not biologically inspired as it assumes the use of data not locally available to the synapses. However, we propose its use as a benchmarking tool, to compare the training performance of local biologically plausible and hardware-mappable learning rules at scale. New rules or combinations will be proposed and tested as needed, based on the obtained results. Progress in this area will also drive the development of novel hardware-mappable algorithms for continual lifelong learning and categorization of new events from few presented examples. This project goes beyond the existing state-of-the-art by looking at large scale realistic neuronal circuits as networks trainable via global optimization methods such as surrogate gradient descent. The objective function of the brain that supports learning is largely unknown, but it is likely that it operates through local learning rules. Studying network trajectories around local minima as proposed in this work represents a useful strategy for understanding whether a network is training by using a specific (set of) learning rule(s). Starting from a completely untrained network is a challenging test since it is difficult to determine how the learning rule affects the trajectory of the network. This interdisciplinary project will help understand what rule governs learning in these regions or if multiple learning rules are involved. The work will develop a robust methodology to measure if the network is converging to the target solution, oscillating around it, or diverging away.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Machine Learning for Mapping Multipactor Susceptibility in RF Systems: Capabilities and Generalization Constraints

Multipactor is a surface-driven electron avalanche phenomenon that degrades the performance and reliability of radio-frequency (RF) systems in particle accelerator and vacuum electronics applications. Multipactor behavior in a given device structure is conventionally assessed through susceptibility charts, which provide a parameter-space characterization of the instability. In this work, we assess the capabilities of machine-learning (ML) models to learn and predict such susceptibility charts and analyze the constraints governing their generalization across materials. Using a simulation-derived dataset spanning six distinct secondary-electron-yield material profiles in a canonical two-surface planar geometry, we train supervised regression models and artificial neural networks to predict the time-averaged electron growth rate, δavg, across the relevant parameter space. Model performance is evaluated using metrics that explicitly probe the structure of susceptibility charts, including Intersection over Union, Structural Similarity Index, and correlation analysis. Tree-based ensemble models outperform neural-network models in reconstructing susceptibility regions and in generalizing across material domains. Principal-component analysis reveals disjoint material feature distributions, indicating that the piecewise mode structure of multipactor susceptibility is difficult to represent with a single global model and that generalization is constrained by data coverage rather than by model complexity. An exhaustive reduced-coverage study further shows that sparse material-space coverage can yield mean performance in the same general range but producing large variability in the susceptibility-region overlap. These results clarify the capabilities of ML-based surrogate models for parameter-space characterization of multipactor discharge. They also provide guidance for their appropriate use in RF system design.

43 PARTICLE ACCELERATORS↗
DOE OSTI2023

Plug & play directed evolution of proteins with gradient-based discrete MCMC

Abstract A long-standing goal of machine-learning-based protein engineering is to accelerate the discovery of novel mutations that improve the function of a known protein. We introduce a sampling framework for evolving proteins in silico that supports mixing and matching a variety of unsupervised models, such as protein language models, and supervised models that predict protein function from sequence. By composing these models, we aim to improve our ability to evaluate unseen mutations and constrain search to regions of sequence space likely to contain functional proteins. Our framework achieves this without any model fine-tuning or re-training by constructing a product of experts distribution directly in discrete protein space. Instead of resorting to brute force search or random sampling, which is typical of classic directed evolution, we introduce a fast Markov chain Monte Carlo sampler that uses gradients to propose promising mutations. We conduct in silico directed evolution experiments on wide fitness landscapes and across a range of different pre-trained unsupervised models, including a 650 M parameter protein language model. Our results demonstrate an ability to efficiently discover variants with high evolutionary likelihood as well as estimated activity multiple mutations away from a wild type protein, suggesting our sampler provides a practical and effective new paradigm for machine-learning-based protein engineering.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Automatic speech recognition predicts contemporaneous earthquake fault displacement

Abstract Significant progress has been made in probing the state of an earthquake fault by applying machine learning to continuous seismic waveforms. The breakthroughs were originally obtained from laboratory shear experiments and numerical simulations of fault shear, then successfully extended to slow-slipping faults. Here we apply the Wav2Vec-2.0 self-supervised framework for automatic speech recognition to continuous seismic signals emanating from a sequence of moderate magnitude earthquakes during the 2018 caldera collapse at the Kīlauea volcano on the island of Hawai’i. We pre-train the Wav2Vec-2.0 model using caldera seismic waveforms and augment the model architecture to predict contemporaneous surface displacement during the caldera collapse sequence, a proxy for fault displacement. We find the model displacement predictions to be excellent. The model is adapted for near-future prediction information and found hints of prediction capability, but the results are not robust. The results demonstrate that earthquake faults emit seismic signatures in a similar manner to laboratory and numerical simulation faults, and artificial intelligence models developed for encoding audio of speech may have important applications in studying active fault zones.

58 GEOSCIENCES↗
DOE OSTI2024

Evaluating the Use of Foundational Chemical Language Models in Multimodal Graph Fusion

Rapid and accurate prediction of the physicochemical properties of molecules given their structures remains a key challenge in cheminformatics. Machine learning approaches offer high-throughput options, but the optimality of inductive biases and data representations are up for debate. For example, BERT-based masked language models (MLMs) can be trained in a self-supervised way on hundreds of millions to billions of readily available SMILES strings. Another option is graph neural networks (GNNs), which can operate directly on molecular structures. Yet, generating accurate molecular geometry is computationally expensive, leading to a relative scarcity in data compared to SMILES strings. It is attractive to combine these two paradigms by pre-training an LM on a large corpus of SMILES strings and embedding these representation into a geometric graph neural network. Despite the promise of such an approach, and contrary to previous studies, we find mixed results with the combination of the LMs and GNNs on several molecule datasets. In particular, we found evidence for improvement on the FreeSolv and QM7 benchmarks, but degraded performance on the ESOL, LIPO and QM9 datasets compared to a GNN baseline.

Francel, Collin [University of Alabama]↗
DOE OSTI2022

Use of Machine Learning on PMU Data for Transmission System Fault Analysis

Synchrophasor technology has been used for monitoring, control, and protection of bulk power system for over 10 years. Deployment of phasor measurement units (PMUs) in the USA power system has surpassed 3000 units installed in the transmission substations as stand-alone intelligent electronic devices (IEDs) or as a software add-on to other devices such as digital protective relays (DPRs) or digital fault recorders (DFRs). By now, thousands of terabytes of PMU data may have been captured and stored by various transmission system operators (TSOs) and independent system operators (ISOs). This creates an opportunity to deploy advanced machine learning (ML) techniques to detect and classify faults recorded by PMUs automatically to be used by the system operators for rapid, critical decision-making when manual analysis of the past or unfolding events is not feasible. In this paper we offer a brief background on how the automated fault analysis may be done using DPR and/or DFR data, and compare some of the legacy approaches to the new ML approaches in the context of the system-wide PMU recordings. We then offer insights from developing practical ML solutions that have been applied on field recordings captured by close to 450 PMUs from all three US interconnections (Western, Eastern and ERCOT) over two years (2016-2017). We identify and illustrate ML challenges we addressed: inaccurate data, data with scarce and temporally imprecise fault labels, data recorded by PMUs sparsely located at substations resulting in the fault records taken afar from the ends of the faulted lines, data containing only positive sequence values, and data taken at different voltage levels. We then illustrate the ML model results for fault analysis under different application scenarios. The novelty of this study is not only in the design, implementation, and performance analysis of the ML algorithms, but also in the use of advanced fault modelling and simulation approaches to improve the training results when developing supervised ML models for fault detection and classification. Extensive simulations of faults were conducted on a 14-bus power system to create a training dataset with over 1400 accurately labelled faults. This dataset was applied to enhance the accuracy of fault detection and classification of machine learning-based models trained with small number of labelled faults in large datasets recorded in the grid interconnections ranging from 5,000 to 70,000 buses.

Synchrophasors, Machine Learning, Fault Analysis, ↗
DOE OSTI2022

Recent Advances toward Efficient Calculation of Higher Nuclear Derivatives in Quantum Chemistry

In this article, we provide an overview of state-of-the-art techniques that are being developed for efficient calculation of second and higher nuclear derivatives of quantum mechanical (QM) energy. Calculations of nuclear Hessians and anharmonic terms incur high costs and memory and scale poorly with system size. Three emerging classes of methods—machine learning (ML), automatic differentiation (AD), and matrix completion (MC)—have demonstrated promise in overcoming these challenges. We illustrate studies that employ unsupervised ML methods to reduce the need for multiple Hessian calculations in dynamics simulations and those that utilize supervised ML to construct approximate potential energy surfaces and estimate Hessians and anharmonic terms at reduced cost. By extension, if electronic structure operations could be written in a manner similar to functions underlying ML methods, rapid differentiation or AD routines can be employed to inexpensively calculate higher arbitrary-order derivatives. While ML approaches are typically black-box, we describe methods such as compressed sensing (CS) and MC, which explicitly leverage problem-specific mathematical properties of higher derivatives such as sparsity and low-rank, to complete higher derivative information using only a small, incomplete sample. The three classes of methods facilitate reliable predictions of observables ranging from infrared spectra to thermal conductivity and constitute a promising way forward in accurately capturing otherwise intractable higher-order responses of QM energy to nuclear perturbations.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
DOE OSTI2024

Multivariate Time Series Intermittent Fault Detectionin Controller Area Network CAN

Fault detection in Controller Area Network (CAN) systems is crucial for ensuring the reliability and safety of automotive and industrial applications. This study investigates and compares the effectiveness of time series classification models for supervised fault detection in CAN data. This repository contains the code and data for our benchmarking experiment aimed at detecting intermittent faults in automotive Controller Area Network (CAN) data. The goal of this project is to compare various machine learning (ML) and deep learning (DL) models using different Time Series Cross-Validation (TSCV) techniques to evaluate their effectiveness in a streaming environment for fault detection.

Hespeler, Steven [Oak Ridge National Laboratory (O↗
DOE OSTI2026

Regularizing INR with Diffusion Prior for Self-Supervised 3D Reconstruction OF Neutron Computed Tomography Data

Recently, generative diffusion priors have made huge strides as inverse problem solvers, including the ability to be adapted for inference on out-of-distribution data. Concurrently, implicit neural representations (INRs) have emerged as fast and lightweight inverse imaging solvers that are amenable to hybrid approaches that combine learned priors with traditional inverse problem formulations. In this paper, we present a diffusive computed tomography (CT) inversion framework for regularizing INRs called Diffusive INR (DINR), designed to enable high-quality reconstruction from sparse-view neutron CT. Pretrained purely on synthetic data, DINR is evaluated on simulated and experimentally obtained observations of concrete microstructures, where traditional reconstruction methods suffer substantial degradation when the number of views is reduced. Our approach delivers superior performance, reduces reconstruction artifacts, and achieves gains in PSNR and SSIM, enabling accurate micro-structural characterization even under extreme data limitations compared to state-of-the-art sparse-view reconstruction techniques.

Hossain, Maliha [ORNL]↗
DOE OSTI2024

Predicting non-linear stress–strain response of mesostructured cellular materials using supervised autoencoder

Recent breakthroughs in advanced manufacturing capabilities have made it possible to design and print sophisticated topologies of cellular structures using diverse engineering materials such as metals, polymers, and ceramics. In these architectured materials, it is often desirable to tailor the mechanical properties by altering the unit cell topology. This necessitates an in-depth understanding of how the topology of the unit cell structure affects the macroscopic behavior of the material in both the linear and the non-linear regimes encountered under large compression. Here, we have developed a machine learning (ML) approach capable of accelerating the prediction of the stress–strain response of a polymer-based cellular structure under uniaxial confined compression. As part of generating the training data for ML, 60,000 mesostructures were generated using a relatively novel approach based on cellular automata, and their corresponding stress–strain responses were obtained from the finite element simulations. Principal component analysis (PCA) was used to reduce the dimensionality of the stress–strain curves. With only 20 principal components, PCA captured 99.89% of the variance in the stress–strain curves while reducing the dimensionality by 5X. ML using supervised autoencoder was able to successfully speed up the prediction of the non-linear stress–strain response of a unit cell by up to 4600X. The proposed method can serve as an efficient data generation tool and a rapid means for predicting the structure–property relationship through accelerated forward modeling of cellular materials under compaction, in cases where the macroscopic stress–strain response is governed by the unit-cell topology.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Model metamers reveal divergent invariances between biological and artificial neural networks

Deep neural network models of sensory systems are often proposed to learn representational transformations with invariances like those in the brain. To reveal these invariances, we generated ‘model metamers’, stimuli whose activations within a model stage are matched to those of a natural stimulus. Metamers for state-of-the-art supervised and unsupervised neural network models of vision and audition were often completely unrecognizable to humans when generated from late model stages, suggesting differences between model and human invariances. Targeted model changes improved human recognizability of model metamers but did not eliminate the overall human–model discrepancy. The human recognizability of a model’s metamers was well predicted by their recognizability by other models, suggesting that models contain idiosyncratic invariances in addition to those required by the task. Metamer recognizability dissociated from both traditional brain-based benchmarks and adversarial vulnerability, revealing a distinct failure mode of existing sensory models and providing a complementary benchmark for model assessment.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Automated phase segmentation and quantification of high-resolution TEM image for alloy design

In the alloy design and development process, a wealth of atomically resolved structural high-resolution transmission electron microscopy (HRTEM) images are produced. Identifying the different nano-precipitate phases and tracking their evolution under various compositions and during manufacturing or post-processing requires hundreds of HRTEM images and thousands of precipitates. The nanoscopic phase information labeling and analysis purely relies on humans are prohibitively costly and time-consuming, sometimes not reliable because of the lack of authoritative knowledge. Here, in this work, we develop a novel unsupervised machine learning approach coupled with adaptive computer vision techniques with features in the Fourier space to automatically determine the number of phases and segment/quantify the phases with nanoscale resolution, allowing for quantitative correlation between nanostructure formation, processing and functional properties. To automate the phase extraction/quantification and ascertain its applicability, we have applied the developed framework to the HRTEM images from several alloy systems, processing conditions, image magnifications, and phase types and morphologies (precipitates, nano-twins, stacking faults, crystalline matrix, and amorphous structures) for verification. This study paves the road for compression, visualization, and translation of raw image structural data into physically relevant information in real-time with minimal human supervision. It shows the promise of enabling high-throughput materials characterization for the acceleration of alloy manufacturing and design.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Training Spiking Neural Networks with Synaptic Plasticity under Integer Representation

Neuromorphic computing is emerging as a promising Beyond Moore computing paradigm that employs event-triggered computation and non-von Neumann hardware. Spike Timing Dependent Plasticity (STDP) is a well-known bio-inspired learning rule that relies on activities of locally connected neurons to adjust the weights of their respective synapses. In this work, we analyze a basic STDP rule and its sensitivity on the different hyperparameters for training spiking neural networks (SNNs) with supervision, customized for a neuromorphic hardware implementation with integer weights. We compare the classification performance on four UCI datasets (iris, wine, breast cancer and digits) that depict varying levels of complexity. We perform a search for optimal set of hyperparameters using both grid search and Bayesian optimization. Through the use of Bayesian optimization, we show the general trends in hyperparameter sensitivity in SNN classification problem. With the best sets of hyperparameters, we achieve accuracies comparable to some of the best performing SNNs on these four datasets. With a highly optimized supervised STDP rule we show that these accuracies can be achieved with just 20 epochs of training.

Kulkarni, Shruti↗
DOE OSTI2022

GeoThermalCloud: Machine Learning for Geothermal Resource Exploration

Geothermal is a renewable energy source that can provide reliable and flexible electricity generation for the world. In the past decade, the U.S. Geological Survey's resource assessments, Play Fairway Analyses (PFA), and GeoVision report by the U.S. Department of Energy's Geothermal Technologies Office provided insights on enormous untapped potential for geothermal energy to contribute to the U.S. domestic energy needs. The past studies identified that geothermal resources without surface expression (e.g., blind/hidden hydrothermal systems) comprise a huge potential. These blind systems can significantly increase power generation. But a primary challenge is locating and quantifying these hidden resources, which do not have any thermal manifestations on the surface. PFA has successfully identified some blind systems in the western USA (e.g., specific locations in the Great Basin region within Nevada). However, a comprehensive search for these blind systems can be time-consuming, expensive, and resource-intensive with a low probability of success. Accelerated discovery of these blind resources is needed with growing energy needs and higher chances of exploration success. Recent advances in machine learning (ML) have shown promise in shortening the timeline for this discovery. This paper presents a novel ML-based methodology for geothermal exploration towards PFA applications. Our methodology is provided through our open-source ML framework called GeoThermalCloud \url{https://github.com/SmartTensors/GeoThermalCloud.jl}. GeoThermalCloud uses a series of unsupervised, supervised, and physics-informed ML methods available in SmartTensors AI platform \url{https://github.com/SmartTensors}. Here, the presented analyses are performed using our unsupervised ML algorithm called NMF$k$, which is available in the SmartTensors AI platform. Our ML algorithm facilitates the discovery of new phenomena, hidden patterns, and mechanisms that helps us to make informed decisions. Moreover, the GeoThermalCloud enhances the collected PFA data and discovers signatures representative of geothermal resources. Through GeoThermalCloud, we were able to identify hidden patterns in the geothermal field data needed for the efficient discovery of blind systems. Crucial geothermal signatures often overlooked in traditional PFA are extracted using GeoThermalCloud and analyzed by the subject matter experts to provide ML-enhanced PFA, which is informative for efficient exploration. We applied our ML methodology on various open-source geothermal datasets within the U.S. (some of these are collected by past PFA work), and the results provide valuable insights on resource types within those explored regions. This ML-enhanced workflow makes GeoThermalCloud attractive for the geothermal community to improve existing datasets and extract valuable information often unnoticed during geothermal exploration.

machine learning (ML), geothermal energy↗
DOE OSTI2026

ResSR: A Computationally Efficient Residual Approach to Super-Resolving Multispectral Images

Multispectral imaging (MSI) plays a critical role in material classification, environmental monitoring, and remote sensing. However, MSI sensors typically have wavelength-dependent resolution, which limits downstream analysis. MSI super-resolution (MSI-SR) methods address this limitation by reconstructing all bands at a common high spatial resolution. Existing methods can achieve high reconstruction quality but often rely on spatially-coupled optimization or large learning-based models, leading to significant computational cost and limiting their use in large-scale or time-critical settings. In this paper, we introduce ResSR, a computationally efficient, model-based MSI-SR method that achieves high-quality reconstruction without supervised training or spatially-coupled optimization. Notably, ResSR decouples spectral and spatial processing into two sequential steps. ResSR first computes a spectrally-informed high-resolution estimate of the MSI using singular value decomposition together with a spatially-decoupled approximate forward model. It then applies a residual correction step to restore low-frequency spatial consistency while preserving high-frequency detail recovered by the spectral reconstruction. ResSR achieves comparable or improved reconstruction quality relative to existing MSI-SR methods while being

Sullivan, Haley [ORNL] (ORCID:0000000274069217)↗