Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,297 records · Page 72
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

A comparison of machine learning surrogate models of street-scale flooding in Norfolk, Virginia

Low-lying coastal cities, exemplified by Norfolk, Virginia, face the challenge of street flooding caused by rainfall and tides, which strain transportation and sewer systems and can lead to personal and property damage. While high-fidelity, physics-based simulations provide accurate predictions of urban pluvial flooding, their computational complexity renders them unsuitable for real-time applications. Using data from Norfolk rainfall events between 2016 and 2018, this study compares the performance of a previous surrogate model based on a random forest algorithm with two deep learning models: Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU). The comparison of deep learning to the random forest algorithm is motivated by the desire to utilize a machine learning architecture that allows for the future inclusion of common uncertainty quantification techniques and the effective integration of relevant, multi-modal features.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

TransformerG2G: Adaptive time-stepping for learning temporal graph embeddings using transformers

Dynamic graph embedding has emerged as a very effective technique for addressing diverse temporal graph analytic tasks (i.e., link prediction, node classification, recommender systems, anomaly detection, and graph generation) in various applications. Such temporal graphs exhibit heterogeneous transient dynamics, varying time intervals, and highly evolving node features throughout their evolution. Hence, incorporating long-range dependencies from the historical graph context plays a crucial role in accurately learning their temporal dynamics. In this paper, we develop a graph embedding model with uncertainty quantification, TransformerG2G, by exploiting the advanced transformer encoder to first learn intermediate node representations from its current state (t) and previous context (over timestamps [t–1,t–l], l is the length of context). Moreover, we employ two projection layers to generate lower-dimensional multivariate Gaussian distributions as each node's latent embedding at timestamp t. We consider diverse benchmarks with varying levels of "novelty" as measured by the TEA (Temporal Edge Appearance) plots. Here, our experiments demonstrate that the proposed TransformerG2G model outperforms conventional multi-step methods and our prior work (DynG2G) in terms of both link prediction accuracy and computational efficiency, especially for high degree of novelty. Furthermore, the learned time-dependent attention weights across multiple graph snapshots reveal the development of an automatic adaptive time stepping enabled by the transformer. Importantly, by examining the attention weights, we can uncover temporal dependencies, identify influential elements, and gain insights into the complex interactions within the graph structure. For example, we identified a strong correlation between attention weights and node degree at the various stages of the graph topology evolution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

A machine learning approach for particle accelerator errant beam prediction using spatial phase deviation

Particle accelerators are extremely complex systems that are expected to operate on high availability. Predicting impending failures only by utilizing data collected from diagnostic equipment already on board can help operators to avoid installing expensive sensors, unscheduled downtime and associated costs. For this purpose we explore the predictive power of Machine Learning algorithms to detect faulty beams prior to the failure. In this study, we propose a Machine Learning approach to model mapping from a pair of sensors located across the accelerator. While the model is trained to represent normal operation, we evaluate the predictive performance on known faulty beam pulses. We also investigate the model performance on unseen data through k-fold cross-validation. Then we recap the analysis with a neural architecture search and hyperparameter optimization study to fine tune our initial model. In conclusion, this paper will also introduce a sustainable framework that can standardize Machine Learning workflow applied to particle accelerators.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Expanded analysis of machine learning models for nuclear transient identification using TPOT

Industries around the world are becoming more and more data driven. The nuclear field is no exception with several different applications being proposed. One popular area of research is the use of machine learning in transient detection. This paper seeks to build upon a previous study which made use of the AutoML package TPOT to train traditional machine learning models to classify transient events occurring with a reactor. Synthetic data was once again collected using a GPWR reactor simulator. Data on 12 different events was collected using 15 different initial conditions. Here, a dataset consisting of over 100,000 data points was compiled and used to train 7 different machine learning models using a pre-defined TPOT dictionary with 12 different preprocessing techniques. Three of the trained models were able to produce validation results in the 90s with the expanded dataset. Once the models were trained, it was possible to look into where during the simulation, misclassifications occurred. Using these three models, analysis was done to determine if TPOT could be used to train models that were effective if important features were missing. The results from this were positive with the newly trained models scoring close to the original models. Finally, to conclude this study, the three high performing models were retrained using different random states to see if there was any major variation when different states were used.

42 ENGINEERING↗
DOE OSTI2023

Predicting the superconducting critical temperature in transition metal carbides and nitrides using machine learning

Transition metal carbides and nitrides have unique mechanical and chemical characteristics. At low temperatures many of them also exhibit superconductivity, which can be controlled by substitutions into both the transition metal and carbon/nitrogen sites. To investigate the factors governing the superconducting state, we apply machine learning methods. Here we collected a dataset containing 147 materials, which was used to create a pipeline for predicting their superconducting critical temperature. When this pipeline is applied to a randomly selected test set, it shows a good performance, with of 0.82 and RMSE of 1.9 K. To explore the limits of the machine learning approach, we also use it to predict entire substitution series within the dataset. This represents a realistic test for the predictive models, which can be extremely useful when applied to new substitutions in materials systems. The performance of the pipeline in this case is much more uneven, with good predictions for some series, while for others the model shows minimal predictive power. We discuss possible reasons for these results, as well as methods to estimate the performance of machine learning on new substitution series.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2020

Time-series learning of latent-space dynamics for reduced-order model closure

In this work, we study the performance of long short-term memory networks (LSTMs) and neural ordinary differential equations (NODEs) in learning latent-space representations of dynamical equations for an advection-dominated problem given by the viscous Burgers equation. Our formulation is devised in a nonintrusive manner with an equation-free evolution of dynamics in a reduced space with the latter being obtained through a proper orthogonal decomposition. In addition, we leverage the sequential nature of learning for both LSTMs and NODEs to demonstrate their capability for closure in systems that are not completely resolved in the reduced space. We assess our hypothesis for two advection-dominated problems given by the viscous Burgers equation. We observe that both LSTMs and NODEs are able to reproduce the effects of the absent scales for our test cases more effectively than does intrusive dynamics evolution through a Galerkin projection. This result empirically suggests that time-series learning techniques implicitly leverage a memory kernel for coarse-grained system closure as is suggested through the Mori–Zwanzig formalism.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Deep Learning enabled spectral energy conversion for in situ exposure measurements

A detector-specific deep learning (DL) approach is presented for spectra-to-exposure conversion using large-format sodium iodide (NaI(Tl)) detectors deployed for in situ environmental radiation measurements in emergency response scenarios. Accurate determination of exposure from NaI spectra is challenging due to poor energy resolution, partial energy absorption, and the strong sensitivity of traditionally deployed analytical conversion methods to calibrated source geometry and pre-deployment assumptions. Here, to address these limitations, a multi-layer perceptron model was trained on a hybrid in situ /Monte Carlo dataset constructed to span a broad range of photon energies, spatial extents, and realistic deployment variability, representative of general in situ emergency response conditions. The DL model was evaluated against commonly fielded analytical approaches under matched simulation conditions, including a single-factor method, a G-function method, and a modeled pressurized ion chamber (PIC) baseline. This study was intentionally computational in scope to enable controlled, like-for-like comparisons between conversion techniques while minimizing confounding real-world variability. Comparison to the modeled PIC provides contextual benchmarking and is not intended as a field inter-comparison with deployed instruments. Across the evaluated 20 keV to 3 MeV energy range, the DL approach consistently exhibited higher accuracy and reduced variance relative to the analytical methods against a deterministically calculated exposure. This may indicate improved robustness to spectral complexity without reliance on source-, geometric-, or spectral region-specific optimization. While results do not represent real-world validation, the presented work demonstrates that deep learning may effectively learn the nonlinear detector response-to-exposure relationship for asymmetric NaI(Tl) detectors and offers a promising pathway for improving in situ exposure estimation using spectroscopic systems already integrated into initial real-time emergency response operations.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2026

Integrating very-high-resolution imagery, Sentinel-2 time-series data, and machine learning to map shrub fractional abundance across arid and semi-arid ecosystems in China

Shrub fractional abundance (SFA), the proportion of shrub cover per unit area, serves as a critical indicator of environmental aridity and ecosystem health in arid and semi-arid regions, particularly across the Mongolian steppe. However, large-scale SFA mapping in Mongolian steppe ecosystems remains challenging due to the small crown size of shrubs, their sparse distribution, and spectral overlap with coexisting low vegetation (e.g., grasses and herbs), which hinders accurate detection using coarser-resolution satellite data or traditional field surveys. To address these challenges, we developed a two-step approach that integrates very-high-resolution (VHR) imagery, time-series Sentinel-2 data, and deep learning techniques. First, we generated high-accuracy benchmark maps of individual shrub crowns from 0.5 m VHR imagery by combining manual segmentation with a hybrid deep learning framework (Dino V2 and convolutional neural networks). Second, we used these shrub crown maps as training data to build an XGBoost model for predicting SFA from 20 m Sentinel-2 time-series data, leveraging phenological information to improve estimation. We validated our approach across 70 sites (1km 2 each) in the Inner Mongolia Autonomous Region, which is representative of Mongolian steppe ecosystems. From VHR imagery, we mapped 1.31 million shrub crowns with an accuracy of R 2 = 0.92. Scaling up with Sentinel-2 data yielded regional SFA maps with an R 2 = 0.60. Further SHAP (SHapley Additive exPlanations) analysis on the developed XGBoost model revealed that phenological metrics (particularly observations in early-May, mid-July, and late-September), which distinguish shrub phenology from that of other land cover types (e.g., grasses and bare soil), were the most influential predictors of SFA. Finally, our regional SFA maps uncovered unimodal relationships between shrub distribution and climate variables, peaking at mean annual minimum temperatures near 0 °C and annual precipitation around 200 mm. Collectively, these findings demonstrate how the integration of multi-source remote sensing and machine learning can overcome historical limitations in SFA mapping, enabling accurate, spatially continuous assessments across vast Inner-Mongolian steppe ecosystems. Our framework has the potential to be applied to other steppe ecosystems and dryland ecosystems across the Mongolian steppe and beyond, offering a foundation for improved monitoring and ecological impact assessments in the face of global climate changes.

Arid and semi-arid landscapes↗
DOE OSTI2025

yProv4ML: Effortless provenance tracking for machine learning systems

The rapid growth in interest in deep learning and foundation models (FMs) in particular, has attracted the attention of a diverse range of researchers thanks to their generalization ability. However, the advent of these techniques has also brought to light the lack of transparency and rigor in the way development is pursued. In particular, the inability to determine the number of epochs and other hyperparameters in advance presents challenges in identifying the best model. To address this challenge, machine learning frameworks such as MLFlow can automate the collection of this type of information. However, these tools capture data using proprietary formats and pose little attention to lineage. This paper proposes yProv4ML, a framework that captures provenance information generated during machine learning processes in PROV-JSON format, with minimal code modification.

Machine learning↗
DOE OSTI2021

Deep Learning Segmentation of Complex Features in Atomic-Resolution Phase-Contrast Transmission Electron Microscopy Images

Phase-contrast transmission electron microscopy (TEM) is a powerful tool for imaging the local atomic structure of materials. TEM has been used heavily in studies of defect structures of two-dimensional materials such as monolayer graphene due to its high dose efficiency. However, phase-contrast imaging can produce complex nonlinear contrast, even for weakly scattering samples. It is, therefore, difficult to develop fully automated analysis routines for phase-contrast TEM studies using conventional image processing tools. For automated analysis of large sample regions of graphene, one of the key problems is segmentation between the structure of interest and unwanted structures such as surface contaminant layers. In this study, we compare the performance of a conventional Bragg filtering method with a deep learning routine based on the U-Net architecture. We show that the deep learning method is more general, simpler to apply in practice, and produces more accurate and robust results than the conventional algorithm. We provide easily adaptable source code for all results in this paper and discuss potential applications for deep learning in fully automated TEM image analysis.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Machine Learning for Materials Scientists: An Introductory Guide toward Best Practices

This Methods/Protocols article is intended for materials scientists interested in performing machine learning-centered research. Herein, we cover broad guidelines and best practices regarding the obtaining and treatment of data, feature engineering, model training, validation, evaluation and comparison, popular repositories for materials data and benchmarking data sets, model and architecture sharing, and finally publication. In addition, we include interactive Jupyter notebooks with example Python code to demonstrate some of the concepts, workflows, and best practices discussed. Overall, the data-driven methods and machine learning workflows and considerations are presented in a simple way, allowing interested readers to more intelligently guide their machine learning research using the suggested references, best practices, and their own materials domain expertise.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Surfactant-Specific AI-Driven Molecular Design: Integrating Generative Models, Predictive Modeling, and Reinforcement Learning for Tailored Surfactant Synthesis

Molecular design is a critical aspect of various scientific and industrial fields, where the properties of molecules hold significant importance. In this study, a 3-fold methodology design is presented that leverages the power of generative artificial intelligence (AI), predictive modeling, and reinforcement learning to create tailored molecules with desired properties. This model synergistically combines deep learning techniques with Self-Referencing Embedded Strings (SELFIES) molecular representation to build a generative model that generates valid molecules and a graphical neural network model that accurately forecasts molecular properties. The Variational Autoencoder (VAE) coupled with reinforcement learning helps refine molecule generation based on targeted attributes. Data from an experimental study involving surfactants were used to test the framework. A validation of the structural integrity of the molecules generated was conducted, and Tanimoto similarities were used to quantify the similarity and diversity between the original and generated molecular structures. Also, saliency maps for the generated surfactants were produced to identify the features explaining the property values. Lastly, molecular dynamics simulations were used to validate the stability of the generated molecules. The results showed that the proposed framework can effectively produce valid molecules within the set property threshold value.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Toward a Diverse Next-Generation Energy Workforce: Teaching Artificial Photosynthesis and Electrochemistry in Elementary Schools through Active Learning

Artificial photosynthesis is a promising approach to generate commodity chemicals using abundant chemical feedstocks and renewable energy sources. Despite its importance, affordable and effective hands-on classroom activities that demonstrate artificial photosynthesis and teach key concepts, especially for primary school students, are lacking. Educating young students on this topic is a critical step in the development of the next-generation energy workforce, especially one that is diverse in race and gender. Here, we hypothesize that an effective approach to educate a broad range of young students on the topic of artificial photosynthesis is through the use of an active learning-based lesson plan that employs cheap and accessible materials. This hypothesis is confirmed by evaluating the understanding of fifth grade students with a survey before and after a lesson plan on artificial photosynthesis that uses active-learning techniques and uses safe and highly accessible materials (baking soda, tap water, plastic jars, Ni coil, alligator clips, and a solar cell) to perform solar-powered water splitting. The lesson plan and survey questions are designed to align with the educational outcomes for fifth grade classrooms in California and to address four general learning objectives: (1) Motivations of Artificial Photosynthesis, (2) Applications of Artificial Photosynthesis, (3) Inputs and Outputs of Artificial Photosynthesis, and (4) Engineering Design for Artificial Photosynthesis. The survey data demonstrate a statistically significant improvement in overall student understanding from the lesson plan. Importantly, the data show that the lesson plan presented here is effective at narrowing the performance gap between minority students and overly represented groups.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Ligand-Based Compound Activity Prediction via Few-Shot Learning

Predicting the activities of new compounds against biophysical or phenotypic assays based on the known activities of one or a few existing compounds is a common goal in early stage drug discovery. This problem can be cast as a “few-shot learning” challenge, and prior studies have developed few-shot learning methods to classify compounds as active versus inactive. However, the ability to go beyond classification and rank compounds by expected affinity is more valuable. We describe Few-Shot Compound Activity Prediction (FS-CAP), a novel neural architecture trained on a large bioactivity data set to predict compound activities against an assay outside the training set, based on only the activities of a few known compounds against the same assay. Our model aggregates encodings generated from the known compounds and their activities to capture assay information and uses a separate encoder for the new compound whose activity is to be predicted. The new method provides encouraging results relative to traditional chemical-similarity-based techniques as well as other state-of-the-art few-shot learning methods in tests on a variety of ligand-based drug discovery settings and data sets.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Hierarchical Reinforcement Learning of a Short-Range Bond-Order Potential for Silica: Analytic Embedding of Coordination with Classical Efficiency

Reinforcement learning (RL) has recently emerged as a data-efficient strategy to parametrize short-range interatomic potentials. Building on our past RL optimization of pairwise silica models, we extend the framework to a bond-order (Tersoff-type) potential that provides an analytic embedding of local coordination through a three-body term. A hierarchical RL workflow combining continuous-action Monte Carlo Tree Search and property-based rewards efficiently explores the 26-dimensional parameter space, sequentially optimizing lattice parameters, densities, angles, and cohesive energies of 21 silica polymorphs. The resulting models, Q-Tersoff and ML-Tersoff, reproduce the energetic ordering of low-energy phases and capture the angular correlations and amorphous structure factors of silica with improved fidelity over pairwise force fields, while remaining orders of magnitude faster than high-dimensional machine-learned potentials. Both models underperform for elastic constants and high-energy frameworks, delineating the limits of the current analytic form. The approach establishes a general and interpretable route to angle-aware, short-range potentials that bridge physics-based and machine-learned descriptions of silicate materials.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Machine Learning Analysis of Direct Dynamics Trajectory Outcomes for Thermal Deazetization of 2,3-Diazabicyclo[2.2.1]hept-2-ene

Experimentally, the thermal gas-phase deazetization of 2,3-diazabicyclo[2.2.1]hept-2-ene (1) results in the loss of N2 and the formation of bicyclo products 3 (exo) and 4 (endo) in a nonstatistical ratio, with preference for the exo product. Here, we report unrestricted M06-2X quasiclassical trajectories initialized from the concerted N2 ejection transition state that were able to replicate the experimental preference to form 3. We found that the 3:4 ratio results from the relative amounts of very fast (ballistic) exotype trajectories versus trajectories that lead to the 1,3-diradical intermediate 2. These quasiclassical trajectories provided a set of transition-state vibrational, velocity, momenta, and geometric features for the machine learning analysis. Additionally, a selection of popular supervised classification algorithms (e.g., random forest) provided poor prediction of trajectory outcomes based on only transition-state vibrational quanta and energy features. However, these machine learning models provided more accurate predictions using atomic velocities and atomic positions, attaining ~70% accuracy using initial conditions and between 85 and 95% accuracy at later reaction time steps. This increased accuracy allowed the feature importance analysis to reveal that, at the later-time analysis, the methylene bridge out-of-plane bending is correlated with trajectory outcomes for the formation of either the exo product or toward the diradical intermediate. Possible reasons for the struggle of machine learning algorithms to classify trajectories based on transition-state features is the heavily overlapping feature values, the finite but very large possible vibrational mode combinations, and the possibility of chaos as trajectories propagate. We examined this chaos by comparing a set of nearly identical trajectories that differed by only a very small scaling of the kinetic energies resulting from the transition-state reaction coordinate.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Insights into Prismatic Loop Formation in Irradiated Fe–Cr Alloys from Hypothesis-Driven Active Learning and Causal Analysis

Neutron and electron irradiation experimental studies conducted on body-centered cubic Fe and Fe–Cr alloys have established two prismatic dislocation loop populations, which have Burgers vectors of either a/2$\langle$111$\rangle$ or a$\langle$100$\rangle$. Here, the loop formation depends on factors such as dose (D), dose rate (D rt ), temperature (T), chromium content (Cr%), and other alloying elements. Hence, it is important to understand how irradiation-induced dislocation loops evolve conditional upon the loop characteristics, such as loop density (DD), average loop size d̅, and irradiation parameters (D, D rt , T, and irradiation type), which is still an active area of research. To understand these complex structure–property relationships, machine learning (ML) is employed in a three-step approach. This includes imputing missing data with a k-nearest neighbor, generating functionalized features, and assessing feature importance with random forest classification and regression. Physics-based features are incorporated in a hypothesis-driven active learning scheme to overcome data unavailability challenges. Insights obtained from ML models (i) to categorize dislocation loop types, show the highest correlation with d̅; (ii) Log(DD), obtained through mathematical formulations involving D, Cr%, d̅, and T (e.g., Log(DD) ~ D + exp(-Cr%) + 1/d̅ and log(DD) ~ D + exp(-Cr%) + 1/T). Hypothesis-driven active learning is able to predict Log(DD) in which the experimental date is not known. Causal models verify cause–effect relationships for dislocation loop classification and irradiation factors in FeCr alloys.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Predicting the Activity and Selectivity of Bimetallic Metal Catalysts for Ethanol Reforming using Machine Learning

Machine learning is ideally suited for the pattern detection in large uniform datasets, but consistent experimental datasets on catalyst studies are often small. Here we demonstrate how a combination of machine learning and first-principles calculations can be used to extract knowledge from a relatively small set of experimental data. The approach is based on combining a complex machine-learning model trained on an extensive computational library of transition-state energies with simple linear regression models of experimental catalytic activities and selectivities from the literature. Using the combined model, we identify the key C–C bond scission reactions involved in ethanol reforming and perform a computational screening for ethanol reforming on monolayer bimetallic catalysts with architectures TM-Pt-Pt(111) and Pt-TM-Pt(111) (TM = 3d transition metals). The model also predicts four promising catalyst compositions for future experimental studies. In conclusion, the approach is not limited to ethanol reforming but is of general use for the interpretation of experimental observations as well as for the computational discovery of novel catalytic materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗