Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural generative models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Confusion-Driven Machine Learning of Structural Phases of a Flexible, Magnetic Stockmayer Polymer

We use a semisupervised, neural-network-based machine learning technique, the confusion method, to investigate structural transitions in magnetic polymers, which we model as chains of magnetic colloidal nanoparticles characterized by dipole–dipole and Lennard-Jones interactions. As input for the neural network, we use the particle positions and magnetic dipole moments of equilibrium polymer configurations, which we generate via replica-exchange Wang–Landau simulations. We demonstrate that by measuring the classification accuracy of neural networks, we can effectively identify transition points between multiple structural phases without any prior knowledge of their existence or location. We corroborate our findings by investigating relevant conventional order parameters. Our study furthermore examines previously unexplored low-temperature regions of the phase diagram, where we find new structural transitions between highly ordered helicoidal polymer configurations.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Machine Learning Classification of Molten Salt Heat Exchanger Channel Plugging using Synthetic Data

This report addresses the requirements of Milestone M3.4 AI capability to identify and predict maintenance events. Development of digital twins (DT) for molten salt reactor (MSR) components is crucial for reducing operating and maintenance costs (O&M) and ensuring commercial viability of these reactors. Our focus is on development of DT for MSR primary system heat exchanger (HX), a critical component, the fault in which can reduce operating efficiency and force reactor shutdown. We are investigating the feasibility of a conceptual DT of HX consisting of internal distributed temperature sensing with fiber optics and machine learning (ML) algorithms to detect and localize faults. To determine the optimal approach to detection and localization of channel plugging, we benchmark seven different ML models: Logistic Regression, K-Nearest Neighbors (KNN), Gaussian Naïve Bayes, Support Vector Machines (SVM), Decision Tree Classifier, Random Forest Tree Classifier, and Feed-Forward Neural Network. ML algorithms are benchmarked using synthetic HX plugging data generated with computational fluid dynamics COMSOL software, with added brown noise to represent experimental noise. We show that the best performance is obtained with the Decision Tree classifier.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1988

Neural networks with hierarchically correlated patterns

The Hopfield model of neural networks is extended to allow for the storage and retrieval of hierarchically correlated patterns. The overlaps between these patterns form an ultrametric tree. Intermediate states, which serve as ancestors for the following levels, are generated at each level of the tree. The states belonging to each level are stored, by a modified learning rule, in a series of identical networks, one for each level. The retrieval of a particular pattern is preceded and assisted by the successive retrieval of its ancestors. The performance of this scheme is studied analytically and numerically.

Gutfreund, H.↗
DOE OSTI2024

Evaluating the Use of Foundational Chemical Language Models in Multimodal Graph Fusion

Rapid and accurate prediction of the physicochemical properties of molecules given their structures remains a key challenge in cheminformatics. Machine learning approaches offer high-throughput options, but the optimality of inductive biases and data representations are up for debate. For example, BERT-based masked language models (MLMs) can be trained in a self-supervised way on hundreds of millions to billions of readily available SMILES strings. Another option is graph neural networks (GNNs), which can operate directly on molecular structures. Yet, generating accurate molecular geometry is computationally expensive, leading to a relative scarcity in data compared to SMILES strings. It is attractive to combine these two paradigms by pre-training an LM on a large corpus of SMILES strings and embedding these representation into a geometric graph neural network. Despite the promise of such an approach, and contrary to previous studies, we find mixed results with the combination of the LMs and GNNs on several molecule datasets. In particular, we found evidence for improvement on the FreeSolv and QM7 benchmarks, but degraded performance on the ESOL, LIPO and QM9 datasets compared to a GNN baseline.

Francel, Collin [University of Alabama]↗
DOE OSTI2021

hippynn Python Package

hippynn is a python package for defining, training, and applying neural networks to atomistic systems. In particular, it focuses on Hierarchical Interacting Particle Neural Networks (HIP-NNs), a deep learning architecture for atomistic systems. HIP-NNs take input data describing the properties of atomistic systems (often generated using ab-initio quantum mechanics) to learn fast and accurate models. A trained HIP-NN can predict potential energy surfaces, atomic charges, and more. hippynn uses PyTorch for portable high-performance code, including both CPU and GPU support. hippynn allows for extensive customization, including user-defined models and loss functions, to facilitate future research into extensions of HIP-NN as well as other atomistic deep learning models.

Lubbers, Nicolas↗
NASA NTRS2022

Microstructure Segmentation with Deep Learning Encoders Pre-Trained on a Large Microscopy Dataset

This study examined the improvement of microscopy segmentation accuracy by transfer learning from a large dataset of microscopy images called MicroNet. Many neural network encoder architectures, including VGG, Inception, and ResNet, were trained on over 100,000 labelled microscopy images from 54 classes. These pre-trained encoders were then embedded into multiple segmentation architectures including U-Net and DeepLabV3+ to evaluate segmentation performance on newly created benchmark microscopy datasets. Compared to ImageNet pre-training, models pre-trained on MicroNet generalized better to out-of-distribution micrographs taken under different imaging and sample conditions and were more accurate with less training data. When training with only a single Ni-superalloy image, pre-training on MicroNet produced a 72.2 percent reduction in relative segmentation error. These results suggest that transfer learning from large in-domain datasets generate models with learned feature representations that are more useful for downstream tasks and will likely improve any microscopy image analysis technique that can leverage pre-trained encoders.

machine learning↗
DOE OSTI2025

Leveraging Natural Language Processing and Generative Models in Molecular Chemistry: Property Prediction and Novel Compound Generation

The accurate prediction of molecular properties is important for the rational design and the advancement of green chemistry and sustainable materials research. However, the predictive power of traditional computational chemistry methods is limited due to computational restrictions. Here, in this study, we examine an alternative approach to the accurate prediction of properties of organic compounds: natural language processing (NLP)-based molecular embedding. Using viscosity, partition coefficient (log P), and enthalpy of vaporization as test properties through a survey of comprehensive datasets comprising 5695 data points for viscosity, 25 870 data points for log P, and 2296 data points for enthalpy of vaporization. These are important properties for the design of greener, safer, and sustainable chemical processes. Models were trained using NLP methods such as Mol2vec and fine-tuned ChemBERTa, and results were compared with traditional input featurization techniques such as Morgan fingerprints and quantum chemistry derived sigma profiles and DFT features. Among the various machine learning models, Mol2vec demonstrated superior predictive capabilities, achieving the highest correlation coefficient (R 2 = 0.945) and lowest RMSE (0.106 mPa s) for viscosity, as well as high accuracy for log P and enthalpy of vaporization predictions. These findings establish the Mol2vec featurization technique, graph-convolutional neural networks (GCNN), and fine-tuned ChemBERTa model as powerful tools for predictive modeling of organic compounds properties, offering a significant improvement over previously used featurization techniques and opening up strategies for very-high-throughput computational screening. Finally, we integrated ML models with hybrid language-model-based generative adversarial networks (LM-GAN) to generate novel molecular sequences with desirable properties for different research applications. The ability to computationally design solvents with lower viscosity, lower log P, and lower enthalpy of vaporization offers a data-driven route to accelerating the discovery of sustainable alternatives to traditionally toxic solvents.

ChemBERTa↗
DOE OSTI2026

From Rules to Reasoning: A Survey of Large Language Model-Based Approaches to Scientific Hypothesis and Idea Generation

Scientific hypothesis generation represents a fundamental challenge in contemporary research due to exponentially expanding literature volumes and increasing disciplinary specialization. Large language models (LLMs) have emerged as transformative tools for automated scientific discovery, moving beyond traditional rule-based and literature-mining approaches. Four paradigmatic approaches define current LLM-driven hypothesis generation: direct prompting and fine-tuning methods, knowledge-enhanced frameworks integrating retrieval-augmented generation (RAG), multi-agent collaborative systems simulating research teams, and reasoning-focused approaches implementing cognitive architectures. Domain-specific applications demonstrate statistical equivalence to human expert performance in social psychology, experimental validation in biomedical research, and near-expert quality in astronomy. Evaluation methodologies encompass human expert assessment, LLM-as-judge frameworks, and comprehensive benchmarking systems. Technical challenges include hallucination management, knowledge integration limitations, and balancing novelty with feasibility. Future directions emphasize hybrid neural-symbolic architectures and sophisticated human-AI collaboration models for responsible scientific discovery acceleration.

AI-driven discovery↗
DOE OSTI2026

Probabilistic Inference of Low-Surface-Brightness Galaxy Morphological Parameters Using Simulation-Based Inference

Low-surface-brightness galaxies (LSBGs) are diffuse, often dark-matter-dominated systems whose faintness makes their structural parameters difficult to measure reliably in wide-field imaging surveys. Robust parameter inference, including uncertainty quantification, is important for population studies and for comparisons with models of galaxy formation, as future surveys are expected to produce increasingly large samples of diffuse galaxies. In practice, LSBG profile modeling is sensitive to sky- background errors, masking choices, contaminating background sources, and the computational cost of obtaining posterior-level uncertainties for large samples. Motivated by these questions, we develop a simulation-based inference (SBI) framework for estimating posterior distributions of LSBG morphological parameters from simulated galaxy images. Using PyImfit, we generate DES-like single-Sersic profile LSBG images with known position angle, ellipticity, Sersic index, effective surface brightness, and effective radius. We then train a normalizing-flow-based neural posterior estimator using the sbi package to infer these parameters from the simulated images. For isolated simulated galaxies, the SBI posterior recovers the true input parameters, produces posterior predictive residuals consistent with the assumed noise model, and shows good empirical calibration in a DES-motivated test regime. We also compare SBI with PyImfit-based MCMC inference and find broadly comparable posterior constraints, while SBI enables substantially faster posterior sampling after training. Finally, we test robustness to compact background contaminants. A model trained only on isolated galaxies produces undercovered posteriors on contaminated images, whereas training on simulations with variable contaminant positions and fluxes improves calibration across contaminated test sets. These results demonstrate the promise of SBI for scalable, uncertainty-aware LSBG morphology inference, while emphasizing that posterior reliability strongly depends on whether training simulations include relevant observational complications.

Batbayar, Bilguun [U. Chicago (main)]↗
DOE OSTI2023

Physics informed machine learning for chemistry tabulation

Modeling of turbulent combustion system requires modeling the underlying chemistry and the turbulent transport. Solving both systems simultaneously is computationally prohibitive. Instead, given the difference in scales at which the two sub-systems evolve, the two sub-systems are typically (re)solved separately. Popular approaches such as the Flamelet Generated Manifolds (FGM) use a two-step strategy where the governing reaction kinetics are pre-computed and mapped to a low-dimensional manifold, characterized by a few reaction progress variables (model reduction) and the manifold is then “looked-up” during the run-time to estimate the high-dimensional system state by the turbulent transport system. While existing works have focused on these two steps independently, in this work we show that joint learning of the progress variables and the look-up model, can yield more accurate results. Here, we build on the base formulation and implementation to include the dynamically generated Thermochemical State Variables (Lower Dimensional Dynamic Source Terms). We discuss the challenges in the implementation of this deep neural network architecture and experimentally demonstrate its superior performance.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Ligand-Based Compound Activity Prediction via Few-Shot Learning

Predicting the activities of new compounds against biophysical or phenotypic assays based on the known activities of one or a few existing compounds is a common goal in early stage drug discovery. This problem can be cast as a “few-shot learning” challenge, and prior studies have developed few-shot learning methods to classify compounds as active versus inactive. However, the ability to go beyond classification and rank compounds by expected affinity is more valuable. We describe Few-Shot Compound Activity Prediction (FS-CAP), a novel neural architecture trained on a large bioactivity data set to predict compound activities against an assay outside the training set, based on only the activities of a few known compounds against the same assay. Our model aggregates encodings generated from the known compounds and their activities to capture assay information and uses a separate encoder for the new compound whose activity is to be predicted. The new method provides encouraging results relative to traditional chemical-similarity-based techniques as well as other state-of-the-art few-shot learning methods in tests on a variety of ligand-based drug discovery settings and data sets.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Machine learning-enabled discovery of ionic liquid–solvent electrolytes exhibiting high ionic conductivity

Ionic liquids (ILs), which are a class of materials with versatile nature and growing popularity, are facing impediments toward widespread usage as electrolytes due to various factors such as low ionic conductivity, high viscosity, high market price etc. One of the ways these limitations can be addressed is by mixing ILs with a molecular solvent. In a combinatorial sense, there exists an immense number of specific IL–solvent combinations. An exhaustive experimental or even simulation-based investigation of the chemical space spanned by such combinations can be extremely time-consuming, expensive, and nearly impossible. An alternative approach is to employ machine learning-based models developed from available databases. Although there exists prior literature that integrates machine learning to investigate mixtures of specific solvents with ILs, these models lack generalization necessitating development of a large number of ML models to handle various solvents. To remedy this shortcoming, as a part of designing green electrolytes with high ionic conductivity that can have potential applications in next-generation batteries and solar cells, this work aims to develop a unified machine learning model to predict ionic conductivity of any IL–solvent mixture system. In this regard, three models, namely, Random Forest, extreme gradient boosting (XGBoost), and artificial neural network (ANN) were formulated using the NIST ILThermo database. The dataset contained 549 unique ionic liquids from 16 cation families and 81 unique solvents, representing a total of 23 712 datapoints. SHAPLEY additive explanation (SHAP) method was used to assess the impact of various features on model prediction and their significance was compared with literature to gain physical insight about the model behavior. Finally, using the developed models, approximately 2.5 million IL–solvent mixtures at five different compositions were screened at room temperature. The high-throughput screening yielded nearly 19 000 IL–solvent mixtures for which ionic conductivity was found to exceed the ionic conductivity of conventional Li-ion battery electrolyte.

25 ENERGY STORAGE↗
DOE OSTI2026

PowerModelsGAT-AI: Physics-Informed Graph Attention for Multi-System Power Flow With Continual Learning

Solving the alternating current power flow equations in real time is essential for secure grid operation, yet classical Newton–Raphson solvers can be slow under stressed conditions. Existing graph neural networks for power flow are typically trained on a single system and often degrade on different systems. We present PowerModelsGAT-AI, a physics-informed graph attention network that predicts bus voltages and generator injections. The model uses bus-type-aware masking to handle different bus types and balances multiple loss terms, including a power-mismatch penalty, using learned weights. We evaluate the model on 14 benchmark systems (4 to 6,470 buses) and train a unified model on 13 of these under contingency conditions with up to two branch outages, achieving an average normalized mean absolute error of 0.89% for voltage magnitudes and R 2 >0.99 for voltage angles. We also show continual learning: when adapting a base model to a new 1,354-bus system, standard fine-tuning causes severe forgetting with error increases exceeding 1000% on base systems, while our experience replay and elastic weight consolidation strategy keeps error increases below 2% and in some cases improves base-system performance. Interpretability analysis shows that learned attention weights correlate with physical branch parameters (susceptance: r=0.38 ; thermal limits: r=0.22 ), and feature importance analysis supports that the model captures established power flow relationships.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Frequency Disturbance Event Detection Based on Synchrophasors and Deep Learning

Power system frequency disturbances are caused by various generation and transmission events including generator trips, load disconnections, line trips, etc. Accurate detections of the events are crucial to bulk power system situation awareness and event investigation. This paper utilizes the recent advances of deep learning to build a convolutional neural network model to detect events in an accurate yet straightforward manner. Herein, the rate of change of frequency and the relative angle shift are converted to images as the inputs of the proposed model. Finally, this paper uses two convolutional neural networks and classifier fusion to achieve the detection result. Compared with the conventional event detection algorithm and the frequency only deep learning model, the proposed model improves the detection accuracy by over 48%. As a promising tool for bulk power system situation awareness, the proposed model requires a short decision time, which is suitable for practical scenarios.

42 ENGINEERING↗
DOE OSTI2026

Machine learning enables reconstruction of past fire regimes from charcoal-derived fire intensity and fuel composition

Background Fire is a foundational ecological process that shapes ecosystem structure, diversity, and resilience. Quantifying paleofire regime attributes such as frequency, severity, and intensity is essential for understanding the historical range of variability in fire behavior and its ecological effects. While frequency and severity are often reconstructed in paleofire studies, quantitative reconstructions of fire intensity remain limited. Recent work has shown that maximum pyrolysis temperature—a proxy for fire intensity—and plant species type can be inferred from charcoal using transmission Fourier-transform infrared (FTIR) spectroscopy. However, the sample preparation for transmission FTIR is destructive and time-consuming, limiting application and reuse of materials for other analyses. We evaluated reflectance FTIR spectroscopy as a non-destructive alternative for reconstructing combustion temperature and plant species from laboratory-generated charcoal. We also examined the influence of contrasting airflow environments (ambient air versus nitrogen-rich) on pyrolysis temperature and plant species reconstruction prediction accuracies and compared predictive performance between a novel, neural network–based deep learning model with the traditional modern analogue technique (MAT) using k-nearest neighbor functions. As proof of concept, we apply our enhanced methodology to ancient charcoal to demonstrate applicability at improving long-term fire regime reconstructions and the ability to link paleofire records with contemporary fire ecology. Results Our analysis shows that transmission and reflectance FTIR spectra yield comparable spectral profiles. However, sample preparation for reflectance FTIR is minimal and non-destructive, unlike transmission FTIR which is destructive. We demonstrate that oxygen environments improved reconstruction accuracy relative to nitrogen-rich conditions. Finally, our deep learning neural network (DL) achieved testing accuracies of 98.7% for temperature and 96.2% for species identification, outperforming MAT’s k-NN approach (89.8% and 65.9%, respectively). A Shapley importance analysis identified 5 key spectral regions that greatly influenced the model’s temperature or species categorization. When applied to ancient charcoal, our results show historic fires from the most recent past primarily burned at low intensities (400–500 °C), reflective of natural fire regimes in ponderosa pine forests. Our results corroborate charcoal morphology data that suggests all ancient charcoal originated from burned woody plant types. Conclusions By combining reflectance FTIR spectroscopy with a deep learning approach, we provide the first accuracies high enough to confidently identify both species and temperature from laboratory-produced charcoal, improving quantitative reconstructions of fire intensity and fuel composition from paleofire records. This opens a wide range of research into the link between fire and larger drivers (i.e., climate or human) and greater ecological understanding of fire regimes beyond that of burn scars or recent observations. These methodological improvements have direct relevance for fire management by improving interpretation of historical fire behavior, informing fuel–fire relationships, and providing a scalable analytical framework applicable to both long-term ecological studies and contemporary fire science.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

A case study in contrastive learning information combination: Application to technical forensics of additive manufacturing filament source identification

Combination of information from disparate data sources into a single decision is a core challenge in many fields, including the field of technical forensics. Technical forensics (TF) utilizes technical characterization of questioned samples to determine properties of that sample; these properties are then used to infer information of forensic interest, such as provenance, age, or attribution. TF is utilized in traditional forensic applications, such as the attribution of material fragments from an explosive, and in nuclear forensic applications, such as the attribution of actinides which have been interdicted out of regulatory control. The challenge of combining information from disparate sources, described alternately by many terms including “Data Fusion” and “Data Integration”, is exacerbated in the technical forensics domain due to at least two factors: the challenge of interpreting each information source singularly, and the relatively small data set sizes available. Extensive literature exists attempting to combine technical forensics information sources, both in manual and automated processes. These attempts are often bespoke to the specific information sources (such as the bi-, tri-, or quad-isotope chart (Moody, Grant, and Hutcheon 2005)), with some emerging examples of simple early- and late- fusion (, respectively). Simultaneous to the information combination efforts described in the previous paragraph, the field of natural language processing attempted (and largely succeeded) in combining information from multiple non-technical information sources. The ecosystem of “multi-modal” language models, which can take text and images as input, and generate text and images as output, became large and diverse by 2025 (Khan et al. 2025). In a generalized sense, many of these methods are trained by learning neural networks which can convert raw text or images into a vector of numbers describing the text or image, hereafter called “embeddings” and the neural networks performing the conversion are called “embedders”. By using a separate embedder for text and images, finding coincident text and images (such as images with their captions), and optimizing the parameters of the embedders such that the embeddings for the text and the image are similar, the field has found a bridge between text and images (Girdhar et al. 2023). It is the contention of the authors of this report that this insight is not limited to text and images but instead can be extended to any modality which can be found coincidently. The subject of the rest of this report is the application of this method to example multi-modal technical forensic data. Some details about the data used in this report are not appropriate for this report, and are included in a companion report (PNNL-38669).

36 MATERIALS SCIENCE↗
DOE OSTI2025

DOE FAIR Surrogate Benchmarks Supporting AI and Simulation Research (SBI Surrogate Benchmark Initiative) (Final Report)

Computational Science is being revolutionized by integrating AI and simulation and, in particular, by deep learning surrogate models that can replace all or part of traditional large‐scale HPC computations. Such surrogates can achieve remarkable performance improvements, as much as several orders of magnitude, and save both compute time and energy. The Surrogate Benchmark Initiative (SBI) project creates a community repository and FAIR (Findable, Accessible, Interoperable, and Reusable) data ecosystem for HPC application surrogate benchmarks. The SBI team comes from Argonne National Laboratory (ANL), Indiana University (IU), Rutgers University, the University of Tennessee, Knoxville (UTK), and the University of Virginia(UVA). SBI repositories include data, code, and all relevant collateral artifacts, that the science and engineering community needs to use and reuse these data sets and surrogates. SBI repositories generate active research from both participants in SBI and the broader AI and domain science communities. This project develops surrogates that use several different neural nets to learn and quickly infer the results of simulations and data systems and capture them as surrogate benchmarks with a rich set of metadata, covering. Data; Model; Metrics specification; Machine specification; Science, Speed, Power Results, We research FAIR metadata for these benchmarks. We develop application surrogate examples as benchmarks across many fields (ANL, UTK, IU, UVA). We also study non Surrogate benchmarks that have many common features and similar issues regarding FAIRness. We work with MLCommons (UVA, UTK), which is a major machine learning benchmarking activity where we get metadata ontologies, software, and benchmarks, benchmarks have datasets, models, and metadata, and they need a technical framework developed by UTK and Rutgers and deployed by UVA. We study features of Surrogates, including performance, training set size, and uncertainty quantification (Rutgers, UVA and IU).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Fast and Accurate Predictions of Total Energy for Solid Solution Alloys with Graph Convolutional Neural Networks

We use graph convolutional neural networks (GCNNs) to produce fast and accurate predictions of the total energy of solid solution binary alloys. GCNNs allow us to abstract the lattice structure of a solid material as a graph, whereby atoms are modeled as nodes and metallic bonds as edges. This representation naturally incorporates information about the structure of the material, thereby eliminating the need for computationally expensive data pre-processing which would be required with standard neural network (NN) approaches. We train GCNNs on ab-initio density functional theory (DFT) for copper-gold (CuAu) and iron-platinum (FePt) data that has been generated by running the LSMS-3 code, which implements a locally self-consistent multiple scattering method, on OLCF supercomputers Titan and Summit. GCNN outperforms the ab-initio DFT simulation by orders of magnitude in terms of computational time to produce the estimate of the total energy for a given atomic configuration of the lattice structure. We compare the predictive performance of GCNN models against a standard NN such as dense feedforward multi-layer perceptron (MLP) by using the root-mean-squared errors to quantify the predictive quality of the deep learning (DL) models. We find that the attainable accuracy of GCNNs is at least an order of magnitude better than that of the MLP.

Lupo Pasini, Massimiliano↗