Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Machine Learning Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 901 records · Page 50
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Correlations between thermal history and keyhole porosity in laser powder bed fusion

Additive manufacturing has the potential to revolutionize the production of metallic components as it yields near net shape parts with complex geometries and minimizes waste. At the present day, additively manufactured components face qualification and certification challenges due to the difficulty in controlling defects. This has driven a significant research effort aimed at better understanding and improving processing controls – yielding a plethora of in-situ measurements aimed at correlating defects with material quality metrics of interest. In this work, we develop machine-learning methods to learn correlations between thermal history and subsurface porosity for a variety of print conditions in laser powder bed fusion. Un-normalized surface temperatures (in the form of black-body radiances) are obtained using high-speed infrared imaging and porosity formation is observed in the sample cross-section through synchrotron x-ray imaging. To demonstrate the predictive power of these features, we present four statistical machine-learning models that correlate temperature histories to subsurface porosity formation in laser fused Ti-6Al4V powder.

36 MATERIALS SCIENCE↗
DOE OSTI2026

A foundation model for non-destructive defect identification from vibrational spectra

Defects are ubiquitous in solids and strongly influence materials’ functional properties. However, non-destructive characterization and quantification of defects, especially when multiple types coexist, remain a long-standing challenge. Here, we introduce DefectNet, a foundation machine learning model that predicts the chemical identity and concentration of substitutional point defects with multiple coexisting elements directly from vibrational spectra, specifically phonon density-of-states (PDoS). Trained on over 16,000 simulated spectra from 2,000 semiconductors, DefectNet employs a tailored attention mechanism to identify up to six distinct defect elements at concentrations ranging from 0.2% to 25%. The model generalizes well to unseen crystals across 56 elements and can be fine-tuned on experimental data. Validation using inelastic scattering measurements of SiGe alloys and MgB 2 superconductor demonstrates its accuracy and transferability. Furthermore, our work establishes vibrational spectroscopy as a viable, non-destructive probe for bulk point defect quantification, and highlights the promise of foundation models in data-driven defect engineering.

artificial intelligence↗
DOE OSTI2022

Convolutional neural networks for intra-hour solar forecasting based on sky image sequences

Accurate and timely solar forecasts play an increasingly critical role in power systems. Compared to longer forecasting timescales, very short-term solar forecasting has lagged behind in both research and practice. In this paper, we propose deep convolutional neural networks (CNNs) to provide operational intra-hour (10-minute-ahead to 60-minute-ahead) solar forecasts. We develop two CNN structures inspired by a widely-used CNN architecture. The CNNs are tailored to our solar forecasting regression tasks and rely solely on sky image sequences. Case studies based on six years of data (over 150,000 data points) demonstrate that the best CNN model has forecast skill scores of 20%-39% over the naive persistence of cloudiness benchmark, even at these very short timescales. The CNNs also have consistently superior performance when compared to shallow machine learning models with meteorological predictors, where the improvement averages around 7%. The sensitivity analyses show that the sky image length, resolution, and weather conditions have impacts on the deep learning model accuracy. In our intra-hour problem with specific setups, two sky images with a 10-minute 128 x 128 resolution yield the most accurate forecasts. Current limitations, future work, and deployment challenges and solutions are also discussed.

14 SOLAR ENERGY↗
DOE OSTI2023

Exploring Randomly Wired Neural Networks for Climate Model Emulation

Exploring the climate impacts of various anthropogenic emissions scenarios is key to making informed decisions for climate change mitigation and adaptation. State-of-the-art Earth system models can provide detailed insight into these impacts but have a large associated computational cost on a per-scenario basis. This large computational burden has driven recent interest in developing cheap machine learning models for the task of climate model emulation. In this paper, we explore the efficacy of randomly wired neural networks for this task. We describe how they can be constructed and compare them with their standard feedforward counterparts using the ClimateBench dataset. Specifically, we replace the serially connected dense layers in multilayer perceptrons, convolutional neural networks, and convolutional long short-term memory networks with randomly wired dense layers and assess the impact on model performance for models with 1 million and 10 million parameters. We find that models with less-complex architectures see the greatest performance improvement with the addition of random wiring (up to 30.4% for multilayer perceptrons). Furthermore, of 24 different model architecture, parameter count, and prediction task combinations, only one had a statistically significant performance deficit in randomly wired networks relative to their standard counterparts, with 14 cases showing statistically significant improvement. We also find no significant difference in prediction speed between networks with standard feedforward dense layers and those with randomly wired layers. These findings indicate that randomly wired neural networks may be suitable direct replacements for traditional dense layers in many standard models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Correlator convolutional neural networks as an interpretable architecture for image-like quantum matter data

Image-like data from quantum systems promises to offer greater insight into the physics of correlated quantum matter. However, the traditional framework of condensed matter physics lacks principled approaches for analyzing such data. Machine learning models are a powerful theoretical tool for analyzing image-like data including many-body snapshots from quantum simulators. Recently, they have successfully distinguished between simulated snapshots that are indistinguishable from one and two point correlation functions. Thus far, the complexity of these models has inhibited new physical insights from such approaches. Here, we develop a set of nonlinearities for use in a neural network architecture that discovers features in the data which are directly interpretable in terms of physical observables. Applied to simulated snapshots produced by two candidate theories approximating the doped Fermi-Hubbard model, we uncover that the key distinguishing features are fourth-order spin-charge correlators. Our approach lends itself well to the construction of simple, versatile, end-to-end interpretable architectures, thus paving the way for new physical insights from machine learning studies of experimental and numerical data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Leveraging Natural Language Processing and Generative Models in Molecular Chemistry: Property Prediction and Novel Compound Generation

The accurate prediction of molecular properties is important for the rational design and the advancement of green chemistry and sustainable materials research. However, the predictive power of traditional computational chemistry methods is limited due to computational restrictions. Here, in this study, we examine an alternative approach to the accurate prediction of properties of organic compounds: natural language processing (NLP)-based molecular embedding. Using viscosity, partition coefficient (log P), and enthalpy of vaporization as test properties through a survey of comprehensive datasets comprising 5695 data points for viscosity, 25 870 data points for log P, and 2296 data points for enthalpy of vaporization. These are important properties for the design of greener, safer, and sustainable chemical processes. Models were trained using NLP methods such as Mol2vec and fine-tuned ChemBERTa, and results were compared with traditional input featurization techniques such as Morgan fingerprints and quantum chemistry derived sigma profiles and DFT features. Among the various machine learning models, Mol2vec demonstrated superior predictive capabilities, achieving the highest correlation coefficient (R 2 = 0.945) and lowest RMSE (0.106 mPa s) for viscosity, as well as high accuracy for log P and enthalpy of vaporization predictions. These findings establish the Mol2vec featurization technique, graph-convolutional neural networks (GCNN), and fine-tuned ChemBERTa model as powerful tools for predictive modeling of organic compounds properties, offering a significant improvement over previously used featurization techniques and opening up strategies for very-high-throughput computational screening. Finally, we integrated ML models with hybrid language-model-based generative adversarial networks (LM-GAN) to generate novel molecular sequences with desirable properties for different research applications. The ability to computationally design solvents with lower viscosity, lower log P, and lower enthalpy of vaporization offers a data-driven route to accelerating the discovery of sustainable alternatives to traditionally toxic solvents.

ChemBERTa↗
DOE OSTI2021

Deciphering enhancer sequence using thermodynamics-based models and convolutional neural networks

Abstract Deciphering the sequence-function relationship encoded in enhancers holds the key to interpreting non-coding variants and understanding mechanisms of transcriptomic variation. Several quantitative models exist for predicting enhancer function and underlying mechanisms; however, there has been no systematic comparison of these models characterizing their relative strengths and shortcomings. Here, we interrogated a rich data set of neuroectodermal enhancers in Drosophila, representing cis- and trans- sources of expression variation, with a suite of biophysical and machine learning models. We performed rigorous comparisons of thermodynamics-based models implementing different mechanisms of activation, repression and cooperativity. Moreover, we developed a convolutional neural network (CNN) model, called CoNSEPT, that learns enhancer ‘grammar’ in an unbiased manner. CoNSEPT is the first general-purpose CNN tool for predicting enhancer function in varying conditions, such as different cell types and experimental conditions, and we show that such complex models can suggest interpretable mechanisms. We found model-based evidence for mechanisms previously established for the studied system, including cooperative activation and short-range repression. The data also favored one hypothesized activation mechanism over another and suggested an intriguing role for a direct, distance-independent repression mechanism. Our modeling shows that while fundamentally different models can yield similar fits to data, they vary in their utility for mechanistic inference. CoNSEPT is freely available at: https://github.com/PayamDiba/CoNSEPT.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

PayamDiba/CoNSEPT

Deciphering the sequence-function relationship encoded in enhancers holds the key to interpreting non-coding variants and understanding mechanisms of transcriptomic variation. Several quantitative models exist for predicting enhancer function and underlying mechanisms; however, there has been no systematic comparison of these models characterizing their relative strengths and shortcomings. Here, we interrogated a rich data set of neuroectodermal enhancers in Drosophila, representing cis- and trans- sources of expression variation, with a suite of biophysical and machine learning models. We performed rigorous comparisons of thermodynamics-based models implementing different mechanisms of activation, repression and cooperativity. Moreover, we developed a convolutional neural network (CNN) model, called CoNSEPT, that learns enhancer ‘grammar’ in an unbiased manner. CoNSEPT is the first general-purpose CNN tool for predicting enhancer function in varying conditions, such as different cell types and experimental conditions, and we show that such complex models can suggest interpretable mechanisms. We found model-based evidence for mechanisms previously established for the studied system, including cooperative activation and short-range repression. The data also favored one hypothesized activation mechanism over another and suggested an intriguing role for a direct, distance-independent repression mechanism. Our modeling shows that while fundamentally different models can yield similar fits to data, they vary in their utility for mechanistic inference.

Dibaeinia, Payam↗
DOE OSTI2020

Estimating Subhourly Inverter Clipping Loss From Satellite-Derived Irradiance Data: Preprint

Photovoltaic system production simulations are conventionally run using hourly weather datasets. Hourly simulations are sufficiently accurate to predict the majority of long-term system behavior but cannot resolve high-frequency effects like inverter clipping caused by short-duration irradiance variability. Direct modeling of this subhourly clipping error is only possible for the few locations with high-resolution irradiance datasets. This paper describes a method of predicting the magnitude of this error using a machine learning model and 30-minute satellite irradiance data. The method predicts a correction for each 30-minute interval with the potential to roll up into 60-minute corrections to match an hourly energy model. The model is trained and validated at locations where the error can be directly simulated from 1-minute ground data. The validation shows low bias at most ground station locations. The model is also applied to gridded satellite irradiance to produce a heatmap of the estimated clipping error across the United States. Finally, the relative importance of each predictor satellite variable is retrieved from the model and discussed.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2025

A machine-learning approach to measure 3D sample properties from 2D Transmission Electron Microscopy images

Transmission Electron Microscopy (TEM) is a powerful tool for the characterization of materials at the nanoscale; however, its inherent two-dimensional (2D) nature poses significant challenges to accurately measure three-dimensional (3D) properties. We introduce a supervised machine-learning model that predicts 3D structural information, such as sample thickness and curvature, from a series of conventional 2D TEM images. The model, a U-Net convolutional neural network, is trained on a large synthetic dataset generated from dynamical diffraction simulations that model TEM’s complex, nonlinear image formation, accounting for sample thickness and curvature. This physically realistic framework enables exploration of a broad parameter space impractical to sample experimentally. We demonstrate that the trained model has accurate predictions for experimental single-crystal silicon samples, achieving performance comparable to established measurement techniques. This work highlights the critical role of robust, simulation-based training in overcoming the limitations of real-world imaging artifacts and inconsistent sample geometries. By integrating machine learning with numerical simulations, we offer an efficient and scalable framework for quantitative TEM analysis, paving the way for more sophisticated 3D characterization of complex materials.

Dynamical diffraction↗
DOE OSTI2024

The ab initio amorphous materials database: Empowering machine learning to decode diffusivity

Amorphous materials exhibit unique properties that make them suitable for various applications in science and technology, ranging from optical and electronic devices and solid-state batteries to protective coatings. However, data-driven ex- ploration and design of amorphous materials is hampered by the absence of a com- prehensive database covering a broad chemical space. In this work, we present the largest computed amorphous materials database to date, generated from sys- tematic and accurate ab initio molecular dynamics (AIMD) calculations. We also show how the database can be used in simple machine-learning models to connect properties to composition and structure, here specifically targeting ionic conductiv- ity. These models predict the Li-ion diffusivity with speed and accuracy, offering a cost-effective alternative to expensive density functional theory (DFT) calculations. Furthermore, the process of computational quenching amorphous materials provides a unique sampling of out-of-equilibrium structures, energies, and force landscape, and we anticipate that the corresponding trajectories will inform future work in uni- versal machine learning potentials, impacting design beyond that of non-crystalline materials.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Improving Effective Mass Estimations in Pu-Metal Annuli

Determining the effective mass of 240 Pu in a plutonium metal item can be achieved through a number of destructive and non-destructive assay techniques. However, these techniques have one or more shortcomings. These include the need for large quantities of plutonium, long measurement time, or lack of sufficient accuracy. While efforts have been made to mitigate these issues by estimating 240 Pu quantities through neutron coincidence counting techniques, these estimates are subject to systematic bias, and their estimates are not well characterized when other factors of the annulus’ physical form and composition are accounted for. In this work, we expand upon these non-destructive assay techniques via the implementation of random forest machine learning models, which produce correction functions that augment and improve the effective mass estimates derived from classical leakage multiplication, singles, doubles, and triples multiplicity counting equations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

High bias machine learning for antineutrino-based safeguards for small reactors

The statistical methods used for antineutrino detection will need to be improved to effectively monitor the inventory of next-generation nuclear reactors. In this sensitivity study, we evaluate machine learning models compared to previously used statistical approaches to identify diversion scenarios in a simulated Advanced Fast Reactor (AFR)-100. A chi-square goodness-of-fit technique, which individually compares the simulated antineutrino yields to the expected antineutrino yield, resulted in precise but low diversion detection probability. Various support vector machine (SVM) models were applied with diverse training datasets to evaluate the robustness of the method towards unexpected or “unseen” diversion scenarios. Furthermore, our results indicate that while the SVM models significantly improved the detection probability of near-field antineutrino-based safeguards, up to a probability of ~0.04, for the simulated small reactor, the detection system still needs improvements to reach the 0.2 detection limit established by the International Atomic Energy Agency.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2025

Non-destructive structural characterization of graphite components using mechanical resonance and deep learning

As compared to conventional nuclear reactors, microreactors have the potential to significantly reduce construction timelines and capital costs, decreasing the barriers for advanced nuclear reactor technologies. However, the lower power output of these microreactors (typically < 20 MWe) creates challenging economics if operation and maintenance costs cannot be sufficiently reduced. The compact size of these designs presents an opportunity for comprehensive in-situ structural health monitoring to provide real-time feedback in order to reduce operational costs associated with maintenance and downtime. Many microreactor concepts use graphite for both in-core neutron moderation and as a structural material, which has typically required some form of periodic and laborious inspection. This report provides a description and assessment of recent work with graphite to couple acoustic-based experimental measurements and characterization with machine learning models to mature structural health monitoring capabilities and generate benefits for the nuclear microreactor industry. With resilient embedded sensors in development in other programs funded by the US Department of Energy’s Office of Nuclear Energy and elsewhere, the work described herein builds upon previously funded efforts to mature non-destructive testing technology that relates measured vibrational signatures to structural changes, using a combination of new experimental measurements and machine learning processing. Building on past successful demonstrations of predictive workflows to identify structural changes in a hexagonal stainless steel test article with excellent acoustic propagation, we first performed baseline characterization on graphite samples with canonical geometries to ensure compatibility and confidence in the applied techniques for a material with distinctly different mechanical properties. In contrast to efforts in previous years, we worked exclusively with unidirectional vibration data that is more comparable to those expected from the existing embedded sensor technologies which are suitable for deployment in a reactor setting. Established acoustic and modern machine-learning-based characterization approaches were applied to the resulting datasets from these simple geometries. Both approaches were found to be highly capable of detecting even small geometric irregularities amongst nominally identical samples. As such, we then moved to testing these approaches for detection of artificial local stress perturbations introduced into a more complex geometry: a hexagonal block with drilled holes. A main outcome of this work is that a generalizable ML workflow can be used to detect and predict the characteristics of small artificial anomalies in a graphite component with a relevant geometry. While this work was performed using surficial vibration data, we expect the approach to be flexible and viable for other monitoring scenarios, such as those with different arrangements or types of sensor arrays. As compared to previously funded efforts, an existing ML workflow based on neural networks was enhanced through the addition of recently developed Fourier neural operators. As applied to previously collected and new vibration datasets, prediction accuracies of anomaly characterizations were greatly improved with minimal added computational cost. As trained on small durations of vibration data (tens of seconds) collected over a realistic number of locations, the model was able to reliably determine the presence of a subtle stress anomaly and begin to provide location estimates. Such an approach is likely to be viable for more relevant reactor damage scenarios for graphite components, such as progressive crack growth or creep.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Localized Defect Detection from Spatially Mapped, In-Situ Process Data With Machine Learning

In powder bed fusion additive manufacturing, machines are often equipped with in-situ sensors to monitor the build environment as well as machine actuators and subsystems. The data from these sensors offer rich information about the consistency of the fabrication process within a build and across builds. This information may be used for process monitoring and defect detection; however, little has been done to leverage this data from the machines for more than just coarse-grained process monitoring. In this work we demonstrate how these inherently temporal data may be mapped spatially by leveraging scan path information. We then train a XGBoost machine learning model to predict localized defects—specifically soot–using only the mapped process data of builds from a laser powder bed fusion process as input features. The XGBoost model offers a feature importance metric that will help to elucidate possible relationships between the process data and observed defects. Finally, we analyze the model performance spatially and rationalize areas of greater and lesser performance.

3D printing↗
DOE OSTI2025

Hyperplane decision trees as piecewise linear surrogate models for chemical process design

Recent trends in chemical engineering research point towards an increasing reliance on data-driven modeling approaches. Neural networks, for instance, have proven to be accurate when data is plentiful and high-dimensional, but in many cases, they require computationally-intensive training procedures. Here, in this work, we describe hyperplane decision trees (HT) as a highly expressive and low-compute machine learning model architecture. These models are locally linear and have linear decision boundaries, resulting in a piecewise linear model of the data. This property allows them to be converted into mixed-integer linear constraints which can be globally optimized. Our open-source PyTorch implementation of this method is a fast, flexible, and accessible way to build accurate piecewise linear models of data.

Decision trees↗
NASA NTRS2019

Atmospheric Chemistry Modeling and Air Quality Forecasting Using Machine Learning

Atmospheric chemistry models are a central tool to study the impact of chemical constituents on the environment, vegetation and human health. These models split the atmosphere in a large number of grid-boxes and consider the emission of compounds into these boxes and their subsequent transport, deposition, and chemical processing. The chemistry is represented through a series of simultaneous ordinary differential equations, one for each compound. Given the difference in life-times between the chemical compounds (milli-seconds for O1D to years for CH4) these equations are numerically stiff and solving them consists of a significant fraction of the computational burden of a chemistry model.We have investigated a machine learning approach to emulate the chemistry instead of solving the differential equations numerically. From a one-month simulation of the GEOS-Chem model we have produced a training dataset consisting of the concentration of compounds before and after the differential equations are solved, together with some key physical parameters for every grid-box and time-step. From this dataset we have trained a machine learning algorithm (regression forest) to be able to predict the concentration of the compounds after the integration step based on the concentrations and physical state at the beginning of the time step. We have then included this algorithm back into the GEOS-Chem model, bypassing the need to integrate the chemistry.This machine learning approach shows many of the characteristics of the full simulation and has the potential to be substantially faster. There are a wide range of application for such an approach - generating boundary conditions, for use in air quality forecasts, chemical data assimilation systems, etc. We discuss speed and accuracy of our approach, and highlight some potential future directions for improving it.

Keller, Christoph A.↗