Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “structured machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,153 records · Page 64
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Machine Learning of Plasma Science for Next Generation Microelectronics (Project Final Report)

Low temperature plasmas (LTPs) are an enabling technology behind reducing device dimensions and the continuation of Moore’s Law. It is estimated that 40-45% of all process steps necessary to manufacture semiconductor devices involve LTPs. However, challenges in plasma process design and continuous incorporation of novel materials for new device architectures are pushing the limits of what is possible with current plasma technology. For example, creating higher aspect ratio structures and etching features at the atomic scale both require finer control of the ion energy/velocity at wafer surfaces. To support these types of future innovations in the plasma processing systems that Sandia and the DOE rely upon, we have developed novel diagnostics, simulations, and machine learning capabilities to discover, characterize, and predict plasma phenomena affecting the ion energy/velocity distribution function (IEDF). These efforts also supported research program development and external collaboration with industry and academia through Sandia’s Plasma Research Facility (PRF). This report will focus on the following topics and accomplishments of this three year LDRD project, briefly summarized.

42 ENGINEERING↗
DOE OSTI2021

Revealing the Chemical Bonding in Adatom Arrays via Machine Learning of Hyperspectral Scanning Tunneling Spectroscopy Data

The adatom arrays on surfaces offer an ideal playground to explore the mechanisms of chemical bonding via changes in the local electronic tunneling spectra. While this information is readily available in hyperspectral scanning tunneling spectroscopy data, its analysis has been considerably impeded by a lack of suitable analytical tools. Here we develop a machine learning based workflow combining supervised feature identification in the spatial domain and unsupervised clustering in the energy domain to reveal the details of structure-dependent changes of the electronic structure in adatom arrays on the Co 3 Sn 2 S 2 cleaved surface. This approach, in combination with first-principles calculations, provides insight for using artificial neural networks to detect adatoms and classifies each based on their local neighborhood comprised of other adatoms. These structurally classified adatoms are further spectrally deconvolved. The unexpected inhomogeneity of electronic structures among adatoms in similar configurations is unveiled using this method, suggesting there is not a single atomic species of adatoms, but rather multiple types of adatoms on the Co 3 Sn 2 S 2 surface. This is further supported by a slight contrast difference in the images (or slight size variation) of the topography of the adatoms.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Hierarchical Reinforcement Learning of a Short-Range Bond-Order Potential for Silica: Analytic Embedding of Coordination with Classical Efficiency

Reinforcement learning (RL) has recently emerged as a data-efficient strategy to parametrize short-range interatomic potentials. Building on our past RL optimization of pairwise silica models, we extend the framework to a bond-order (Tersoff-type) potential that provides an analytic embedding of local coordination through a three-body term. A hierarchical RL workflow combining continuous-action Monte Carlo Tree Search and property-based rewards efficiently explores the 26-dimensional parameter space, sequentially optimizing lattice parameters, densities, angles, and cohesive energies of 21 silica polymorphs. The resulting models, Q-Tersoff and ML-Tersoff, reproduce the energetic ordering of low-energy phases and capture the angular correlations and amorphous structure factors of silica with improved fidelity over pairwise force fields, while remaining orders of magnitude faster than high-dimensional machine-learned potentials. Both models underperform for elastic constants and high-energy frameworks, delineating the limits of the current analytic form. The approach establishes a general and interpretable route to angle-aware, short-range potentials that bridge physics-based and machine-learned descriptions of silicate materials.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Matminer: An open source toolkit for materials data mining

As materials data sets grow in size and scope, the role of data mining and statistical learning methods to analyze these materials data sets and build predictive models is becoming more important. This manuscript introduces matminer, an open-source, Python-based software platform to facilitate data-driven methods of analyzing and predicting materials properties. Matminer provides modules for retrieving large data sets from external databases such as the Materials Project, Citrination, Materials Data Facility, and Materials Platform for Data Science. It also provides implementations for an extensive library of feature extraction routines developed by the materials community, with 47 featurization classes that can generate thousands of individual descriptors and combine them into mathematical functions. Finally, matminer provides a visualization module for producing interactive, shareable plots. These functions are designed in a way that integrates closely with machine learning and data analysis packages already developed and in use by the Python data science community. We explain the structure and logic of matminer, provide a description of its various modules, and showcase several examples of how matminer can be used to collect data, reproduce data mining studies reported in the literature, and test new methodologies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Unsupervised word embeddings capture latent knowledge from materials science literature

The overwhelming majority of scientific knowledge is published as text, which is difficult to analyse by either traditional statistical analysis or modern machine learning methods. By contrast, the main source of machine-interpretable data for the materials research community has come from structured property databases, which encompass only a small fraction of the knowledge present in the research literature. Beyond property values, publications contain valuable knowledge regarding the connections and relationships between data items as interpreted by the authors. To improve the identification and use of this knowledge, several studies have focused on the retrieval of information from scientific literature using supervised natural language processing, which requires large hand-labelled datasets for training. Here we show that materials science knowledge present in the published literature can be efficiently encoded as information-dense word embeddings (vector representations of words) without human labelling or supervision. Without any explicit insertion of chemical knowledge, these embeddings capture complex materials science concepts such as the underlying structure of the periodic table and structure-property relationships in materials. Furthermore, we demonstrate that an unsupervised method can recommend materials for functional applications several years before their discovery. This suggests that latent knowledge regarding future discoveries is to a large extent embedded in past publications. Lastly, our findings highlight the possibility of extracting knowledge and relationships from the massive body of scientific literature in a collective manner, and point towards a generalized approach to the mining of scientific literature.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Extensive Attention Mechanisms in Graph Neural Networks for Materials Discovery

We present our research where attention mechanism is extensively applied to various aspects of graph neural net- works for predicting materials properties. As a result, surrogate models can not only replace costly simulations for materials screening but also formulate hypotheses and insights to guide further design exploration. We predict formation energy of the Materials Project and gas adsorption of crystalline adsorbents, and demonstrate the superior performance of our graph neural networks. Moreover, attention reveals important substructures that the machine learning models deem important for a material to achieve desired target properties. Our model is based solely on standard structural input files containing atomistic descriptions of the adsorbent material candidates. We construct novel methodological extensions to match the prediction accuracy of state-of-the-art models some of which were built with hundreds of features at much higher computational cost. We show that sophisticated neural networks can obviate the need for elaborate feature engineering. Our approach can be more broadly applied to optimize gas capture processes at industrial scale.

Cong, Guojing↗
DOE OSTI2024

A Causal Approach to Model Validation and Calibration

This poster presents a novel method for validation and verification that focuses on identifying causal relationships between data elements, moving beyond traditional statistical and machine learning approaches. These methods employ causal discovery techniques to reveal the underlying mechanisms of data generation. The research utilizes structural causal models and directed acyclic graphs to depict causal relationships. This approach assists in achieving alignment between simulation models and reality.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Hierarchical, rotation‐equivariant neural networks to select structural models of protein complexes

Abstract Predicting the structure of multi‐protein complexes is a grand challenge in biochemistry, with major implications for basic science and drug discovery. Computational structure prediction methods generally leverage predefined structural features to distinguish accurate structural models from less accurate ones. This raises the question of whether it is possible to learn characteristics of accurate models directly from atomic coordinates of protein complexes, with no prior assumptions. Here we introduce a machine learning method that learns directly from the 3D positions of all atoms to identify accurate models of protein complexes, without using any precomputed physics‐inspired or statistical terms. Our neural network architecture combines multiple ingredients that together enable end‐to‐end learning from molecular structures containing tens of thousands of atoms: a point‐based representation of atoms, equivariance with respect to rotation and translation, local convolutions, and hierarchical subsampling operations. When used in combination with previously developed scoring functions, our network substantially improves the identification of accurate structural models among a large set of possible models. Our network can also be used to predict the accuracy of a given structural model in absolute terms. The architecture we present is readily applicable to other tasks involving learning on 3D structures of large atomic systems.

Eismann, Stephan↗
DOE OSTI2024

Evaluation of GlassNet for physics-informed machine learning of glass stability and glass-forming ability

Glassy materials form the basis of many modern applications, including nuclear waste immobilization, touch-screen displays, and optical fibers, and also hold great potential for future medical and environmental applications. However, their structural complexity and large composition space make design and optimization challenging for certain applications. Of particular importance for glass processing and design is an estimate of a given composition's glass-forming ability (GFA). However, there remain many open questions regarding the underlying physical mechanisms of glass formation, especially in oxide glasses. It is apparent that a proxy for GFA would be highly useful in glass processing and design, but identifying such a surrogate property has proven itself to be difficult. While glass stability (GS) parameters have historically been used as a GFA surrogate, recent research has demonstrated that most of these parameters are not accurate predictors of the GFA of oxide glasses. Here, in this work, we explore the application of an open-source pre-trained neural network model, GlassNet, that can predict the characteristic temperatures necessary to compute GS with reasonable performance and assess the feasibility of using these physics-informed machine learning (PIML)-predicted GS parameters to estimate GFA. In doing so, we track the uncertainties at each step of the computation—from the original ML prediction errors to the compounding of errors during GS estimation, and finally to the final estimation of GFA. While GlassNet exhibits reasonable accuracy on all individual properties, we observe a large compounding of error in the combination of these individual predictions for the PIML prediction of GS, finding that random forest models offer similar accuracy to GlassNet. We also break down the performance of GlassNet on different glass families and find that the error in GS prediction is correlated with the error in crystallization peak temperature prediction. Lastly, we utilize this finding to assess the relationship between top-performing GS parameters and GFA for two ternary glass systems: sodium borosilicate and sodium iron phosphate glasses. We conclude that to obtain true ML predictive capability of GFA, significantly more data needs to be collected.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Multi defect detection and analysis of electron microscopy images with deep learning

Electron microscopy is widely used to explore defects in crystal structures, but human detecting of defects is often time-consuming, error-prone, and unreliable, and is not scalable to large numbers of images or real-time analysis. In this work, we discuss the application of machine learning approaches to find the location and geometry of different defect clusters in irradiated steels. We show that a deep learning based Faster R-CNN analysis system has a performance comparable to human analysis with relatively small training data sets. Furthermore, this study proves the promising ability to apply deep learning to assist the development of automated microscopy data analysis even when multiple features are present and paves the way for fast, scalable, and reliable analysis systems for massive amounts of modern electron microscopy data.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Cyote-attack Chain Estimator

Attack Chain Estimator (ACE) Application Overview The Attack Chain Estimator (ACE) Application is a sophisticated tool designed for the ingestion, classification, sequencing, and enrichment of cybersecurity threat reports. This application leverages advanced machine learning models and extensive historical data to provide comprehensive insights into cyber threats, specifically targeting Industrial Control Systems (ICS). Purpose The primary functions of the ACE Application include: Ingestion of Cybersecurity Threat Reporting: Capable of ingesting text-based threat reports in markdown or text file format. Supports ingestion of structured data from other sources in STIX/JSON format. Classification of Report’s Text-Based Events: Utilizes a DeBERTa classifier, specifically trained on cybersecurity data, to map the events to MITRE ATT&CK for ICS Tactics and Techniques. Classification is performed using multiple Jupyter notebooks and machine learning workflows hosted as FastAPI microservices: regex_data deberta_base_35_train_hft_classifier_mlflow.ipynb hft_regex_classifier_mlflow.ipynb param_train_hft_classifier_mlflow.ipynb regex_tactic_tech.ipynb Ordering of Tactics, Techniques, and Observable Events: Sequences the identified tactics, techniques, and events to form a coherent attack chain. Enrichment with Historical Attack Chain Details: Enhances the attack chain with details from historical attacks using a Markov model developed from CyOTE Precursor Analysis Report data. The Markov model is available as a FastAPI endpoint for seamless integration. Enrichment with Adversary Emulation Capabilities Data: Integrates adversary emulation capabilities data using MITRE Caldera for OT adversary abilities UUIDs. Export of Output Files: Provides options to export the enriched attack chain in JSON or CSV formats. Routing of Output to Other Applications: Facilitates routing of output to various platforms and applications, including: Threat Intelligence Platforms COREII Scout for Threat Intelligence Analysis COREII Modeling and Simulation for Adversary Emulation Technical Description The ACE Application is an advanced cybersecurity tool designed to provide detailed threat analysis and sequence generation. It is built on a robust architecture that integrates natural language processing, machine learning, and historical data modeling. Key Components: Data Ingestion Module: Handles the input of threat reports and data from various formats, ensuring flexibility in data sources. Classification Engine: Employs DeBERTa-based classifiers hosted as FastAPI microservices to analyze and classify threat report events in accordance with the MITRE ATT&CK framework for ICS. Sequence Generator: Orders the classified events into a logical attack chain, providing clear insight into the sequence of tactics and techniques used in the threat. Enrichment Engine: Integrates historical data and adversary emulation capabilities to enhance the attack chain with valuable context and additional details. The historical data enrichment is powered by a Markov model, which is available as a FastAPI endpoint. Export and Routing Module: Facilitates the export of the enriched attack chain in multiple formats and routes the output to designated applications for further analysis or emulation.

Paul, Tony [Idaho National Laboratory (INL), Idaho↗
DOE OSTI2023

Airfoil Computational Fluid Dynamics - 2k shapes, 25 AoA's, 3 Re numbers

This dataset contains aerodynamic quantities - including flow field values (momentum, energy, and vorticity) and summary values (coefficients of lift, drag, and momentum) - for 1,830 airfoil shapes computed using the HAM2D CFD (computational fluid dynamics) model. The airfoil shapes were designed using the separable shape tensor parameterization that encodes two-dimensional shapes as elements of the Grassmann manifold. This data-driven approach learns two independent spaces of parameter from a collection of sample airfoils. The first captures large-scale, linear perturbations, and the second defines small-scale, higher-order perturbations. For this dataset, we used the G2Aero database of over 19,000 airfoil shapes to learn a parameter space that captured a wide array of shape characteristics. We sampled airfoil designs over both parameter spaces to explore the full range of possible shape variations. The aerodynamic quantities for the generated airfoil were obtained using the HAM2D code, which is a finite-volume Reynolds-averaged Navier-Stokes (RANS) flow solver. We employ a fifth-order WENO scheme for spatial reconstruction with Roe's flux difference scheme for inviscid flux and second-order central differencing for viscous flux. A preconditioned GMRES method is applied for implicit integration. The Spalart-Allmaras 1-eq turbulence model is used for the turbulence closure, and the Medida-Baeder 2-eq transition model is applied to account for the effects of laminar turbulent transition. The airfoil grid is generated with a total of 400 points on the airfoil surface, the initial wall-normal spacing of y+ = 1, and an outer boundary located at 300 chord lengths away from the wall. The CFD simulations are performed at a freestream Mach number of 0.1, for or three different Reynolds' numbers (3M, 6M, and 9M), and for 25 angles of attack from -4 deg. to 20 deg. with 1 degree increments. Across all these various parameters, this dataset includes the results from over 250,000 CFD simulations. The simulations were performed using the Bridges-2 system at the Pittsburgh Supercomputing Center in February 2023 as part of the INTEGRATE project funded by the Advanced Research Projects Agency - Energy, in the U.S. Department of Energy. The data was collected, reformatted, and preprocessed for this OEDI submission in July 2023 under the Foundational AI for Wind Energy project funded by the U.S. Department of Energy Wind Energy Technologies Office. This dataset is intended to serve as a benchmark against which new artificial intelligence (AI) or machine learning (ML) tools may be tested. Baseline AI/ML methods for analyzing this dataset have been implemented, and a link to their repository containing those models has been provided. The .h5 data file structure can be found in the GitHub Repository resource under explore_airfoil_2k_data.ipynb.

2k↗
DOE OSTI2023

Airfoil Computational Fluid Dynamics - 9k shapes, 2 AoA's

This dataset contains aerodynamic quantities - including flow field values (momentum, energy, and vorticity) and summary values (coefficients of lift, drag, and momentum) - for 8,996 airfoil shapes, computed using the HAM2D CFD (computational fluid dynamics) model. The airfoil shapes were designed using the separable shape tensor parameterization that encodes two-dimensional shapes as elements of the Grassmann manifold. This data-driven approach learns two independent spaces of parameter from a collection of sample airfoils. The first captures large-scale, linear perturbations, and the second defines small-scale, higher-order perturbations. For this data, we used the G2Aero database of over 19,000 airfoil shapes to learn a parameter space that captured a wide array of shape characteristics. We fixed the linear deformations to be the mean over the database and sampled new shapes over a four-dimensional parameter space of higher-order perturbation. This sampling approaches allows for isolated analysis of non-linear airfoil shape deformations while holding other aspects (e.g., airfoil thickness) approximately constant. The aerodynamic quantities for the generated airfoil were obtained using the HAM2D code, which is a finite-volume Reynolds-averaged Navier-Stokes (RANS) flow solver. We employ a fifth-order WENO scheme for spatial reconstruction with Roe's flux difference scheme for inviscid flux and second-order central differencing for viscous flux. A preconditioned GMRES method is applied for implicit integration. The Spalart-Allmaras 1-eq turbulence model is used for the turbulence closure, and the Medida-Baeder 2-eq transition model is applied to account for the effects of laminar turbulent transition. The airfoil grid is generated with a total of 400 points on the airfoil surface, the initial wall-normal spacing of y+ = 1, and an outer boundary located at 300 chord lengths away from the wall. The CFD simulations are performed at a freestream Mach number of 0.1, Reynolds number of 9M, and at two angles of attack, 4 deg. and 12 deg. The simulations were performed using the Bridges-2 system at the Pittsburgh Supercomputing Center in February 2023 as part of the INTEGRATE project funded by the Advanced Research Projects Agency - Energy in the U.S. Department of Energy. The data was collected, reformatted, and preprocessed for this OEDI submission in July 2023 under the Foundational AI for Wind Energy project funded by the U.S. Department of Energy Wind Energy Technologies Office. This dataset is intended to serve as a benchmark against which new artificial intelligence (AI) or machine learning (ML) tools may be tested. Baseline AI/ML methods for analyzing this dataset have been implemented, and a link to their repository containing those models has been provided. The .h5 data file structure can be found in the GitHub Repository resource under explore_airfoil_9k_data.ipynb.

9k↗
DOE OSTI2023

Improving Protein–Ligand Interaction Modeling with cryo-EM Data, Templates, and Deep Learning in 2021 Ligand Model Challenge

Elucidating protein–ligand interaction is crucial for studying the function of proteins and compounds in an organism and critical for drug discovery and design. The problem of protein–ligand interaction is traditionally tackled by molecular docking and simulation, which is based on physical forces and statistical potentials and cannot effectively leverage cryo-EM data and existing protein structural information in the protein–ligand modeling process. In this work, we developed a deep learning bioinformatics pipeline (DeepProLigand) to predict protein–ligand interactions from cryo-EM density maps of proteins and ligands. DeepProLigand first uses a deep learning method to predict the structure of proteins from cryo-EM maps, which is averaged with a reference (template) structure of the proteins to produce a combined structure to add ligands. The ligands are then identified and added into the structure to generate a protein–ligand complex structure, which is further refined. The method based on the deep learning prediction and template-based modeling was blindly tested in the 2021 EMDataResource Ligand Challenge and was ranked first in fitting ligands to cryo-EM density maps. These results demonstrate that the deep learning bioinformatics approach is a promising direction for modeling protein–ligand interactions on cryo-EM data using prior structural information.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Cluster expansion by transfer learning for phase stability predictions

Recent progress towards universal machine-learned interatomic potentials holds considerable promise for materials discovery. Yet the accuracy of these potentials for predicting phase stability may still be limited. In contrast, cluster expansions provide accurate phase stability predictions but are computationally demanding to parameterize from first principles, especially for structures of low dimension or with a large number of components, such as interfaces or multimetal catalysts. We overcome this trade-off via transfer learning. Using Bayesian inference, we incorporate prior statistical knowledge from machine-learned and physics-based potentials, enabling us to sample the most informative configurations and to efficiently fit first-principles cluster expansions. Furthermore, this algorithm is tested on Pt:Ni, showing robust convergence of the mixing energies as a function of sample size with reduced statistical fluctuations.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Comparing Mapper Graphs of Artificial Neuron Activations

The mapper graph is a popular tool from topological data analysis that provides a graphical summary of point cloud data. It has been used to study data from cancer research, sports analytics, neurosciences, and machine learning. In particular, mapper graphs have been used recently to visualize the topology of high-dimensional artificial neural activations from convolutional neural networks and large language models. However, a key question that arises from using mapper graphs across applications is how to compare mapper graphs to study their structural differences. In this paper, we introduce a distance between mapper graphs using tools from optimal transport. We demonstrate the utility of such a distance by studying the topological changes of neural activations across convolutional layers in deep learning, as well as by capturing the loss of structural information for multiscale mapper.

mapper graphs, computational topology, machine lea↗
DOE OSTI2026

Charge accumulation and solvation in $β$-NiOOH: Surface chemistry of an OER catalyst from ML-aided simulations

Electrochemical water splitting is a key technology for a sustainable energy transition, providing a route to store surplus electricity from renewable sources. A central bottleneck is the sluggish oxygen evolution reaction (OER), which drives the search for catalysts that are active, stable, and inexpensive enough for large-scale deployment. Within this context, pure and doped NiO x H y combine high activity with low cost, making them prime candidates for alkaline OER. Yet, despite extensive study, the atomistic structure of NiOOH under operando conditions and the associated reaction mechanisms remain debated. Here, we investigate the structural complexity of pure β-NiOOH, the scaffold for its doped derivatives. We systematically investigate the oxidation of the surface adsorbates via proton-coupled electron transfer steps across relevant facets and sites, identifying the most probable sequence of deprotonation events. Our results reveal asymmetric charge accumulation on Wulff-relevant surfaces and show how applied potential can promote morphological restructuring. Explicit solvation is included through machine-learning interatomic potential molecular dynamics of the NiOOH/water interface, which allows us to resolve the hydrophobic and hydrophilic character of different surfaces and the associated interfacial water structure. Together, these insights demonstrate how surface chemistry and solvation jointly govern the stability of NiOOH and the accumulation of surface charge, with possible implications for catalytic performance.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Machine learning-assisted crystal engineering of a zeolite

Abstract It is shown that Machine Learning (ML) algorithms can usefully capture the effect of crystallization composition and conditions (inputs) on key microstructural characteristics (outputs) of faujasite type zeolites (structure types FAU, EMT, and their intergrowths), which are widely used zeolite catalysts and adsorbents. The utility of ML (in particular, Geometric Harmonics) toward learning input-output relationships of interest is demonstrated, and a comparison with Neural Networks and Gaussian Process Regression, as alternative approaches, is provided. Through ML, synthesis conditions were identified to enhance the Si/Al ratio of high purity FAU zeolite to the hitherto highest level (i.e., Si/Al = 3.5) achieved via direct (not seeded), and organic structure-directing-agent-free synthesis from sodium aluminosilicate sols. The analysis of the ML algorithms’ results offers the insight that reduced Na 2 O content is key to formulating FAU materials with high Si/Al ratio. An acid catalyst prepared by partial ion exchange of the high-Si/Al-ratio FAU (Si/Al = 3.5) exhibits improved proton reactivity (as well as specific activity, per unit mass of catalyst) in propane cracking and dehydrogenation compared to the catalyst prepared from the previously reported highest Si/Al ratio (Si/Al = 2.8).

36 MATERIALS SCIENCE↗