Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “graph convolutional networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

FPGA Acceleration of GCN in Light of the Symmetry of Graph Adjacency Matrix

Graph Convolutional Neural Networks (GCNs) are widely used to process large-scale graph data. Different from deep neural networks (DNNs), GCNs are sparse, irregular, and unstructured, posing unique challenges to hardware acceleration with regular processing elements (PEs). In particular, the adjacency matrix of a GCN is extremely sparse, leading to frequent but irregular memory access, low spatial/temporal data locality and poor data reuse. Furthermore, a realistic graph usually consists of unstructured data (e.g., unbalanced distributions), creating significantly different processing times and imbalanced workload for each node in GCN acceleration. To overcome these challenges, we propose an end-to-end hardware-software co-design to accelerate GCNs on resource-constrained FPGAs with the features including: (1) A custom dataflow that leverages symmetry along the diagonal of the adjacency matrix to accelerate feature aggregation for undirected graphs. We utilize either the upper or the lower triangular matrix of the adjacency matrix to perform aggregation in GCN to improve data reuse. (2) Unified compute cores for both aggregation and transform phases, with full support to the symmetry-based dataflow. These cores can be dynamically reconfigured to the systolic mode for transformation or as individual accumulators for aggregation in GCN processing. (3) Preprocessing of the graph in software to rearrange the edges and features to match the custom dataflow. This step improves the regularity in memory access and data reuse in the aggregation phase. Moreover, we quantize the GCN precision from FP32 to INT8 to reduce the memory footprint without losing the inference accuracy. We implement our accelerator design in Intel Stratix10 MX FPGA board with HBM2, and demonstrate 1.3x-110.5x improvement in end-to-end GCN latency as compared to the state-of the-art FPGA implementations, on the graph datasets of Cora, Pubmed, Citeseer and Reddit.

Nair, Gopikrishnan R.↗
DOE OSTI2022

Multi-task graph neural networks for simultaneous prediction of global and atomic properties in ferromagnetic systems *

Abstract We introduce a multi-tasking graph convolutional neural network, HydraGNN, to simultaneously predict both global and atomic physical properties and demonstrate with ferromagnetic materials. We train HydraGNN on an open-source ab initio density functional theory (DFT) dataset for iron-platinum with a fixed body centered tetragonal lattice structure and fixed volume to simultaneously predict the mixing enthalpy (a global feature of the system), the atomic charge transfer, and the atomic magnetic moment across configurations that span the entire compositional range. By taking advantage of underlying physical correlations between material properties, multi-task learning (MTL) with HydraGNN provides effective training even with modest amounts of data. Moreover, this is achieved with just one architecture instead of three, as required by single-task learning (STL). The first convolutional layers of the HydraGNN architecture are shared by all learning tasks and extract features common to all material properties. The following layers discriminate the features of the different properties, the results of which are fed to the separate heads of the final layer to produce predictions. Numerical results show that HydraGNN effectively captures the relation between the configurational entropy and the material properties over the entire compositional range. Overall, the accuracy of simultaneous MTL predictions is comparable to the accuracy of the STL predictions. In addition, the computational cost of training HydraGNN for MTL is much lower than the original DFT calculations and also lower than training separate STL models for each property.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2023

Graph neural networks predict energetic and mechanical properties for models of solid solution metal alloy phases

Here, we developed a PyTorch-based architecture called HydraGNN that implements graph convolutional neural networks (GCNNs) to predict the formation energy and the bulk modulus for models of solid solution alloys for various atomic crystal structures and relaxed volumes. We trained the GCNN surrogate model on a dataset for nickel–niobium (NiNb) generated by the embedded atom model (EAM) empirical interatomic potential for demonstration purposes. The dataset was generated by calculating the formation energy and the bulk modulus as a prototypical elastic property for optimized geometries starting from initial body-centered cubic (BCC), face-centered cubic (FCC), and hexagonal compact packed (HCP) crystal structures, with configurations spanning the possible compositional range for each of the three types of initial crystal structures. Numerical results show that the GCNN model effectively predicts both the formation energy and the bulk modulus as function of the optimized crystal structure, relaxed volume, and configurational entropy of the model structures for solid solution alloys.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Accurate Prediction of Voltage of Battery Electrode Materials Using Attention-Based Graph Neural Networks

Performing first-principles calculations to discover electrodes’ properties in the large chemical space is a challenging task. While machine learning (ML) has been applied to effectively accelerate those discoveries, most of the applied methods ignore the materials’ spatial information and only use predefined features: based only on chemical compositions. Here, we propose two attention-based graph convolutional neural network techniques to learn the average voltage of electrodes. Our proposed methods, which combine both atomic composition and atomic coordinates in 3D-space, improve the accuracy in voltage prediction significantly when compared to composition-based ML models. The first model directly learns the chemical reaction of electrodes and metal ions to predict their average voltage, whereas the second model combines electrodes’ ML predicted formation energy (E form ) to compute their average voltage. Our E form -based model demonstrates improved accuracy in transferability from our subset of learned Li ions to Na ions. Moreover, we predicted the theoretical voltage of 10 Na x MPO 4 F (M = Ti, Cr, Fe, Cu, Mn, Co, and Ni) fluorophosphate battery frameworks, which are unavailable in the Material Project database. It could be shown that we can expect average voltages higher than 3.1 V from those Na battery frameworks except from the NaTiPO 4 F and TiPO 4 F pair of electrodes, which offer an average voltage of 1.32 V.

25 ENERGY STORAGE↗
DOE OSTI2024

Comparison of Machine Learning Approaches for Prediction of the Equivalent Alkane Carbon Number for Microemulsions Based on Molecular Properties

The chemical properties of oils are vital in the design of microemulsion systems. The hydrophilic–lipophilic difference equation used to predict microemulsions’ phase behavior expresses the oils’ physiochemical properties as the equivalent alkane carbon number (EACN). The experimental determination of EACN requires knowledge of the temperature dependence of the microemulsion system and the effects of different surfactant concentrations. Thus, the experimental determination is time-intensive and tedious, requiring days to months for proper separations. Furthermore, the experiments require high purity of chemicals because microemulsions are sensitive to impurities. Our work focuses on the quick and reliable predictions of the EACN with machine learning (ML) models. Due to the immaturity of ML chemical predictions, we compare three graph neural networks (GNNs) and a gradient-boosted tree algorithm, known as XGBoost. The GNNs use the molecular structures represented as simplified molecular-input line-entry system (SMILES) codes for the initial input, which allows us to assess whether geometry optimization is necessary for reliable results. The XGBoost model also begins with the SMILES representations of the molecules but uses molecular descriptors instead of geometry optimizations. As a result, the best model tested (crystal graph convolutional neural network with Merck molecular force field-94) has an error of 1.15 EACN units of the true EACN for unknown data with the errors skewed toward zero and an R² score of 0.9

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Improving Variational Autoencoders for New Physics Detection at the LHC With Normalizing Flows

We investigate how to improve new physics detection strategies exploiting variational autoencoders and normalizing flows for anomaly detection at the Large Hadron Collider. As a working example, we consider the DarkMachines challenge dataset. We show how different design choices (e.g., event representations, anomaly score definitions, network architectures) affect the result on specific benchmark new physics models. Once a baseline is established, we discuss how to improve the anomaly detection accuracy by exploiting normalizing flow layers in the latent space of the variational autoencoder.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Development of Whole System Digital Twins for Advanced Reactors: Leveraging Graph Neural Networks and SAM Simulations

Here, in this work, we introduce a novel method to develop whole system digital twins (DTs) for advanced nuclear reactors. This method treats a complex reactor system as a heterogeneous graph: with the system components as different types of graph nodes and their physical interconnections as edges. Based on the heterogeneous graph, a graph neural network combining graph convolution and temporal node attention is developed as the DT, facilitating a comprehensive understanding of the system's dynamic behavior. By utilizing the System Analysis Module (SAM) code for simulating various operational transients, we develop a graph-based database that trains the DT. This DT is characterized by two primary functions: It can infer the entire system's status using sparse node information, and it can predict the progress of transients based on current and historical system information. Our approach is validated through case studies on the Experimental Breeder Reactor II (EBR-II) system and a generic Fluoride-salt-cooled High-temperature Reactor (gFHR), demonstrating the DT's accuracy in forecasting operational transients. The DT's rapid computation capabilities enhance its potential for supporting advanced reactor operations, offering benefits in intelligent simulation, autonomous control, and anomaly detection, paving the way for improved safety analysis and intelligent component health management for advanced reactor systems and reducing their operations and maintenance cost.

EBR-II↗
DOE OSTI2024

Transferring predictions of formation energy across lattices of increasing size*

In this study, we show the transferability of graph convolutional neural network (GCNN) predictions of the formation energy of the nickel-platinum solid solution alloy across atomic structures of increasing sizes. The original dataset was generated with the large-scale atomic/molecular massively parallel simulator using the second nearest-neighbor modified embedded-atom method empirical interatomic potential. Geometry optimization was performed on the initially randomly generated face centered cubic crystal structures and the formation energy has been calculated at each step of the geometry optimization, with configurations spanning the whole compositional range. Using data from various steps of the geometry optimization, we first trained our open-source, scalable implementation of GCNN called HydraGNN on a lattice of 256 atoms, which accounts well for the short-range interactions. Using this data, we predicted the formation energy for lattices of 864 atoms and 2048 atoms, which resulted in lower-than-expected accuracy due to the long-range interactions present in these larger lattices. We accounted for the long-range interactions by including a small amount of training data representative for those two larger sizes, whereupon the predictions of HydraGNN scaled linearly with the size of the lattice. Therefore, our strategy ensured scalability while reducing significantly the computational cost of training on larger lattice sizes.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Explainable Synthesizability Prediction of Inorganic Crystal Polymorphs Using Large Language Models

Abstract We evaluate the ability of machine learning to predict whether a hypothetical crystal structure can be synthesized and explain those predictions to scientists. Fine‐tuned large language models (LLMs) trained on a human‐readable text description of the target crystal structure perform comparably to previous bespoke convolutional graph neural network methods, but better prediction quality can be achieved by training a positive‐unlabeled learning model on a text‐embedding representation of the structure. An LLM‐based workflow can then be used to generate human‐readable explanations for the types of factors governing synthesizability, extract the underlying physical rules, and assess the veracity of those rules. These explanations can guide chemists in modifying or optimizing non‐synthesizable hypothetical structures to make them more feasible for materials design.

Kim, Seongmin [Department of Chemical and Biologic↗
DOE OSTI2025

Explainable Synthesizability Prediction of Inorganic Crystal Polymorphs Using Large Language Models

Abstract We evaluate the ability of machine learning to predict whether a hypothetical crystal structure can be synthesized and explain those predictions to scientists. Fine‐tuned large language models (LLMs) trained on a human‐readable text description of the target crystal structure perform comparably to previous bespoke convolutional graph neural network methods, but better prediction quality can be achieved by training a positive‐unlabeled learning model on a text‐embedding representation of the structure. An LLM‐based workflow can then be used to generate human‐readable explanations for the types of factors governing synthesizability, extract the underlying physical rules, and assess the veracity of those rules. These explanations can guide chemists in modifying or optimizing non‐synthesizable hypothetical structures to make them more feasible for materials design.

Kim, Seongmin [Department of Chemical and Biologic↗
DOE OSTI2025

Machine learning guided search for energetically favorable metal borocarbide ternary compounds

In this work, we employ machine-learning (ML) combined with first principles calculations to efficiently search for the energetically favorable metal borocarbide (M-B-C) ternary compounds with M being the group 1–3 metal elements. Using a crystal graph convolutional neural network (CGCNN) ML approach followed by first-principles calculations, we predicted 47 energetically favorable stable and metastable ternary Na-B-C, Ca-B-C, and La-B-C ternary compounds with their decomposition energy (E d ) below or within 100 meV/atom from the currently known convex hulls. Phonon spectra and electronic structures of the 14 energetically favorable stable structures are also investigated by first-principles calculations. By substituting the metal atoms in the 29 energetically favorable non-equivalent template structures of Na (Ca, La)-B-C with other group 1–3 elements in the periodic table, we further obtain 22 stable structures and 52 metastable structures (E d ≤100 meV/atom with respect to the known convex hulls) for Li-B-C, K-B-C, Rb-B-C, Mg-B-C, Sr-B-C, Ba-B-C, Sc-B-C and Y-B-C ternary compounds. New convex hulls including our newly predicted stable ternary structures and the known stable structures are constructed for the M-B-C systems. The results obtained by our ML guided first-principles calculations enrich our knowledge in the structure and energy landscape of metal borocarbide ternary compounds and provide useful guidance for further experimental synthesis and discovery.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Search for a boosted Higgs boson decaying to bottom quark pairs in association with a W or Z boson in proton-proton collisions at $\sqrt{s}$ = 13TeV

A search is conducted for standard model Higgs bosons with large transverse momentum ($p_\mathrm{T}$) decaying to bottom quark pairs and produced in association with a hadronically decaying W or Z boson at the LHC. The result is based on a dataset of proton-proton collisions at a center-of-mass energy of 13 TeV collected with the CMS detector in 2016$-$2018, corresponding to an integrated luminosity of 138 fb$^{-1}$. Boosted Higgs, W, and Z boson decays are reconstructed using large-radius jets with $p_\mathrm{T}$$\gt$ 450 GeV and identified with heavy-flavor classifiers based on a graph convolutional neural network. The observed signal strength relative to the standard model expectation is $μ$ = 0.72 $^{+0.75}_{-0.71}$ including statistical and systematic uncertainties.

CMS↗
DOE OSTI2025

Search for Stable and Low-Energy Ce–Co–Cu Ternary Compounds Using Machine Learning

Cerium-based intermetallics have garnered significant research attention as potential new permanent magnets. In this study, we explore the compositional and structural landscape of Ce−Co−Cu ternary compounds using a machine learning (ML)- guided framework integrated with first-principles calculations. We employ a crystal graph convolutional neural network (CGCNN), which enables efficient screening for promising candidates, significantly accelerating the material discovery process. With this approach, we predict five stable compounds, Ce 3 Co 3 Cu, CeCoCu 2 , Ce 12 Co 7 Cu, Ce 11 Co 9 Cu, and Ce 10 Co 11 Cu 4 , with formation energies below the convex hull, along with hundreds of low-energy (possibly metastable) Ce−Co−Cu ternary compounds. Firstprinciples calculations reveal that several structures are both energetically and dynamically stable. Notably, two Co-rich low-energy compounds, Ce 4 Co 33 Cu and Ce 4 Co 31 Cu 3 , are predicted to have high magnetizations.

Chemical structure↗
DOE OSTI2025

Leveraging Natural Language Processing and Generative Models in Molecular Chemistry: Property Prediction and Novel Compound Generation

The accurate prediction of molecular properties is important for the rational design and the advancement of green chemistry and sustainable materials research. However, the predictive power of traditional computational chemistry methods is limited due to computational restrictions. Here, in this study, we examine an alternative approach to the accurate prediction of properties of organic compounds: natural language processing (NLP)-based molecular embedding. Using viscosity, partition coefficient (log P), and enthalpy of vaporization as test properties through a survey of comprehensive datasets comprising 5695 data points for viscosity, 25 870 data points for log P, and 2296 data points for enthalpy of vaporization. These are important properties for the design of greener, safer, and sustainable chemical processes. Models were trained using NLP methods such as Mol2vec and fine-tuned ChemBERTa, and results were compared with traditional input featurization techniques such as Morgan fingerprints and quantum chemistry derived sigma profiles and DFT features. Among the various machine learning models, Mol2vec demonstrated superior predictive capabilities, achieving the highest correlation coefficient (R 2 = 0.945) and lowest RMSE (0.106 mPa s) for viscosity, as well as high accuracy for log P and enthalpy of vaporization predictions. These findings establish the Mol2vec featurization technique, graph-convolutional neural networks (GCNN), and fine-tuned ChemBERTa model as powerful tools for predictive modeling of organic compounds properties, offering a significant improvement over previously used featurization techniques and opening up strategies for very-high-throughput computational screening. Finally, we integrated ML models with hybrid language-model-based generative adversarial networks (LM-GAN) to generate novel molecular sequences with desirable properties for different research applications. The ability to computationally design solvents with lower viscosity, lower log P, and lower enthalpy of vaporization offers a data-driven route to accelerating the discovery of sustainable alternatives to traditionally toxic solvents.

ChemBERTa↗
DOE OSTI2022

Adsorbate chemical environment-based machine learning framework for heterogeneous catalysis

Abstract Heterogeneous catalytic reactions are influenced by a subtle interplay of atomic-scale factors, ranging from the catalysts’ local morphology to the presence of high adsorbate coverages. Describing such phenomena via computational models requires generation and analysis of a large space of atomic configurations. To address this challenge, we present Adsorbate Chemical Environment-based Graph Convolution Neural Network (ACE-GCN), a screening workflow that accounts for atomistic configurations comprising diverse adsorbates, binding locations, coordination environments, and substrate morphologies. Using this workflow, we develop catalyst surface models for two illustrative systems: (i) NO adsorbed on a Pt 3 Sn(111) alloy surface, of interest for nitrate electroreduction processes, where high adsorbate coverages combined with low symmetry of the alloy substrate produce a large configurational space, and (ii) OH* adsorbed on a stepped Pt(221) facet, of relevance to the Oxygen Reduction Reaction, where configurational complexity results from the presence of irregular crystal surfaces, high adsorbate coverages, and directionally-dependent adsorbate-adsorbate interactions. In both cases, the ACE-GCN model, trained on a fraction (~10%) of the total DFT-relaxed configurations, successfully describes trends in the relative stabilities of unrelaxed atomic configurations sampled from a large configurational space. This approach is expected to accelerate development of rigorous descriptions of catalyst surfaces under in-situ conditions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Scale-invariant machine-learning model accelerates the discovery of quaternary chalcogenides with ultralow lattice thermal conductivity

We design an advanced machine-learning (ML) model based on crystal graph convolutional neural network that is insensitive to volumes (i.e., scale) of the input crystal structures to discover novel quaternary chalcogenides, AMM'Q 3 (A/M/M' = alkali, alkaline earth, post-transition metals, lanthanides, and Q = chalcogens). These compounds are shown to possess ultralow lattice thermal conductivity (κ l ), a desired requirement for thermal-barrier coatings and thermoelectrics. Upon screening the thermodynamic stability of ~1 million compounds using the ML model iteratively and performing density-functional theory (DFT) calculations for a small fraction of compounds, we discover 99 compounds that are validated to be stable in DFT. Taking several DFT-stable compounds, we calculate their κ l using Peierls–Boltzmann transport equation, which reveals ultralow κ l (<2 Wm -1 K -1 at room temperature) due to their soft elasticity and strong phonon anharmonicity. Our work demonstrates the high efficiency of scale-invariant ML model in predicting novel compounds and presents experimental-research opportunities with these new compounds.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Deep learning workflow for the inverse design of molecules with specific optoelectronic properties

The inverse design of novel molecules with a desirable optoelectronic property requires consideration of the vast chemical spaces associated with varying chemical composition and molecular size. First principles-based property predictions have become increasingly helpful for assisting the selection of promising candidate chemical species for subsequent experimental validation. However, a brute-force computational screening of the entire chemical space is decidedly impossible. To alleviate the computational burden and accelerate rational molecular design, we here present an iterative deep learning workflow that combines (i) the density-functional tight-binding method for dynamic generation of property training data, (ii) a graph convolutional neural network surrogate model for rapid and reliable predictions of chemical and physical properties, and (iii) a masked language model. As proof of principle, we employ our workflow in the iterative generation of novel molecules with a target energy gap between the highest occupied molecular orbital (HOMO) and the lowest unoccupied molecular orbital (LUMO).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Machine learning assisted search for Fe–Co–C ternary compounds with high magnetic anisotropy

We employ a machine learning (ML)-guided framework to explore rare earth free magnetic materials, specifically focusing on Fe–Co–C ternary compounds for potential use in permanent magnets. Utilizing a specifically trained crystal graph convolutional neural network model, we efficiently screen a vast space of nearly a million substitutional structures to select 620 promising structures for further investigation by first-principles calculation. We predict five low-energy metastable Fe–Co–C compounds with formation energy less than 150 meV/atom above the convex hull. These compounds exhibit high magnetization (Js > 1.0 T) and significant magnetic anisotropy (K1 > 1.0 MJ/m3), making them promising candidates for permanent magnet applications. The phonon calculations indicate these compounds are dynamically stable. Our ML-guided framework demonstrates the utility of rapidly identifying novel materials with tailored magnetic properties.

36 MATERIALS SCIENCE↗