Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “machine-learned potentials”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

Even Higher-Level Synthesis: An Exploration of AI Hardware Accelerators using HLS4ML

With the rise of artificial intelligence, the popularization of deep learning, and a constantly evolving industry, the demand for flexible and efficient tools has never been greater. As algorithms grow more complex, their runtime and energy consumption increase exponentially. Customized hardware accelerators, long used for specific mathematical operations, remain essential for managing modern applications' computational and power demands. Hardware accelerators can speed up complex computations by orders of magnitude, but their manual design and verification processes are often challenging and time-consuming. High-Level Synthesis (HLS) provides a solution by transforming high-level algorithm descriptions, typically written in C++ or SystemC, into synthesizable RTL suitable for hardware implementation. This approach reduces development time for RTL engineers while offering flexibility beyond what traditional handwritten RTL can provide. We extended this capability to the machine-learning domain with the open-source framework hls4ml, which allows neural networks trained in Python frameworks like Tensorflow or PyTorch to be synthesized into efficient hardware representations for the traditional FPGA and ASIC flows. This breakthrough addresses the growing need for reduced design turnaround and easy verification of ML hardware accelerators with low latency and power efficiency constraints. During this tutorial, we will demonstrate how Python complements HLS by simplifying the ML design process, bridging the gap between software and hardware development. Attendees will explore how we translate neural networks modeled in Python into fixed-point C++ models suitable for HLS workflows. We will dive into strategies like Value-Range Analysis and Quantization-Aware Training, which optimize these designs for deployment and evaluate their accuracy, power consumption, and energy efficiency. To exemplify these concepts, experts from Fermilab will share their experiences applying this technology to high-energy physics experiments, where real-time, low-latency processing is critical. Over the years, Fermilab engineers have demonstrated how deep neural networks, optimized for hardware using hls4ml, can meet the stringent requirements of trigger systems at the CERN Large Hadron Collider. These systems rely on rapid decision-making to process immense data volumes while retaining only the most relevant events for further analysis. The application of hls4ml has also been extended to innovative technologies like smart pixel arrays. These smart pixels integrate ML inference capabilities directly into sensor devices, enabling localized data processing at the pixel level. This approach drastically reduces the need to transmit raw data to external processing units, significantly decreasing power consumption and latency. By embedding neural networks within the pixel architecture, the smart pixels can identify and prioritize relevant data in real time, providing a highly efficient solution for edge computing in scenarios such as particle detectors and imaging systems. Fermilab's work highlights the potential of hardware-accelerated ML in scenarios where both speed and power efficiency are mission-critical. Through this tutorial, attendees will gain valuable insights into the challenges and solutions of deploying ML in hardware. Understanding how HLS and hls4ml streamline the development of neural network-based hardware accelerators is fundamental for the industry's future. Participants will learn how these technologies are shaping the future of AI and scientific computing.

Di Guglielmo, Giuseppe [Fermilab]↗

A Graph Dynamical neural network approach for decoding dynamical states in ferroelectrics.

Ferroelectric materials such as BaTiO 3 show tremendous potential for emerging advances in memory devices, particular neuromorphic type devices. High density of memory can be obtained by stabilising polar domain walls at the nanoscale, regions of discontinuity between the well-defined polarization order parameter, but little is known about what controls their structure and dynamics in real nanoscale materials. Indeed, chiral polar domain walls have been observed in heterogeneous ferroelectrics, such as oxygen-deficient BaTiO 3 , but very little is known about how such polar-domains walls interact with defects. Indeed, a critical understanding of how dynamics of domain-walls depend on point-defects is crucial to create engineered ferroelectric memory devices. For this work, we perform large-scale simulations of nansocale domain-wall dynamics in pristine and defective BaTiO 3 using reactive force-field developed by us earlier (Phys. Chem. Chem. Phys., 2019, 21, 18240–18249), and capture their dynamical dependence on point defects using a graph dynamical neural-network approach, which we adapted to interrogate solids with well-defined order-parameters, and implemented using Pytorch based libraries. Our machine learning (ML) approach goes beyond the traditional post-processing methods to capture both spatial and temporal heterogeneities of large-scale molecular dynamics simulations of complex defective ferroelectric oxide materials. We crucially find that isolated oxygen vacancies introduce very localized spatial regions (~1–2 unit-cell in length) that show slow dipole relaxation due to formation of defect-dipoles, and that these defect-dipoles in turn slow the intrinsic dynamics of domain walls. Further, the roughness of domain walls, also influenced by vacancies, introduce dynamic heterogeneity along the domain-wall. As such we find a novel mechanism by which quenched disorder due to defects introduce dynamic heterogeneity thereby influencing response to external fields (particularly time varying fields) in a ferroelectric. Our study also emphasizes the need for creating digital twins of dynamical quantities to achieve autonomous in operando control of nanoscale switching.

42 ENGINEERING↗

Acceleration of Graph Neural Network-Based Prediction Models in Chemistry via Co-Design Optimization on Intelligence Processing Units

Atomic structure prediction and associated property calculations are the bedrock of chemical physics. Since high-fidelity ab initio modeling techniques for computing the structure and properties can be prohibitively expensive, this motivates the development of machine-learning (ML) models that make these predictions more efficiently. Training graph neural networks over large atomistic databases introduces unique computational challenges such as the need to process millions of small graphs with variable size and support communication patterns that are distinct from learning over large graphs such as social networks. We demonstrate a novel hardware-software co-design approach to scale up the training of atomistic graph neural networks (GNN) for structure and property prediction. First, to eliminate redundant computation and memory associated with alternative padding techniques and to improve throughput via minimizing communication, we formulate the effective coalescing of the batches of variable-size atomistic graphs as the bin packing problem and introduce a hardware-agnostic algorithm to pack these batches. In addition, we propose hardware-specific optimizations including a planner and vectorization for the gather-scatter operations targeted for Graphcore’s Intelligence Processing Unit (IPU), as well as model-specific optimizations such as merged communication collectives and optimized softplus. Putting these all together, we demonstrate the effectiveness of the proposed co-design approach by providing an implementation of a well-established atomistic GNN on the Graphcore IPUs. We evaluate the training performance on multiple atomistic graph databases with varying degrees of graph counts, sizes and sparsity. Here, we demonstrate that such a co-design approach can reduce the training time of atomistic GNNs and can improve the performance by up to 1.5× compared to the baseline implementation of the model on the IPUs. Additionally, we compare our IPU implementation with a Nvidia GPU-based implementation and show that our atomistic GNN implementation on the IPUs can run 1.8× faster on average compared to the execution time on the GPUs.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Crowdsourcing the Frontier: Advancing Hybrid Physics‐ML Climate Simulation via a $\$$50,000 Kaggle Competition

Subgrid machine-learning (machine learning [ML]) parameterizations have the potential to introduce a new generation of climate models that incorporate the effects of higher-resolution physics without incurring the prohibitive computational cost associated with more explicit physics-based simulations. However, important issues, ranging from online instability to inconsistent online performance, have limited their operational use for long-term climate projections. To more rapidly drive progress in solving these issues, domain scientists and ML researchers opened up the offline aspect of this problem to the broader ML and data science community with the release of ClimSim, a NeurIPS Data sets and Benchmarks publication, and an associated Kaggle competition. This paper reports on the downstream results of the Kaggle competition by coupling emulators inspired by the winning teams' architectures to an interactive climate model (including full cloud microphysics, a regime historically prone to online instability) and systematically evaluating their online performance. Our results demonstrate that online stability in the low-resolution real-geography setting is reproducible across multiple diverse architectures, which we consider a key milestone. All tested architectures exhibit strikingly similar offline and online biases, though their responses to architecture-agnostic design choices (e.g., expanding the list of input variables) can differ significantly. Multiple Kaggle-inspired architectures achieve state-of-the-art results on certain metrics such as zonal mean bias patterns and global Root Mean Squared Error, indicating that crowdsourcing the essence of the offline problem is one path to improving online performance in hybrid physics-AI climate simulation.

Environmental sciences↗

Machine-learning-assisted material discovery of oxygen-rich highly porous carbon active materials for aqueous supercapacitors

Abstract Porous carbons are the active materials of choice for supercapacitor applications because of their power capability, long-term cycle stability, and wide operating temperatures. However, the development of carbon active materials with improved physicochemical and electrochemical properties is generally carried out via time-consuming and cost-ineffective experimental processes. In this regard, machine-learning technology provides a data-driven approach to examine previously reported research works to find the critical features for developing ideal carbon materials for supercapacitors. Here, we report the design of a machine-learning-derived activation strategy that uses sodium amide and cross-linked polymer precursors to synthesize highly porous carbons (i.e., with specific surface areas > 4000 m 2 /g). Tuning the pore size and oxygen content of the carbonaceous materials, we report a highly porous carbon-base electrode with 0.7 mg/cm 2 of electrode mass loading that exhibits a high specific capacitance of 610 F/g in 1 M H 2 SO 4 . This result approaches the specific capacitance of a porous carbon electrode predicted by the machine learning approach. We also investigate the charge storage mechanism and electrolyte transport properties via step potential electrochemical spectroscopy and quasielastic neutron scattering measurements.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Atomistic learning in the electronically grand-canonical ensemble

Abstract A strategy is presented for the machine-learning emulation of electronic structure calculations carried out in the electronically grand-canonical ensemble. The approach relies upon a dual-learning scheme, where both the system charge and the system energy are predicted for each image. The scheme is shown to be capable of emulating basic electrochemical reactions at a range of potentials, and coupling it with a bootstrap-ensemble approach gives reasonable estimates of the prediction uncertainty. The method is also demonstrated to accelerate saddle-point searches, and to extrapolate to systems with one to five water layers. We anticipate that this method will allow for larger length- and time-scale simulations necessary for electrochemical simulations.

36 MATERIALS SCIENCE↗

The ab initio non-crystalline structure database: empowering machine learning to decode diffusivity

Non-crystalline materials exhibit unique properties that make them suitable for various applications in science and technology, ranging from optical and electronic devices and solid-state batteries to protective coatings. However, data-driven exploration and design of non-crystalline materials is hampered by the absence of a comprehensive database covering a broad chemical space. In this work, we present the largest computed non-crystalline structure database to date, generated from systematic and accurate ab initio molecular dynamics (AIMD) calculations. We also show how the database can be used in simple machine-learning models to connect properties to composition and structure, here specifically targeting ionic conductivity. These models predict the Li-ion diffusivity with speed and accuracy, offering a cost-effective alternative to expensive density functional theory (DFT) calculations. Furthermore, the process of computational quenching non-crystalline structures provides a unique sampling of out-of-equilibrium structures, energies, and force landscape, and we anticipate that the corresponding trajectories will inform future work in universal machine learning potentials, impacting design beyond that of non-crystalline materials. In addition, combining diffusion trajectories from our dataset with models that predict liquidus viscosity and melting temperature could be utilized to develop models for predicting glass-forming ability.

36 MATERIALS SCIENCE↗

Machine-learning for designing nanoarchitectured materials by dealloying

Machine learning-augmented materials design is an emerging method for rapidly developing new materials. It is especially useful for designing new nanoarchitectured materials, whose design parameter space is often large and complex. Metal-agent dealloying, a materials design method for fabricating nanoporous or nanocomposite from a wide range of elements, has attracted significant interest. Here, a machine learning approach is introduced to explore metal-agent dealloying, leading to the prediction of 132 plausible ternary dealloying systems. A machine learning-augmented framework is tested, including predicting dealloying systems and characterizing combinatorial thin films via automated and autonomous machine learning-driven synchrotron techniques. This work demonstrates the potential to utilize machine learning-augmented methods for creating nanoarchitectured thin films.

36 MATERIALS SCIENCE↗

On ultrafast x-ray scattering methods for magnetism

With the introduction of x-ray free electron laser sources around the world, new scientific approaches for visualizing matter at fundamental length and time-scales have become possible. As it relates to magnetism and ‘magnetic-type’ systems, advanced methods are being developed for studying ultrafast magnetic responses on the time-scales at which they occur. We describe three capabilities which have the potential to seed new directions in this area and present original results from each: pump-probe x-ray scattering with low energy excitation, x-ray photon fluctuation spectroscopy, and ultrafast diffuse x-ray scattering. By combining these experimental techniques with advanced modeling together with machine learning, we describe how the combination of these domains allows for a new understanding in the field of magnetism. Finally, we give an outlook for future areas of investigation and the newly developed instruments which will take us there.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Hamiltonian learning using machine-learning models trained with continuous measurements

Here, we build upon recent work on the use of machine-learning models to estimate Hamiltonian parameters using continuous weak measurement of qubits as input. We consider two settings for the training of our model: (1) supervised learning, where the weak-measurement training record can be labeled with known Hamiltonian parameters, and (2) unsupervised learning, where no labels are available. The first has the advantage of not requiring an explicit representation of the quantum state, thus potentially scaling very favorably to a larger number of qubits. The second requires the implementation of a physical model to map the Hamiltonian parameters to a measurement record, which we implement using an integrator of the physical model with a recurrent neural network to provide a model-free correction at every time step to account for small effects not captured by the physical model. We test our construction on a system of two qubits and demonstrate accurate prediction of multiple physical parameters in both the supervised context and the unsupervised context. We demonstrate that the model benefits from larger training sets, establishing that it is “learning,” and we show robustness regarding errors in the assumed physical model by achieving accurate parameter estimation in the presence of unanticipated single-particle relaxation.

97 MATHEMATICS AND COMPUTING↗

Anomaly detection in collider physics via factorized observables

To maximize the discovery potential of high-energy colliders, experimental searches should be sensitive to unforeseen new physics scenarios. This goal has motivated the use of machine learning for unsupervised anomaly detection. In this paper, we introduce a new anomaly detection strategy called : factorized observables for regressing conditional expectations. Our approach is based on the inductive bias of factorization, which is the idea that the physics governing different energy scales can be treated as approximately independent. Assuming factorization holds separately for signal and background processes, the appearance of nontrivial correlations between low- and high-energy observables is a robust indicator of new physics. Under the most restrictive form of factorization, a machine-learned model trained to identify such correlations will in fact converge to the optimal new physics classifier. We test on a benchmark anomaly detection task for the Large Hadron Collider involving collimated sprays of particles called jets. By teasing out correlations between the kinematics and substructure of jets, our method can reliably extract percent-level signal fractions. This strategy for uncovering new physics adds to the growing toolbox of anomaly detection methods for collider physics with a complementary set of assumptions. Published by the American Physical Society 2024

Astronomy & Astrophysics↗

Learning to Predict Arbitrary Quantum Processes

We present an efficient machine-learning (ML) algorithm for predicting any unknown quantum process ℰ over 𝑛 qubits. For a wide range of distributions 𝒟 on arbitrary 𝑛-qubit states, we show that this ML algorithm can learn to predict any local property of the output from the unknown process ℰ, with a small average error over input states drawn from 𝒟. The ML algorithm is computationally efficient even when the unknown process is a quantum circuit with exponentially many gates. Our algorithm combines efficient procedures for learning properties of an unknown state and for learning a low-degree approximation to an unknown observable. The analysis hinges on proving new norm inequalities, including a quantum analogue of the classical Bohnenblust-Hille inequality, which we derive by giving an improved algorithm for optimizing local Hamiltonians. Numerical experiments on predicting quantum dynamics with evolution time up to 10 6 and system size up to 50 qubits corroborate our proof. Overall, our results highlight the potential for ML models to predict the output of complex quantum dynamics much faster than the time needed to run the process itself.

quantum computation↗

The ab initio amorphous materials database: Empowering machine learning to decode diffusivity

Amorphous materials exhibit unique properties that make them suitable for various applications in science and technology, ranging from optical and electronic devices and solid-state batteries to protective coatings. However, data-driven ex- ploration and design of amorphous materials is hampered by the absence of a com- prehensive database covering a broad chemical space. In this work, we present the largest computed amorphous materials database to date, generated from sys- tematic and accurate ab initio molecular dynamics (AIMD) calculations. We also show how the database can be used in simple machine-learning models to connect properties to composition and structure, here specifically targeting ionic conductiv- ity. These models predict the Li-ion diffusivity with speed and accuracy, offering a cost-effective alternative to expensive density functional theory (DFT) calculations. Furthermore, the process of computational quenching amorphous materials provides a unique sampling of out-of-equilibrium structures, energies, and force landscape, and we anticipate that the corresponding trajectories will inform future work in uni- versal machine learning potentials, impacting design beyond that of non-crystalline materials.

36 MATERIALS SCIENCE↗

Communication-Constrained Robust Control and Learning of Grid-Connected

The electric grid of things (EGoT) promises great potential for innovative grid services by tapping into vast load flexibility. However, the unique characteristics of EGoT, being a part of the cyber-physical electric power system, present both opportunities and challenges, especially concerning supply-demand balancing, stability, and communication constraints. Traditionally, centralized control was employed to ensure balance and stability in power systems. However, with the massive influx of EGoT devices, new strategies are needed to efficiently coordinate and control these distributed devices for optimal grid operations. While some studies have explored efficiency and economic models, there remains a gap in ensuring reliability under everyday operations and resilience during extreme conditions. Addressing this gap, this project develops the technology for an Energy Service Interface (ESI) that includes novel pricing, control, learning, and distributed optimization algorithms, which will enable utilities to recruit EGoT assets for crucial grid services such as load flexibility, voltage regulation, and situation-awareness. The key novelty of the proposed technology is the careful distribution of learning and control functions across utility and EGoT asset owners such that provably efficient and resilient grid operations are attained while respecting communication and information-exchange constraints. Specifically, the project team develops machine-learning enhanced load modeling methods to allow EGoT asset owners to learn their load capability and flexibility, and develops pricing-based and decentralized learning-based control so that asset owners can coordinate to meet system-wide demand-supply balance and reliability goals. For extreme situations involving high-impact, low-probability catastrophic events (termed the “black-sky” events), the team also develops (1) a “Feeder-Operating Center-on-a-Laptop” (FOCAL) software that can assist utility personnel in leveraging EGoT assets to accelerate the service recovery of damaged feeders, and (2) distributed optimization algorithms that can coordinate the operation points of EGoT devices under severe communication constraints. The proposed technology has been extensively tested and evaluated through simulations and on a testbed. In summary, as we transition into a more interconnected and digital power grid era, our project’s findings and developments offer a pivotal step toward guaranteeing both efficiency and resilience in the face of both everyday operations and rare “black-sky” events.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Predicting Failure Using Deep Learning SAND Report

Accurate prediction of ductile failure is critical to Sandia’s NW mission, but the models are computationally heavy. The costs of including high-fidelity physics and mechanics that are germane to the failure mechanisms are often too burdensome for analysts either because of the person-hours it requires to input them or because of the additional computational time, or both. In an effort to deliver analysts a tool for representing these phenomena with minimal impact to their existing workflow, our project sought to develop modern data-driven methods that would add microstructural information to business-as-usual calculations and expedite failure predictions. The goal is a tool that receives as input a structural model with stress and strain fields, as well as a machine-learned model, and output predictions of structural response in time, including failure. As such, our project spent substantial time performing high-fidelity, three-dimensional experiments to elucidate materials mechanisms of void nucleation and evolution. We developed crystal-plasticity finite-element models from the experimental observations to enrich the findings with fields not readily measured. We developed engineering length-scale simulations of replicated test specimens to understand how the engineering fields evolve in the presence of fine-scale defects. Finally, we developed deep learning convolutional neural networks, and graph-based neural networks to encode the findings of the experiments and simulations and make forward predictions in time for structural performance. This project demonstrated the power of data-driven methods for model development, which have the potential to vastly increase both the accuracy and speed of failure predictions. These benefits and the methods necessary to develop them are highlighted in this report. However, many challenges remain to implementing these in real applications, and these are discussed along with potential methods for overcoming them.

97 MATHEMATICS AND COMPUTING↗

Next-Generation Materials Design: Quantum Mechanics and Data-Driven Modeling

The future of materials design is rapidly advancing through the combination of quantum mechanics and data-driven modeling. These approaches integrate quantum principles with advanced data analysis, enabling precise insights into material behavior. This talk will highlight recent progress in using these methods for computational design, particularly in high-entropy alloy catalysts, emphasizing the role of hierarchical machine-learning architectures for accurate predictions. Additionally, I will discuss our work on developing machine learning interatomic potentials (MLPs) for single-element metals, metal oxides, and alloys under extreme conditions, focusing on melting behavior and phase properties at high temperatures and pressures. We have also refined our MLP models to capture dynamic surface interactions, such as CO2 and CO adsorption on MgO, using both static and molecular dynamics simulations. These models maintain high accuracy while significantly reducing computational costs compared to first-principles calculations. By enabling efficient and accurate simulations, this work supports broader community adoption, optimizes datasets for materials discovery, and extends the accessible time, size, and environmental conditions beyond the limits of experiments and traditional simulations.

machine learning↗

Investigation of the Effect of Framework Flexibility on Adsorption in SIFSIX-3-Cu using a Machine-Learned Force Field

Metal-organic frameworks (MOFs) are a promising class of adsorbents. The performance of MOF sorbents relies on high selectivity and low regeneration energy. This work focuses on the use of machine learned force fields (MLFFs) to model adsorption in a flexible MOF, SIFSIX-3-Cu. A DeePMD-based MLFF was trained to reproduce DFT (PBE+D3) energies, forces, and stresses, using an iterative sampling scheme combining sampling based on molecular dynamics, Monte Carlo, and geometry optimization to capture both attractive and repulsive regions of the potential energy surface. Flexibility of the MOF was explicitly included in this model. Hybrid Monte Carlo/molecular dynamics (MC/MD) simulations using the MLFF predicted adsorption isotherms in good agreement with experimental data for a range of pressures (40 Pa – 104 Pa) in contrast to rigid models, which overpredict CO2 adsorption at low pressures. The improvement was the result of a description of the variability of fluorine-fluorine diagonal distances at adsorption sites. This detailed description of flexibility afforded by the MLFF resulted in more accurate predictions adsorption isotherms when compared to the experimentally measured values. These results underscore the importance of including framework flexibility when modeling adsorption phenomena in MOFs, particularly for low pressure applications and provide a robust procedure for training MLFF models for MOFs.

Atomistic Simulation↗

Fingerprinting Interactions between Proteins and Ligands for Facilitating Machine Learning in Drug Discovery

Molecular recognition is fundamental in biology, underpinning intricate processes through specific protein–ligand interactions. This understanding is pivotal in drug discovery, yet traditional experimental methods face limitations in exploring the vast chemical space. Computational approaches, notably quantitative structure–activity/property relationship analysis, have gained prominence. Molecular fingerprints encode molecular structures and serve as property profiles, which are essential in drug discovery. While two-dimensional (2D) fingerprints are commonly used, three-dimensional (3D) structural interaction fingerprints offer enhanced structural features specific to target proteins. Machine learning models trained on interaction fingerprints enable precise binding prediction. Recent focus has shifted to structure-based predictive modeling, with machine-learning scoring functions excelling due to feature engineering guided by key interactions. Notably, 3D interaction fingerprints are gaining ground due to their robustness. Various structural interaction fingerprints have been developed and used in drug discovery, each with unique capabilities. This review recapitulates the developed structural interaction fingerprints and provides two case studies to illustrate the power of interaction fingerprint-driven machine learning. The first elucidates structure–activity relationships in β2 adrenoceptor ligands, demonstrating the ability to differentiate agonists and antagonists. The second employs a retrosynthesis-based pre-trained molecular representation to predict protein–ligand dissociation rates, offering insights into binding kinetics. Despite remarkable progress, challenges persist in interpreting complex machine learning models built on 3D fingerprints, emphasizing the need for strategies to make predictions interpretable. Binding site plasticity and induced fit effects pose additional complexities. Interaction fingerprints are promising but require continued research to harness their full potential.

3D structural interaction fingerprints↗