Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Physics Informed Machine Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49

Arbitrary Polynomial Separations in Trainable Quantum Machine Learning

Recent theoretical results in quantum machine learning have demonstrated a general trade-off between the expressive power of quantum neural networks (QNNs) and their trainability; as a corollary of these results, practical exponential separations in expressive power over classical machine learning models are believed to be infeasible as such QNNs take a time to train that is exponential in the model size. We here circumvent these negative results by constructing a hierarchy of efficiently trainable QNNs that exhibit unconditionally provable, polynomial memory separations of arbitrary constant degree over classical neural networks—including state-of-the-art models, such as Transformers—in performing a classical sequence modeling task. This construction is also computationally efficient, as each unit cell of the introduced class of QNNs only has constant gate complexity. We show that contextuality—informally, a quantitative notion of semantic ambiguity—is the source of the expressivity separation, suggesting that other learning tasks with this property may be a natural setting for the use of quantum learning algorithms.

Anschuetz, Eric R. [California Institute of Techno↗

Outcomes of the DOE Workshop on Atmospheric Challenges for the Wind Energy Industry

The U.S. Department of Energy-funded Mesoscale-to-Microscale Coupling (MMC) project team planned and conducted a virtual Workshop on Atmospheric Challenges for the Wind Energy Industry on October 19 and 20, 2020. The goal of the workshop was to forge a dialog with the community, including industry representatives, on how modeling tools are currently being used, the present active atmospheric modeling research in support of wind energy, and required advancements in capabilities and technology to continue to advance wind energy deployment. The workshop was planned in collaboration with an industry advisory panel that included representatives from wind power plant developers, turbine manufacturers, and companies that provide resource assessment and forecasting services. The format of the workshop included panels from government research sponsors, visionaries from industry, and mixed panels of researchers discussing research status and needs. A shared keynote presentation from the Technical University of Denmark experts anchored the second day of the workshop. An emphasis was placed on understanding the research needs in the offshore environment. In addition, breakout opportunities were provided each day. On the first day, the breakout discussions addressed predesigned questions configured to elicit participants’ thoughts on needed research directions. The second-day breakouts treated three important technical topics through a combination of presentations and group conversations. Each workshop participant chose their breakout preference from among downscaling details, modeling for turbines, and using artificial intelligence for atmospheric modeling. The discussions were robust and productive. The outcomes of the workshop include archiving a series of recommendations from industry and the research community on research directions required to further advance wind energy deployment. Discussions confirmed the need for high-fidelity modeling but that there are specific areas of applicability and other areas where the time and cost of computation is prohibitive. In those cases, the high-fidelity models can inform low-order models that are more practical for real-time or widely deployed applications. Industry must consider the financial cost of performing more expensive modeling approaches, but industry engineers and researchers are using these approaches where there appears to be a return on investment. An emerging type of low-order model is based on machine learning (ML). Participants confirmed that there are many atmospheric phenomena that need to be modeled better, including low-level jets, cold air outbreaks, land-sea induced circulations, diurnal variability, thin stable boundary layers, dynamic changes such as from frontal passage, interaction of wakes and blockage, and more. For the offshore environment, there is wide agreement that some level of ocean-wave-atmospheric coupling is necessary to capture variations in rotor-level winds needed to plan and operate offshore wind plants. Another recurring recommendation is that more observations are needed, particularly for the offshore environment. Those observations should consider the needs for model improvement, both for physically based models and for ML models. Observations must capture atmospheric profiles of variables that are important to understanding and modeling atmospheric and oceanic phenomena that impact boundary layer winds. Models must be validated with data and the uncertainty quantified, particularly those that are sensitive to initial and boundary conditions. Finally, a repeated request was to consider the holistic needs of hybrid plants of wind, solar, and storage resources because those types of plants are likely to be the wave of the future. In addition, industry wishes to understand impacts of the resource under a changing climate for long-term planning.

17 WIND ENERGY↗

ChIMES: A Machine-Learned Interatomic Model Targeting Improved Description of Condensed Phase Chemistry in Energetic Materials

In this report we detail completion of a Physics and Engineering Model Level Two Milestone targeting improved reactive interatomic potentials (IAPs) for energetic materials (EM) through machine learning. The specific goals of this milestone were to develop, validate, and document a new reactive molecular dynamics method for EM, based on machine learning by (1) generating databases of first-principles-derived forces, stresses, and energies for HN3 and 3,4-bis(3-nitrofurazan- 4-yl)furoxan (DNTF) (2) generate atomistic force fields from these databases via ML, and (3) benchmark model performance against first principles calculations. These goals were achieved by (1) further developing a machine learned reactive IAP and generation approach (i.e. the Chebyshev Interaction Model for Efficient Simulation or “ChIMES”), for which resulting IAPs can approach the predictive power of quantum-mechanical approaches at a fraction of the computational expense, and (2) applying the ChIMES framework to develop models for HN3 and DNTF. We find that for simple energetic materials like HN3, high accuracy ChIMES models can be obtained through application of a fitting approach that does not use active machine learning. We demonstrate the suitability of ChIMES models for simulations involving EM by using the HN3 model in multiscale shock technique simulations to predict the HN3 Chapman-Jouguet detonation state and investigate chemical evolution out to 1 ns following shock compression. This model is then used in larger direct shock (DS) simulations for a preliminary investigation of how bubbles (i.e. voids) influence material response under shock compression. We find that more complex EM (i.e. DNTF) necessitate a more sophisticated fitting approach, and develop a new active learning method and python tool to meet this challenge. We demonstrate that this fitting approach yields ChIMES models that out-perform commonly used standard reactive IAPs as well as semi-empirical quantum methods, and discuss the systematic improvability of these actively learned ChIMES models. We also describe challenges related to model development for EM such as DNTF, for which few experimental or previous simulation data are available (e.g. which could otherwise inform generation of training data). To overcome this issue, we establish a semi-empirical quantum ChIMES capability which can be used to efficiently map out relevant thermodynamic and configurational space, and generate ChIMES-IAP training data in a multiscale manner. We also show that these semi-empirical quantum ChIMES models can be used to generate predictions for the shock Hugoniot (the Hugoniot is the locus of thermodynamic states found in a shocked material) equation of state, investigate related thermochemistry, and explore carbon condensation following shock compression. This work represents a substantial advance in our atomistic modeling capability for EM that will provide much needed information on the chemistry of detonation for continued development of continuum models based on the Cheetah thermochemical code.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Neural network surrogate models for equations of state

Equation of state (EOS) data provide necessary information for accurate multiphysics modeling, which is necessary for fields such as inertial confinement fusion. Here, we suggest a neural network surrogate model of energy and entropy and use thermodynamic relationships to derive other necessary thermodynamic EOS quantities. We incorporate phase information into the model by training a phase classifier and using phase-specific regression models, which improves the modal prediction accuracy. Our model predicts energy values to 1% relative error and entropy to 3.5% relative error in a log-transformed space. Although sound speed predictions require further improvement, the derived pressure values are accurate within 10% relative error. Our results suggest that neural network models can effectively model EOS for inertial confinement fusion simulation applications.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Communication-Constrained Robust Control and Learning of Grid-Connected

The electric grid of things (EGoT) promises great potential for innovative grid services by tapping into vast load flexibility. However, the unique characteristics of EGoT, being a part of the cyber-physical electric power system, present both opportunities and challenges, especially concerning supply-demand balancing, stability, and communication constraints. Traditionally, centralized control was employed to ensure balance and stability in power systems. However, with the massive influx of EGoT devices, new strategies are needed to efficiently coordinate and control these distributed devices for optimal grid operations. While some studies have explored efficiency and economic models, there remains a gap in ensuring reliability under everyday operations and resilience during extreme conditions. Addressing this gap, this project develops the technology for an Energy Service Interface (ESI) that includes novel pricing, control, learning, and distributed optimization algorithms, which will enable utilities to recruit EGoT assets for crucial grid services such as load flexibility, voltage regulation, and situation-awareness. The key novelty of the proposed technology is the careful distribution of learning and control functions across utility and EGoT asset owners such that provably efficient and resilient grid operations are attained while respecting communication and information-exchange constraints. Specifically, the project team develops machine-learning enhanced load modeling methods to allow EGoT asset owners to learn their load capability and flexibility, and develops pricing-based and decentralized learning-based control so that asset owners can coordinate to meet system-wide demand-supply balance and reliability goals. For extreme situations involving high-impact, low-probability catastrophic events (termed the “black-sky” events), the team also develops (1) a “Feeder-Operating Center-on-a-Laptop” (FOCAL) software that can assist utility personnel in leveraging EGoT assets to accelerate the service recovery of damaged feeders, and (2) distributed optimization algorithms that can coordinate the operation points of EGoT devices under severe communication constraints. The proposed technology has been extensively tested and evaluated through simulations and on a testbed. In summary, as we transition into a more interconnected and digital power grid era, our project’s findings and developments offer a pivotal step toward guaranteeing both efficiency and resilience in the face of both everyday operations and rare “black-sky” events.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Accurate Thermochemistry of Complex Lignin Structures via Density Functional Theory, Group Additivity, and Machine Learning

A molecular-level understanding of lignin structures and bond dissociation energies could facilitate depolymerization technologies. Still, this information is currently limited due to the lack of databases and the simplification of surrogate models. Here, substitution effects on seven common linkages in lignin polymers are systematically investigated. An automated reaction network generator is employed to create a database of structures. A new group additivity (GA) model based on principal component analysis (PCA) descriptors is introduced and trained on gas-phase density functional theory data of 4100 species at the M06-2X/6-311++G(d,p) level. Hydrogen bonds, local steric, and nonaromatic ring contributions are also incorporated. Lastly, we improve the accuracy of the group additivity model to reach the G4 theory by computing a data set of 770 species at this level and using a data fusion approach.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Local Bayesian Dirichlet mixing of imperfect models

Abstract To improve the predictability of complex computational models in the experimentally-unknown domains, we propose a Bayesian statistical machine learning framework utilizing the Dirichlet distribution that combines results of several imperfect models. This framework can be viewed as an extension of Bayesian stacking. To illustrate the method, we study the ability of Bayesian model averaging and mixing techniques to mine nuclear masses. We show that the global and local mixtures of models reach excellent performance on both prediction accuracy and uncertainty quantification and are preferable to classical Bayesian model averaging. Additionally, our statistical analysis indicates that improving model predictions through mixing rather than mixing of corrected models leads to more robust extrapolations.

97 MATHEMATICS AND COMPUTING↗

From sequence to protein structure and conformational dynamics with artificial intelligence/machine learning

The 2024 Nobel Prize in Chemistry was awarded in part for de novo protein structure prediction using AlphaFold2, an artificial intelligence/machine learning (AI/ML) model trained on vast amounts of sequence and three-dimensional structure data. AlphaFold2 and related models, including RoseTTAFold and ESMFold, employ specialized neural network architectures driven by attention mechanisms to infer relationships between sequence and structure. At a fundamental level, these AI/ML models operate on the long-standing hypothesis that the structure of a protein is determined by its amino acid sequence. More recently, AlphaFold2 has been adapted for the prediction of multiple protein conformations by subsampling multiple sequence alignments. Herein, we provide an overview of the deterministic relationship between sequence and structure, which was hypothesized over half a century ago with profound implications for the biological sciences ever since. We postulate that protein conformational dynamics are also determined, at least in part, by amino acid sequence and that this relationship may be leveraged for construction of AI/ML models dedicated to predicting protein conformational ensembles. Accordingly, we describe a conceptual model architecture, which may be trained on sequence data in combination with conformationally sensitive structural information, coming primarily from nuclear magnetic resonance (NMR) spectroscopy. Notwithstanding certain limitations in this context, NMR offers abundant structural heterogeneity conducive to conformational ensemble prediction. As NMR and other data continue to accumulate, sequence-informed prediction of protein structural dynamics with AI/ML has the potential to emerge as a transformative capability across the biological sciences.

Artificial intelligence↗

Accurate and confident prediction of electron beam longitudinal properties using spectral virtual diagnostics

Abstract Longitudinal phase space (LPS) provides a critical information about electron beam dynamics for various scientific applications. For example, it can give insight into the high-brightness X-ray radiation from a free electron laser. Existing diagnostics are invasive, and often times cannot operate at the required resolution. In this work we present a machine learning-based Virtual Diagnostic (VD) tool to accurately predict the LPS for every shot using spectral information collected non-destructively from the radiation of relativistic electron beam. We demonstrate the tool’s accuracy for three different case studies with experimental or simulated data. For each case, we introduce a method to increase the confidence in the VD tool. We anticipate that spectral VD would improve the setup and understanding of experimental configurations at DOE’s user facilities as well as data sorting and analysis. The spectral VD can provide confident knowledge of the longitudinal bunch properties at the next generation of high-repetition rate linear accelerators while reducing the load on data storage, readout and streaming requirements.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Reaction processes at step edges on S-decorated Cu(111) and Ag(111) surfaces: MD analysis utilizing machine learning derived potentials

We report a variety of complexation, reconstruction, and sulfide formation processes can occur at step edges on the {111} surfaces of coinage metals (M) in the presence of adsorbed S under ultra-high vacuum conditions. Given the cooperative many-atom nature of these reaction processes, Molecular Dynamics (MD) simulation of the associated dynamics is instructive. However, only quite restricted Density Functional Theory (DFT)-level ab initio MD is viable. Thus, for M = Ag and Cu, we instead utilize the DeePMD framework to develop machine-learning derived potentials, retaining near-DFT accuracy for the M–S systems, which should have broad applicability. These potentials are validated by comparison with DFT predictions for various key quantities related to the energetics of S on M(111) surfaces. The potentials are then utilized to perform extensive MD simulations elucidating the above diverse restructuring and reaction processes at step edges. Key observations from MD simulations include the formation of small metal–sulfur complexes, especially MS 2 ; development of a local reconstruction at A-steps featuring an S-decorated {100} motif; and 3D sulfide formation. Additional analysis yields further information on the kinetics for metal–sulfur complex formation, where these complexes can strongly enhance surface mass transport, and on the propensity for sulfide formation.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Machine Learning Full NMR Chemical Shift Tensors of Silicon Oxides with Equivariant Graph Neural Networks

The nuclear magnetic resonance (NMR) chemical shift tensor is a highly sensitive probe of the electronic structure of an atom and furthermore its local structure. Recently, machine learning has been applied to NMR in the prediction of isotropic chemical shifts from a structure. Current machine learning models, however, often ignore the full chemical shift tensor for the easier-to-predict isotropic chemical shift, effectively ignoring a multitude of structural information available in the NMR chemical shift tensor. Here we use an equivariant graph neural network (GNN) to predict full 29 Si chemical shift tensors in silicate materials. The equivariant GNN model predicts full tensors to a mean absolute error of 1.05 ppm and is able to accurately determine the magnitude, anisotropy, and tensor orientation in a diverse set of silicon oxide local structures. When compared with other models, the equivariant GNN model outperforms the state-of-the-art machine learning models by 53%. The equivariant GNN model also outperforms historic analytical models by 57% for isotropic chemical shift and 91% for anisotropy. The software is available as a simple-to-use open-source repository, allowing similar models to be created and trained with ease.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Machine learning methods for fission product identification from Bragg curves

A fission time projection chamber (fission-TPC) was developed to provide precise neutron-induced fission measurements for several major actinides. As fission fragments lose energy in one of the gas volumes of the fission-TPC, energy loss information is captured and may be used to determine fission product yields as the stopping power of an ion is dependent on the atomic number. The work presented here demonstrates the ability to apply machine learning techniques for Bragg curve classification. A set of one million energy loss curves for 24 different fission-fragment elements was generated using common stopping power software. A ResNet architecture optimized for 1D data was used to train, test, and validate a model for light and heavy fission fragments using the simulated data. The resultant classification accuracy for the light and heavy fragments indicates that this could be a viable method for elemental classification of data from the fission-TPC.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Quantitative and interpretable order parameters for phase transitions from persistent homology

Here, we apply modern methods in computational topology to the task of discovering and characterizing phase transitions. As illustrations, we apply our method to four two-dimensional lattice spin models: the Ising, square ice, XY, and fully frustrated XY models. In particular, we use persistent homology, which computes the births and deaths of individual topological features as a coarse-graining scale or sublevel threshold is increased, to summarize multiscale and high-point correlations in a spin configuration. We employ vector representations of this information called persistence images to formulate and perform the statistical task of distinguishing phases. For the models we consider, a simple logistic regression on these images is sufficient to identify the phase transition. Interpretable order parameters are then read from the weights of the regression. This method suffices to identify magnetization, frustration, and vortex-antivortex structure as relevant features for phase transitions in our models. We also define “persistence” critical exponents and study how they are related to those critical exponents usually considered.

36 MATERIALS SCIENCE↗

Harnessing interpretable and unsupervised machine learning to address big data from modern X-ray diffraction

The information content of crystalline materials becomes astronomical when collective electronic behavior and their fluctuations are taken into account. In the past decade, improvements in source brightness and detector technology at modern X-ray facilities have allowed a dramatically increased fraction of this information to be captured. Now, the primary challenge is to understand and discover scientific principles from big datasets when a comprehensive analysis is beyond human reach. We report the development of an unsupervised machine learning approach, X-ray diffraction (XRD) temperature clustering (X-TEC), that can automatically extract charge density wave order parameters and detect intraunit cell ordering and its fluctuations from a series of high-volume X-ray diffraction measurements taken at multiple temperatures. We benchmark X-TEC with diffraction data on a quasi-skutterudite family of materials, (Ca x Sr 1–x ) 3 Rh 4 Sn 13 , where a quantum critical point is observed as a function of Ca concentration. We apply X-TEC to XRD data on the pyrochlore metal, Cd 2 Re 2 O 7 , to investigate its two much-debated structural phase transitions and uncover the Goldstone mode accompanying them. We demonstrate how unprecedented atomic-scale knowledge can be gained when human researchers connect the X-TEC results to physical principles. Specifically, we extract from the X-TEC–revealed selection rules that the Cd and Re displacements are approximately equal in amplitude but out of phase. This discovery reveals a previously unknown involvement of 5d 2 Re, supporting the idea of an electronic origin to the structural order. Our approach can radically transform XRD experiments by allowing in operando data analysis and enabling researchers to refine experiments by discovering interesting regions of phase space on the fly.

36 MATERIALS SCIENCE↗

Terahertz Circular Dichroism Spectroscopy of Molecular Assemblies and Nanostructures

Chemical, physical, biological and materials engineering disciplines use a variety of chiroptical spectroscopies to probe geometrical and optical asymmetry in molecules and particles. Electronic (ECD) and vibrational (VCD) circular dichroism are the most common of these techniques and collectively enable the studies of electronic and vibronic transitions with energies between 0.1 and 5.0 eV. The vibrational states with characteristic energies in the range of 0.001–0.01 eV carry valuable information about concerted intermolecular motions in molecules and crystals involving multiple atoms. These vibronic transitions located in the terahertz (THz) part of the spectrum become increasingly more important for the chemistry, physics, and biology of complex molecules and materials However, the methodology and hardware of THz circular dichroism (TCD) are much less developed than the chiroptical spectroscopies for ultraviolet, visible, near- and mid infrared photons. In this report we provide theoretical foundations, practical implementations, comparative assessments, and exemplary applications of TCD spectroscopy. We show that the sign, intensity, and position of TCD peaks are highly sensitive to the three-dimensional structure and long-range organization of molecular crystals, which offer unique capabilities to study (bio) molecules, their crystals, and nanoscale assemblies and apply the novel data processing methodologies. TCD also offers a convenient toolbox to identify new physical phenomena, such as chiral phonons and their propagation in nanostructured matter. We also discuss the major challenges, emerging opportunities and promising research directions, including broad investigation of chiral phonons observed in chiral (nano) crystals and emerging machine learning methodologies for TCD in biological and nanoscale structures. Ubiquity of low-frequency vibrations with rotational components in biomolecular structures, combined with sharpness of peaks in TCD spectra, enables a variety of technological translations.

36 MATERIALS SCIENCE↗

Comparison of multifidelity machine learning models for potential energy surfaces

Multifidelity modeling is a technique for fusing the information from two or more datasets into one model. It is particularly advantageous when one dataset contains few accurate results and the other contains many less accurate results. Within the context of modeling potential energy surfaces, the low-fidelity dataset can be made up of a large number of inexpensive energy computations that provide adequate coverage of the N-dimensional space spanned by the molecular internal coordinates. The high-fidelity dataset can provide fewer but more accurate electronic energies for the molecule in question. Here, we compare the performance of several neural network-based approaches to multifidelity modeling. We show that the four methods (dual, Δ-learning, weight transfer, and Meng–Karniadakis neural networks) outperform a traditional implementation of a neural network, given the same amount of training data. We also show that the Δ-learning approach is the most practical and tends to provide the most accurate model.

Chemistry↗

Constructing a new predictive scaling formula for ITER's divertor heat-load width informed by a simulation-anchored machine learning

Understanding and predicting divertor heat-load width λq is a critically important problem for an easier and more robust operation of ITER with high fusion gain. Previous predictive simulation data for λ q using the extreme-scale edge gyrokinetic code XGC1 [S. Ku et al., Phys. Plasmas 25, 056107 (2018)] in the electrostatic limit under attached divertor plasma conditions in three major US tokamaks [C. S. Chang et al., Nucl. Fusion 57, 116023 (2017)] reproduced the Eich and Goldston attached-divertor formula results [formula #14 in T. Eich et al., Nucl. Fusion 53, 093031 (2013) and R. J. Goldston, Nucl. Fusion 52, 013009 (2012)] and furthermore predicted over six times wider λ q than the maximal Eich and Goldston formula predictions on a full-power (Q = 10) scenario ITER plasma. After adding data from further predictive simulations on a highest current JET and highest-current Alcator C-Mod, a machine learning program is used to identify a new scaling formula for λ q as a simple modification to the Eich formula #14, which reproduces the Eich scaling formula for the present tokamaks and which embraces the wide λ q XGC for the full-current Q = 10 ITER plasma. Additionally, the new formula is then successfully tested on three more ITER plasmas: two corresponding to long burning scenarios with Q = 5 and one at low plasma current to be explored in the initial phases of ITER operation. The new physics that gives rise to the wider λ q XGC is identified to be the weakly collisional, trapped-electron-mode turbulence across the magnetic separatrix, which is known to be an efficient transporter of the electron heat and mass. Electromagnetic turbulence and high-collisionality effects on the new formula are the next study topics for XGC1.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Three-dimensional coherent X-ray diffraction imaging via deep convolutional neural networks

Abstract As a critical component of coherent X-ray diffraction imaging (CDI), phase retrieval has been extensively applied in X-ray structural science to recover the 3D morphological information inside measured particles. Despite meeting all the oversampling requirements of Sayre and Shannon, current phase retrieval approaches still have trouble achieving a unique inversion of experimental data in the presence of noise. Here, we propose to overcome this limitation by incorporating a 3D Machine Learning (ML) model combining (optional) supervised learning with transfer learning. The trained ML model can rapidly provide an immediate result with high accuracy which could benefit real-time experiments, and the predicted result can be further refined with transfer learning. More significantly, the proposed ML model can be used without any prior training to learn the missing phases of an image based on minimization of an appropriate ‘loss function’ alone. We demonstrate significantly improved performance with experimental Bragg CDI data over traditional iterative phase retrieval algorithms.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗