Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural state space models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Designing Monte Carlo Simulation and an Optimal Machine Learning to Optimize and Model Space Missions

This paper investigates applying artificial intelligence (AI) algorithms to attitude control system of satellites to optimally tune the controller using high performance computing. This methodology is applied to the Virtual Telescope for X-ray Observation mission, which is a precise formation of two separate spacecraft observing multiple objects in the space in the X-ray domain. The mission is divided into phases based on the instrumentation and the mission goal. To reach an stable precise formation robust to stochastic slew and slew rate (i.e., Euler angles and angular velocities) in a minimal constrained time T , consumed energy of the attitude control system, denoted as E, and root-mean-square state error of attitude control system, denoted as e, are minimized. Monte-Carlo simulation is used for the sensitivity analysis of optimization and designing a controller. Deep neural networks (DNN), Gaussian processes (GP), and support vector regression (SVR) learn this optimization as a surrogate model, while their hyperparameters are optimized in a novel approach. THETA supercomputer at Argonne Leadership Computing Facility (ALCF) is used for optimizing the hyperparameters of DNN. The surrogate model meets the requirements of the mission, and it shows a better performance over the optimization and Monte-Carlo. The optimal DNN can satisfy the mission requirements e and T while reducing E for 90% compared to the other given methods.

42 ENGINEERING↗
DOE OSTI2021

Navigating Transition-Metal Chemical Space: Artificial Intelligence for First-Principles Design

Conspectus The variability of chemical bonding in open-shell transition-metal complexes not only motivates their study as functional materials and catalysts but also challenges conventional computational modeling tools. Here, tailoring ligand chemistry can alter preferred spin or oxidation states as well as electronic structure properties and reactivity, creating vast regions of chemical space to explore when designing new materials atom by atom. Although first-principles density functional theory (DFT) remains the workhorse of computational chemistry in mechanism deduction and property prediction, it is of limited use here. DFT is both far too computationally costly for widespread exploration of transition-metal chemical space and also prone to inaccuracies that limit its predictive performance for localized d electrons in transition-metal complexes. These challenges starkly contrast with the well-trodden regions of small-organic-molecule chemical space, where the analytical forms of molecular mechanics force fields and semiempirical theories have for decades accelerated the discovery of new molecules, accurate DFT functional performance has been demonstrated, and gold-standard methods from correlated wavefunction theory can predict experimental results to chemical accuracy. The combined promise of transition-metal chemical space exploration and lack of established tools has mandated a distinct approach. In this Account, we outline the path we charted in exploration of transition-metal chemical space starting from the first machine learning (ML) models (i.e., artificial neural network and kernel ridge regression) and representations for the prediction of open-shell transition-metal complex properties. The distinct importance of the immediate coordination environment of the metal center as well as the lack of low-level methods to accurately predict structural properties in this coordination environment first motivated and then benefited from these ML models and representations. Once developed, the recipe for prediction of geometric, spin state, and redox potential properties was straightforwardly extended to a diverse range of other properties, including in catalysis, computational “feasibility”, and the gas separation properties of periodic metal–organic frameworks. Interpretation of selected features most important for model prediction revealed new ways to encapsulate design rules and confirmed that models were robustly mapping essential structure–property relationships. Encountering the special challenge of ensuring that good model performance could generalize to new discovery targets motivated investigation of how to best carry out model uncertainty quantification. Distance-based approaches, whether in model latent space or in carefully engineered feature space, provided intuitive measures of the domain of applicability. With all of these pieces together, ML can be harnessed as an engine to tackle the large-scale exploration of transition-metal chemical space needed to satisfy multiple objectives using efficient global optimization methods. In practical terms, bringing these artificial intelligence tools to bear on the problems of transition-metal chemical space exploration has resulted in ML-model assessments of large, multimillion compound spaces in minutes and validated new design leads in weeks instead of decades.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Predicting path-dependent diffusion barrier spectra in vast compositional space of multi-principal element alloys via convolutional neural networks

The emergent multi-principal element alloys (MPEAs) provide a vast compositional space to search for novel materials for technological advances. How to screen promising compositions from such an ample design space for targeted properties is a grand challenge. In this report, we demonstrate the state-of-the-art deep learning technology—convolutional neural network (CNN)—in predicting path-dependent vacancy migration energy barrier spectra, which are critical to diffusion behavior and many high-temperature properties, in the hyperdimensional composition space of MPEAs. The developed CNN model, fully capturing local chemical features surrounding each vacancy, accurately and efficiently predicts migration energy barrier of MPEAs with different degrees of chemical short-range order and at any unseen compositions. By varying the size of the local region encapsulating vacancy in the CNN model, we reveal that the length scale influencing vacancy migration is surprisingly extensive, up to its six nearest neighboring shells. The efforts of the CNN model make it promising for developing a database of diffusion barriers for various MPEA systems, which would have profound implications for accelerating alloy screening and discovering new compositions with desirable properties.

36 MATERIALS SCIENCE↗
DOE OSTI2025

The design space of E(3)-equivariant atom-centred interatomic potentials

Abstract Molecular dynamics simulation is an important tool in computational materials science and chemistry, and in the past decade it has been revolutionized by machine learning. This rapid progress in machine learning interatomic potentials has produced a number of new architectures in just the past few years. Particularly notable among these are the atomic cluster expansion, which unified many of the earlier ideas around atom-density-based descriptors, and Neural Equivariant Interatomic Potentials (NequIP), a message-passing neural network with equivariant features that exhibited state-of-the-art accuracy at the time. Here we construct a mathematical framework that unifies these models: atomic cluster expansion is extended and recast as one layer of a multi-layer architecture, while the linearized version of NequIP is understood as a particular sparsification of a much larger polynomial model. Our framework also provides a practical tool for systematically probing different choices in this unified design space. An ablation study of NequIP, via a set of experiments looking at in- and out-of-domain accuracy and smooth extrapolation very far from the training data, sheds some light on which design choices are critical to achieving high accuracy. A much-simplified version of NequIP, which we call BOTnet (for body-ordered tensor network), has an interpretable architecture and maintains its accuracy on benchmark datasets.

Computer Science↗
DOE OSTI2022

Configuration Space Integration for Adsorbate Partition Functions: The Effect of Anharmonicity on the Thermophysical Properties of CO–Pt(111) and CH 3 OH–Cu(111)

A method for computing anharmonic thermophysical properties for adsorbates on metal surfaces has been extended to include libration, or frustrated rotation. Classical phase space integration is used with Monte Carlo sampling of the configuration space to obtain the partition function of CO on Pt(111) and CH 3 OH on Cu(111). A minima-preserving neural network potential energy surrogate is used within the integration routines. Direct state counting using discrete variable representation is used to benchmark the results. We find that the phase space integration approach is in excellent agreement with the direct state counting results. Comparison with standard models such as the harmonic oscillator indicates that anharmonicity contributes significantly to the thermodynamic properties of CH 3 OH on Cu(111). We find that there is also a considerable difference between the harmonic oscillator and phase space integration for CO on Pt(111), although the discrepancy can largely be attributed to the presence of multiple binding sites within the unit cell. We demonstrate that a multisite harmonic oscillator model might be sufficient for CO-Pt(111). A more thorough description of the potential energy surface, which can be achieved with phase space integration, is necessary for weakly bound adsorbates such as CH 3 OH. In conclusion, the thermophysical properties were used to calculate free energies of adsorption on the respective metals, and subsequently the equilibrium constants and Langmuir isotherms in relevant temperature ranges. The results show that the choice of model to obtain partition functions greatly affects the resulting surface coverages in kinetic models.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Uncertainty Quantification and Sensitivity Analysis of Low-Dimensional Manifold via Co-Kurtosis PCA in Combustion Modeling

For multi-scale multi-physics applications e.g., the turbulent combustion code Pele, robust and accurate dimensionality reduction is crucial to solving problems at exascale and beyond. A recently developed technique, Co-Kurtosis based Principal Component Analysis (CoK-PCA) which leverages principal vectors of co-kurtosis, is a promising alternative to traditional PCA for complex chemical systems. To improve the effectiveness of this approach, we employ Artificial Neural Networks for reconstructing thermo-chemical scalars, species production rates, and overall heat release rates corresponding to the full state space. Our focus is on bolstering confidence in this deep learning based non-linear reconstruction through Uncertainty Quantification (UQ) and Sensitivity Analysis (SA). UQ involves quantifying uncertainties in inputs and outputs, while SA identifies influential inputs. One of the noteworthy challenges is the computational expense inherent in both endeavors. To address this, we employ the Monte Carlo methods to effectively quantify and propagate uncertainties in our reduced spaces while managing computational demands. Our research carries profound implications not only for the realm of combustion modeling but also for a broader audience in UQ. By showcasing the reliability and robustness of CoK-PCA in dimensionality reduction and deep learning predictions, we empower researchers and decision-makers to navigate complex combustion systems with greater confidence.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Co-optimized machine-learned manifold models for large eddy simulation of turbulent combustion

Many modeling approaches in large eddy simulation (LES) of turbulent combustion employ a projection of the thermochemical state onto a low-dimensional manifold within state space to reduce the number of transported variables and hence computational cost. Flamelet-generated manifolds (FGM) is an example of a well-established, physics-based approach, but increasingly, principal component analysis (PCA) is being used as a data-driven method for generating manifold models. For both approaches, the nonlinear relationship between the location on the predefined manifold and the outputs of interest, such as reaction rates, can be tabulated or encoded in a neural network. This work proposes a new approach for manifold modeling that extends these existing approaches. A modified neural network structure simultaneously encodes the definition of the manifold variables, the nonlinear mapping, and the subfilter closure for LES. This allows all three of these aspects of the model to be co-optimized, generating a model from any source of combustion thermochemical state data. The manifold parameterizing variables are constrained to be linear combinations of species, as in FGM and PCA-based models, to aid in interpretability and implementation. For LES, subfilter variances of the manifold variables are also included as inputs. Two types of a priori analysis are performed to evaluate the new approach. In the first, the model is trained on data from one-dimensional premixed flames. In this case, the approach recovers the behavior of flamelet-based manifold approaches, and in fact slightly improves performance by identifying an optimized progress variable. The approach is also applied to data from direct numerical simulations of spherical ignition kernels in isotropic turbulence. For any specified manifold dimensionality, the new approach provides substantially lower prediction errors than a PCA-based model developed from the same data set. Additionally, the LES formulation of the new approach can provide accurate predictions for filtered reaction rates across a variety of filter widths.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Short-term nodal load forecasting based on machine learning techniques

This paper introduces an advanced Short-term Nodal Load Forecasting (STNLF) method that forecasts nodal load profiles for the next day in power systems, based on the combined use of three machine learning techniques. Least Absolute Shrinkage and Selection Operator (LASSO) is employed to reduce the number of features for a single nodal load forecasting. Principal Component Analysis (PCA) is used to capture the features of historical loads in low-dimensional space compared to the original high-dimensional load space where features are barely possible to depict. Additionally, Bayesian Ridge Regression (BRR) is utilized to decide the parameters of the prediction model from a statistics perspective. Tests based on modified PJM load data demonstrate the effectiveness of the proposed STNLF method compared to the state-of-the-art General Regression Neural Network (GRNN) method. Moreover, the reliability of the day-ahead Unit Commitment (UC) solution is shown to have been improved, based on the forecasted load data using the proposed STNLF method.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Neural Lyapunov Differentiable Predictive Control

We present a learning-based predictive control methodology using the differentiable programming framework with sufficient Lyapunov-based guarantees. The neural Lyapunov differentiable predictive control (NLDPC) learns the policy by constructing a computational graph encompassing the system dynamics, state, and input constraints, and the necessary Lyapunov certification constraints, and thereafter using the automatic differentiation to update the neural policy parameters. In conjunction, our approach jointly learns a Lyapunov function that certifies the regions of state-space with stable dynamics. We also provide an initial condition sampling-based statistical guarantee for the training of NLDPC. Our offline training approach provides a computationally efficient and scalable alternative to classical model predictive control solutions, which now directly comes with Lyapunov based guarantees as shown in this paper. We substantiate the advantages of the proposed approach with simulations to stabilize the double integrator model and on an example of controlling an aircraft model.

Mukherjee, Sayak↗
DOE OSTI2020

Deep transfer learning for star cluster classification: I. application to the PHANGS– HST survey

ABSTRACT We present the results of a proof-of-concept experiment that demonstrates that deep learning can successfully be used for production-scale classification of compact star clusters detected in Hubble Space Telescope(HST) ultraviolet-optical imaging of nearby spiral galaxies ($D\lesssim 20\, \textrm{Mpc}$) in the Physics at High Angular Resolution in Nearby GalaxieS (PHANGS)–HST survey. Given the relatively small nature of existing, human-labelled star cluster samples, we transfer the knowledge of state-of-the-art neural network models for real-object recognition to classify star clusters candidates into four morphological classes. We perform a series of experiments to determine the dependence of classification performance on neural network architecture (ResNet18 and VGG19-BN), training data sets curated by either a single expert or three astronomers, and the size of the images used for training. We find that the overall classification accuracies are not significantly affected by these choices. The networks are used to classify star cluster candidates in the PHANGS–HST galaxy NGC 1559, which was not included in the training samples. The resulting prediction accuracies are 70 per cent, 40 per cent, 40–50 per cent, and 50–70 per cent for class 1, 2, 3 star clusters, and class 4 non-clusters, respectively. This performance is competitive with consistency achieved in previously published human and automated quantitative classification of star cluster candidate samples (70–80 per cent, 40–50 per cent, 40–50 per cent, and 60–70 per cent). The methods introduced herein lay the foundations to automate classification for star clusters at scale, and exhibit the need to prepare a standardized data set of human-labelled star cluster classifications, agreed upon by a full range of experts in the field, to further improve the performance of the networks introduced in this study.

Wei, Wei↗
DOE OSTI2025

Neural network approaches for parameterized optimal control

Here, we consider numerical approaches for deterministic, finite-dimensional optimal control problems whose dynamics depend on unknown or uncertain parameters. We seek to amortize the solution over a set of relevant parameters in an offline stage to enable rapid decision-making and be able to react to changes in the parameter in the online stage. To tackle the curse of dimensionality arising when the state and/or parameter are high-dimensional, we represent the policy using neural networks. We compare two training paradigms: First, our model-based approach leverages the dynamics and definition of the objective function to learn the value function of the parameterized optimal control problem and obtain the policy using a feedback form. Second, we use actor-critic reinforcement learning to approximate the policy in a data-driven way. Using an example involving a two-dimensional convection-diffusion equation, which features high-dimensional state and parameter spaces, we investigate the accuracy and efficiency of both training paradigms. While both paradigms lead to a reasonable approximation of the policy, the model-based approach is more accurate and considerably reduces the number of PDE solves.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Data-driven profile prediction for DIII-D

A new, fully data-driven algorithm has been developed that uses neural networks to predict plasma pro les on a scale of τ E into the future given actuators and the present plasma state. The model was trained and tested on DIII-D data from the 2013-2018 experimental campaigns. The model is accurate on average, with q predictions the worst and pressure predictions the best. The model can run in milliseconds and is very simple to use. This makes it a potentially useful tool for operators and physicists when planning plasma scenarios. It also is a candidate for doing phase-space exploration without going through the DIIID database or complicated and computationally expensive simulation codes. Here, a reduced model using only realtime diagnostics has also been developed and formed the basis for a model-predictive control algorithm implemented and successfully tested on DIII-D.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Highly sensitive 2D X-ray absorption spectroscopy via physics informed machine learning

Abstract Improving the spatial and spectral resolution of 2D X-ray near-edge absorption structure (XANES) has been a decade-long pursuit to probe local chemical reactions at the nanoscale. However, the poor signal-to-noise ratio in the measured images poses significant challenges in quantitative analysis, especially when the element of interest is at a low concentration. In this work, we developed a post-imaging processing method using deep neural network to reliably improve the signal-to-noise ratio in the XANES images. The proposed neural network model could be trained to adapt to new datasets by incorporating the physical features inherent in the latent space of the XANES images and self-supervised to detect new features in the images and achieve self-consistency. Two examples are presented in this work to illustrate the model’s robustness in determining the valence states of Ni and Co in the LiNi x Mn y Co 1-x-y O 2 systems with high confidence.

36 MATERIALS SCIENCE↗
DOE OSTI2025

A generative artificial intelligence framework for long-time plasma turbulence simulations

Generative deep learning techniques are employed in a novel framework for the construction of surrogate models capturing the spatiotemporal dynamics of 2D plasma turbulence. The proposed Generative Artificial Intelligence Turbulence (GAIT) framework enables the acceleration of turbulence simulations for long-time transport studies. GAIT leverages a convolutional variational auto-encoder and a recurrent neural network to generate new turbulence data from existing simulations, extending the time horizon of transport studies with minimal computational cost. The application of the GAIT framework to plasma turbulence using the Hasegawa–Wakatani (HW) model is presented, evaluating its performance via various analyses. Very good agreement is found between the GAIT and the HW models in the spatiotemporal Fourier and Proper Orthogonal Decomposition spectra, the flow topology characterized by the Okubo–Weiss parameter, and the time autocorrelation function of turbulent fluctuations. Excellent agreement has also been obtained in the probability distribution function of particle displacements and the effective turbulent diffusivity. In-depth analyses of the latent space of turbulent states, choice of hyperparameters and alternative deep learning models for the time prediction are presented. Our results highlight the potential of Artificial Intelligence-based surrogate models to overcome the computational challenges in turbulence simulation, which can be extended to other situations such as geophysical fluid dynamics.

Artificial intelligence↗
DOE OSTI2024

Deep energy-pressure regression for a thermodynamically consistent EOS model

Abstract In this paper, we aim to explore novel machine learning (ML) techniques to facilitate and accelerate the construction of universal equation-Of-State (EOS) models with a high accuracy while ensuring important thermodynamic consistency. When applying ML to fit a universal EOS model, there are two key requirements: (1) a high prediction accuracy to ensure precise estimation of relevant physics properties and (2) physical interpretability to support important physics-related downstream applications. We first identify a set of fundamental challenges from the accuracy perspective, including an extremely wide range of input/output space and highly sparse training data. We demonstrate that while a neural network (NN) model may fit the EOS data well, the black-box nature makes it difficult to provide physically interpretable results, leading to weak accountability of prediction results outside the training range and lack of guarantee to meet important thermodynamic consistency constraints. To this end, we propose a principled deep regression model that can be trained following a meta-learning style to predict the desired quantities with a high accuracy using scarce training data. We further introduce a uniquely designed kernel-based regularizer for accurate uncertainty quantification. An ensemble technique is leveraged to battle model overfitting with improved prediction stability. Auto-differentiation is conducted to verify that necessary thermodynamic consistency conditions are maintained. Our evaluation results show an excellent fit of the EOS table and the predicted values are ready to use for important physics-related tasks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Charting the chemical space of Zintl phases with graph neural networks and bonding insights

A large number of Zintl phases have been discovered by solid-state chemists driven by empirical knowledge, chemical intuition and in some cases, through serendipitous accidents. These discoveries have only scratched the surface, given the vast compositional and structural diversity that Zintl phases can accommodate. The large chemical space of Zintl phases, as well as intermetallic compounds in general, remain under-explored. Here, we use graph neural networks and the upper bound energy minimization approach to efficiently scan a large chemical space of >90 000 hypothetical Zintl phases and accurately discover 1810 new thermodynamically stable phases with 90% precision, as validated with first-principles calculations. We show that our approach is more than 2× more accurate in predicting DFT stability than M3GNet (40% precision) on the same dataset. Using a random forest model and SHAP analysis, we demonstrate the critical role of ionic bonding in the thermodynamic stability of Zintl phases. Our results not only expand the known chemical landscape of Zintl phases but also highlight the efficacy of machine learning frameworks combined with domain knowledge in uncovering chemically meaningful insights across complex intermetallics.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Probabilistic locked mode predictor in the presence of a resistive wall and finite island saturation in tokamaks

We present a framework for estimating the probability of locking to an error field in a rotating tokamak plasma. This leverages machine learning methods trained on data from a mode-locking model, including an error field, resistive magnetohydrodynamics modeling of the plasma, a resistive wall, and an external vacuum region, leading to a fifth-order ordinary differential equation (ODE) system. It is an extension of the model without a resistive wall introduced by Akçay et al. [Phys. Plasmas 28, 082106 (2021)]. Tearing mode saturation by a finite island width is also modeled. We vary three pairs of control parameters in our studies: the momentum source plus either the error field, the tearing stability index, or the island saturation term. The order parameters are the time-asymptotic values of the five ODE variables. Normalization of them reduces the system to 2D and facilitates the classification into locked (L) or unlocked (U) states, as illustrated by Akçay et al., [Phys. Plasmas 28, 082106 (2021)]. This classification splits the control space into three regions: L̂, with only L states; Û, with only U states; and a hysteresis (hysteretic) region Ĥ, with both L and U states. In regions L̂ and Û, the cubic equation of torque balance yields one real root. Region Ĥ has three roots, allowing bifurcations between the L and U states. The classification of the ODE solutions into L/U is used to estimate the locking probability, conditional on the pair of the control parameters, using a neural network. We also explore estimating the locking probability for a sparse dataset, using a transfer learning method based on a dense model dataset.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

1D Modeling for the Development of a Digital Twin for Hydropower Systems with Applications for a Testing Rig

Modeling and simulation constitute two important parts in constructing a sensible digital twin to mimic the dynamics of hydropower systems. Because of the physical nature of a hydropower system, the basic modeling should cover the water flow systems from the reservoir to the penstock (inlet water pipes), penstock to hydro turbine, and hydro turbine to generator and from the linkage of the hydropower systems to the grid. This report describes initial attempts to model these dynamic components, including the formulation of the linearized state space model for hydro turbine systems using the well-known six-coefficients linearization method and the formulation of the voltage and power dynamical models of a synchronous generator. To validate the accuracy of the proposed model, data collected from Norwegian University of Science and Technology were used to obtain relevant parameters for the model, and the desired simulation results were obtained. Additionally, neural network modeling and learning were also developed and applied to model the generation torque and water flow rate subsystems to demonstrate a potential learning scheme, which can be used in the development of a digital twin.

13 HYDRO ENERGY↗