Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “constrained learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

TPSAS-NF1676L-35322-DND

This talk discusses emerging methods that seek to fuse and integrate physics-based modeling with machine learning. With the recent rise of machine learning and artificial intelligence, there has been a huge surge in data-driven approaches to solve computational science and engineering problems. However, neglecting a priori knowledge of established physical laws and relying solely on data-driven methods can yield unreliable, less interpretable, and/or non-physical results, especially when data is sparse or predictions are required outside of the training data domain. This two part talk presents two distinct approaches for accelerating predictions with machine learning that are grounded and constrained by relevant physics and their application to problems at NASA.

Julian Cuevas Paniagua↗
DOE OSTI2021

Unsupervised machine learning for unbiased chemical classification in X-ray absorption spectroscopy and X-ray emission spectroscopy

Here we report a comprehensive computational study of unsupervised machine learning for extraction of chemically relevant information in X-ray absorption near edge structure (XANES) and in valence-to-core X-ray emission spectra (VtC-XES) for classification of a broad ensemble of sulphorganic molecules. By progressively decreasing the constraining assumptions of the unsupervised machine learning algorithm, moving from principal component analysis (PCA) to a variational autoencoder (VAE) to t-distributed stochastic neighbour embedding (t-SNE), we find improved sensitivity to steadily more refined chemical information. Surprisingly, when embedding the ensemble of spectra in merely two dimensions, t-SNE distinguishes not just oxidation state and general sulphur bonding environment but also the aromaticity of the bonding radical group with 87% accuracy as well as identifying even finer details in electronic structure within aromatic or aliphatic sub-classes. We find that the chemical information in XANES and VtC-XES is very similar in character and content, although they unexpectedly have different sensitivity within a given molecular class. We also discuss likely benefits from further effort with unsupervised machine learning and from the interplay between supervised and unsupervised machine learning for X-ray spectroscopies. Our overall results, i.e., the ability to reliably classify without user bias and to discover unexpected chemical signatures for XANES and VtC-XES, likely generalize to other systems as well as to other one-dimensional chemical spectroscopies.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Enhanced physics-constrained deep neural networks for modeling vanadium redox flow battery

Numerical simulation has become indispensable in advancing cost-effective process optimization and control of flow batteries. We propose an enhanced version of the physics-constrained deep neural network (PCDNN) approach to provide high-accuracy voltage predictions in the vanadium redox flow batteries (VRFBs). The purpose of the PCDNN approach is to enforce the physics-based zero-dimensional (0D) VRFB model in a neural network to assure model generalization for various battery operation conditions. However, limited by the simplifications of the 0D model, the PCDNN cannot capture sharp voltage changes in the extreme SOC regions. To improve the accuracy of voltage prediction at extreme ranges, we introduce a second (enhanced) DNN to mitigate the prediction errors carried from the 0D model itself and call the resulting approach enhanced PCDNN (ePCDNN). By comparing with experimental data, we demonstrate that the ePCDNN approach can accurately capture the voltage response throughout the charge–discharge cycle, including the tail region of the voltage discharge curve. The loss function for training the ePCDNN is designed to be flexible by adjusting the weights of the physics-constrained DNN and the enhanced DNN. In conclusion, this allows the ePCDNN framework to be transferable to battery systems with variable physical model fidelity.

25 ENERGY STORAGE↗
DOE OSTI2023

Domain Adaptive Graph Neural Networks for Constraining Cosmological Parameters Across Multiple Data Sets

Deep learning models have been shown to outperform methods that rely on summary statistics, like the power spectrum, in extracting information from complex cosmological data sets. However, due to differences in the subgrid physics implementation and numerical approximations across different simulation suites, models trained on data from one cosmological simulation show a drop in performance when tested on another. Similarly, models trained on any of the simulations would also likely experience a drop in performance when applied to observational data. Training on data from two different suites of the CAMELS hydrodynamic cosmological simulations, we examine the generalization capabilities of Domain Adaptive Graph Neural Networks (DA-GNNs). By utilizing GNNs, we capitalize on their capacity to capture structured scale-free cosmological information from galaxy distributions. Moreover, by including unsupervised domain adaptation via Maximum Mean Discrepancy (MMD), we enable our models to extract domain-invariant features. We demonstrate that DA-GNN achieves higher accuracy and robustness on cross-dataset tasks. Using data visualizations, we show the effects of domain adaptation on proper latent space data alignment. This shows that DA-GNNs are a promising method for extracting domain-independent cosmological information, a vital step toward robust deep learning for real cosmic survey data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Apparatus and method for optimizing quantifiable behavior in configurable devices and systems

An apparatus and method are provided to perform constrained optimization of a constrained property of an apparatus, which is complex due to having several components, and these components are configurable in real-time. The optimization is achieved by detecting values of the constrained property and a plurality of other properties of the apparatus when the apparatus is configured in a first subset of the plurality of configurations. A model is learned using the detected values of the constrained property. The model represents the constrained property and can also represent other properties as a function of the configurations. The model can also include estimated uncertainties of the constrained property in the model. Then, using the d model and the estimated uncertainties, the optimal configuration can be selected to minimize an error value (e.g., the difference between a desired value and an observed value of the at least one constrained property).

Hoffmann, Henry↗
DOE OSTI2023

Domain Adaptive Graph Neural Networks for Constraining Cosmological Parameters Across Multiple Data Sets

Deep learning models have been shown to outperform methods that rely on summary statistics, like the power spectrum, in extracting information from complex cosmological data sets. However, due to differences in the subgrid physics implementation and numerical approximations across different simulation suites, models trained on data from one cosmological simulation show a drop in performance when tested on another. Similarly, models trained on any of the simulations would also likely experience a drop in performance when applied to observational data. Training on data from two different suites of the CAMELS hydrodynamic cosmological simulations, we examine the generalization capabilities of Domain Adaptive Graph Neural Networks (DA-GNNs). By utilizing GNNs, we capitalize on their capacity to capture structured scale-free cosmological information from galaxy distributions. Moreover, by including unsupervised domain adaptation via Maximum Mean Discrepancy (MMD), we enable our models to extract domain-invariant features. We demonstrate that DA-GNN achieves higher accuracy and robustness on cross-dataset tasks (up to $28\%$ better relative error and up to almost an order of magnitude better $\chi^2$). Using data visualizations, we show the effects of domain adaptation on proper latent space data alignment. This shows that DA-GNNs are a promising method for extracting domain-independent cosmological information, a vital step toward robust deep learning for real cosmic survey data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Physics-constrained Deep Recurrent Neural Models of Building Thermal Dynamics

We develop physics-constrained and control-oriented predictive deep learning models for the thermal dynamics of a real-world commercial office building. The proposed method is based on the systematic encoding of physics-based prior knowledge into a structured recurrent neural architecture. Specifically, our model mimics the structure of the building thermal dynamics model and leverages penalty methods to model inequality constraints. Additionally, we use constrained matrix parameterization based on the Perron-Frobenius theorem to bound the eigenvalues of the learned network weights. We interpret the stable eigenvalues as dissipativeness of the learned building thermal model. We demonstrate the effectiveness of the proposed approach on a dataset obtained from an office building with $20$ thermal zones.

Building Energy, structured neural networks↗
DOE OSTI2025

Computational investigation of water glasses using machine-learning potentials

The molecular origins of water’s anomalous properties have long been a subject of scientific inquiry. The liquid–liquid phase transition hypothesis, which posits the existence of distinct low-density and high-density liquid states separated by a first-order phase transition terminating at a critical point, has gained increasing experimental and computational support and offers a thermodynamically consistent framework for many of water’s anomalies. However, experimental challenges in avoiding crystallization near the postulated liquid–liquid critical point have focused attention to water’s canonical glassy states: low-density and high-density amorphous ice. Here, we use two Deep Potential machine-learning models, trained on the Strongly Constrained and Appropriately Normed density functional and the highly accurate Many-Body Polarizable potential, to conduct an investigation of water’s glassy phenomenology based on quantum mechanical calculations. Despite not being explicitly trained on amorphous ices, both models accurately capture the structure and transformation of the water glasses, including their interconversion along different thermodynamic paths. Isobaric quenching of liquid water at various pressures generates a continuum of intermediate amorphous ices and density fluctuations increase near the liquid–liquid critical pressure. The glass transition temperatures of the amorphous ices produced at different pressures exhibit two distinct branches, corresponding to low-density and high-density amorphous ice behaviors, consistent with experiment and the liquid–liquid transition hypothesis. Extrapolating transformation pressures from isothermal compressions to experimental compression rates brings our simulations into excellent agreement with data. Our findings demonstrate that machine-learning potentials trained on equilibrium phases can effectively model nonequilibrium glassy behavior and pave the way for studying long-timescale, out-of-equilibrium processes with quantum mechanical accuracy.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Neural network approaches for parameterized optimal control

Here, we consider numerical approaches for deterministic, finite-dimensional optimal control problems whose dynamics depend on unknown or uncertain parameters. We seek to amortize the solution over a set of relevant parameters in an offline stage to enable rapid decision-making and be able to react to changes in the parameter in the online stage. To tackle the curse of dimensionality arising when the state and/or parameter are high-dimensional, we represent the policy using neural networks. We compare two training paradigms: First, our model-based approach leverages the dynamics and definition of the objective function to learn the value function of the parameterized optimal control problem and obtain the policy using a feedback form. Second, we use actor-critic reinforcement learning to approximate the policy in a data-driven way. Using an example involving a two-dimensional convection-diffusion equation, which features high-dimensional state and parameter spaces, we investigate the accuracy and efficiency of both training paradigms. While both paradigms lead to a reasonable approximation of the policy, the model-based approach is more accurate and considerably reduces the number of PDE solves.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Observational Constraints on Warm Cloud Microphysical Processes Using Machine Learning and Optimization Techniques

We introduce new parameterizations for autoconversion and accretion rates that greatly improve representation of the growth processes of warm rain. The new parameterizations capitalize on machine‐learning and optimization techniques and are constrained by in situ cloud probe measurements from the recent Atmospheric Radiation Measurement Program field campaign at Azores. The uncertainty in the new estimates of autoconversion and accretion rates is about 15% and 5%, respectively, outperforming existing parameterizations. Our results confirm that cloud and drizzle water content are the most important factors for determining accretion rates. However, for autoconversion, in addition to cloud water content and droplet number concentration, we discovered a key role of drizzle number concentration that is missing in current parameterizations. The robust relation between autoconversion rate and drizzle number concentration is surprising but real, and furthermore supported by theory. Thus, drizzle number concentration should be considered in parameterizations for improved representation of the autoconversion process.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Constrained or unconstrained? Neural-network-based equation discovery from data

Throughout many fields, practitioners often rely on differential equations to model systems. Yet, for many applications, the theoretical derivation of such equations and/or the accurate resolution of their solutions may be intractable. Instead, recently developed methods, including those based on parameter estimation, operator subset selection, and neural networks, allow for the data-driven discovery of both ordinary and partial differential equations (PDEs), on a spectrum of interpretability. The success of these strategies is often contingent upon the correct identification of representative equations from noisy observations of state variables and, as importantly and intertwined with that, the mathematical strategies utilized to enforce those equations. Specifically, the latter has been commonly addressed via unconstrained optimization strategies. Representing the PDE as a neural network, we propose to discover the PDE (or the associated operator) by solving a constrained optimization problem and using an intermediate state representation similar to a physics-informed neural network (PINN). The objective function of this constrained optimization problem promotes matching the data, while the constraints require that the discovered PDE is satisfied at a number of spatial collocation points. We present a penalty method and a widely used trust-region barrier method to solve this constrained optimization problem, and we compare these methods on numerical examples. Our results on several example problems demonstrate that the latter constrained method outperforms the penalty method, particularly for higher noise levels or fewer collocation points. This work motivates further exploration into using sophisticated constrained optimization methods in scientific machine learning, as opposed to their commonly used, penalty-method or unconstrained counterparts. For both of these methods, we solve these discovered neural network PDEs with classical methods, such as finite difference methods, as opposed to PINNs-type methods relying on automatic differentiation. Here, we briefly highlight how simultaneously fitting the data while discovering the PDE improves the robustness to noise and other small, yet crucial, implementation details.

Data-driven discovery↗
DOE OSTI2022

Qualitative trend analysis based on a mixed-integer representation

Shape constrained spline fitting is a useful method to impose prior knowledge onto flexible semi-parametric models during parameter estimation. Most typically, the function shape is imposed through order restrictions on the regression coefficients. The intended shape is considered known or selected based on heuristic rules. In this study, we present a method to estimate the optimal set of order restrictions to segment a univariate data series into episodes with distinct shapes. This is also known as the qualitative trend analysis (QTA) problem. The obtained solution uses a trade-off between lack-of-fit and model complexity. Further, our practical implementation takes inspiration from the generalized order restricted information criterion (GORIC) for inequality-constrained model selection. From this, one learns (a) that QTA can be formulated as a mixed-integer quadratic program (MIQP) and (b) that the newly proposed mixed order restricted information criterion (MORIC) enables optimal segmentation. This is illustrated through didactic case studies.

42 ENGINEERING↗
DOE OSTI2024

Probing the state of hydrogen in δ − AlOOH at mantle conditions with machine learning potential

Hydrous and nominally anhydrous minerals are a fundamental class of solids of enormous significance to geophysics. They are the water carriers in the deep geological water cycle and impact structural, elastic, plastic, and thermodynamic properties and phase relations in Earth's forming aggregates (rocks). They play a critical role in the geochemical and geophysical processes that shape the planet. Their complexity has prevented predictive calculations of their properties, but progress in materials simulations ushered by machine-learning potentials is transforming this state of affairs. Here, we adopt a hybrid approach that combines deep learning potentials (DPs) with the strongly constrained and appropriately normed meta-generalized gradient approximation functional to simulate a prototypical hydrous system. We illustrate the success of this approach to simulate δ − AlOOH ( δ ) , a phase capable of transporting water down to near the core-mantle boundary of the Earth ( ∼ 2900 km depth and ∼ 135 GPa ) in subducting slabs. A high-throughput sampling of phase space using molecular dynamics simulations with DPs sheds light on the hydrogen-bond behavior and proton diffusion at geophysical conditions. These simulations provide a pathway for a deeper understanding of these crucial components that shape Earth's internal state. Published by the American Physical Society 2024

Luo, Chenxing (ORCID:0000000341166851)↗
DOE OSTI2022

ML-PSA

The computer code uses a parallel simulated annealing framework with embedded machine learning components to solve multi-constrained optimization problems. The software automatically balances the execution of low and high fidelity physics models within the optimization procedure. The low fidelity model is used to rapidly explore the design space while the high fidelity physics model is executed sparingly to account for complex design constraints that are not resolved by the quickly executing low fidelity model.

Gurecky, William↗
DOE OSTI2021

Combining artificial intelligence, Earth observations, and climate models to improve predictability of ice-biogeochemistry interactions

Focal Area: Predictive Modeling. We describe how artificial intelligence (AI) can be combined with state-of-the-science Earth system models to better predict future regional climate responses. To demonstrate, we describe a case study in biogeochemical interactions with sea ice. Science Challenge: Biogeochemical models are poorly constrained for high latitude systems. Machine learning methods and edge computing can be combined with Earth system models, such as the Energy Exascale Earth System Model (E3SM), to gain insight into ice-biogeochemical interactions and improve sea ice extent prediction.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

LLNL 5-2428: Fracture Permeability and Seismic Slip Behavior

Our goal is to develop, apply and validate a holistic thermal, hydrologic, mechanical, and chemical (THMC) workflow that also includes evaluation of induced seismic slip in EGS reservoirs. We will integrate experimental and modelling approaches to reduce parameteruncertainty and better predict and mitigate seismic hazard at Utah FORGE and future EGS sites.We propose a novel approach that incorporates 3D physics-based Earthquake simulations in THMC models, herein referred to as “THMC+E” models. This capability will enable improvedengineering decisions at Utah FORGE and move EGS operations toward repeatable, robust, economically viable, and socially accepted development. Utah FORGE management and future EGS operators may employ results of THMC+E models for decision making purposes throughout the lifetime of the field operation. For example, before production well installation, our THMC+E models will predict circulation scenarios and related seismic hazard for a suite ofpossible well locations and flow rates, thus enabling evaluation of optimal production well placement. Such efforts will be conducted throughout the lifetime of the project, whereby additional laboratory experiments will constrain key model parameters and machine learning (ML) will reduce the size of the parameter space and the associated uncertainty. THMC+E simulations will enable exploration various circumstances that may hinder EGS success and develop mitigation strategies.

58 GEOSCIENCES↗
NASA NTRS2019

Understanding Heating in Active Region Cores through Machine Learning. I. Numerical Modeling and Predicted Observables

To adequately constrain the frequency of energy deposition in active region cores in the solar corona, systematic comparisons between detailed models and observational data are needed. In this paper, we describe a pipeline for forward modeling active region emission using magnetic field extrapolations and field-aligned hydrodynamic models. We use this pipeline to predict time-dependent emission from active region NOAA 1158 for low-, intermediate-, and high-frequency nanoflares. In each pixel of our predicted multi-wavelength, time-dependent images, we compute two commonly used diagnostics: the emission measure slope and the time lag. We find that signatures of the heating frequency persist in both of these diagnostics. In particular, our results show that the distribution of emission measure slopes narrows and the mean decreases with decreasing heating frequency and that the range of emission measure slopes is consistent with past observational and modeling work. Furthermore, we find that the time lag becomes increasingly spatially coherent with decreasing heating frequency while the distribution of time lags across the whole active region becomes more broad with increasing heating frequency. In a follow-up paper, we train a random forest classifier on these predicted diagnostics and use this model to classify real observations of NOAA 1158 in terms of the underlying heating frequency.

UV radiation↗
NASA NTRS1986

KBGIS-2: A knowledge-based geographic information system

The architecture and working of a recently implemented knowledge-based geographic information system (KBGIS-2) that was designed to satisfy several general criteria for the geographic information system are described. The system has four major functions that include query-answering, learning, and editing. The main query finds constrained locations for spatial objects that are describable in a predicate-calculus based spatial objects language. The main search procedures include a family of constraint-satisfaction procedures that use a spatial object knowledge base to search efficiently for complex spatial objects in large, multilayered spatial data bases. These data bases are represented in quadtree form. The search strategy is designed to reduce the computational cost of search in the average case. The learning capabilities of the system include the addition of new locations of complex spatial objects to the knowledge base as queries are answered, and the ability to learn inductively definitions of new spatial objects from examples. The new definitions are added to the knowledge base by the system. The system is currently performing all its designated tasks successfully, although currently implemented on inadequate hardware. Future reports will detail the performance characteristics of the system, and various new extensions are planned in order to enhance the power of KBGIS-2.

Smith, T.↗