Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “physics regularization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Anisotropic physics-regularized interpretable machine learning of microstructure evolution

Anisotropic Physics-Regularized Interpretable Machine Learning Microstructure Evolution (APRIMME) is a general-purpose machine learning solution for grain growth simulations. In prior work, PRIMME employed a deep neural network to predict site-specific migration as a function of its neighboring sites to model normal, isotropic, grain growth behavior. This work aims to extend this method by incorporating grain boundary misorientation-based grain growth behavior. APRIMME is trained on anisotropic simulations created using the Monte Carlo-Potts (MCP) model. Furthermore, the results of this work are compared statistically using grain radius, number of sides per grain, mean neighborhood misorientations, and the standard deviation of triple junction dihedral angles, and are found to match in most cases. The exceptions are small and seem to be related to two causes: (1) the deterministic model of APRIMME is learning from the stochastic simulations of MCP, which seems to accentuate triple junction behaviors; and, (2) a bias against very small grains is made evident in a quicker decrease in grains than expected at the beginning of an APRIMME simulation. APRIMME is also evaluated for its general ability to capture anisotropic grain growth behavior by first investigating different test case initial conditions, including a circle grain, three grain, and hexagonal grain microstructures.

36 MATERIALS SCIENCE↗

Physically regularized machine learning emulators of aerosol activation

Abstract. The activation of aerosol into cloud droplets is an important step in the formation of clouds and strongly influences the radiative budget of the Earth. Explicitly simulating aerosol activation in Earth system models is challenging due to the computational complexity required to resolve the necessary chemical and physical processes and their interactions. As such, various parameterizations have been developed to approximate these details at reduced computational cost and accuracy. Here, we explore how machine learning emulators can be used to bridge this gap in computational cost and parameterization accuracy. We evaluate a set of emulators of a detailed cloud parcel model using physically regularized machine learning regression techniques. We find that the emulators can reproduce the parcel model at higher accuracy than many existing parameterizations. Furthermore, physical regularization tends to improve emulator accuracy, most significantly when emulating very low activation fractions. This work demonstrates the value of physical constraints in machine learning model development and enables the implementation of improved hybrid physical and machine learning models of aerosol activation into next-generation Earth system models.

58 GEOSCIENCES↗

Physics-informed regularization and structure preservation for learning stable reduced models from data with operator inference

Operator inference learns low-dimensional dynamical-system models with polynomial nonlinear terms from trajectories of high-dimensional physical systems (non-intrusive model reduction). Here, this work focuses on the large class of physical systems that can be well described by models with quadratic and cubic nonlinear terms and proposes a regularizer for operator inference that induces a stability bias onto learned models. The proposed regularizer is physics informed in the sense that it penalizes higher-order terms with large norms and so explicitly leverages the polynomial model form that is given by the underlying physics. This means that the proposed approach judiciously learns from data and physical insights combined, rather than from either data or physics alone. Additionally, a formulation of operator inference is proposed that enforces model constraints for preserving structure such as symmetry and definiteness in linear terms. Numerical results demonstrate that models learned with operator inference and the proposed regularizer and structure preservation are accurate and stable even in cases where using no regularization and Tikhonov regularization leads to models that are unstable.

97 MATHEMATICS AND COMPUTING↗

Physiological Adaptations to Progressive Endurance Exercise Training in Adult and Aged Rats: Insights from the Molecular Transducers of Physical Activity Consortium (MoTrPAC)

While regular physical activity is a cornerstone of health, wellness, and vitality, the impact of endurance exercise training on molecular signaling within and across tissues remains to be delineated. The Molecular Transducers of Physical Activity Consortium (MoTrPAC) was established to characterize molecular networks underlying the adaptive response to exercise. Here, we describe the endurance exercise training studies undertaken by the Preclinical Animal Sites Studies component of MoTrPAC, in which we sought to develop and implement a standardized endurance exercise protocol in a large cohort of rats. To this end, Adult (6-mo) and Aged (18-mo) female (n = 151) and male (n = 143) Fischer 344 rats were subjected to progressive treadmill training (5 d/wk, ~70%–75% VO 2 max) for 1, 2, 4, or 8 wk; sedentary rats were studied as the control group. A total of 18 solid tissues, as well as blood, plasma, and feces, were collected to establish a publicly accessible biorepository and for extensive omics-based analyses by MoTrPAC. Treadmill training was highly effective, with robust improvements in skeletal muscle citrate synthase activity in as little as 1–2 wk and improvements in maximum run speed and maximal oxygen uptake by 4–8 wk. For body mass and composition, notable age- and sex-dependent responses were observed. This work in mature, treadmill-trained rats represents the most comprehensive and publicly accessible tissue biorepository, to date, and provides an unprecedented resource for studying temporal-, sex-, and age-specific responses to endurance exercise training in a preclinical rat model.

60 APPLIED LIFE SCIENCES↗

Recurrent neural network-based multiaxial plasticity model with regularization for physics-informed constraints

We report a recurrent neural network (RNN) based model is developed as a surrogate to predict nonlinear plastic response under multiaxial loading. The RNN-based model is trained and tested on stress versus strain curves generated using a numerical solution based on the classical radial return method. Besides simply learning the basic constitutive relationship, a novel approach is taken to enforce certain physical conditions. Specifically, regularization is employed to maintain non-negative plastic power density throughout the loading history thereby ensuring monotonically increasing plastic work and thermodynamic consistency. Enforcing physics in this manner permits coupling of the data-driven RNN approach with physics-based knowledge and laws. This has the effect of reducing the necessary amount of data and ensuring known physical laws are not violated. Since, once trained, the model need not perform the expensive task of solving nonlinear equations, its efficiency is orders of magnitude greater than its numerical counterpart. The RNN-based model has been trained on varied sets of data and the accuracy on test datasets validated. The developed model is general and robust and has widespread application such as in the simulation of metal forming, large scale plasticity, and part life prediction.

42 ENGINEERING↗

Nonideal Rayleigh-Taylor Mixing

Rayleigh-Taylor mixing is a classical hydrodynamic instability that occurs when a light fluid pushes against a heavy fluid. The two main sources of nonideal behavior in Rayleigh-Taylor (RT) mixing are regularizations (physical and numerical), which produce deviations from a pure Euler equation, scale invariant formulation, and nonideal (i.e., experimental) initial conditions. The Kolmogorov theory of turbulence predicts stirring at all length scales for the Euler fluid equations without regularization. We interpret mathematical theories of existence and nonuniqueness in this context, and we provide numerical evidence for dependence of the RT mixing rate on nonideal regularizations; in other words, indeterminacy when modeled by Euler equations. Operationally, indeterminacy shows up as nonunique solutions for RT mixing, parametrized by Schmidt and Prandtl numbers, in the large Reynolds number (Euler equation) limit. Verification and validation evidence is presented for the large eddy simulation algorithm used here. Mesh convergence depends on breaking the nonuniqueness with explicit use of the laminar Schmidt and Prandtl numbers and their turbulent counterparts, defined in terms of subgrid scale models. The dependence of the mixing rate on the Schmidt and Prandtl numbers and other physical parameters will be illustrated. We demonstrate numerically the influence of initial conditions on the mixing rate. Both the dominant short wavelength initial conditions and long wavelength perturbations are observed to play a role. By examination of two classes of experiments, we observe the absence of a single universal explanation, with long and short wavelength initial conditions, and the various physical and numerical regularizations contributing in different proportions in these two different contexts.

97 MATHEMATICS AND COMPUTING↗

Threshold and spin factors in the yield of bremsstrahlung-induced reactions

Relative yields of photon-induced reactions are systematized in a function of the threshold parameter for moderately heavy targets at the bremsstrahlung end-point energy lower than 30 MeV. Regular dependence is established for the group of (γ, n), (γ, p), and (γ, d) reactions, while the yields of (γ, 2n) and (γ,α) reactions deviate from the regularity. Physical conclusions are discussed and possible application of this systematic for data processing is proposed. In particular, the constituent threshold and spin factors in the isomer-to-ground-state ratio could be isolated separately. For spin dependence of the yields, a new regularization parameter is introduced and previously hidden peculiarities are concluded.

74 ATOMIC AND MOLECULAR PHYSICS↗

Recipes for when physics fails: recovering robust learning of physics informed neural networks

Abstract Physics-informed neural networks (PINNs) have been shown to be effective in solving partial differential equations by capturing the physics induced constraints as a part of the training loss function. This paper shows that a PINN can be sensitive to errors in training data and overfit itself in dynamically propagating these errors over the domain of the solution of the PDE. It also shows how physical regularizations based on continuity criteria and conservation laws fail to address this issue and rather introduce problems of their own causing the deep network to converge to a physics-obeying local minimum instead of the global minimum. We introduce Gaussian process (GP) based smoothing that recovers the performance of a PINN and promises a robust architecture against noise/errors in measurements. Additionally, we illustrate an inexpensive method of quantifying the evolution of uncertainty based on the variance estimation of GPs on boundary data. Robust PINN performance is also shown to be achievable by choice of sparse sets of inducing points based on sparsely induced GPs. We demonstrate the performance of our proposed methods and compare the results from existing benchmark models in literature for time-dependent Schrödinger and Burgers’ equations.

97 MATHEMATICS AND COMPUTING↗

Accelerated deep self-supervised ptycho-laminography for three-dimensional nanoscale imaging of integrated circuits

Three-dimensional inspection of nanostructures such as integrated circuits is important for security and reliability assurance. Two scanning operations are required: ptychographic to recover the complex transmissivity of the specimen, and rotation of the specimen to acquire multiple projections covering the 3D spatial frequency domain. Two types of rotational scanning are possible: tomographic and laminographic. For flat, extended samples, for which the full 180° coverage is not possible, the latter is preferable because it provides better coverage of the 3D spatial frequency domain compared to limited-angle tomography. It is also because the amount of attenuation through the sample is approximately the same for all projections. However, both techniques are time consuming because of extensive acquisition and computation time. Here, we demonstrate the acceleration of ptycho-laminographic reconstruction of integrated circuits with 16 times fewer angular samples and 4.67 times faster computation by using a physics-regularized deep self-supervised learning architecture. We check the fidelity of our reconstruction against a densely sampled reconstruction that uses full scanning and no learning. As already reported elsewhere [ Opt. Express 28 , 12872 ( 2020 ) OPEXFF 1094-4087 10.1364/OE.379200 ], we observe improvement of reconstruction quality even over the densely sampled reconstruction, due to the ability of the self-supervised learning kernel to fill the missing cone.

47 OTHER INSTRUMENTATION↗

Elucidating Abnormal Grain Growth in Thermomagnetic Processed Materials with Transfer Learning and Reinforcement Learning

The goal of this research program is to establish the mechanism governing local grain boundary motion, which is needed to design and process desirable microstructures for better performance, by identifying the relative contributions of grain boundary (GB) energy and mobility to grain growth. Classical models for grain growth assume that the primary mechanism for reducing the total interfacial energy is area reduction and that GB restructuring is not significant. This assumption implies that grain growth is locally driven by curvature. However, recent experimental observations using new non-destructive 3D x-ray diffraction microscopy techniques (3D-XRM) reveal that classic descriptors (i.e., curvature, number of neighbors, grain size) do not predict real grain growth. Instead, local GB motion appears to be governed by its energy relative to its neighbors such that low-energy boundaries replace those of higher energy. However, simulations that incorporate GB energy anisotropy still fail to reproduce these observations. These discrepancies suggest that the common assumption for grain growth theory must be re-examined to predict and, thus, control microstructure evolution in real polycrystals. A significant challenge to testing this assumption is due to anisotropic GB mobility. Mobility may cause abnormal grain growth or affect the final grain shapes or growth rate but its true contributions are unknown because it is difficult to measure. For example, observations in Fe have found that grains associated with high energy and high mobility boundaries tend to experience abnormal grain growth, whereas abnormal grain growth is associated with low energy and high mobility boundaries in alumina. As mobility and energy both control GB motion, it is challenging to isolate the local driving forces necessary to test the common assumption that the primary mechanism is area reduction. The novelty of this work is the use of machine learning tools to capture GB mobility and energy from 3D-XRM measurements in polycrystals to test the common assumption used in grain growth models. Machine learning can capture high-order correlations in dynamic systems like those found in the evolving GB topology. The PIs have developed a physics-regularized interpretable machine learning microstructure evolution (PRIMME) model that accurately replicates the grain growth behavior of its trained data set.

36 MATERIALS SCIENCE↗

GRIDS-Net: Inverse shape design and identification of scatterers via geometric regularization and physics-embedded deep learning

This study presents a deep learning based methodology for both remote sensing and design of acoustic scatterers. The ability to determine the shape of a scatterer, either in the context of material design or sensing, plays a critical role in many practical engineering problems. This class of inverse problems is extremely challenging due to their high-dimensional, nonlinear, and ill-posed nature. To overcome these technical hurdles, we introduce a geometric regularization approach for deep neural networks (DNN) based on non-uniform rational B-splines (NURBS) and capable of predicting complex 2D scatterer geometries in a parsimonious dimensional representation. Then, this geometric regularization is combined with physics-embedded learning and integrated within a robust convolutional autoencoder (CAE) architecture to accurately predict the shape of 2D scatterers in the context of identification and inverse design problems. Further, an extensive numerical study is presented in order to showcase the remarkable ability of this approach to handle complex scatterer geometries while generating physically-consistent acoustic fields. The study also assesses and contrasts the role played by the (weakly) embedded physics in the convergence of the DNN predictions to a physically consistent inverse design.

42 ENGINEERING↗

Deep Learning with Physics Priors as Generalized Regularizers

In various scientific and engineering applications, there is typically an approximate model of the underlying complex system, even though it contains both aleatoric and epistemic uncertainties. In this paper, we present a principled method to incorporate these approximate models as physics priors in modeling, to prevent overfitting and enhancing the generalization capabilities of the trained models. Utilizing the structural risk minimization (SRM) inductive principle pioneered by Vapnik, this approach structures the physics priors into generalized regularizers. The experimental results demonstrate that our method achieves up to two orders of magnitude of improvement in testing accuracy.

Liu, Frank↗

A kinetic-based regularization method for data science applications

We propose a physics-based regularization technique for function learning, inspired by statistical mechanics. By drawing an analogy between optimizing the parameters of an interpolator and minimizing the energy of a system, we introduce corrections that impose constraints on the lower-order moments of the data distribution. This minimizes the discrepancy between the discrete and continuum representations of the data, in turn allowing to access more favorable energy landscapes, thus improving the accuracy of the interpolator. Our approach improves performance in both interpolation and regression tasks, even in high-dimensional spaces. Unlike traditional methods, it does not require empirical parameter tuning, making it particularly effective for handling noisy data. We also show that thanks to its local nature, the method offers computational and memory efficiency advantages over Radial Basis Function interpolators, especially for large datasets.

97 MATHEMATICS AND COMPUTING↗

A physics-informed and hierarchically regularized data-driven model for predicting fluid flow through porous media

This paper presents a new deep learning data-driven model for predicting structure dependent pore-fluid velocity fields in rock. The model is based on a Convolutional Auto-Encoder (CAE) artificial neural network capable of learning from image data generated by direct numerical simulations of fluid flow through pore-structures, such as by Lattice Boltzmann or molecular dynamics methods. The main novelty of the model in comparison to previous CAE-based data-driven approaches consists of three parts. The first is a methodology for decomposing the full-domain of the porous media into sub-regions, or “sub-domains”, in order to reduce the overall size of the CAE, batch process the sub-domains in parallel, and enable the CAE to learn local and generalizable nonlinear mappings of pore-fluid velocities. The second consists of embedding the finite difference solutions of the incompressible Navier-Stokes and continuity equations into convolutional layers prior to the CAE in order to provide the CAE with knowledge of fluid dynamics physics (PhyFlow). The third main novelty is that the training of the CAE is regularized with a hierarchical loss function that encourages the learning of fluid flow patterns (in a way similar to ranked modes in principal component analysis), ranking from most to least important. This is shown to increase the stability in learning, reduce over-fitting, and promote interpretability of the CAE neural network layers (HierCAE). The comprehensive new data-driven model, which we call the PhyFlow-HierCAE model, is shown to exhibit improved accuracy and generalizability of flow field predictions over conventional CAE models, attributable to the embedded physical knowledge and the hierarchical regularization, as well as realize orders of magnitude speed-ups in computation times as a surrogate for the direct numerical simulations. Examples of training and forward predictions on unseen pore-structures are provided and evaluated for data from Lattice Boltzmann and molecular dynamics simulations of pore-fluid flow. The model is shown to be a fast and accurate emulator (or “surrogate”) for predicting effective permeability of unseen pore-structures based on learning from relatively small direct numerical simulation datasets.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Regularizing cosmological singularities by varying physical constants

Varying physical constant cosmologies were claimed to solve standard cosmological problems such as the horizon, the flatness and the Λ-problem. In this paper, we suggest yet another possible application of these theories: solving the singularity problem. By specifying some examples we show that various cosmological singularities may be regularized provided the physical constants evolve in time in an appropriate way.

79 ASTRONOMY AND ASTROPHYSICS↗

Differentiable but exact formulation of density-functional theory

The universal density functional F of density-functional theory is a complicated and ill-behaved function of the density—in particular, F is not differentiable, making many formal manipulations more complicated. While F has been well characterized in terms of convex analysis as forming a conjugate pair (E, F) with the ground-state energy E via the Hohenberg–Kohn and Lieb variation principles, F is nondifferentiable and subdifferentiable only on a small (but dense) subset of its domain. In this article, we apply a tool from convex analysis, Moreau–Yosida regularization, to construct, for any ε > 0, pairs of conjugate functionals ({sup ε}E, {sup ε}F) that converge to (E, F) pointwise everywhere as ε → 0{sup +}, and such that {sup ε}F is (Fréchet) differentiable. For technical reasons, we limit our attention to molecular electronic systems in a finite but large box. It is noteworthy that no information is lost in the Moreau–Yosida regularization: the physical ground-state energy E(v) is exactly recoverable from the regularized ground-state energy {sup ε}E(v) in a simple way. All concepts and results pertaining to the original (E, F) pair have direct counterparts in results for ({sup ε}E, {sup ε}F). The Moreau–Yosida regularization therefore allows for an exact, differentiable formulation of density-functional theory. In particular, taking advantage of the differentiability of {sup ε}F, a rigorous formulation of Kohn–Sham theory is presented that does not suffer from the noninteracting representability problem in standard Kohn–Sham theory.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Extreme sparsification of physics-augmented neural networks for interpretable model discovery in mechanics

Data-driven constitutive modeling with neural networks has received increased interest in recent years due to its ability to easily incorporate physical and mechanistic constraints and to overcome the challenging and time-consuming task of formulating phenomenological constitutive laws that can accurately capture the observed material response. However, even though neural network-based constitutive laws have been shown to generalize proficiently, the generated representations are not easily interpretable due to their high number of trainable parameters. Sparse regression approaches exist that allow for obtaining interpretable expressions, but the user is tasked with creating a library of model forms which by construction limits their expressiveness to the functional forms provided in the libraries. Here, in this work, we propose to train regularized physics-augmented neural network-based constitutive models utilizing a smoothed version of $L^0$-regularization. This aims to maintain the trustworthiness inherited by the physical constraints, but also enables interpretability which has not been possible thus far on any type of machine learning-based constitutive model where model forms were not assumed a priori but were actually discovered. During the training process, the network simultaneously fits the training data and penalizes the number of active parameters, while also ensuring constitutive constraints such as thermodynamic consistency. We show that the method can reliably obtain interpretable and trustworthy constitutive models for compressible and incompressible hyperelasticity, yield functions, and hardening models for elastoplasticity, using synthetic and experimental data. This work aims to set a new paradigm for interpretable machine learning models in the broad area of solid mechanics where low and limited data is available along with prior knowledge of physical constraints that the learned maps need to obey. This paradigm can potentially be extended to a broader spectrum of scientific exploration.

Data-driven constitutive models↗