NeurAM: nonlinear dimensionality reduction for uncertainty quantification through neural active manifolds
Explore the source record for details and available documents.
Schiavazzi, Daniele [University of Notre Dame]↗
SEARCH · Engineering Papers
Results for “nonlinear manifolds”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Cost function for low-dimensional manifold topology assessment
Abstract In reduced-order modeling, complex systems that exhibit high state-space dimensionality are described and evolved using a small number of parameters. These parameters can be obtained in a data-driven way, where a high-dimensional dataset is projected onto a lower-dimensional basis. A complex system is then restricted to states on a low-dimensional manifold where it can be efficiently modeled. While this approach brings computational benefits, obtaining a good quality of the manifold topology becomes a crucial aspect when models, such as nonlinear regression, are built on top of the manifold. Here, we present a quantitative metric for characterizing manifold topologies. Our metric pays attention to non-uniqueness and spatial gradients in physical quantities of interest, and can be applied to manifolds of arbitrary dimensionality. Using the metric as a cost function in optimization algorithms, we show that optimized low-dimensional projections can be found. We delineate a few applications of the cost function to datasets representing argon plasma, reacting flows and atmospheric pollutant dispersion. We demonstrate how the cost function can assess various dimensionality reduction and manifold learning techniques as well as data preprocessing strategies in their capacity to yield quality low-dimensional projections. We show that improved manifold topologies can facilitate building nonlinear regression models.
42 ENGINEERING↗
Statistical Learning for Nonlinear Model Reduction from Local Simulations of Stochastic and Particle- and Agent-Based Systems
Stochastic physical systems across the sciences that have very high-dimensional state spaces, with a large number of fast degrees of freedom that force direct simulators to proceed by integration steps that are orders of magnitude smaller than events of interests (e.g., particle collisions). Examples range from molecular motion to dynamics of large populations of cells. A grand challenge in the simulation and understanding of such systems is the systematic construction of accurate, interpretable, reduced models, enabling faster simulations, revealing fundamental properties of the dynamics, and predicting phenomena of interest that the original simulator could not reached with sufficient accuracy or within a given computational budget. In this projected we developed novel statistical estimation/machine learning techniques for analyzing and building empirical reduced models for important families of high-dimensional stochastic systems, in particular: - we developed techniques for estimating interaction kernels in interacting particle- and agent-based systems, which are ubiquitous in Physics, Biology and many other sciences, given observed trajectories of the system; - we developed techniques for nonlinear model reduction for high-dimensional stochastic systems that have a small number of unknown, nonlinear slow variables, and a large number of fast modes, that are possibly of large magnitude, given observed short trajectories of the system in the form of bursts of trajectories from different initial conditions; - we developed novel techniques for estimating linear dynamical systems on graphs when both the dynamics and the underlying graph are unknown, and we have a sparse set of space-time observations; - we considered the problem of estimating an unknown nonlinear observation function of a standard process (e.g. Brownian motion), so that we can recognized if an observed dynamics is "just" a nonlinear version of a known dynamics; we also developed benchmarks for learning algorithms aimed at learning and classifying diffusion processes.
97 MATHEMATICS AND COMPUTING↗
Normal stability of slow manifolds in nearly periodic Hamiltonian systems
Kruskal [J. Math. Phys. 3, 806 (1962)] showed that each nearly periodic dynamical system admits a formal U(1) symmetry, generated by the so-called roto-rate. We prove that such systems also admit nearly invariant manifolds of each order, near which rapid oscillations are suppressed. We study the nonlinear normal stability of these slow manifolds for nearly periodic Hamiltonian systems on barely symplectic manifolds—manifolds equipped with closed, non-degenerate 2-forms that may be degenerate to leading order. In particular, we establish a sufficient condition for long-term normal stability based on second derivatives of the well-known adiabatic invariant. We use these results to investigate the problem of embedding guiding center dynamics of a magnetized charged particle as a slow manifold in a nearly periodic system. Here, we prove that one previous embedding and two new embeddings enjoy long-term normal stability and thereby strengthen the theoretical justification for these models.
97 MATHEMATICS AND COMPUTING↗
Riemannian Optimization Applied to AC Optimal Power Flow: Preprint
The nonlinear, nonconvex AC optimal power flow problem is of growing importance as the nature of the power grid evolves. This problem can be difficult to solve for interior point methods. However, the advent of optimization algorithms over smooth Riemannian manifolds presents an alternative approach. The nonlinear, nonconvex constraints in the AC power flow problem form an embedded submanifold of Euclidean space. In this paper, the authors explore the performance of Riemannian optimization algorithms for the ACOPF problem where the optimization is performed directly on the AC power flow manifold. They demonstrate that these are viable computational alternatives to interior point methods. This is done by using Julia and the packages PowerModels.jl and Manopt.jl.
manifold optimization↗
Riemannian Optimization Applied to AC Optimal Power Flow
The nonlinear, nonconvex AC optimal power flow problem is of growing importance as the nature of the power grid evolves. This problem can be difficult to solve for interior point methods. However, the advent of optimization algorithms over smooth Riemannian manifolds presents an alternative approach. The nonlinear, nonconvex constraints in the AC power flow problem form an embedded submanifold of Euclidean space. In this paper, the authors explore the performance of Riemannian optimization algorithms for the ACOPF problem where the optimization is performed directly on the AC power flow manifold. This is done by using the Julia programming language and the Julia packages PowerModels.jl and Manopt.jl.
AC optimal power flow↗
NS5-brane backgrounds and coset CFT partition functions
Worldsheet string theory is solvable for a variety of backgrounds involving Neveu-Schwarz fivebranes, in terms of gauged nonlinear sigma models on group manifolds. We compute the worldsheet torus partition function of these models, and propose gauging of null isometries as a unifying principle and conceptual framework for this large family of string backgrounds. In the process, we explain how partition functions of asymmetrically gauged Wess-Zumino-Witten models can be computed from the path integral, and organize and systematize various results scattered throughout the literature.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
Automated and efficient local adaptive regression for principal component-based reduced-order modeling of turbulent reacting flows
Principal Component Analysis can be used to reduce the cost of Computational Fluid Dynamics simulations of turbulent reacting flows by reducing the dimensionality of the transported variables through projection of the thermochemical state onto a lower-dimensional manifold. However, because of the nonlinearity of the principal component source terms, nonlinear regression techniques must be utilized for the source terms in terms of the principal components. Unfortunately, widely available and utilized nonlinear regression techniques can have prohibitive computational requirements and/or accuracy that is highly dependent on user experience in ad hoc tuning of model architecture and hyperparameters. Here, in this work, a new nonlinear regression approach is proposed that is both computationally efficient and automated so does not require any user input. The approach is evaluated through a priori prediction of principal component source terms using data from a Direct Numerical Simulation of a turbulent nonpremixed n-heptane/air jet flame. In particular, the proposed framework consists of local regressions whose complexity is adapted according to the local nonlinearity of the data: local linear regression when accurate enough and local Artificial Neural Networks when nonlinear regression is required. The number of local clusters for local regression is determined automatically using the Davies-Bouldin index. In addition, Bayesian optimization is utilized for model training (i.e., to select the best architectures and hyperparameters of the nonlinear regressions in an unsupervised fashion), eliminating ad hoc hand-tuning and/or expensive grid searches. Overall, compared to a single, global neural network, the new local adaptive regression approach is shown to have comparable accuracy but 69% less training time due to the utilization of local linear regression and faster training of local neural networks.
42 ENGINEERING↗
Exploring regular and turbulent flow states in active nematic channel flow via Exact Coherent Structures and their invariant manifolds
This work is a unified study of stable and unstable steady states of 2D active nematic channel flow using the framework of Exact Coherent Structures (ECS). ECS are stationary, periodic, quasiperiodic, or traveling wave solutions of the governing equations that, together with their invariant manifolds, organize the dynamics of nonlinear continuum systems. We extend our earlier work on ECS in the preturbulent regime by performing a comprehensive study of stable and unstable ECS for a wide range of activity values spanning the preturbulent and turbulent regimes. In the weakly turbulent regime, we compute more than 200 unstable ECS that co-exist at a single set of parameters, and uncover the role of symmetries in organizing the phase space geometry. We provide conclusive numerical evidence that in the preturbulent regime, generic trajectories shadow a series of unstable ECS before settling onto an attractor. Lastly, our studies hint at shadowing of quasiperiodic type ECS in the turbulent regime.
75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
Manifold Learning: What, How, and Why
Manifold learning (ML), also known as nonlinear dimension reduction, is a set of methods to find the low-dimensional structure of data. Dimension reduction for large, high-dimensional data is not merely a way to reduce the data; the new representations and descriptors obtained by ML reveal the geometric shape of high-dimensional point clouds and allow one to visualize, denoise, and interpret them. This review presents the underlying principles of ML, its representative methods, and their statistical foundations, all from a practicing statistician's perspective. It describes the trade-offs and what theory tells us about the parameter and algorithmic choices we make in order to obtain reliable conclusions.
Mathematics↗
Exact Coherent Structures and Phase Space Geometry of Preturbulent 2D Active Nematic Channel Flow
Confined active nematics exhibit rich dynamical behavior, including spontaneous flows, periodic defect dynamics, and chaotic “active turbulence.” Here, we study these phenomena using the framework of exact coherent structures, which has been successful in characterizing the routes to high Reynolds number turbulence of passive fluids. Exact coherent structures are stationary, periodic, quasiperiodic, or traveling wave solutions of the hydrodynamic equations that, together with their invariant manifolds, serve as an organizing template of the dynamics. We compute the dominant exact coherent structures and connecting orbits in a preturbulent active nematic channel flow, which enables a fully nonlinear but highly reduced-order description in terms of a directed graph. Using this reduced representation, we compute instantaneous perturbations that switch the system between disparate spatiotemporal states occupying distant regions of the infinite-dimensional phase space. Overall, our results lay the groundwork for a systematic means of understanding and controlling active nematic flows in the moderate- to high-activity regime.
36 MATERIALS SCIENCE↗
Machine-Learned Manifold-Based Models for Large Eddy Simulation of Turbulent Combustion
Reduced-order manifold combustion models are commonly used to lower the cost of reacting Large Eddy Simulations (LES) and may be derived either from physical principles as in flamelet models or from data-driven methods like Principal Component Analysis (PCA). In either case, neural networks are increasingly used as part of these models to provide a nonlinear mapping between a small set of pre-defined variables that parameterize the manifold and outputs of interest, such as reaction rates. In this work, we propose a new manifold-based modeling approach that combines the definition of the manifold-parameterizing variables (linear combinations of species), the nonlinear mapping to the outputs, and closure of filtered quantities for LES into the structure of a single neural network. This allows the process used to train the neural network to simultaneously optimize both the functional form of the model and the identities of the inputs to the model. The new approach can flexibly incorporate thermochemical data from any combustion system; if trained on data from 1D flames it can be interpreted as an optimized flamelet model, but it can also be used to learn models from data from more complex configurations. This work presents a priori evaluations of the new approach in both contexts. Evaluation using data from 1D premixed flames demonstrates the physical interpretability of the manifold variables generated by the new approach. Evaluation using data from direct numerical simulations of turbulent flames shows improved predictions relative to either flamelet or PCA-based models in a more complex configuration.
47 OTHER INSTRUMENTATION↗
Generative learning for slow manifolds and bifurcation diagrams
In dynamical systems characterized by separation of time scales, the approximation of so called “slow manifolds”, on which the long term dynamics lie, is a useful step for model reduction. Initializing on such slow manifolds is a useful step in modeling, since it circumvents fast transients, and is crucial in multiscale algorithms (like the equation-free approach) alternating between fine scale (fast) and coarser scale (slow) simulations. In a similar spirit, when one studies the infinite time dynamics of systems depending on parameters, the system attractors (e.g., its steady states) lie on bifurcation diagrams (curves for one-parameter continuation, and more generally, on manifolds in state parameter space. Sampling these manifolds gives us representative attractors (here, steady states of ODEs or PDEs) at different parameter values. Algorithms for the systematic construction of these manifolds (slow manifolds, bifurcation diagrams) are required parts of the “traditional” numerical nonlinear dynamics toolkit. In more recent years, as the field of Machine Learning develops, conditional score-based generative models (cSGMs) have been demonstrated to exhibit remarkable capabilities in generating plausible data from target distributions that are conditioned on some given label. It is tempting to exploit such generative models to produce samples of data distributions (points on a slow manifold, steady states on a bifurcation surface) conditioned on (consistent with) some quantity of interest (QoI, observable). In this work, we present a framework for using cSGMs to quickly (a) initialize on a low-dimensional (reduced-order) slow manifold of a multi-time-scale system consistent with desired value(s) of a QoI (a “label”) on the manifold, and (b) approximate steady states in a bifurcation diagram consistent with a (new, out-of-sample) parameter value. This conditional sampling can help uncover the geometry of the reduced slow-manifold and/or approximately “fill in” missing segments of steady states in a bifurcation diagram. Finally, the quantity of interest, which determines how the sampling is conditioned, is either known a priori or identified using manifold learning-based dimensionality reduction techniques applied to the training data.
Dynamical systems↗
Variational Autoencoders for Learning Nonlinear Dynamics of Physical Systems
We develop data-driven methods for incorporating physical information for priors to learn parsimonious representations of nonlinear systems arising from parameterized PDEs and mechanics. Our approach is based on Variational Autoencoders (VAEs) for learning nonlinear state space models from observations. We develop ways to incorporate geometric and topological priors through general manifold latent space representations. We investigate the performance of our methods for learning low dimensional representations for the nonlinear Burgers equation and constrained mechanical systems.
97 MATHEMATICS AND COMPUTING↗
Grassmannian Diffusion Maps--Based Dimension Reduction and Classification for High-Dimensional Data
This work introduces the Grassmannian diffusion maps (GDMaps), a novel nonlinear dimensionality reduction technique that defines the affinity between points through their representation as low-dimensional subspaces corresponding to points on the Grassmann manifold. Here, the method is designed for applications, such as image recognition and data-based classification of constrained high-dimensional data where each data point itself is a high-dimensional object (i.e., a large matrix) that can be compactly represented in a lower-dimensional subspace. The GDMaps is composed of two stages. The first is a pointwise linear dimensionality reduction wherein each high-dimensional object is mapped onto the Grassmann manifold representing the low-dimensional subspace on which it resides. The second stage is a multipoint nonlinear kernel-based dimension reduction using diffusion maps to identify the subspace structure of the points on the Grassmann manifold. To this end, an appropriate Grassmannian kernel is used to construct the transition matrix of a random walk on a graph connecting points on the Grassmann manifold. Spectral analysis of the transition matrix yields low-dimensional Grassmannian diffusion coordinates embedding the data into a low-dimensional reproducing kernel Hilbert space. Further, a novel data classification/recognition technique is developed based on the construction of an overcomplete dictionary of reduced dimension whose atoms are given by the Grassmannian diffusion coordinates. Three examples are considered. First, a "toy" example shows that the GDMaps can identify an appropriate parametrization of structured points on the unit sphere. The second example demonstrates the ability of the GDMaps to revealing the intrinsic subspace structure of high-dimensional random field data. In the last ex- ample, a face recognition problem is solved considering face images subject to varying illumination conditions, changes in face expressions, and occurrence of occlusions. The technique presented high recognition rates (i.e., 95% in the best case) using a fraction of the data required by conventional methods.
42 ENGINEERING↗
Variational nonlinear WKB in the Eulerian frame
Nonlinear WKB is a multiscale technique for studying locally plane-wave solutions of nonlinear partial differential equations (PDEs). Its application comprises two steps: (1) replacement of the original PDE with an extended system separating the large scales from the small and (2) reduction of the extended system to its slow manifold. In the context of variational fluid theories with particle relabeling symmetry, nonlinear WKB in the mean Eulerian frame is known to possess a variational structure. This much has been demonstrated using, for instance, the theoretical apparatus known as the generalized Lagrangian mean. On the other hand, the variational structure of nonlinear WKB in the conventional Eulerian frame remains mysterious. By exhibiting a variational principle for the extended equations from step (1) above, here we demonstrate that nonlinear WKB in the Eulerian frame is in fact variational. Remarkably, the variational principle for the extended system admits loops of relabeling transformations as a symmetry group. Noether’s theorem therefore implies that the extended Eulerian equations possess a family of circulation invariants parameterized by S 1 . As an illustrative example, we use our results to systematically deduce a variational model of high-frequency acoustic waves interacting with a larger-scale compressible isothermal flow.
97 MATHEMATICS AND COMPUTING↗
Preserving nonlinear constraints in variational flow filtering data assimilation
Data assimilation aims to estimate the states of a dynamical system by optimally combining sparse and noisy observations of the physical system with uncertain forecasts produced by a computational model. The states of many dynamical systems of interest obey nonlinear physical constraints, and the corresponding dynamics is confined to a certain sub-manifold of the state space. Standard data assimilation techniques applied to such systems yield posterior states lying outside the manifold, violating the physical constraints. This work focuses on particle flow filters which use stochastic differential equations to evolve state samples from a prior distribution to samples from an observation-informed posterior distribution. The variational Fokker-Planck (VFP)—a generic particle flow filtering framework—is extended to incorporate non-linear, equality state constraints in the analysis. To this end, two algorithmic approaches that modify the VFP stochastic differential equation are discussed: (i) VFPSTAB, to inexactly preserve constraints with the addition of a stabilizing drift term, and (ii) VFPDAE, to exactly preserve constraints by treating the VFP dynamics as a stochastic differential-algebraic equation (SDAE). Additionally, an implicit-explicit time integrator is developed to evolve the VFPDAE dynamics. The strength of the proposed approach for constraint preservation in data assimilation is demonstrated on three test problems: the double pendulum, Korteweg-de-Vries, and the incompressible Navier-Stokes equations.
97 MATHEMATICS AND COMPUTING↗
An adaptive space-time phase field formulation for dynamic fracture of brittle shells based on LR NURBS
Here, we present an adaptive space-time phase field formulation for dynamic fracture of brittle shells. Their deformation is characterized by the Kirchhoff–Love thin shell theory using a curvilinear surface description. All kinematical objects are defined on the shell’s mid-plane. The evolution equation for the phase field is determined by the minimization of an energy functional based on Griffith’s theory of brittle fracture. Membrane and bending contributions to the fracture process are modeled separately and a thickness integration is established for the latter. The coupled system consists of two nonlinear fourth-order PDEs and all quantities are defined on an evolving two-dimensional manifold. Since the weak form requires C 1 -continuity, isogeometric shape functions are used. The mesh is adaptively refined based on the phase field using Locally Refinable (LR) NURBS. Time is discretized based on a generalized-α method using adaptive time-stepping, and the discretized coupled system is solved with a monolithic Newton–Raphson scheme. Finally, the interaction between surface deformation and crack evolution is demonstrated by several numerical examples showing dynamic crack propagation and branching.
42 ENGINEERING↗