SEARCH · Engineering Papers

Results for “Convergent”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Local practically safe extremum seeking with assignable rate of attractivity to the safe set

We present Assignably Safe Extremum Seeking (ASfES), an algorithm designed to minimize a measured, static objective function while maintaining a measured, static metric of safety (a control barrier function or CBF) to be positive in a practical sense. We ensure that for trajectories with safe initial conditions, the violation of safety can be made arbitrarily small through appropriately chosen design constants. We also guarantee an assignable “attractivity” rate: from unsafe initial conditions, the trajectories approach the safe set, in the sense of the measured CBF, at a rate no slower than a user-assigned rate. Similarly, from safe initial conditions, the trajectories approach the unsafe set, in the sense of the CBF, no faster than the assigned attractivity rate. The feature of assignable attractivity is not present in the semiglobal version of safe extremum seeking, where the semiglobality of convergence is achieved by slowing the adaptation. We also demonstrate local convergence of the parameter to a neighborhood of the minimum of a quadratic objective function constrained to the safe set with a linear CBF. The ASfES algorithm and analysis are multivariable, but we also extend the algorithm to a Newton-Based ASfES scheme which we show is only useful in the scalar case. The proven properties of the designs are illustrated through simulation examples.

42 ENGINEERING↗

DOE OSTI2023

An adaptive sampling augmented Lagrangian method for stochastic optimization with deterministic constraints

The primary goal of this paper is to provide an efficient solution algorithm based on the augmented Lagrangian framework for optimization problems with a stochastic objective function and deterministic constraints. Our main contribution is combining the augmented Lagrangian framework with adaptive sampling, resulting in an efficient optimization methodology validated with practical examples. To achieve the presented efficiency, here we consider inexact solutions for the augmented Lagrangian subproblems, and through an adaptive sampling mechanism, we control the variance in the gradient estimates. Furthermore, we analyze the theoretical performance of the proposed scheme by showing equivalence to a gradient descent algorithm on a Moreau envelope function, and we prove sublinear convergence for convex objectives and linear convergence for strongly convex objectives with affine equality constraints. The worst-case sample complexity of the resulting algorithm, for an arbitrary choice of penalty parameter in the augmented Lagrangian function, is $\mathscr{O}$(ϵ -3-δ ) , where ϵ > 0 is the expected error of the solution and δ > 0 is a user-defined parameter. If the penalty parameter is chosen to be $\mathscr{O}$(ϵ -1 ), we demonstrate that the result can be improved to $\mathscr{O}$(ϵ -2 ) , which is competitive with the other methods employed in the literature. Moreover, if the objective function is strongly convex with affine equality constraints, we obtain $\mathscr{O}$(ϵ -1 log(1/ϵ)) complexity. Finally, we empirically verify the performance of our adaptive sampling augmented Lagrangian framework in machine learning optimization and engineering design problems, including topology optimization of a heat sink with environmental uncertainty.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

GMRES with embedded ensemble propagation for the efficient solution of parametric linear systems in uncertainty quantification of computational models

In a previous work, embedded ensemble propagation was proposed to improve the efficiency of sampling-based uncertainty quantification methods of computational models on emerging computational architectures. It consists of simultaneously evaluating the model for a subset of samples together, instead of evaluating them individually. A first approach introduced to solve parametric linear systems with ensemble propagation is ensemble reduction. In Krylov methods for example, this reduction consists in coupling the samples together using an inner product that sums the sample contributions. Ensemble reduction has the advantages of being able to use optimized implementations of BLAS functions and having a stopping criterion which involves only one scalar. However, the reduction potentially decreases the rate of convergence due to the gathering of the spectra of the samples. In this paper, we investigate a second approach: ensemble propagation without ensemble reduction in the case of GMRES. This second approach solves each sample simultaneously but independently to improve the convergence compared to ensemble reduction. This raises two new issues which are solved in this paper: the fact that optimized implementations of BLAS functions cannot be used anymore and that ensemble divergence, whereby individual samples within an ensemble must follow different code execution paths, can occur. We tackle those issues by implementing a high-performing ensemble GEMV and by using masks. The proposed ensemble GEMV leads to a similar cost per GMRES iteration for both approaches, i.e. with and without reduction. For illustration, we study the performances of the new linear solver in the context of a mesh tying problem. Furthermore, this example demonstrates improved ensemble propagation speed-up without reduction.

BLAS↗

DOE OSTI2020

PPINN: Parareal physics-informed neural network for time-dependent PDEs

Physics-informed neural networks (PINNs) encode physical conservation laws and prior physical knowledge into the neural networks, ensuring the correct physics is represented accurately while alleviating the need for supervised learning to a great degree. While effective for relatively short-term time integration, when long time integration of the time-dependent PDEs is sought, the time–space domain may become arbitrarily large and hence training of the neural network may become prohibitively expensive. To this end, we develop a parareal physics-informed neural network (PPINN), hence decomposing a long-time problem into many independent short-time problems supervised by an inexpensive/fast coarse-grained (CG) solver. In particular, the serial CG solver is designed to provide approximate predictions of the solution at discrete times, while initiate many fine PINNs simultaneously to correct the solution iteratively. There is a two-fold benefit from training PINNs with small-data sets rather than working on a large-data set directly, i.e., training of individual PINNs with small-data is much faster, while training the fine PINNs can be readily parallelized. Consequently, compared to the original PINN approach, the proposed PPINN approach may achieve a significant speed-up for long-time integration of PDEs, assuming that the CG solver is fast and can provide reasonable predictions of the solution, hence aiding the PPINN solution to converge in just a few iterations. To investigate the PPINN performance on solving time-dependent PDEs, we first apply the PPINN to solve the Burgers equation, and subsequently we apply the PPINN to solve a two-dimensional nonlinear diffusion–reaction equation. Furthermore, our results demonstrate that PPINNs converge in a few iterations with significant speed-ups proportional to the number of time-subdomains employed.

42 ENGINEERING↗

DOE OSTI2020

On differentiable local bounds preserving stabilization for Euler equations

This work presents the design of nonlinear stabilization techniques for the finite element discretization of Euler equations in both steady and transient form. Implicit time integration is used in the case of the transient form. A differentiable local bounds preserving method has been developed, which combines a Rusanov artificial diffusion operator and a differentiable shock detector. Nonlinear stabilization schemes are usually stiff and highly nonlinear. This issue is mitigated by the differentiability properties of the proposed method. Moreover, in order to further improve the nonlinear convergence, we also propose a continuation method for a subset of the stabilization parameters. The resulting method has been successfully applied to steady and transient problems with complex shock patterns. Numerical experiments show that it is able to provide sharp and well resolved shocks. Furthermore, the importance of the differentiability is assessed by comparing the new scheme with its non-differentiable counterpart. Numerical experiments suggest that, for up to moderate nonlinear tolerances, the method exhibits improved robustness and nonlinear convergence behavior for steady problems. Additionally, in the case of transient problem, we also observe a reduction in the computational cost.

42 ENGINEERING↗

DOE OSTI2020

The second-generation Shifted Boundary Method and its numerical analysis

Recently, the Shifted Boundary Method (SBM) was proposed within the class of unfitted (or immersed, or embedded) finite element methods. By reformulating the original boundary value problem over a surrogate (approximate) computational domain, the SBM avoids integration over cut cells and the associated problematic issues regarding numerical stability and matrix conditioning. Accuracy is maintained by modifying the original boundary conditions using Taylor expansions. Hence the name of the method, that shifts the location and values of the boundary conditions. In this article, we present enhanced variational SBM formulations for the Poisson and Stokes problems with improved flexibility and robustness. These simplified variational forms allow to relax some of the assumptions required by the mathematical proofs of stability and convergence of earlier implementations. First, we show that these new SBM implementations can be proved asymptotically stable and convergent even without the rather restrictive assumption that the inner product between the normals to the true and surrogate boundaries is positive. Second, we show that it is not necessary to introduce a stabilization term involving the tangential derivatives of the solution at Dirichlet boundaries, therefore avoiding the calibration of an additional stabilization parameter. Finally, we prove enhanced L 2 -estimates without the cumbersome assumption – of earlier proofs – that the surrogate domain is convex. Instead we rely on a conventional assumption that the boundary of the true domain is smooth, which can also be replaced by requiring convexity of the true domain. The aforementioned improvements open the way to a more general and efficient implementation of the Shifted Boundary Method, particularly in complex three-dimensional geometries. We complement these theoretical developments with numerical experiments in two and three dimensions.

42 ENGINEERING↗

DOE OSTI2020

A coupled multipoint stress–multipoint flux mixed finite element method for the Biot system of poroelasticity

In this work, we present a mixed finite element method for a five-field formulation of the Biot system of poroelasticity that reduces to a cell-centered pressure–displacement system on simplicial and quadrilateral grids. A mixed stress–displacement–rotation formulation for elasticity with weak stress symmetry is coupled with a mixed velocity–pressure Darcy formulation. The spatial discretization is based on combining the multipoint stress mixed finite element (MSMFE) method for elasticity and the multipoint flux mixed finite element (MFMFE) method for Darcy flow. It uses the lowest order Brezzi–Douglas–Marini mixed finite element spaces for the poroelastic stress and Darcy velocity, piecewise constant displacement and pressure, and continuous piecewise linear or bilinear rotation. A vertex quadrature rule is applied to the velocity, stress, and stress–rotation bilinear forms, which block-diagonalizes the corresponding matrices and allows for local velocity, stress, and rotation elimination. This leads to a cell-centered positive-definite system for pressure and displacement at each time step. We perform error analysis for the semidiscrete and fully discrete formulations, establishing first order convergence for all variables in their natural norms. The numerical tests confirm the theoretical convergence rates and illustrate the locking-free property of the method.

42 ENGINEERING↗

DOE OSTI2024

Respecting causality for training physics-informed neural networks

While the popularity of physics-informed neural networks (PINNs) is steadily rising, to this date PINNs have not been successful in simulating dynamical systems whose solution exhibits multi-scale, chaotic or turbulent behavior. In this work we attribute this shortcoming to the inability of existing PINNs formulations to respect the spatio-temporal causal structure that is inherent to the evolution of physical systems. We argue that this is a fundamental limitation and a key source of error that can ultimately steer PINN models to converge towards erroneous solutions. We address this pathology by proposing a simple re-formulation of PINNs loss functions that can explicitly account for physical causality during model training. We demonstrate that this simple modification alone is enough to introduce significant accuracy improvements, as well as a practical quantitative mechanism for assessing the convergence of a PINNs model. Here we provide state-of-the-art numerical results across a series of benchmarks for which existing PINNs formulations fail, including the chaotic Lorenz system, the Kuramoto–Sivashinsky equation in the chaotic regime, and the Navier–Stokes equations. To the best of our knowledge, this is the first time that PINNs have been successful in simulating such systems, introducing new opportunities for their applicability to problems of industrial complexity.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Hutchinson Trace Estimation for high-dimensional and high-order Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) have proven effective in solving partial differential equations (PDEs), especially when some data are available by seamlessly blending data and physics. However, extending PINNs to high-dimensional and even high-order PDEs encounters significant challenges due to the computational cost associated with automatic differentiation in the residual loss function calculation. Herein, we address the limitations of PINNs in handling high-dimensional and high-order PDEs by introducing the Hutchinson Trace Estimation (HTE) method. Starting with the second-order high-dimensional PDEs, which are ubiquitous in scientific computing, HTE is applied to transform the calculation of the entire Hessian matrix into a Hessian vector product (HVP). This approach not only alleviates the computational bottleneck via Taylor-mode automatic differentiation but also significantly reduces memory consumption from the Hessian matrix to an HVP’s scalar output. We further showcase HTE’s convergence to the original PINN loss and its unbiased behavior under specific conditions. Comparisons with the Stochastic Dimension Gradient Descent (SDGD) highlight the distinct advantages of HTE, particularly in scenarios with significant variability and variance among dimensions. We further extend the application of HTE to higher-order and higher-dimensional PDEs, specifically addressing the biharmonic equation. By employing tensor-vector products (TVP), HTE efficiently computes the colossal tensor associated with the fourth-order high-dimensional biharmonic equation, saving memory and enabling rapid computation. The effectiveness of HTE is illustrated through experimental setups, demonstrating comparable convergence rates with SDGD under memory and speed constraints. Additionally, HTE proves valuable in accelerating the Gradient-Enhanced PINN (gPINN) version as well as the Biharmonic equation. Overall, HTE opens up a new capability in scientific machine learning for tackling high-order and high-dimensional PDEs.

Curse of dimensionality↗

DOE OSTI2024

Tackling the curse of dimensionality in fractional and tempered fractional PDEs with physics-informed neural networks

Fractional and tempered fractional partial differential equations (PDEs) are effective models of long-range interactions, anomalous diffusion, and non-local effects. Traditional numerical methods for these problems are mesh-based, thus struggling with the curse of dimensionality (CoD). Physics-informed neural networks (PINNs) offer a promising solution due to their universal approximation, generalization ability, and mesh-free training. In principle, Monte Carlo fractional PINN (MC-fPINN) estimates fractional derivatives using Monte Carlo methods and thus could lift CoD. However, this may cause significant variance and errors, hence affecting convergence; in addition, MC-fPINN is sensitive to hyperparameters. In general, numerical methods and specifically PINNs for tempered fractional PDEs are under-developed. Herein, we extend MC-fPINN to tempered fractional PDEs to address these issues, resulting in the Monte Carlo tempered fractional PINN (MC-tfPINN). To reduce possible high variance and errors from Monte Carlo sampling, we replace the one-dimensional (1D) Monte Carlo with 1D Gaussian quadrature, applicable to both MC-fPINN and MC-tfPINN. We validate our methods on various forward and inverse problems of fractional and tempered fractional PDEs, scaling up to 100,000 dimensions. Our improved MC-fPINN/MC-tfPINN using quadrature consistently outperforms the original versions in accuracy and convergence speed in very high dimensions.

42 ENGINEERING↗

DOE OSTI2024

Pressure stability in explicitly coupled simulations of poromechanics with application to CO 2 sequestration

We study in detail the pressure stabilizing effects of the non-iterated fixed-stress splitting in poromechanical problems which are nearly undrained and incompressible. When applied in conjunction with a spatial discretization which does not satisfy the discrete inf–sup condition, namely a mixed piecewise linear–piecewise constant spatial discretization, the explicit fixed-stress scheme can have a pressure stabilizing effect in transient problems. This effect disappears, however, upon time step refinement or the attainment of steady state. The interpretation of the scheme as an Augmented Lagrangian method similar to Uzawa iteration for incompressible flow helps explain these results. Moreover, due to the slowly evolving solution within undrained seal regions, we show that the explicit fixed-stress scheme requires very large time steps to reveal its pressure stabilizing effect in examples of geologic CO 2 sequestration. We note that large time steps can result in large errors in drained regions, such as the aquifer or reservoir regions of these examples, and can prevent convergence of nonlinear solvers in the case of multiphase flows, which can make the explicit scheme an unreliable source of pressure stabilization. We conclude by demonstrating that pressure jump stabilization is as effective in the explicit fixed-stress setting as in the fully implicit setting for undrained problems, while maintaining the stability and convergence of the fixed-stress split for drained problems.

58 GEOSCIENCES↗

DOE OSTI2025

Optimizing the optimizer for physics-informed neural networks and Kolmogorov-Arnold networks

Physics-Informed Neural Networks (PINNs) have revolutionized the computation of PDE solutions by integrating partial differential equations (PDEs) into the neural network’s training process as soft constraints, becoming an important component of the scientific machine learning (SciML) ecosystem. More recently, physics-informed Kolmogorv-Arnold networks (PIKANs) have also shown to be effective and comparable in accuracy with PINNs. In their current implementation, both PINNs and PIKANs are mainly optimized using first-order methods like Adam, as well as quasi-Newton methods such as BFGS and its low-memory variant, L-BFGS. However, these optimizers often struggle with highly nonlinear and non-convex loss landscapes, leading to challenges such as slow convergence, local minima entrapment, and (non)degenerate saddle points. In this study, we investigate the performance of Self- Scaled BFGS (SSBFGS), Self-Scaled Broyden (SSBroyden) methods and other advanced quasi-Newton schemes, including BFGS and L-BFGS with different line search strategies. These methods dynamically rescale updates based on historical gradient information, thus enhancing training efficiency and accuracy. We systematically compare these optimizers – using both PINNs and PIKANs – on key challenging PDEs, including the Burgers, Allen-Cahn, Kuramoto-Sivashinsky, Ginzburg-Landau, and Stokes equations. Additionally, we evaluate the performance of SSBFGS and SSBroyden for Deep Operator Network (DeepONet) architectures, demonstrating their effectiveness for data-driven operator learning. Our findings provide state-of-the-art results with orders-of-magnitude accuracy improvements without the use of adaptive weights or any other enhancements typically employed in PINNs. More broadly, our work reveal insights into the effectiveness of quasi-Newton optimization strategies in significantly improving the convergence and accurate generalization of PINNs and PIKANs.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Saddle point search with dynamic active volume

Sampling potential energy surface (PES) is critical for many problems in materials science, chemistry, physics, and biology and requires highly efficient saddle point searches (SPS). In the study, we introduce the concept of dynamic active volume (DAV) in addition to the active volume in self-evolving atomistic kinetic Monte Carlo (SEAKMC). The DAV method has further reduced the dimensionality of the PES at the elevation stage of a SPS. At the subsequent converging stage, the dynamic boundary in DAV is lifted to allow the system to converge to the right location of a saddle point. Coupled with the dimer method, the DAV method not only significantly reduces the time cost for a given search attempt, but also dramatically increases the probability of finding relevant saddle points for PES sampling. Further, a Python software package within the framework SEAKMC (SEAKMC_py) with the DAV method has been developed.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Physics-guided neural networks with engineering domain knowledge for hybrid process modeling

As neural networks are more frequently used to solve problems in science and engineering, the methods used to incorporate scientific knowledge into these networks are becoming increasingly complex. Here, this work breaks down these complicated techniques into a set of basic strategies which can easily be applied to diverse situations. Several novel neural networks are built using the categories laid out in this work. These networks are tested on simulated data from a continuous stirred tank reactor (CSTR) model to evaluate the advantages provided by each network. The three points demonstrated in this work are: (1) architectural hybrid models can speed up convergence and reduce the amount of data necessary to train a model; (2) adding a physics-guided loss function can improve model generalization and make models more physically consistent; (3) using physics-guided initialization and transfer learning improves accuracy and speeds up convergence, but can harm generalizability if used incorrectly.

42 ENGINEERING↗

DOE OSTI2025

Applying a Compact Porous Media Model to Numerically Derive Resistance Coefficients for Lattice Structures

Additive Manufacturing allows for exploring various geometries to achieve specific engineering criteria. Lattices are one geometry with unique properties, including being periodically repeating structures which allow flow through them to be represented as a porous media according to Darcy-Forchheimer equations. These equation’s coefficients are generally experimentally derived, but this work demonstrates the ability to numerically derive them with CFD. Simulations were performed using three-dimensional stead state Reynolds-averaged Navier-Stokes with a k-ω Shear Stress Transport turbulence model using Ansys Fluent. Three lattice geometries were investigated and drag coefficients were derived. The method was validated against externally published data for similar geometries demonstrating strong agreement, and grid convergence for all simulations was calculated with a Grid Convergence Index method. Wall roughness is demonstrated to have a non-negligible impact on results and roughness values are considered for the primary focus Octahedral geometry where both smooth wall and rough wall coefficients were derived. The porosity coefficients for the Octahedral geometry at 1.0 [m/s] were found to be 2.89×10 6 and 2.90×10 6 [1/(Pa*m*s)] for the permeability coefficients, 6.37×10 1 and 5.44×10 1 [m 2 /kg] for the inertial resistance coefficients, and with a max pressure drop of 5116.7 [Pa] and 4429.5 [Pa] for the smooth walls and rough walls, respectively. The derived numerical method enables rapid exploration and optimization of new lattice designs for diverse engineering applications.

42 ENGINEERING↗

DOE OSTI2021

A multi-center quadrature scheme for the molecular continuum

We report a common way to evaluate electronic integrals for polyatomic molecules is to use Becke’s partitioning scheme (Becke and Chem, 1988) in conjunction with overlapping grids centered at each atomic site. The Becke scheme was designed for integrands that fall off rapidly at large distances, such as those approximating bound electronic states. When applied to states in the electronic continuum, however, Becke scheme exhibits slow convergence and it is highly redundant. Here, we present a modified version of Becke scheme that is applicable to functions of the electronic continuum, such as those involved in molecular photoionization and electron–molecule scattering, and which ensures convergence and efficiency comparable to those realized in the calculation of bound states. In this modified scheme, the atomic weights already present in Becke’s partition are smoothly switched off within a range of few bond lengths from their respective nuclei, and complemented by an asymptotically unitary weight. The atomic integrals are evaluated on small spherical grids, centered on each atom, with size commensurate to the support of the corresponding atomic weight. The residual integral of the interstitial and long-range region is evaluated with a central master grid. The accuracy of the method is demonstrated by evaluating integrals involving integrands containing Gaussian Type Orbitals and Yukawa potentials, on the atomic sites, as well as spherical Bessel functions centered on the master grid. These functions are representative of those encountered in realistic electron-scattering and photoionization calculations in polyatomic molecules.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2023

Verification and validation of the open-source plasma fluid code: Zapdos

Zapdos is an open-source finite element plasma fluid solver based on the MOOSE multiphysics framework. This paper outlines Zapdos verification, benchmarking, and validation efforts for 1D and 2D RF capacitively coupled plasma discharge models for mid-range pressures (0.1 - 1 Torr). The verification process involved using the method of manufactured solutions to assess Zapdos spatial and temporal error convergence. L2 errors ranged from 10 -2 to 10 -4 , while the convergence’s slope were in agreement with the predicted slopes for the tested variable and time integration orders. The benchmarking process involved comparisons to previously results from the validated finite element code, LSODI. These works included 1D and 2D simulations for a range of plasma parameters (densities, temperatures, voltage, etc.). For the 1D cases, Zapdos and LSODI results were in very good agreement. In the 2D cases, variable behaviors matched, with slight discrepancies in peak values. The validation process involved comparisons to experimental works including electron density measurements by microwave interferometry and metastable density measurements by planar laser-induced fluorescence imaging. Results shown reasonable agreement at higher pressure, with results starting to diverge at low pressures. Furthermore, probable causes for this diverges are the limitation of the fluid assumption for plasmas at low pressure, or the need for more robust boundary conditions. Overall, Zapdos shown reasonable results for the verification, benchmarking, and validation efforts, and Zapdos can be downloaded at https://github.com/shannon-lab/zapdos.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2026

Fast solvers for tokamak fluid models with PETSc

Multigrid (MG) is widely recognized as a highly effective solver for the model problem, the Laplacian, but textbook MG fails on most problems of interest. MG methods have been applied to complex, real-world applications with careful consideration of the physical model and discretization. In this work we develop the first step in applying MG methods to science and engineering relevant magnetohydrodynamics (MHD) tokamak models in the M3D-C1 (https://m3dc1.pppl.gov) fusion energy science code. The semi-implicit time integrator in M3D-C1 is composed of many linear solves. The implicit advance of the momentum equation is the most challenging and is the focus of this work. The current production solver in M3D-C1 is a block Jacobi (BJ) preconditioner within a Krylov solver, where blocks group degrees of freedom on planes of constant toroidal coordinate. BJ convergence degrades as the number of planes increases due to the spectral properties of the matrix preconditioned with BJ. The partially magnetic field-aligned, regular toroidal grid structure in M3D-C1 is amenable to semi-coarsening geometric MG in the toroidal direction. This paper develops such a solver and demonstrates competitive performance on a runaway electron model of a SPARC (https://cfs.energy/technology/sparc) disruption, and superior robustness on a stellarator model on which the BJ solver fails to converge.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗