SEARCH · Engineering Papers

Results for “Quasi-Newton method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

66 records · Page 4

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Projective Integral Updates for High-Dimensional Variational Inference

Variational inference is an approximation framework for Bayesian inference that seeks to improve quantified uncertainty in predictions by optimizing a simplified distribution over parameters to stand in for the full posterior. Capturing model variations that remain consistent with training data enables more robust predictions by reducing parameter sensitivity. This work introduces a fixed-point optimization for variational inference that is applicable when every feasible log density can be expressed as a linear combination of functions from a given basis. In such cases, the optimizer becomes a fixed-point of projective integral updates. When the basis spans univariate quadratics in each parameter, the feasible distributions are Gaussian mean-fields and the projective integral updates yield quasi-Newton variational Bayes (QNVB). Other bases and updates are also possible. Since these updates require high-dimensional integration, this work begins by proposing an efficient quasirandom sequence of quadratures for mean-field distributions. Each iterate of the sequence contains two evaluation points that combine to correctly integrate all univariate quadratic functions and, if the mean-field factors are symmetric, all univariate cubics. More importantly, averaging results over short subsequences achieves periodic exactness on a much larger space of multivariate polynomials of quadratic total degree. The corresponding variational updates require four loss evaluations with standard (not second-order) backpropagation to eliminate error terms from over half of all multivariate quadratic basis functions. Furthermore, this integration technique is motivated by first proposing stochastic blocked mean-field quadratures, which may be useful in other contexts. A PyTorch implementation of QNVB allows for better control over model uncertainty during training than competing methods. Experiments demonstrate superior generalizability for multiple learning problems and architectures.

Gaussian mean-field↗

NASA NTRS2003

Optimized Shapes of Ocsillating Resonators for Generating High-Amplitude Pressure Waves

It is well known that the resonator geometry strongly influences the resonant frequencies of an acoustical resonator and the generated nonlinear standing pressure waveform. Maximizing the ratio of maximum to minimum gas pressure at an end of an oscillating resonator by optimizing the cavity contour is investigated numerically. A quasi-Newton type scheme is used to find optimized axisymmetric resonator shapes to achieve the maximum pressure compression ratio. The acoustical field is solved using a one-dimensional model, and the resonance frequency shift and hysteresis effects are obtained through an automation scheme based on continuation methods. Results are presented from optimizing cone, horn-cone, and cosine resonator geometries. Significant performance improvement is found in the optimized shapes over others previously published. Different optimized shapes are found when starting with different initial guesses, indicating multiple local extrema. The numerical model is validated by comparing with the experimental results of a horn-cone shaped resonator.

Li, Xiao-Fan↗

NASA NTRS1991

Aerodynamic shape optimization of arbitrary hypersonic vehicles

A new method was developed to optimize, in terms of aerodynamic wave drag minimization, arbitrary (nonaxisymmetric) hypersonic vehicles in modified Newtonian flow, while maintaining the initial volume and length of the vehicle. This new method uses either a surface fitted Fourier series to represent the vehicle's geometry or an independent point motion algorithm. In either case, the coefficients of the Fourier series or the spatial locations of the points defining each cross section were varied and a numerical optimization algorithm based on a quasi-Newton gradient search concept was used to determine the new optimal configuration. Results indicate a significant decrease in aerodynamic wave drag for simple and complex geometries at relatively low CPU costs. In the case of a cone, the results agreed well with known analytical optimum ogive shapes. The procedure is capable of accepting more complex flow field analysis codes.

Dulikravich, George S.↗

NASA NTRS1993

An efficient nonlinear relaxation technique for the three-dimensional, Reynolds-averaged Navier-Stokes equations

An efficient implicit method for the computation of steady, three-dimensional, compressible Navier-Stokes flowfields is presented. A nonlinear iteration strategy based on planar Gauss-Seidel sweeps is used to drive the solution toward a steady state, with approximate factorization errors within a crossflow plane reduced by the application of a quasi-Newton technique. A hybrid discretization approach is employed, with flux-vector splitting utilized in the streamwise direction and central differences with artificial dissipation used for the transverse fluxes. Convergence histories and comparisons with experimental data are presented for several 3-D shock-boundary layer interactions. Both laminar and turbulent cases are considered, with turbulent closure provided by a modification of the Baldwin-Barth one-equation model. For the problems considered (175,000-325,000 mesh points), the algorithm provides steady-state convergence in 900-2000 CPU seconds on a single processor of a Cray Y-MP.

Edwards, Jack R.↗

NASA NTRS2003

Nonlinear Resonant Oscillations of Gas in Optimized Acoustical Resonators and the Effect of Central Blockage

Optimizing resonator shapes for maximizing the ratio of maximum to minimum gas pressure at an end of the resonator is investigated numerically. It is well known that the resonant frequencies and the nonlinear standing waveform in an acoustical resonator strongly depend on the resonator geometry. A quasi-Newton type scheme was used to find optimized axisymmetric resonator shapes achieving the maximum pressure compression ratio with an acceleration of constant amplitude. The acoustical field was solved using a one-dimensional model, and the resonance frequency shift and hysteresis effects were obtained through an automation scheme based on continuation method. Results are presented for optimizing three types of geometry: a cone, a horn-cone and a half cosine-shape. For each type, different optimized shapes were found when starting with different initial guesses. Further, the one-dimensional model was modified to study the effect of an axisymmetric central blockage on the nonlinear standing wave.

Li, Xiaofan↗

NASA NTRS2003

Nonlinear Resonant Oscillations of Gas in Optimized Acoustical Resonators and the Effect of Central Blockage

Optimizing resonator shapes for maximizing the ratio of maximum to minimum gas pressure at an end of the resonator is investigated numerically. It is well known that the resonant frequencies and the nonlinear standing waveform in an acoustical resonator strongly depend on the resonator geometry. A quasi-Newton type scheme was used to find optimized axisymmetric resonator shapes achieving the maximum pressure compression ratio with an acceleration of constant amplitude. The acoustical field was solved using a one-dimensional model, and the resonance frequency shift and hysteresis effects were obtained through an automation scheme based on continuation method. Results are presented for optimizing three types of geometry: a cone, a horn-cone and a half cosine- shape. For each type, different optimized shapes were found when starting with different initial guesses. Further, the one-dimensional model was modified to study the effect of an axisymmetric central blockage on the nonlinear standing wave.

Li, Xiao-Fan↗

NASA NTRS2002

Human motion planning based on recursive dynamics and optimal control techniques

This paper presents an efficient optimal control and recursive dynamics-based computer animation system for simulating and controlling the motion of articulated figures. A quasi-Newton nonlinear programming technique (super-linear convergence) is implemented to solve minimum torque-based human motion-planning problems. The explicit analytical gradients needed in the dynamics are derived using a matrix exponential formulation and Lie algebra. Cubic spline functions are used to make the search space for an optimal solution finite. Based on our formulations, our method is well conditioned and robust, in addition to being computationally efficient. To better illustrate the efficiency of our method, we present results of natural looking and physically correct human motions for a variety of human motion tasks involving open and closed loop kinematic chains.

Non-NASA Center↗

DOE OSTI2020

Global adjoint tomography—model GLAD-M25

SUMMARY Building on global adjoint tomography model GLAD-M15, we present transversely isotropic global model GLAD-M25, which is the result of 10 quasi-Newton tomographic iterations with an earthquake database consisting of 1480 events in the magnitude range 5.5 ≤ Mw ≤ 7.2, an almost sixfold increase over the first-generation model. We calculated fully 3-D synthetic seismograms with a shortest period of 17 s based on a GPU-accelerated spectral-element wave propagation solver which accommodates effects due to 3-D anelastic crust and mantle structure, topography and bathymetry, the ocean load, ellipticity, rotation and self-gravitation. We used an adjoint-state method to calculate Fréchet derivatives in 3-D anelastic Earth models facilitated by a parsimonious storage algorithm. The simulations were performed on the Cray XK7 ‘Titan’ and the IBM Power 9 ‘Summit’ at the Oak Ridge Leadership Computing Facility. We quantitatively evaluated GLAD-M25 by assessing misfit reductions and traveltime anomaly histograms in 12 measurement categories. We performed similar assessments for a held-out data set consisting of 360 earthquakes, with results comparable to the actual inversion. We highlight the new model for a variety of plumes and subduction zones.

58 GEOSCIENCES↗

DOE OSTI2026

RegularizedOptimization.jl: A Julia framework for regularized and nonsmooth optimization

RegularizedOptimization.jl is a Julia package that implements families of quadratic regularization and trust-region methods for solving the nonsmooth optimization problem $^{\textrm{minimize}}_{𝑥∈ℝ^𝑛}$ 𝑓(𝑥) + ℎ(𝑥) subject to 𝑐(𝑥) = 0, (1) where 𝑓 ∶ ℝ 𝑛 → ℝ and 𝑐 ∶ ℝ 𝑛 → ℝ 𝑚 are continuously differentiable, and ℎ ∶ ℝ 𝑛 → ℝ∪{+∞} is lower semi-continuous. The nonsmooth objective ℎ can be a regularizer, such as a sparsity inducing penalty, model simple constraints, such as 𝑥 belonging to a simple convex set, or can be a combination of both. All 𝑓, ℎ, and 𝑐 can be nonconvex. RegularizedOptimization.jl provides a modular and extensible framework for solving (1), and developing novel solvers. Currently, the following solvers are implemented: • Trust-region solvers TR and TRDH (Aravkin et al., 2022; Leconte & Orban, 2025) • Quadratic regularization solvers R2, R2DH and R2N (Aravkin et al., 2022; Diouane, Habiboullah, et al., 2024) • Levenberg-Marquardt solvers LM and LMTR (Aravkin et al., 2024) used when 𝑓 is a least-squares residual. • Augmented Lagrangian solver AL (De Marchi et al., 2023). All solvers rely on first derivatives of 𝑓 and 𝑐, and optionally on their second derivatives in the form of Hessian-vector products. If second derivatives are not available, quasi-Newton approximations can be used. In addition, the proximal mapping of the nonsmooth part ℎ, or adequate models thereof, must be evaluated. At each iteration, a step is computed by solving a subproblem of the form (1) inexactly, in which 𝑓, ℎ, and 𝑐 are replaced with appropriate models around the current iterate. The solvers R2, R2DH, and TRDH are particularly well suited to solve the subproblems, though they are general enough to solve (1). All solvers are allocation-free, so re-solves incur no additional allocations. To illustrate our claim of extensibility, a first version of the AL solver was implemented by an external contributor. Furthermore, a nonsmooth penalty approach, described in Diouane, Gollier, et al. (2024), is currently being developed, that relies on the library to efficiently solve the subproblems.

Gollier, Maxence [Polytechnique Montréal, QC (Cana↗

DOE OSTI2025

A randomized sketching trust-region secant method for low-memory dynamic optimization

The numerical solution of dynamic optimization problems is often limited by the memory required to store the state trajectory, which is used to evaluate the objective function and its derivatives. Recently, [R. Muthukumar et al., SIAM Journal on Optimization 31(2), pp. 1242–1275 (2021)] introduced a trust-region method for dynamic optimization that employs randomized sketching to compress the state trajectory, resulting in inexact derivative computations. By adaptively learning the sketch rank, the trust-region algorithm achieves rigorous convergence guarantees. Here, we extend this approach to use secant Hessian approximations. Due to the randomness introduced by the sketch, the traditional secant update formulae can produce poor Hessian approximations. In particular, the difference of two gradients, computed from two different sketches, may be inconsistent. To overcome this, we employ a sketched approximation of the Hessian application, in lieu of computing the gradient difference. We numerically demonstrate the improved stability of this approach on an example from PDE-constrained optimization.

dynamic optimization↗

DOE OSTI2020

Extended Lagrangian Born–Oppenheimer molecular dynamics using a Krylov subspace approximation

It is shown how the electronic equations of motion in extended Lagrangian Born–Oppenheimer molecular dynamics simulations can be integrated using low-rank approximations of the inverse Jacobian kernel. This kernel determines the metric tensor in the harmonic oscillator extension of the Lagrangian that drives the evolution of the electronic degrees of freedom. The proposed kernel approximation is derived from a pseudoinverse of a low-rank estimate of the Jacobian, which is expressed in terms of a generalized set of directional derivatives with directions that are given from a Krylov subspace approximation. The approach allows a tunable and adaptive approximation that can take advantage of efficient preconditioning techniques. The proposed kernel approximation for the integration of the electronic equations of motion makes it possible to apply extended Lagrangian first-principles molecular dynamics simulations to a broader range of problems, including reactive chemical systems with numerically sensitive and unsteady charge solutions. This can be achieved without requiring exact full calculations of the inverse Jacobian kernel in each time step or relying on iterative non-linear self-consistent field optimization of the electronic ground state prior to the force evaluations as in regular direct Born–Oppenheimer molecular dynamics. We note the low-rank approximation of the Jacobian is directly related to Broyden’s class of quasi-Newton algorithms and Jacobian-free Newton–Krylov methods and provides a complementary formulation for the solution of nonlinear systems of equations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2020

Parameter Sensitivity Analysis of the SparTen High Performance Sparse Tensor Decomposition Software (Extended Analysis)

Tensor decomposition models play an increasingly important role in modern data science applications. One problem of particular interest is fitting a low-rank Canonical Polyadic (CP) tensor decomposition model when the tensor has sparse structure and the tensor elements are nonnegative count data. SparTen is a high-performance C++ library which computes a low-rank decomposition using different solvers: a first-order quasi-Newton or a second-order damped Newton method, along with the appropriate choice of runtime parameters. Since default parameters in SparTen are tuned to experimental results in prior published work on a single real-world dataset conducted using MATLAB implementations of these methods, it remains unclear if the parameter defaults in SparTen are appropriate for general tensor data. Furthermore, it is unknown how sensitive algorithm convergence is to changes in the input parameter values. This report addresses these unresolved issues with large-scale experimentation on three benchmark tensor data sets. Experiments were conducted on several different CPU architectures and replicated with many initial states to establish generalized profiles of algorithm convergence behavior.

97 MATHEMATICS AND COMPUTING↗