Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “nonlinear conjugate gradient”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Accelerated Sparse Recovery via Gradient Descent with Nonlinear Conjugate Gradient Momentum

This paper applies an idea of adaptive momentum for the nonlinear conjugate gradient to accelerate optimization problems in sparse recovery. Specifically, we consider two types of minimization problems: a (single) differentiable function and the sum of a non-smooth function and a differentiable function. In the first case, we adopt a fixed step size to avoid the traditional line search and establish the convergence analysis of the proposed algorithm for a quadratic problem. This acceleration is further incorporated with an operator splitting technique to deal with the non-smooth function in the second case. As a result, we use the convex ι 1 and the nonconvex ι 1 – ι 2 functionals as two case studies to demonstrate the efficiency of the proposed approaches over traditional methods.

97 MATHEMATICS AND COMPUTING↗

Improving Deep Neural Networks’ Training for Image Classification With Nonlinear Conjugate Gradient-Style Adaptive Momentum

Momentum is crucial in stochastic gradient-based optimization algorithms for accelerating or improving training deep neural networks (DNNs). In deep learning practice, the momentum is usually weighted by a well-calibrated constant. However, tuning the hyperparameter for momentum can be a significant computational burden. In this article, we propose a novel adaptive momentum for improving DNNs training; this adaptive momentum, with no momentum-related hyperparame- ter required, is motivated by the nonlinear conjugate gradient (NCG) method. Stochastic gradient descent (SGD) with this new adaptive momentum eliminates the need for the momentum hyperparameter calibration, allows using a significantly larger learning rate, accelerates DNN training, and improves the final accuracy and robustness of the trained DNNs. For example, SGD with this adaptive momentum reduces classification errors for training ResNet110 for CIFAR10 and CIFAR100 from 5.25% to 4.64% and 23.75% to 20.03%, respectively. Furthermore, SGD, with the new adaptive momentum, also benefits adversarial training and, hence, improves the adversarial robustness of the trained DNNs.

97 MATHEMATICS AND COMPUTING↗

A nonlinearly preconditioned conjugate gradient algorithm for rank‐ R canonical tensor approximation

Summary Alternating least squares (ALS) is often considered the workhorse algorithm for computing the rank‐ R canonical tensor approximation, but for certain problems, its convergence can be very slow. The nonlinear conjugate gradient (NCG) method was recently proposed as an alternative to ALS, but the results indicated that NCG is usually not faster than ALS. To improve the convergence speed of NCG, we consider a nonlinearly preconditioned NCG (PNCG) algorithm for computing the rank‐ R canonical tensor decomposition. Our approach uses ALS as a nonlinear preconditioner in the NCG algorithm. Alternatively, NCG can be viewed as an acceleration process for ALS. We demonstrate numerically that the convergence acceleration mechanism in PNCG often leads to important pay‐offs for difficult tensor decomposition problems, with convergence that is significantly faster and more robust than for the stand‐alone NCG or ALS algorithms. We consider several approaches for incorporating the nonlinear preconditioner into the NCG algorithm that have been described in the literature previously and have met with success in certain application areas. However, it appears that the nonlinearly PNCG approach has received relatively little attention in the broader community and remains underexplored both theoretically and experimentally. Thus, this paper serves several additional functions, by providing in one place a concise overview of several PNCG variants and their properties that have only been described in a few places scattered throughout the literature, by systematically comparing the performance of these PNCG variants for the tensor decomposition problem, and by drawing further attention to the usefulness of nonlinearly PNCG as a general tool. In addition, we briefly discuss the convergence of the PNCG algorithm. In particular, we obtain a new convergence result for one of the PNCG variants under suitable conditions, building on known convergence results for non‐preconditioned NCG. Copyright © 2014 John Wiley & Sons, Ltd.

Sterck, Hans De↗

Efficient proximal subproblem solvers for a nonsmooth trust-region method

In [R. J. Baraldi and D. P. Kouri, Mathematical Programming, (2022), pp. 1-40], we introduced an inexact trust-region algorithm for minimizing the sum of a smooth nonconvex and nonsmooth convex function. The principle expense of this method is in computing a trial iterate that satisfies the so-called fraction of Cauchy decrease condition—a bound that ensures the trial iterate produces sufficient decrease of the subproblem model. In this paper, we expound on various proximal trust-region subproblem solvers that generalize traditional trust-region methods for smooth unconstrained and convex-constrained problems. We introduce a simplified spectral proximal gradient solver, a truncated nonlinear conjugate gradient solver, and a dogleg method. Finally, we compare algorithm performance on examples from data science and PDE-constrained optimization.

97 MATHEMATICS AND COMPUTING↗

Poblano v1.0 : a Matlab toolbox for gradient-based optimization.

We present Poblano v1.0, a Matlab toolbox for solving gradient-based unconstrained optimization problems. Poblano implements three optimization methods (nonlinear conjugate gradients, limited-memory BFGS, and truncated Newton) that require only first order derivative information. In this paper, we describe the Poblano methods, provide numerous examples on how to use Poblano, and present results of Poblano used in solving problems from a standard test collection of unconstrained optimization problems.

99 GENERAL AND MISCELLANEOUS↗

A New Capability of E4D For 3D Parallel Joint Inversion of DC Resistivity And Traveltime Data on Unstructured Mesh

A major challenge in interpreting geophysical data is how to derive consistent three-dimensional (3D) earth models of different physical properties from spatially and temporally limited measurements. Joint inversion with cross-gradient constraints is an approach to find such models by imposing structural similarities between different physical parameters. We have developed a parallel distributed-memory joint inversion code for direct-current (DC) resistivity and traveltime data using the cross-gradient constraint on unstructured mesh. The code utilizes existing E4D framework for parallel forward simulation, distributed storage and computation of the Jacobian matrix of forward operator, and parallel execution of matrix-vector multiplication during inversion. Besides, the joint inversion is solved by nonlinear conjugate gradient algorithm parallelized for DC resistivity and traveltime data. The joint inversion capability of E4D was tested using synthetic data from cross-borehole DC resistivity and traveltime data. The results indicate that the shape and size of the anomalies from the joint inversion are more reliable than those from separate inversions.

58 GEOSCIENCES↗

Optipack

Used to solve basic unconstrained optimization problems that can produce smooth accurate gradients. These are standard published unconstrained optimization such as are described in the book Nocedal and Wright Numerical Optimization: 2nd edition, 2006. This software is generic to any mathematical optimization problem where smooth derivatives exist. There is no specific connection or mention whatsoever to any specific application, period. You cannot find more general mathematical software.OptiPack contains some simple optimization algorithms built on Thyra. Currently, only a few different nonlinear conjugate gradient methods are implemented.

Bartlett, Roscoe↗

Optipack

Used to solve basic unconstrained optimization problems that can produce smooth accurate gradients. These are standard published unconstrained optimization such as are described in the book Nocedal and Wright Numerical Optimization: 2nd edition, 2006. This software is generic to any mathematical optimization problem where smooth derivatives exist. There is no specific connection or mention whatsoever to any specific application, period. You cannot find more general mathematical software.OptiPack contains some simple optimization algorithms built on Thyra. Currently, only a few different nonlinear conjugate gradient methods are implemented.

Bartlett, Roscoe↗

Charge order–superfluidity transition in a two-dimensional system of hard-core bosons and emerging domain structures

We have used high-performance parallel computations by NVIDIA graphics cards applying the method of nonlinear conjugate gradients and Monte Carlo method to observe directly the developing ground state configuration of a two-dimensional hard-core boson system with decrease in temperature, and its evolution with deviation from a half-filling. This has allowed us to explore unconventional features of a charge order—superfluidity phase transition, specifically, formation of an irregular domain structure, emergence of a filamentary superfluid structure that condenses within of the charge-ordered phase domain antiphase boundaries, and formation and evolution of various topological structures.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Development and Clinical Evaluation of a Three-Dimensional Cone-Beam Computed Tomography Estimation Method Using a Deformation Field Map

Purpose: To develop a three-dimensional (3D) cone-beam computed tomography (CBCT) estimation method using a deformation field map, and to evaluate and optimize the efficiency and accuracy of the method for use in the clinical setting. Methods and Materials: We propose a method to estimate patient CBCT images using prior information and a deformation model. Patients' previous CBCT data are used as the prior information, and the new CBCT volume to be estimated is considered as a deformation of the prior image volume. The deformation field map is solved by minimizing deformation energy and maintaining new projection data fidelity using a nonlinear conjugate gradient method. This method was implemented in 3D form using hardware acceleration and multi-resolution scheme, and it was evaluated for different scan angles, projection numbers, and scan directions using liver, lung, and prostate cancer patient data. The accuracy of the estimation was evaluated by comparing the organ volume difference and the similarity between estimated CBCT and the CBCT reconstructed from fully sampled projections. Results: Results showed that scan direction and number of projections do not have significant effects on the CBCT estimation accuracy. The total scan angle is the dominant factor affecting the accuracy of the CBCT estimation algorithm. Larger scan angles yield better estimation accuracy than smaller scan angles. Lung cancer patient data showed that the estimation error of the 3D lung tumor volume was reduced from 13.3% to 4.3% when the scan angle was increased from 60 Degree-Sign to 360 Degree-Sign using 57 projections. Conclusions: The proposed estimation method is applicable for 3D DTS, 3D CBCT, four-dimensional CBCT, and four-dimensional DTS image estimation. This method has the potential for significantly reducing the imaging dose and improving the image quality by removing the organ distortion artifacts and streak artifacts shown in images reconstructed by the conventional Feldkamp-Davis-Kress (FDK) algorithm.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Improved proton computed tomography by dual modality image reconstruction

Purpose: Proton computed tomography (CT) is a promising image modality for improving the stopping power estimates and dose calculations for particle therapy. However, the finite range of about 33 cm of water of most commercial proton therapy systems limits the sites that can be scanned from a full 360° rotation. In this paper the authors propose a method to overcome the problem using a dual modality reconstruction (DMR) combining the proton data with a cone-beam x-ray prior. Methods: A Catphan 600 phantom was scanned using a cone beam x-ray CT scanner. A digital replica of the phantom was created in the Monte Carlo code Geant4 and a 360° proton CT scan was simulated, storing the entrance and exit position and momentum vector of every proton. Proton CT images were reconstructed using a varying number of angles from the scan. The proton CT images were reconstructed using a constrained nonlinear conjugate gradient algorithm, minimizing total variation and the x-ray CT prior while remaining consistent with the proton projection data. The proton histories were reconstructed along curved cubic-spline paths. Results: The spatial resolution of the cone beam CT prior was retained for the fully sampled case and the 90° interval case, with the MTF = 0.5 (modulation transfer function) ranging from 5.22 to 5.65 linepairs/cm. In the 45° interval case, the MTF = 0.5 dropped to 3.91 linepairs/cm For the fully sampled DMR, the maximal root mean square (RMS) error was 0.006 in units of relative stopping power. For the limited angle cases the maximal RMS error was 0.18, an almost five-fold improvement over the cone beam CT estimate. Conclusions: Dual modality reconstruction yields the high spatial resolution of cone beam x-ray CT while maintaining the improved stopping power estimation of proton CT. In the case of limited angles, the use of prior image proton CT greatly improves the resolution and stopping power estimate, but does not fully achieve the quality of a 360° proton CT scan.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Three-dimensional induced polarization data inversion for complex resistivity

The conductive and capacitive material properties of the subsurface can be quantified through the frequency-dependent complex resistivity. However, the routine three-dimensional (3D) interpretation of voluminous induced polarization (IP) data sets still poses a challenge due to large computational demands and solution nonuniqueness. We have developed a flexible methodology for 3D (spectral) IP data inversion. Our inversion algorithm is adapted from a frequency-domain electromagnetic (EM) inversion method primarily developed for large-scale hydrocarbon and geothermal energy exploration purposes. The method has proven to be efficient by implementing the nonlinear conjugate gradient method with hierarchical parallelism and by using an optimal finite-difference forward modeling mesh design scheme. The method allows for a large range of survey scales, providing a tool for both exploration and environmental applications. We experimented with an image focusing technique to improve the poor depth resolution of surface data sets with small survey spreads. The algorithm's underlying forward modeling operator properly accounts for EM coupling effects; thus, traditionally used EM coupling correction procedures are not needed. The methodology was applied to both synthetic and field data. We tested the benefit of directly inverting EM coupling contaminated data using a synthetic large-scale exploration data set. Afterward, we further tested the monitoring capability of our method by inverting time-lapse data from an environmental remediation experiment near Rifle, Colorado. Similar trends observed in both our solution and another 2D inversion were in accordance with previous findings about the IP effects due to subsurface microbial activity.

15 GEOTHERMAL ENERGY↗

An inverse problem strategy based on forward model evaluations: Gradient-based optimization without adjoint solves

This study presents a new nonlinear programming formulation for the solution of inverse problems. First, a general inverse problem formulation based on the compliance error functional is presented. The proposed error functional enables the computation of the Lagrange multipliers, and thus the first order derivative information, at the expense of just one model evaluation. Therefore, the calculation of the Lagrange multipliers does not require the solution of the computationally intensive adjoint problem. This leads to significant speedups for large-scale, gradient-based inverse problems.

42 ENGINEERING↗

Comparative Convergence Analysis of Nonlinear AMLI-Cycle Multigrid

The purpose of our paper is to provide a comprehensive convergence analysis of the nonlinear algebraic multilevel iteration (AMLI)-cycle multigrid (MG) method for symmetric positive definite problems. We show that the nonlinear AMLI-cycle MG method is uniformly convergent, based on classical assumptions for approximation and smoothing properties. Furthermore, under only the assumption that the smoother is convergent, we show that the nonlinear AMLI-cycle method is always better (or not worse) than the respective V-cycle MG method. Finally, numerical experiments are presented to illustrate the theoretical results.

97 MATHEMATICS AND COMPUTING↗

Mechanical properties of granular materials: A variational approach to grain-scale simulations

The mechanical properties of cohesionless granular materials are evaluated from grain-scale simulations. A three-dimensional pack of spherical grains is loaded by incremental displacements of its boundaries. The deformation is described as a sequence of equilibrium configurations. Each configuration is characterized by a minimum of the total potential energy. This minimum is computed using a modification of the conjugate gradient algorithm. Our simulations capture the nonlinear, path-dependent behavior of granular materials observed in experiments. Micromechanical analysis provides valuable insight into phenomena such as hysteresis, strain hardening and stress-induced anisotropy. Estimates of the effective bulk modulus, obtained with no adjustment of material parameters, are in agreement with published experimental data. The model is applied to evaluate the effects of hydrate dissociation in marine sediments. Weakening of the sediment is quantified as a reduction in the effective elastic moduli.

54 ENVIRONMENTAL SCIENCES↗

Nonlinear Acceleration Methods for Even-Parity Neutron Transport

Convergence acceleration methods for even-parity transport were developed that have the potential to speed up transport calculations and provide a natural avenue for an implicitly coupled multiphysics code. An investigation was performed into the acceleration properties of the introduction of a nonlinear quasi-diffusion-like tensor in linear and nonlinear solution schemes. Using the tensor reduced matrix as a preconditioner for the conjugate gradients method proves highly efficient and effective. The results for the linear and nonlinear case serve as the basis for further research into the application in a full three-dimensional spherical-harmonics even-parity transport code. Once moved into the nonlinear solution scheme, the implicit coupling of the convergence accelerated transport method into codes for other physics can be done seamlessly, providing an efficient, fully implicitly coupled multiphysics code with high order transport.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Full waveform inversion for time-distance helioseismology

Inferring interior properties of the Sun from photospheric measurements of the seismic wavefield constitutes the helioseismic inverse problem. Deviations in seismic measurements (such as wave travel times) from their fiducial values estimated for a given model of the solar interior imply that the model is inaccurate. Contemporary inversions in local helioseismology assume that properties of the solar interior are linearly related to measured travel-time deviations. It is widely known, however, that this assumption is invalid for sunspots and active regions and is likely for supergranular flows. Here, we introduce nonlinear optimization, executed iteratively, as a means of inverting for the subsurface structure of large-amplitude perturbations. Defining the penalty functional as the L {sub 2} norm of wave travel-time deviations, we compute the total misfit gradient of this functional with respect to the relevant model parameters at each iteration around the corresponding model. The model is successively improved using either steepest descent, conjugate gradient, or the quasi-Newton limited-memory Broyden-Fletcher-Goldfarb-Shanno algorithm. Performing nonlinear iterations requires privileging pixels (such as those in the near field of the scatterer), a practice that is not compliant with the standard assumption of translational invariance. Measurements for these inversions, although similar in principle to those used in time-distance helioseismology, require some retooling. For the sake of simplicity in illustrating the method, we consider a two-dimensional inverse problem with only a sound-speed perturbation.

79 ASTRONOMY AND ASTROPHYSICS↗