Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “gradient methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
NASA NTRS1988

Optimization with artificial neural network systems - A mapping principle and a comparison to gradient based methods

General formulae for mapping optimization problems into systems of ordinary differential equations associated with artificial neural networks are presented. A comparison is made to optimization using gradient-search methods. The performance measure is the settling time from an initial state to a target state. A simple analytical example illustrates a situation where dynamical systems representing artificial neural network methods would settle faster than those representing gradient-search. Settling time was investigated for a more complicated optimization problem using computer simulations. The problem was a simplified version of a problem in medical imaging: determining loci of cerebral activity from electromagnetic measurements at the scalp. The simulations showed that gradient based systems typically settled 50 to 100 times faster than systems based on current neural network optimization methods.

Leong, Harrison Monfook↗
DOE OSTI2022

The Gutzwiller conjugate gradient minimization method for correlated electron systems

In this report we review our recent work on the Gutzwiller conjugate gradient minimization method, an ab initio approach developed for correlated electron systems. The complete formalism has been outlined that allows for a systematic understanding of the method, followed by a discussion of benchmark studies of dimers, one- and two-dimensional single-band Hubbard models. In the end, we present some preliminary results of multi-band Hubbard models and large-basis calculations of F 2 to illustrate our efforts to further reduce the computational complexity.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2021

An adaptive Hessian approximated stochastic gradient MCMC method

Bayesian approaches have been successfully integrated into training deep neural networks. One popular family is stochastic gradient Markov chain Monte Carlo methods (SG-MCMC), which have gained increasing interest due to their ability to handle large datasets and the potential to avoid overfitting. Although standard SG-MCMC methods have shown great performance in a variety of problems, they may be inefficient when the random variables in the target posterior densities have scale differences or are highly correlated. Here, we present an adaptive Hessian approximated stochastic gradient MCMC method to incorporate local geometric information while sampling from the posterior. The idea is to apply stochastic approximation (SA) to sequentially update a preconditioning matrix at each iteration. The preconditioner possesses second-order information and can guide the random walk of a sampler efficiently. Instead of computing and saving the full Hessian of the log posterior, we use limited memory of the samples and their stochastic gradients to approximate the inverse Hessian-vector multiplication in the updating formula. Moreover, by smoothly optimizing the preconditioning matrix via SA, our proposed algorithm can asymptotically converge to the target distribution with a controllable bias under mild conditions. To reduce the training and testing computational burden, we adopt a magnitude-based weight pruning method to enforce the sparsity of the network. Our method is user-friendly and demonstrates better learning results compared to standard SG-MCMC updating rules. The approximation of inverse Hessian alleviates storage and computational complexities for large dimensional models. Numerical experiments are performed on several problems, including sampling from 2D correlated distribution, synthetic regression problems, and learning the numerical solutions of heterogeneous elliptic PDE. The numerical results demonstrate great improvement in both the convergence rate and accuracy.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A Nonlocal-Gradient Descent Method for Inverse Design in Nanophotonics

Local-gradient-based optimization approaches lack nonlocal exploration abilityrequired for escaping from local minima when searching non-convex landscapes.A directional Gaussian smoothing (DGS) approach was recently proposed in [29]and used to define a truly nonlocal gradient, referred to as the DGS gradient, inorder to enable nonlocal exploration in high-dimensional black-box optimization.Promising results show that replacing the traditional local gradient with the nonlocalDGS gradient can significantly improve the performance of gradient-based methodsin optimizing highly multi-modal loss functions. However, the current DGS methodis designed for unbounded and uncontrained optimization problems, making itinapplicable to real-world engineering optimization problems where the tuningparameters are often bounded and the loss function is usually constrained byphysical processes. In this work, we propose to extend to the DGS approachto the constrained inverse design framework in order to find better optima ofmulti-modal loss functions. A series of adaptive strategies for smoothing radiusand learning rate updating are developed to improve the computational efficiencyand robustness. Our methodology is demonstrated by an example of designing ananoscale wavelength demultiplexer, and shows superior performance compared tothe state-of-the-art approaches. By incorporating volume constraints, the optimizeddesign achieves an equivalently high performance but significantly reduces theamount of material usage.

Bi, Sirui↗
DOE OSTI2021

Solid Phase Gradient Alloying Method via ShAPE

Alloying has been used to confer desirable properties to various metallic systems since the bronze age. For example, Ni, Mn, and Cr are alloyed into an Fe matrix to make steel, which has improved strength and corrosion resistance (as displayed in Fig. 1). In addition to the type of alloying elements, the amount of alloying elements added into a matrix is also very critical for a material’s mechanical property, physical property, machinability, and cost. Conventionally, new alloys are discovered by combining alloy precursors and elemental constituents into a mixture and melting them together. The solidified product or ingot represents a usually non-homogeneous combination of the chemistry with a microstructure dictated by the physics of solidification. It is time and energy intensive to optimize the alloying amounts via the casting process and hence, economically not conducive to rapid alloy discovery. In addition, the cast microstructure seen in the as-cast ingot is often not the ideal, nor even desired structure for optimized performance. Cast materials often need further processing such as homogenization, annealing, and deformation work put in through rolling, forging, extruding, etc. to have favorable microstructures and properties. A faster method to discover new alloy combinations and evaluate them in their worked or processed form is needed. Recently, Laser Engineered Net Shaping was applied to fabricate gradient compositional material for alloy designing. However, the mechanical properties of this gradient alloy were poor due to the existence of impurities, oxidation and cracks that are inherent in the melt-solidification process. Because of these limitations, we propose to use a solid-phase process (ShAPE) to create bulk materials (extrudates) that vary in composition from one end of the extruded solid to the other. Various manufacturing methods for alloy design including conventional casting method, laser based combinatorial method and current solid phase gradient alloying method are displayed and compared in Fig. 2. ShAPE machine and a schematic of ShAPE process are displayed in Fig. 3. Starting material is processed by a rotating and plunging die to form an extrudate. We are proposing through this methodology to invent a new, bulk-scale combinatorial technique. With this novel technique, alloying element can be dissolved into the matrix with a continued gradient without bulk melting. Formation of inter-metallics and defects originating from melt processing can be avoided. In addition, the ShAPE process creates the mixed alloy chemistry, and meanwhile subjects the material to severe plastic strain, which induces the favorable “worked” microstructure. Products from the ShAPE process can be tested in hardness directly providing a path to rapid evaluation of mechanical properties. The ability to create graded structures using friction stir processing (FSP) has been demonstrated, however using the ShAPE process we believe will lead to much faster and higher fidelity results. When combined with high throughput screening methods of physical, mechanical and microstructural property characterization, efficiency and accuracy of alloy design can be significantly improved.

36 MATERIALS SCIENCE↗
NASA NTRS1983

A robust Feasible Directions algorithm for design synthesis

A nonlinear optimization algorithm is developed which combines the best features of the Method of Feasible Directions and the Generalized Reduced Gradient Method. This algorithm utilizes the direction-finding sub-problem from the Method of Feasible Directions to find a search direction which is equivalent to that of the Generalized Reduced Gradient Method, but does not require the addition of a large number of slack variables associated with inequality constraints. This method provides a core-efficient algorithm for the solution of optimization problems with a large number of inequality constraints. Further optimization efficiency is derived by introducing the concept of infrequent gradient calculations. In addition, it is found that the sensitivity of the optimum design to changes in the problem parameters can be obtained using this method without the need for second derivatives or Lagrange multipliers. A numerical example is given in order to demonstrate the efficiency of the algorithm and the sensitivity analysis.

Vanderplaats, G. N.↗
NASA NTRS1991

Applications of the conjugate gradient FFT method in scattering and radiation including simulations with impedance boundary conditions

The theoretical and computational aspects related to the application of the Conjugate Gradient FFT (CGFFT) method in computational electromagnetics are examined. The advantages of applying the CGFFT method to a class of large scale scattering and radiation problems are outlined. The main advantages of the method stem from its iterative nature which eliminates a need to form the system matrix (thus reducing the computer memory allocation requirements) and guarantees convergence to the true solution in a finite number of steps. Results are presented for various radiators and scatterers including thin cylindrical dipole antennas, thin conductive and resistive strips and plates, as well as dielectric cylinders. Solutions of integral equations derived on the basis of generalized impedance boundary conditions (GIBC) are also examined. The boundary conditions can be used to replace the profile of a material coating by an impedance sheet or insert, thus, eliminating the need to introduce unknown polarization currents within the volume of the layer. A general full wave analysis of 2-D and 3-D rectangular grooves and cavities is presented which will also serve as a reference for future work.

Barkeshli, Kasra↗
NASA NTRS1999

A Single-Lap Joint Adhesive Bonding Optimization Method Using Gradient and Genetic Algorithms

A natural process for any engineer, scientist, educator, etc. is to seek the most efficient method for accomplishing a given task. In the case of structural design, an area that has a significant impact on the structural efficiency is joint design. Unless the structure is machined from a solid block of material, the individual components which compose the overall structure must be joined together. The method for joining a structure varies depending on the applied loads, material, assembly and disassembly requirements, service life, environment, etc. Using both metallic and fiber reinforced plastic materials limits the user to two methods or a combination of these methods for joining the components into one structure. The first is mechanical fastening and the second is adhesive bonding. Mechanical fastening is by far the most popular joining technique; however, in terms of structural efficiency, adhesive bonding provides a superior joint since the load is distributed uniformly across the joint. The purpose of this paper is to develop a method for optimizing single-lap joint adhesive bonded structures using both gradient and genetic algorithms and comparing the solution process for each method. The goal of the single-lap joint optimization is to find the most efficient structure that meets the imposed requirements while still remaining as lightweight, economical, and reliable as possible. For the single-lap joint, an optimum joint is determined by minimizing the weight of the overall joint based on constraints from adhesive strengths as well as empirically derived rules. The analytical solution of the sin-le-lap joint is determined using the classical Goland-Reissner technique for case 2 type adhesive joints. Joint weight minimization is achieved using a commercially available routine, Design Optimization Tool (DOT), for the gradient solution while an author developed method is used for the genetic algorithm solution. Results illustrate the critical design variables as a function of adhesive properties and convergences of different joints based on the two optimization methods.

Smeltzer, Stanley S., III↗
DOE OSTI2020

A benchmark of Gutzwiller conjugate gradient minimization method in ground state energy calculations of dimers

Herein we present numerical results of ground-state energies of 9 molecules in the well-established G2 molecule set given by the Gutzwiller conjugate gradient minimization (GCGM) method. The method, beyond the commonly used Gutzwiller approximation, was recently developed based on Gutzwiller variational wave functions. We find that compared to benchmark data given by full configuration interaction, GCGM total energies are reasonably well reproduced with the minimum basis set. To include the dynamical correlation beyond the minimal basis calculations, we adopt the local density approximation for the dynamical correlation energy $E_c$. By comparing the results with benchmark data given by experiments and large-basis configuration interaction, the GCGM total energies with $E_c$ are in general better reproduced, but discrepancies are still observed for some dimers.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1987

Experience in grid optimization

Two optimization methods for solving a variational problem in grid generation are described and evaluated. The smoothness, cell volumes, and orthogonality of the variational integrals are examined. The Jacobi-Newton iterative method is compared to the Fletcher-Reeves conjugate gradient method. It is observed that a combination of the Jacobi-Newton iteration and the direct solution of the variational problem produces an algorithm which is easy to program and requires less storage and computer time/iteration than the conjugate gradient method.

Mastin, C. W.↗
DOE OSTI2022

A posteriori superlinear convergence bounds for block conjugate gradient

In this paper, we extend to the block case the a posteriori bound showing superlinear convergence of the conjugate gradient method developed by van der Vorst and Vuik in [J. Comput. Applied Math., 48 (1993), pp. 327–341]. That is, we obtain similar bounds but now for the block conjugate gradient method. We also present a series of computational experiments, illustrating the validity of the bound developed here as well as the bound by Simoncini and Szyld from [SIAM Review, 47 (2005), pp. 247–272] using angles between subspaces. Using these bounds, we make some observations on the onset of superlinearity and how this onset depends on the eigenvalue distribution and the block size.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1973

Optimal control of a variable spin speed CMG system for space vehicles

Many future NASA programs require very high accurate pointing stability. These pointing requirements are well beyond anything attempted to date. This paper suggests a control system which has the capability of meeting these requirements. An optimal control law for the suggested system is specified. However, since no direct method of solution is known for this complicated system, a computation technique using successive approximations is used to develop the required solution. The method of calculus of variations is applied for estimating the changes of index of performance as well as those constraints of inequality of state variables and terminal conditions. Thus, an algorithm is obtained by the steepest descent method and/or conjugate gradient method. Numerical examples are given to show the optimal controls.

Liu, T. C.↗
NASA NTRS2004

Putting Priors in Mixture Density Mercer Kernels

This paper presents a new methodology for automatic knowledge driven data mining based on the theory of Mercer Kernels, which are highly nonlinear symmetric positive definite mappings from the original image space to a very high, possibly infinite dimensional feature space. We describe a new method called Mixture Density Mercer Kernels to learn kernel function directly from data, rather than using predefined kernels. These data adaptive kernels can en- code prior knowledge in the kernel using a Bayesian formulation, thus allowing for physical information to be encoded in the model. We compare the results with existing algorithms on data from the Sloan Digital Sky Survey (SDSS). The code for these experiments has been generated with the AUTOBAYES tool, which automatically generates efficient and documented C/C++ code from abstract statistical model specifications. The core of the system is a schema library which contains template for learning and knowledge discovery algorithms like different versions of EM, or numeric optimization methods like conjugate gradient methods. The template instantiation is supported by symbolic- algebraic computations, which allows AUTOBAYES to find closed-form solutions and, where possible, to integrate them into the code. The results show that the Mixture Density Mercer-Kernel described here outperforms tree-based classification in distinguishing high-redshift galaxies from low- redshift galaxies by approximately 16% on test data, bagged trees by approximately 7%, and bagged trees built on a much larger sample of data by approximately 2%.

Srivastava, Ashok N.↗
NASA NTRS2005

An Ensemble Approach to Building Mercer Kernels with Prior Information

This paper presents a new methodology for automatic knowledge driven data mining based on the theory of Mercer Kernels, which are highly nonlinear symmetric positive definite mappings from the original image space to a very high, possibly dimensional feature space. we describe a new method called Mixture Density Mercer Kernels to learn kernel function directly from data, rather than using pre-defined kernels. These data adaptive kernels can encode prior knowledge in the kernel using a Bayesian formulation, thus allowing for physical information to be encoded in the model. Specifically, we demonstrate the use of the algorithm in situations with extremely small samples of data. We compare the results with existing algorithms on data from the Sloan Digital Sky Survey (SDSS) and demonstrate the method's superior performance against standard methods. The code for these experiments has been generated with the AUTOBAYES tool, which automatically generates efficient and documented C/C++ code from abstract statistical model specifications. The core of the system is a schema library which contains templates for learning and knowledge discovery algorithms like different versions of EM, or numeric optimization methods like conjugate gradient methods. The template instantiation is supported by symbolic-algebraic computations, which allows AUTOBAYES to find closed-form solutions and, where possible, to integrate them into the code.

Srivastava, Ashok N.↗
NASA NTRS1999

A Novel Method Of Gradient Forming and Fluid Manipulation in Reduced Gravity Environments

The use of magnetic fields to control the motion and position of non-conducting liquids has received growing interest in recent times. The possibility of using the forces exerted by a nonuniform magnetic field on a ferrofluid to not only achieve fluid manipulation but also to actively control fluid motion makes it an attractive candidate for applications such as heat transfer in space systems. Terrestrial heat transfer equipment often relies on the normal gravitational force to hold liquid in a desired position or to provide a buoyant force to enhance the heat transfer rate. The residual gravitational force present in a space environment may no longer serve these useful functions and other forces, such as surface tension, can play a significant role in determining heat transfer rates. Although typically overwhelmed by gravitational forces in terrestrial applications, the body force induced in a ferrofluid by a nonuniform magnetic field can help to achieve these objectives in a microgravity environment. This paper will address the fluid manipulation aspect and will comprise of results from model fluid experiments and numerical modeling of the problem. Results from a novel method of forming concentration gradients that are applicable to low gravity applications will be presented. The ground based experiments are specifically tailored to demonstrate the magnetic manipulation capability of a ferrofluid and show that gravitational effects can be countered in carefully designed systems. The development of governing equations for the system will be presented along with a sampling of numerical results.

Ramachandran N.↗
DOE OSTI2021

A Scalable Gradient Free Method for Bayesian Experimental Design with Implicit Models

Bayesian experimental design (BED) is to answer the question that how to choose designs that maximize the information gathering. For implicit models, where the likelihood is intractable but sampling is possible, conventional BED methods have difficulties in efficiently estimating the posterior distribution and maximizing the mutual information (MI) between data and parameters. Recent work proposed the use of gradient ascent to maximize a lower bound on MI to deal with these issues. However, the approach requires a sampling path to compute the pathwise gradient of the MI lower bound with respect to the design variables, and such a pathwise gradient is usually inaccessible for implicit models. In this paper, we propose a novel approach that leverages recent advances in stochastic approximate gradient ascent incorporated with a smoothed variational MI estimator for efficient and robust BED. Without the necessity of pathwise gradients, our approach allows the design process to be achieved through a unified procedure with an approximate gradient for implicit models. Several experiments show that our approach outperforms baseline methods, and significantly improves the scalability of BED in high-dimensional problems.

Zhang, Jiaxin↗
NASA NTRS1979

Induced electric currents in the Alaska oil pipeline measured by gradient, fluxgate, and SQUID magnetometers

The field gradient method for observing the electric currents in the Alaska pipeline provided consistent values for both the fluxgate and SQUID method of observation. These currents were linearly related to the regularly measured electric and magnetic field changes. Determinations of pipeline current were consistent with values obtained by a direct connection, current shunt technique at a pipeline site about 9.6 km away. The gradient method has the distinct advantage of portability and buried- pipe capability. Field gradients due to the pipe magnetization, geological features, or ionospheric source currents do not seem to contribute a measurable error to such pipe current determination. The SQUID gradiometer is inherently sensitive enough to detect very small currents in a linear conductor at 10 meters, or conversely, to detect small currents of one amphere or more at relatively great distances. It is fairly straightforward to achieve imbalance less than one part in ten thousand, and with extreme care, one part in one million or better.

Campbell, W. H.↗