Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Gradient information”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS1985

Variational assimilation of VAS data into a mesoscale model Assimilation method and sensitivity experiments

The assimilation of temperature and moisture gradient information from the Visible IR Spin-Scan Radiometer's Atmospheric Sounder (VAS) into a mesoscale model is presently undertaken by means of a variational method and followed by tests of the sensitivity of both diabatic and adiabatic versions of the model to VAS data assimilations for the case of July 20-21, 1981. The synoptic scale effects of the assimilation of VAS data are noted to be negligible; the greatest impact was instead on mesoscale forecasts of convective instability patterns. The additional assimilation of relative humidity gradients did not significantly change the patterns of the forecast instabilities. The greatest improvements from assimilation resulted from the resolution of the strong mesoscale temperature gradients by the asynoptic VAS data.

Cram, J. M.↗
NASA NTRS1988

Estimating noise levels of remotely sensed measurements from satellites using spatial structure analysis

A technique is presented whereby the noise level of satellite measurements of the atmosphere and earth can be estimated. The technique analyzes a spatial array of data measured by a satellite instrument. A minimum of about 200 satellite measurements is required, preferably in a regular pattern. Statistical structure analysis is used to describe a combination of the mean gradient and noise in the data. The noise level is then estimated by separating out the gradient information and leaving only the noise. Results are presented for four satellite sounding instruments, and effective blackbody or brightness temperature noise levels were compared to prelaunch specifications or inflight calibrations for each instrument. Comparisons showed that in the absence of cloud-contaminated measurements (in the case of infrared data) and away from the highly variable ground surface, the noise level of various satellite instruments can be obtained without the need for calibration data. The noise levels imply how much spatial averaging is possible, without smearing the detected geophysical gradient, and how much is necessary, to meet the absolute signal accuracy requirements for the intended use of the satellite measurements.

Hillger, Donald W.↗
NASA NTRS1973

A nonlinear programming approach for optimizing two-stage lifting vehicle ascent to orbit

An optimal atmospheric flight branched trajectory-shaping capability is presented based on the Davidon-Fletcher-Powell variable metric parameter optimization technique. Gradient information is generated using finite difference methods. A typical atmospheric flight branched optimization problem is analyzed which requires the determination of 31 parameters. This parameter set includes the three-dimensional description of vehicle attitude control angles for three branches of flight: first-stage ascent, second-stage ascent, and first-stage flyback. The important inflight inequality contraints required to maintain the integrity of the vehicles are considered. Some of the numerical methods employed are discussed, along with several new auxiliary techniques developed to improve the compatibility of the numerical gradient and iterator.

Kamm, J. L.↗
NASA NTRS2004

A Study of Penalty Function Methods for Constraint Handling with Genetic Algorithm

COMETBOARDS (Comparative Evaluation Testbed of Optimization and Analysis Routines for Design of Structures) is a design optimization test bed that can evaluate the performance of several different optimization algorithms. A few of these optimization algorithms are the sequence of unconstrained minimization techniques (SUMT), sequential linear programming (SLP) and the sequential quadratic programming techniques (SQP). A genetic algorithm (GA) is a search technique that is based on the principles of natural selection or "survival of the fittest". Instead of using gradient information, the GA uses the objective function directly in the search. The GA searches the solution space by maintaining a population of potential solutions. Then, using evolving operations such as recombination, mutation and selection, the GA creates successive generations of solutions that will evolve and take on the positive characteristics of their parents and thus gradually approach optimal or near-optimal solutions. By using the objective function directly in the search, genetic algorithms can be effectively applied in non-convex, highly nonlinear, complex problems. The genetic algorithm is not guaranteed to find the global optimum, but it is less likely to get trapped at a local optimum than traditional gradient-based search methods when the objective function is not smooth and generally well behaved. The purpose of this research is to assist in the integration of genetic algorithm (GA) into COMETBOARDS. COMETBOARDS cast the design of structures as a constrained nonlinear optimization problem. One method used to solve constrained optimization problem with a GA to convert the constrained optimization problem into an unconstrained optimization problem by developing a penalty function that penalizes infeasible solutions. There have been several suggested penalty function in the literature each with there own strengths and weaknesses. A statistical analysis of some suggested penalty functions is performed in this study. Also, a response surface approach to robust design is used to develop a new penalty function approach. This new penalty function approach is then compared with the other existing penalty functions.

Ortiz, Francisco↗
DOE OSTI2025

Optimizing the optimizer for physics-informed neural networks and Kolmogorov-Arnold networks

Physics-Informed Neural Networks (PINNs) have revolutionized the computation of PDE solutions by integrating partial differential equations (PDEs) into the neural network’s training process as soft constraints, becoming an important component of the scientific machine learning (SciML) ecosystem. More recently, physics-informed Kolmogorv-Arnold networks (PIKANs) have also shown to be effective and comparable in accuracy with PINNs. In their current implementation, both PINNs and PIKANs are mainly optimized using first-order methods like Adam, as well as quasi-Newton methods such as BFGS and its low-memory variant, L-BFGS. However, these optimizers often struggle with highly nonlinear and non-convex loss landscapes, leading to challenges such as slow convergence, local minima entrapment, and (non)degenerate saddle points. In this study, we investigate the performance of Self- Scaled BFGS (SSBFGS), Self-Scaled Broyden (SSBroyden) methods and other advanced quasi-Newton schemes, including BFGS and L-BFGS with different line search strategies. These methods dynamically rescale updates based on historical gradient information, thus enhancing training efficiency and accuracy. We systematically compare these optimizers – using both PINNs and PIKANs – on key challenging PDEs, including the Burgers, Allen-Cahn, Kuramoto-Sivashinsky, Ginzburg-Landau, and Stokes equations. Additionally, we evaluate the performance of SSBFGS and SSBroyden for Deep Operator Network (DeepONet) architectures, demonstrating their effectiveness for data-driven operator learning. Our findings provide state-of-the-art results with orders-of-magnitude accuracy improvements without the use of adaptive weights or any other enhancements typically employed in PINNs. More broadly, our work reveal insights into the effectiveness of quasi-Newton optimization strategies in significantly improving the convergence and accurate generalization of PINNs and PIKANs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Analytical gradient-based optimization of CALPHAD model parameters

The calibration of CALPHAD (CALculation of PHAse Diagrams) models involves the solution of a very challenging high-dimensional multiobjective optimization problem. Traditional approaches to parameter fitting predominantly rely on gradient-free methods, which while robust, are computationally inefficient and often scale poorly with model complexity. In this work, we introduce and demonstrate a generalizable framework for analytic gradient-based optimization of the parameters of the CALPHAD model enabled by the recently formalized Jansson derivative technique. This method allows for efficient evaluation of gradients of thermodynamic properties at equilibrium with respect to model parameters, even in the presence of arbitrarily complex internal degrees of freedom. Leveraging these semi-analytic gradients, we employ the conjugate gradient (CG) method to optimize thermodynamic model parameters for four binary alloy systems: Cu-Mg, Fe-Ni, Cr-Ni, and Cr-Fe. Across all systems, CG achieves comparable or superior optimality relative to Bayesian ensemble Markov Chain Monte Carlo (MCMC) with improvements in computational efficiency ranging from one to three orders of magnitude. Furthermore, our results establish a new paradigm for CALPHAD assessments in which high fidelity data-rich model calibration becomes tractable using deterministic gradient-informed algorithms.

CALPHAD↗
DOE OSTI2023

Adaptive, problem-tailored variational quantum eigensolver mitigates rough parameter landscapes and barren plateaus

Abstract Variational quantum eigensolvers (VQEs) represent a powerful class of hybrid quantum-classical algorithms for computing molecular energies. Various numerical issues exist for these methods, however, including barren plateaus and large numbers of local minima. In this work, we consider the Adaptive, Problem-Tailored Variational Quantum Eiegensolver (ADAPT-VQE) ansätze, and examine how they are impacted by these local minima. We find that while ADAPT-VQE does not remove local minima, the gradient-informed, one-operator-at-a-time circuit construction accomplishes two things: First, it provides an initialization strategy that can yield solutions with over an order of magnitude smaller error compared to random initialization, and which is applicable in situations where chemical intuition cannot help with initialization, i.e., when Hartree-Fock is a poor approximation to the ground state. Second, even if an ADAPT-VQE iteration converges to a local trap at one step, it can still “burrow” toward the exact solution by adding more operators, which preferentially deepens the occupied trap. This same mechanism helps highlight a surprising feature of ADAPT-VQE: It should not suffer optimization problems due to barren plateaus and random initialization. Even if such barren plateaus appear in the parameter landscape, our analysis suggests that ADAPT-VQE avoids such regions by design.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Differentiable Simulation of a Liquid Argon Time Projection Chamber

Liquid argon time projection chambers (LArTPCs) are widely used in particle detection for their tracking and calorimetric capabilities. The particle physics community actively builds and improves high-quality simulators for such detectors in order to develop physics analyses in a realistic setting. The ability of these simulators to mimic real, measured data is limited by the modeling of the physical detectors used for data collection. This modeling can be improved by performing dedicated calibration measurements. Conventional approaches calibrate individual detector parameters or processes one at a time. However, the impact of detector processes is entangled, making this a poor description of the underlying physics. We introduce a differentiable simulator that enables a gradient-based optimization, allowing for the first time a simultaneous calibration of all detector parameters. We describe the procedure of making a differentiable simulator, highlighting the challenges of retaining the physics quality of the standard, non-differentiable version while providing meaningful gradient information. We further discuss the advantages and drawbacks of using our differentiable simulator for calibration. Finally, we provide a starting point for extensions to our approach, including applications of the differentiable simulator to physics analysis pipelines.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Gaussian processes meet NeuralODEs: a Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data

We present a machine learning framework (GP-NODE) for Bayesian model discovery from partial, noisy and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling and Gaussian Process priors over the observed system states. This allows us to exploit temporal correlations in the observed data, and efficiently infer posterior distributions over plausible models with quantified uncertainty. The use of the Finnish Horseshoe as a sparsity-promoting prior for free model parameters also enables the discovery of parsimonious representations for the latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed GP-NODE method including predator–prey systems, systems biology and a 50-dimensional human motion dynamical system. This article is part of the theme issue ‘Data-driven prediction in dynamical systems’.

Science & Technology - Other Topics↗
DOE OSTI2024

MULTI-LEADER: MULTI-source LEarning-Accelerated Design of high-Efficiency multi-stage compRessor (Final Technical Report)

The objective of MULTI-LEADER is to cut design costs by 80% while generating more energy-efficient designs of multi-stage compressors by developing and implementing novel machine learning (ML) techniques, which enable faster and fewer design iterations, improved solver performance, and concurrent multi-disciplinary design. Current industrial practices for the design of multi-stage compressors involve simulation-based design optimization with successive levels of model fidelity, iteratively evaluated between distinct disciplines, one stage at a time to tackle the high dimensional design variations. This project addresses these key design challenges: (1) concurrent optimization of multiple stages under many non-linear constraints; (2) multitude of evaluation of high-fidelity and expensive solvers and their gradients during optimization convergence in high-dimensional design; (3) multi-disciplinary design to maximize aerodynamic performance while guaranteeing structural integrity and additive manufacturability; (4) utilization of multiple fidelity of solvers with disparate parameterization and modeling assumptions. MULTI-LEADER achieved more than 5x speed up in detailed design of more energy-efficient compressors via these machine learning (ML) innovations: (i) rapid design surrogates by multi-source learning from diverse fidelities across multiple disciplines, (ii) physics-constrained data-augmented modeling for improved empiricism, (iii) generative manifold embedding for high dimensional concurrent design without gradient information; (iv) budget-constrained fidelity-adaptive sampling towards fewer design iterations.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2024

Differentiable Multiphysics Codes: A Breakthrough Technology for Simulation and Computing

This document summarizes the findings of a strategic planning exercise commissioned by the Weapons Simulation and Computing, Computational Physics (WSC/CP) program at the Lawrence Livermore National Laboratory (LLNL) in FY24. During the year, the committee met with multiple stakeholder communities to gather input, opinions, suggestions and concerns which have been incorporated throughout this document. The key findings from this exercise are summarized: • The development of multiphysics modelling and simulation (mod/sim) codes and software technologies, their deployment on exascale compute platforms, and their broad adoption across the NNSA is a major success of the Advanced Simulation and Computing (ASC) program and the Exascale Computing Project (ECP). Sustained investment in these core technologies is essential. • Today’s state of the art involves running ensembles of O(100K) simulations to perform uncertainty quantification (UQ) and design studies using multiple statistical methods such as Bayesian optimization to understand sensitivities of our models and explore parameterized design spaces. Even with exascale computing, we are practically limited to O(10) parameters in these studies since the number of simulations required to sample the space scales exponentially with the number of design parameters. • The data from these simulation ensembles is increasingly being used to train machine learned (ML) surrogates (or reduced order models, ROMs) which can then be used for optimization or real time design exploration. However, the trained surrogates are still limited in the number of parameters they can represent due to the sampling limitations previously noted. • Augmenting our suite of integrated multiphysics simulation codes, both current and emerging, with the ability to compute gradients (solution derivatives) of arbitrary simulation outputs with respect to (some or all) simulation inputs would be a breakthrough technology, opening the door to a new era of efficient and automated inverse design based on verified and validated mod/sim capabilities. • This capability, which we refer to as differentiable multiphysics codes (DMCs), would revolutionize both UQ and optimization studies by breaking the curse of dimensionality that presently limits our “gradient-free” ensemble based computing approach. A similar breakthrough occurred in the AI/ML community once the ability to compute gradients of arbitrary loss functions using back-propagation became commonplace. Gradient information from the multiphysics codes can also be used to dramatically improve the efficiency and scale of training of ML/ROM surrogates for rapid assessments. • Achieving this in our suite of codes will be a grand challenge, similar to the amount of effort that was required to transition from CPU to GPU computing. It will require buy-in from the entire WSC/CP program and beyond, including all integrated codes, physics and engineering models, third-party library dependencies and performance portability abstractions. It will also require investment in research and development of numerical methods for computing adjoints of coupled physics across multiple adaptively refined moving meshes and of stochastic (Monte Carlo) and mesh free (SPH) methods. • New software and numerical techniques, largely pioneered by the AI/ML community, make this feasible. Chief among these is automatic differentiation (AD), the ability to employ AD at point-wise locations in a physics calculation (instead of traditional black-box approaches) and the ability to perform “back-propagation in time” (or reverse mode AD) for non-linear partial differential equations (PDEs). Fundamentally, the conclusion of this strategic planning exercise is that the time is right to undertake a large scale effort in WSC, centered on the existing integrated codes, to continue the natural evolution of mod/sim in the age of AI/ML. Instead of attempting to replace mod/sim with purely data driven AI/ML models, we believe the key to success is to integrate AI/ML by building on top of the decades of hard-won knowledge and the verified/validated multiphysics modelling capability that is the hallmark of the ASC program.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Automated tuning for HMC mass ratios

We extended previous work on tuning HMC parameters using gradient information to include Hasenbusch mass ratios. The inclusion of mass ratios adds many more parameters that need to be tuned, and also allows for lots of variations in the choice of integrator pattern. We investigate the effectiveness of automatically tuning a large number of HMC parameters and compare the optimally tuned versions over a range of integrator variants.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1973

Structural optimization by methods of feasible directions.

A general design algorithm based on methods of feasible directions is presented. Zoutendijk's method of feasible directions is first presented as applied to structural design. This method is modified to improve numerical stability of the design process and is then further modified to deal efficiently with infeasible designs. The algorithm requires the analytic gradient of the objective function and the constraint functions which are active at a given stage in the design process. Gradient information is not required for nonactive constraints. Complex constraint functions may be ignored in the initial design stages because violation of these constraints is efficiently overcome later in the design process. The algorithm is demonstrated with elastic design of redundant trusses.

Vanderplaats, G. N.↗
NASA NTRS1973

Utilizing remote sensing data for land use decisions for Indian lands in South Dakota.

Evaluation of photographic and thermal infrared imagery acquired at an altitude of 11,500 ft above ground level (in southwestern South Dakota) on Oct. 15, 1970, June 30, 1971, and Aug. 2, 1971, for use in mapping range sites and soils. A density slicing system was used to enhance film optical density differences associated with range sites and soils. The range site boundaries in native rangeland areas were found to be delineated best by a density slicing analysis of color infrared film obtained in August 1971. The range site maps produced by density slicing were superior to existing range inventory maps. The density slicing analysis of color infrared film from the August 1971 flight was best for locating soil boundaries in rangeland areas. To adequately map soils in rangeland areas, data based on density slicing used in conjunction with slope gradient information derived from stereoscopic analyses provided more detailed and accurate information than is presently available from range site and soil maps of the area.

Frazee, C. J.↗
NASA NTRS1974

Study of genetic direct search algorithms for function optimization

The results are presented of a study to determine the performance of genetic direct search algorithms in solving function optimization problems arising in the optimal and adaptive control areas. The findings indicate that: (1) genetic algorithms can outperform standard algorithms in multimodal and/or noisy optimization situations, but suffer from lack of gradient exploitation facilities when gradient information can be utilized to guide the search. (2) For large populations, or low dimensional function spaces, mutation is a sufficient operator. However for small populations or high dimensional functions, crossover applied in about equal frequency with mutation is an optimum combination. (3) Complexity, in terms of storage space and running time, is significantly increased when population size is increased or the inversion operator, or the second level adaptation routine is added to the basic structure.

Zeigler, B. P.↗
NASA NTRS1984

Efficient calculation of optimum design sensitivity

Two methods for obtaining the sensitivity of an optimized design to changes in problem parameters are presented. The first-order method uses gradient information to estimate the required sensitivity. The second-order method uses second derivatives of the design objective and constraint functions to provide a quadratic approximation to the new design which results from changing the specified parameter. It is shown that the sensitivity of the optimized design may be discontinuous with respect to the new parameter. This discontinuity is accounted for in the present methods. These methods are compared to a previous approach based on the Kuhn-Tucker necessary conditions for optimality. It is shown that the present methods provide the appropriate sensitivity information in an efficient manner. The methods are demonstrated by a structural synthesis example.

Vanderplaats, G. N.↗
NASA NTRS1984

Hurricane Debby - Analysis and numerical forecasts using VAS soundings

The utility of VISSR Atmospheric Sounder (VAS) temperature and moisture soundings in defining the storm and its surroundings at subsynoptic scales has been examined using a numerical analysis and prognosis system. In particular, VAS temperature and moisture soundings and cloud and water vapor motion winds have been used in numerical analysis for three time periods. It is shown that the VAS temperature and moisture data which specify temperature and moisture well in cloud free regions are complemented by cloud and water vapor wind data which provide horizontal gradient information for the cloudy areas. The loss of analysis integrity due to the reduction of VAS data density in the cloudy regions associated with synoptic activity is ameliorated by using cloud and water vapor motion winds. The improvement in numerical forecasts resulting from the addition of these data to the numerical data base is also recorded.

Le Marshall, J. F.↗
NASA NTRS1985

Hurricane Debby - An illustration of the complementary nature of VAS soundings and cloud and water vapor motion winds

The utility of VISSR Atmospheric Sounder (VAS) temperature and moisture soundings and cloud and water vapor motion winds in defining a storm and its surroundings at subsynoptic scales has been examined using a numerical analysis and prognosis system. It is shown that the VAS temperature and moisture data, which specify temperature and moisture well in cloud-free areas, are complemented by cloud and water vapor motion data generated in the cloudy areas. The cloud and water vapor 'winds' provide thermal gradient information for interpolating the soundings across cloudy regions. The loss of analysis integrity due to the reduction of VAS sounding density in the cloudy regions associated with synoptic activity is ameliorated by using cloud and water vapor motion winds. The improvement in numerical forecasts resulting from the addition of these data to the numerical analysis is recorded.

Le Marshall, J. F.↗