SEARCH · Engineering Papers

Results for “gradient-based optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Convergence analysis for a nonlocal gradient descent method via directional Gaussian smoothing

We analyze the convergence of a nonlocal gradient descent method for minimizing a class of high-dimensional non-convex functions, where a directional Gaussian smoothing (DGS) is proposed to define the nonlocal gradient (also referred to as the DGS gradient). The method was first proposed in [Zhang et al., Enabling long-range exploration in minimization of multimodal functions, UAI 2021], in which multiple numerical experiments showed that replacing the traditional local gradient with the DGS gradient can help the optimizers escape local minima more easily and significantly improve their performance. However, a rigorous theory for the efficiency of the method on nonconvex landscape is lacking. In this work, we investigate the scenario where the objective function is composed of a convex function, perturbed by deterministic oscillating noise. We provide a convergence theory under which the iterates exponentially converge to a tightened neighborhood of the solution, whose size is characterized by the noise wavelength. Here, we also establish a correlation between the optimal values of the Gaussian smoothing radius and the noise wavelength, thus justifying the advantage of using moderate or large smoothing radii with the method. Furthermore, if the noise level decays to zero when approaching the global minimum, we prove that DGS-based optimization converges to the exact global minimum with linear rates, similarly to standard gradient-based methods in optimizing convex functions. Several numerical experiments are provided to confirm our theory and illustrate the superiority of the approach over those based on the local gradient.

Tran, Hoang [Oak Ridge National Laboratory (ORNL),↗

DOE OSTI2024

Reduced-dimension Bayesian optimization for model calibration of transient vapor compression cycles

Development and calibration of first-principles dynamic models of vapor compression cycles (VCCs) is of critical importance for applications that include control design and fault detection and diagnostics. Nevertheless, the inherent complexity of models that are represented by large systems of differential–algebraic equations leads to significant challenges for model calibration processes that utilize classical gradient-based methods. Bayesian optimization (BO) is a sample-efficient and gradient-free approach using a probabilistic surrogate model and optimal search over a feasible parameter space. Despite the benefits of BO in reducing computational costs, challenges remain in dealing with a high-dimensional calibration task resulting from a large set of parameters that have significant impacts on system behavior and need to be calibrated simultaneously. This paper presents a reduced-dimension BO framework for calibrating transient VCCs models where the calibration space is projected to a low-dimensional subspace for accelerating convergence of the solution algorithm and consequently reducing the number of transient simulations. The proposed approach was demonstrated via two case studies associated with different VCC applications where 10 parameters were calibrated in each case using laboratory measurements. The reduced-dimension BO framework only required 1 / 8 th of the iterations associated with a standard BO method that deals with high-dimensional calibration parameters for converged solutions and yielded comparable accuracy. Furthermore, both calibrated models revealed significant accuracy improvements compared to uncalibrated models.

Ma, Jiacheng↗

DOE OSTI2024

FLOWERS AEP: An Analytical Model for Wind Farm Layout Optimization

Annual energy production (AEP) is commonly used in objective functions for wind farm layout optimization. AEP is proportional to wind farm power production integrated over an annual distribution of free-stream wind conditions. Physics-based estimates of wind farm power production typically rely on low-fidelity engineering wake models that approximate the steady-state wind farm flow field. AEP estimates are then obtained by performing independent simulations for discrete wind conditions and using rectangular quadrature to account for each condition's expected frequency of occurrence. Depending on the number of simulated discrete wind conditions, this numerical integral could be hampered by poor accuracy or high computational costs. The FLOWERS AEP model instead poses an analytical integral of the engineering wake model over the variable wind conditions, yielding a closed-form, analytical function for wind farm AEP. This paper derives the analytical functions for FLOWERS AEP and its derivatives with respect to turbine position, which are useful for gradient-based wind farm layout optimization, in nondimensional form. We then analyze the benefits of the FLOWERS AEP model over conventional reference models, focusing on its low cost, adequate wake loss predictions, and smooth design space. Although the FLOWERS approach is found to predict the exact value of AEP with some error relative to the reference model (within 14% on average), it dramatically reduces computation time by an order of magnitude, produces a qualitatively similar design space at relatively low resolution, and yields comparable optimal layouts. This significant speed improvement is critical in layout optimization applications, where determining an optimal layout in an efficient manner is more important than precise AEP prediction.

17 WIND ENERGY↗

DOE OSTI2025

DEVELOPMENT AND APPLICATION OF RISK ANALYSIS TOOLKIT FOR PLANT RESOURCE OPTIMIZATION

This paper presents the development of methods and tools that are being designed to optimize plant operations (e.g., maintenance/replacement schedules and optimal maintenance postures for plant components) in a manner that is more cost effective than current approaches and makes better use of available component health and cost data. These methods include both data- and model-based optimization methods. Model-based optimization methods directly include reliability and cost models to determine an optimal plant operational strategy. We consider gradient-based and evolutionary (based on genetic algorithms) optimization methods. The second class of methods target more specific use cases (e.g., project schedule optimization) and are not based on reliability models directly, but they require specific component reliability and cost data. This class of methods is based on variants of the knapsack problem with an aim to determine an optimal project schedule that maximizes the overall NPV. This paper also presents multi-objective methods designed to identify an optimal maintenance posture based on a Pareto frontier analysis. Rather than dictating the “right” tradeoff (i.e., identify the absolute best posture), we show how it is possible to perform a trade space exploration approach (i.e., identify value and costs of several postures and let the analysis account for desired value and cost metrics). This is performed by identifying maintenance postures that maximize value (e.g., system availability) and minimize operational costs, i.e., the Pareto frontier in a value-cost trade space. For all these methods we present detailed applicative examples that show their validity from a decision-making perspective.

97 - MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Binary Quantum Control Optimization with Uncertain Hamiltonians

Optimizing the controls of quantum systems plays a crucial role in advancing quantum technologies. The time-varying noises in quantum systems and the widespread use of inhomogeneous quantum ensembles raise the need for high-quality quantum controls under uncertainties. In this paper, we consider a stochastic discrete optimization formulation of a discretized binary optimal quantum control problem involving Hamiltonians with predictable uncertainties. We propose a sample-based reformulation that optimizes both risk-neutral and risk-averse measurements of control policies, and solve these with two gradient-based algorithms using sum-up-rounding approaches. Furthermore, we discuss the differentiability of the objective function and prove upper bounds of the gaps between the optimal solutions to binary control problems and their continuous relaxations. We conduct numerical simulations on various sized problem instances based on two applications of quantum pulse optimization; we evaluate different strategies to mitigate the impact of uncertainties in quantum systems. In conclusion, we demonstrate that the controls of our stochastic optimization model achieve significantly higher quality and robustness compared with the controls of a deterministic model.

conditional value-at-risk (CVaR)↗

DOE OSTI2026

Optimization techniques in self-similar compressible flow

We investigate the one-dimensional (1D) inviscid compressible flow equations for an ideal gas through the lens of optimization techniques. It is the case that, to our knowledge, optimization analysis applied to the so-called “linear velocity” solutions of the Euler compressible flow equations has not been previously conducted. Through both gradient-based and variational techniques, new variants of well-studied flow scenarios, i.e., self-similar, 1D, linear velocity solution class to idealized inviscid compressible flow equations, are determined, as encoded in both the kinematic and thermodynamic properties of this self-similar solution class. With the kinematics of the said solutions being driven by a self-similar “scale radius” and the thermodynamics being driven separately through the appearance of an arbitrary function, a myriad of new solution classes is possible. Acting as a guide to more realistic physical circumstances as well as discovery, it is the hope that the presented cases serve as the framework for future investigations into the intersection of self-similarity and optimization techniques. Fields of study that may find this work to be of interest include aerodynamic design, flow control, inertial confinement fusion, physics-informed neural networks, and other related areas of interest.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

NASA NTRS1991

The effect of model uncertainty on some optimal routing problems

The effect of model uncertainties on optimal routing in a system of parallel queues is examined. The uncertainty arises in modeling the service time distribution for the customers (jobs, packets) to be served. For a Poisson arrival process and Bernoulli routing, the optimal mean system delay generally depends on the variance of this distribution. However, as the input traffic load approaches the system capacity the optimal routing assignment and corresponding mean system delay are shown to converge to a variance-invariant point. The implications of these results are examined in the context of gradient-based routing algorithms. An example of a model-independent algorithm using online gradient estimation is also included.

Mohanty, Bibhu↗

DOE OSTI2025

Automated Fire Detection for Industrial Settings with Pretrained Convolutional Networks

Early fire detection in industrial environments is critical to preventing equipment damage, personal injury, and operational disruptions. Traditional smoke detectors, while effective, often experience delays due to the time required for smoke to reach sensors, allowing fires to spread. Manual fire watch operations and human surveillance of camera feeds are resource-intensive and prone to human error. To address these challenges, this paper explores the application of convolutional neural networks for automated fire detection, specifically in industrial settings. By leveraging 11 different pre-trained machine vision models from TensorFlow and enhancing them with transfer learning on a custom-built industrial fire dataset, we optimized fire detection performance. Here, we analyzed each machine vision model architecture in terms of its depth, width, and input image resolution, considering both resource requirements and detection accuracy. We further explored the option of combining multiple models into an ensemble classifier to evaluate whether the performance improvements could justify the much greater computational complexity and other practical impacts. A cost-benefit analysis is presented to evaluate the trade-offs between performance and computational expense. Our findings identify that EfficientNetV2L, specifically tailored for industrial applications, provides the optimal balance between costs involved in training and using the model versus the overall fire detection performance. Additionally, we present a qualitative analysis of model performance using the technique of gradient-based class activation mapping to provide explainability by visualizing model decisions.

artificial intelligence↗

DOE OSTI2025

Robustness of Deep Learning Classification to Adversarial Input on GPUs: Asynchronous Parallel Accumulation Is a Source of Vulnerability

The ability of machine learning (ML) classification models to resist small, targeted input perturbations—known as adversarial attacks—is a key measure of their safety and reliability. We show that floating-point non associativity (FPNA) coupled with asynchronous parallel programming on GPUs is sufficient to result in misclassification, without any perturbation to the input. Additionally, we show that this misclassification is particularly significant for inputs close to the decision boundary and that standard adversarial robustness results may be overestimated up to 4.6 when not considering machine-level details. We first study a linear classifier, before focusing on standard Graph Neural Network (GNN) architectures and datasets used in robustness assessments. We develop a novel black-box attack using Bayesian optimization to discover external workloads that can change the instruction scheduling which bias the output of reductions on GPUs and reliably lead to misclassification. Motivated by these results, we present a new learnable permutation (LP) gradient-based approach to learning floating-point operation orderings that lead to misclassifications. The LP approach provides a worst-case estimate in a computationally efficient manner, avoiding the need to run identical experiments tens of thousands of times over a potentially large set of possible GPU states or architectures. Finally, using instrumentation-based testing, we investigate parallel reduction ordering across different GPU architectures under external background workloads, when utilizing multi-GPU virtualization, and when applying power capping. Our results demonstrate that parallel reduction ordering varies significantly across architectures under the first two conditions, substantially increasing the search space required to fully test the effects of this parallel scheduler-based vulnerability. These results and the methods developed here can help to include machine-level considerations into adversarial robustness assessments, which can make a difference in safety and mission critical applications.

Shanmugavelu, Sanjif [Maxeler Technologies, a Groq↗

DOE OSTI2025

Response tailoring of elasto-plastic trusses

Abstract In this work we tailor the response of trusses loaded beyond their yield limit. The truss structures are modeled using finite strain theory and rate-independent elasto-plasticity. We design trusses with a tailored mechanical response that is between “elastic” and “elastic-ideal-plastic” subject to the volume constraint. The design updates are generated by the gradient-based Method of Moving Asymptotes (MMA) solver and the sensitivities of the response functions are computed using a path-dependent adjoint sensitivity analysis. The computations are performed in Matlab.

42 ENGINEERING↗

DOE OSTI2026

Extremized nonlinear and linearized responses in soft metamaterials enabled by gradient-based design and grayscale digital light processing

In this study, we develop a gradient-based design approach that exploits grayscale digital light processing (DLP) 3D printing for extremizing the nonlinear and linearized response of soft metamaterials — materials that harness engineered geometric instabilities to undergo large and programmable changes in configuration. Grayscale DLP approaches modulate local mechanical properties at the pixel scale by tuning the light intensity within a single grayscale image, unlocking an exceptionally large design space. To effectively navigate this space, we develop smooth mappings between local light intensity values and global quantities of interest that characterize the behavior of soft metamaterials. Enabling these smooth mappings are robust and differentiable nonlinear finite element simulations powered by a trust region solver. A PDE-constrained optimization problem is then solved to invert these mappings and produce light intensity distributions that endow the printed part with varying stiffness and flexibility in distinctive regions. It is shown that optimizing the distribution of soft and stiff phases throughout a metamaterial structure results in markedly different buckling and self-contact configurations to drive extremized nonlinear compression and linearized vibration responses. Optimized light intensity distributions are translated to grayscale images and directly used to print soft metamaterial samples, showing remarkable agreement between the buckling and self-contact response in simulated and measured deformed configurations.

Additive manufacturing↗

DOE OSTI2024

Proximal Galerkin: A Structure-Preserving Finite Element Method for Pointwise Bound Constraints

The proximal Galerkin finite element method is a high-order, low iteration complexity, nonlinear numerical method that preserves the geometric and algebraic structure of pointwise bound constraints in infinite-dimensional function spaces. This paper introduces the proximal Galerkin method and applies it to solve free boundary problems, enforce discrete maximum principles, and develop a scalable, mesh-independent algorithm for optimal design with pointwise bound constraints. This paper also introduces the latent variable proximal point (LVPP) algorithm, from which the proximal Galerkin method derives. When analyzing the classical obstacle problem, we discover that the underlying variational inequality can be replaced by a sequence of second-order partial differential equations (PDEs) that are readily discretized and solved with, e.g., the proximal Galerkin method. Throughout this work, we arrive at several contributions that may be of independent interest. These include (1) a semilinear PDE we refer to as the entropic Poisson equation; (2) an algebraic/geometric connection between high-order positivity-preserving discretizations and certain infinite-dimensional Lie groups; and (3) a gradient-based, bound-preserving algorithm for two-field, density-based topology optimization. The complete proximal Galerkin methodology combines ideas from nonlinear programming, functional analysis, tropical algebra, and differential geometry and can potentially lead to new synergies among these areas as well as within variational and numerical analysis. Open-source implementations of our methods accompany this work to facilitate reproduction and broader adoption.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Adaptive Cybersecurity for Distributed Energy Resources (AdCyDER): Online Reinforcement Learning with Stackelberg-Optimized Defenses — Pipeline Architecture, Evaluation Methodology, and Findings from a Synthetic-Data Evaluation

This report documents the design and evaluation of an integrated online-learning pipeline developed within the AdCyDER project for Distributed Energy Resource (DER) cybersecurity. The pipeline couples a Reinforcement Learning (RL) attack classifier — which produces an attack-type probability distribution — with a Stackelberg game-theoretic (GT) defense selector that consumes those distributions alongside SME-encoded priors over (defense, attack) effectiveness pairings and perdefense costs to choose grid-health-preserving defenses. The objective is not attack classification per se but production of distributions that drive effective defense selection through the Stackelberg layer, learned from delayed grid-health feedback rather than labeled attack data. AdCyDER as a whole is broader than the work presented here; this report covers the specific RL/GT loop integration and its evaluation. We present the integrated pipeline (SCADA telemetry with Fronius inverter physics, Suricata IDS, time-windowed aggregation, per-facility LSTM classifier, Stackelberg optimizer, OpenC2 actuators), an experimental campaign of 28 eight-hour iterations across three baseline modes, and a pipeline-ordered diagnostic protocol. The protocol identifies two distinct failure modes within the loop: paired supervised ceilings on the same features establish that the deployed online RL classifier (macro F1 ≈ 0.07) sits at least 4.7× below a same-architecture supervised LSTM (≈ 0.34) and 10–11× below a linear feature-signal ceiling (≈ 0.70–0.79 depending on per-facility isolation), localizing the dominant failure to the training procedure; and the reward signal driving online updates carries weak directional coupling with classifier correctness in the methodology-expected direction (multi-lens convergent: top-decile P(true) records produce more frequent state changes and slightly larger improvements, top-vs-bot Cohen’s 𝑑 ≈ −0.19), but at effect magnitudes too small to drive gradient-based learning at the campaign sample size. The original learning hypothesis is not supported by the data. The primary contributions are the diagnostic methodology — proposed as a transferable falsification protocol for online RL/GT defense pipelines learning from delayed environmental reward — and the open, reproducible experimental infrastructure. We outline reward reformulation as the highest-priority aspirational next step given the underpowered-but-aligned Q6 reading, with hardware-in-the-loop evaluation as the broadest scope-expansion option.

Blakely, Benjamin [Argonne National Laboratory (AN↗

DOE OSTI2025

A direct-adjoint approach for material point model calibration with application to plasticity

Here, this paper proposes a new approach for the calibration of material parameters in local elastoplastic constitutive models. The calibration is posed as a constrained optimization problem, where the constitutive model evolution equations for a single material point serve as constraints. The objective function quantifies the mismatch between the stress predicted by the model and corresponding experimental measurements. To improve calibration efficiency, a novel direct-adjoint approach is presented to compute the Hessian of the objective function, which enables the use of second-order optimization algorithms. Automatic differentiation is used for gradient and Hessian computations. Two numerical examples are employed to validate the Hessian matrices and to demonstrate that the Newton–Raphson algorithm consistently outperforms gradient-based algorithms such as L-BFGS-B.

36 MATERIALS SCIENCE↗

DOE OSTI2025

Efficient wind farm layout optimization with the FLOWERS AEP model and analytic gradients

Wind farm layout optimization (WFLO) studies often aim to maximize the annual energy production (AEP) of a wind farm by choosing an arrangement of turbines that minimizes wake interactions. One way to reduce the cost of WFLO studies is by using more computationally efficient AEP models. The cost of standard AEP modeling approaches, based on the numerical integration of low-fidelity engineering wake models, scales poorly with the number of simulated discrete wind conditions. A second way to reduce cost when using a gradient-based algorithm is to supply exact gradient information instead of finite-difference estimates. However, analytical functions for the derivatives of AEP with respect to turbine positions are not always available in the conventional modeling approach. FLOWERS is a computationally inexpensive, analytical model for wind farm AEP that is specifically developed for WFLO applications. In this paper, we analyze the performance of the FLOWERS AEP model with analytic gradients in a layout optimization study compared with a reference optimization framework across three wind farm case studies. We find that the FLOWERS-based approach reduces computation time by a factor of 50–4000 and improves optimal AEP by about 0.3% with less than half of the variability in AEP across instances with randomized initial conditions. We also find the optimal layouts to be insensitive to model parameter tuning, making FLOWERS-based layout optimization a streamlined, user-friendly approach.

17 WIND ENERGY↗

DOE OSTI2025

Learning with Adaptive Conservativeness for Distributionally Robust Optimization: Incentive Design for Voltage Regulation

Information asymmetry between the Distribution System Operator (DSO) and Distributed Energy Resource Aggregators (DERAs) obstructs designing effective incentives for voltage regulation. To capture this effect, we employ a Stackelberg game-theoretic framework, where the DSO seeks to overcome the information asymmetry and refine its incentive strategies by learning from DERA behavior over multiple iterations. We introduce a model-based online learning algorithm for the DSO, aimed at inferring the relationship between incentives and DERA responses. Given the uncertain nature of these responses, we also propose a distributionally robust incentive design model to control the probability of voltage regulation failure and then reformulate it into a convex problem. This model allows the DSO to periodically revise distribution assumptions on uncertain parameters in the decision model of the DERA. Finally, we present a gradient-based method that permits the DSO to adaptively modify its conservativeness level, measured by the size of a Wasserstein metric-based ambiguity set, according to historical voltage regulation performance. The effectiveness of our proposed method is demonstrated through numerical experiments.

adaptation models↗

NASA NTRS1992

Observations on computational methodologies for use in large-scale, gradient-based, multidisciplinary design incorporating advanced CFD codes

How a combination of various computational methodologies could reduce the enormous computational costs envisioned in using advanced CFD codes in gradient based optimized multidisciplinary design (MdD) procedures is briefly outlined. Implications of these MdD requirements upon advanced CFD codes are somewhat different than those imposed by a single discipline design. A means for satisfying these MdD requirements for gradient information is presented which appear to permit: (1) some leeway in the CFD solution algorithms which can be used; (2) an extension to 3-D problems; and (3) straightforward use of other computational methodologies. Many of these observations have previously been discussed as possibilities for doing parts of the problem more efficiently; the contribution here is observing how they fit together in a mutually beneficial way.

Newman, P. A.↗

NASA NTRS1992

A conjugate gradients/trust regions algorithms for training multilayer perceptrons for nonlinear mapping

This paper addresses the issue of applying a globally convergent optimization algorithm to the training of multilayer perceptrons, a class of Artificial Neural Networks. The multilayer perceptrons are trained towards the solution of two highly nonlinear problems: (1) signal detection in a multi-user communication network, and (2) solving the inverse kinematics for a robotic manipulator. The research is motivated by the fact that a multilayer perceptron is theoretically capable of approximating any nonlinear function to within a specified accuracy. The algorithm that has been employed in this study combines the merits of two well known optimization algorithms, the Conjugate Gradients and the Trust Regions Algorithms. The performance is compared to a widely used algorithm, the Backpropagation Algorithm, that is basically a gradient-based algorithm, and hence, slow in converging. The performances of the two algorithms are compared with the convergence rate. Furthermore, in the case of the signal detection problem, performances are also benchmarked by the decision boundaries drawn as well as the probability of error obtained in either case.

Madyastha, Raghavendra K.↗