Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “gradient-based algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Gradient-based optimization for regression in the functional tensor-train format

Predictive analysis of complex computational models, such as uncertainty quantification (UQ), must often rely on using an existing database of simulation runs. In this paper we consider the task of performing low-multilinear-rank regression on such a database. Specifically we develop and analyze an efficient gradient computation that enables gradient-based optimization procedures, including stochastic gradient descent and quasi-Newton methods, for learning the parameters of a functional tensor-train (FT). We compare our algorithms with 22 other nonparametric and parametric regression methods on 10 real-world data sets and show that for many physical systems, exploiting low-rank structure facilitates efficient construction of surrogate models. Here, we use a number of synthetic functions to build insight into behavior of our algorithms, including the rank adaptation and group-sparsity regularization procedures that we developed to reduce overfitting. Finally we conclude the paper by building a surrogate of a physical model of a propulsion plant on a naval vessel.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Learning with Delayed Rewards—A Case Study on Inverse Defect Design in 2D Materials

Defect dynamics in materials are of central importance to a broad range of technologies from catalysis to energy storage systems to microelectronics. Material functionality depends strongly on the nature and organization of defects-their arrangements often involve intermediate or transient states that present a high barrier for transformation. The lack of knowledge of these intermediate states and the presence of this energy barrier presents a serious challenge for inverse defect design, especially for gradient-based approaches. Here, we present a reinforcement learning (RL) [Monte Carlo Tree Search (MCTS)] based on delayed rewards that allow for efficient search of the defect configurational space and allows us to identify optimal defect arrangements in low-dimensional materials. Using a representative case of two-dimensional MoS 2 , we demonstrate that the use of delayed rewards allows us to efficiently sample the defect configurational space and overcome the energy barrier for a wide range of defect concentrations (from 1.5 to 8% S vacancies)-the system evolves from an initial randomly distributed S vacancies to one with extended S line defects consistent with previous experimental studies. Detailed analysis in the feature space allows us to identify the optimal pathways for this defect transformation and arrangement. Comparison with other global optimization schemes like genetic algorithms suggests that the MCTS with delayed rewards takes fewer evaluations and arrives at a better quality of the solution. The implications of the various sampled defect configurations on the 2H to 1T phase transitions in MoS 2 are discussed. In this study, we introduce a RL strategy employing delayed rewards that can accelerate the inverse design of defects in materials for achieving targeted functionality.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Robust wind farm layout optimization

Wake interactions in wind farms cause losses in annual energy production (AEP) on the order of 10%. Wind farm designers optimize the layout of the farm to mitigate wake losses, especially in the dominant site-specific wind directions. As wind turbines and wind farms grow in scale, optimization becomes more complex. Offshore wind farms regularly comprise more than 100 wind turbines and are characterized by complex boundaries due to shipping lanes, neighboring wind farms, and other constraints. Layout optimization methods are broadly split between gradient-based and gradient-free approaches. Gradient-based approaches can converge quickly and perform well for smaller, academic problems but are often sensitive to initial conditions and tuning parameters and require expert knowledge to use. On the other hand, gradient-free approaches can be more robust to problem complexities. We present a robust layout optimization approach based on a random search algorithm. The algorithm is intended for those who are not optimization experts and has few tuning parameters that need specification to achieve satisfactory results. Unlike off-the-shelf methods, which use generally available, non-domain-specific optimization routines that accept as inputs an optimization function and constraint definitions, this approach takes advantage of the relative computational costs of the different evaluations by evaluating cheaper computations first (boundary and minimum distance constraints) and running expensive AEP evaluations only if all other checks pass. Moreover, an outer genetic algorithm allows multiple solutions to evolve in parallel, enabling rapid solution development on high-performance computers. We discuss the relative ease of selecting necessary tuning parameters and demonstrate the efficacy of the genetic random search on a complex layout problem consisting of placing 70 turbines in a nonconvex and unconnected boundary region.

17 WIND ENERGY↗
DOE OSTI2021

Natural evolutionary strategies for variational quantum computation

Abstract Natural evolutionary strategies (NES) are a family of gradient-free black-box optimization algorithms. This study illustrates their use for the optimization of randomly initialized parameterized quantum circuits (PQCs) in the region of vanishing gradients. We show that using the NES gradient estimator the exponential decrease in variance can be alleviated. We implement two specific approaches, the exponential and separable NES, for parameter optimization of PQCs and compare them against standard gradient descent. We apply them to two different problems of ground state energy estimation using variational quantum eigensolver and state preparation with circuits of varying depth and length. We also introduce batch optimization for circuits with larger depth to extend the use of ES to a larger number of parameters. We achieve accuracy comparable to state-of-the-art optimization techniques in all the above cases with a lower number of circuit evaluations. Our empirical results indicate that one can use NES as a hybrid tool in tandem with other gradient-based methods for optimization of deep quantum circuits in regions with vanishing gradients.

Anand, Abhinav (ORCID:0000000280812310)↗
DOE OSTI2022

Training neural networks using Metropolis Monte Carlo and an adaptive variant

Abstract We examine the zero-temperature Metropolis Monte Carlo (MC) algorithm as a tool for training a neural network by minimizing a loss function. We find that, as expected on theoretical grounds and shown empirically by other authors, Metropolis MC can train a neural net with an accuracy comparable to that of gradient descent (GD), if not necessarily as quickly. The Metropolis algorithm does not fail automatically when the number of parameters of a neural network is large. It can fail when a neural network’s structure or neuron activations are strongly heterogenous, and we introduce an adaptive Monte Carlo algorithm (aMC) to overcome these limitations. The intrinsic stochasticity and numerical stability of the MC method allow aMC to train deep neural networks and recurrent neural networks in which the gradient is too small or too large to allow training by GD. MC methods offer a complement to gradient-based methods for training neural networks, allowing access to a distinct set of network architectures and principles.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Searching for an optimal control in the presence of saddles on the quantum-mechanical observable landscape

The broad success of theoretical and experimental quantum optimal control is intimately connected to the topology of the underlying control landscape. For several common quantum control goals, including the maximization of an observable expectation value, the landscape has been shown to lack local optima if three assumptions are satisfied: (i) the quantum system is controllable, (ii) the Jacobian of the map from the control field to the evolution operator is full rank, and (iii) the control field is not constrained. In the case of the observable objective, this favorable analysis shows that the associated landscape also contains saddles, i.e., critical points that are not local suboptimal extrema. In this paper, we investigate whether the presence of these saddles affects the trajectories of gradient-based searches for an optimal control. We show through simulations that both the detailed topology of the control landscape and the parameters of the system Hamiltonian influence whether the searches are attracted to a saddle. For some circumstances with a special initial state and target observable, optimizations may approach a saddle very closely, reducing the efficiency of the gradient algorithm. Encounters with such attractive saddles are found to be quite rare. Neither the presence of a large number of saddles on the control landscape nor a large number of system states increases the likelihood that a search will closely approach a saddle. Finally, even for applications that encounter a saddle, well-designed gradient searches with carefully chosen algorithmic parameters will readily locate optimal controls.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2023

Optimal Power Flow With State Estimation in the Loop for Distribution Networks

Here in this article, we propose a framework for running optimal control-estimation synthesis in distribution networks. Our approach combines a primal-dual gradient-based optimal power flow solver with a state estimation feedback loop based on a limited set of sensors for system monitoring, instead of assuming exact knowledge of all states. The estimation algorithm reduces uncertainty on unmeasured grid states based on certain online state measurements and noisy "pseudomeasurements." We analyze the convergence of the proposed algorithm and quantify the statistical estimation errors based on a weighted least-squares estimator. The numerical results on a 4521-node network demonstrate that this approach can scale to extremely large networks and provide robustness to both large pseudomeasurement variability and inherent sensor measurement noise.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Dynamic Differential Image Circle Diameter Measurement Precision Assessment: Application to Burning Droplets

Dynamic measurement precision assessment has been achieved for a differential circle measurement application. Differential circle diameter measurement, in image analysis, typically requires fitting a circle model that optimizes for image distortions, defects or occlusions. The differential task occurs when precise measurements of diameter change are required given object size variation with time. An automated system was designed to provide diameter measurements and associated measurement precision of images of a fuel droplet undergoing combustion in zero gravity for the FLEX-2 dataset. An image gradient-based, least-squares boundary point fitting method to a circle or ellipse model is used for diameter measurement. The presence of soot aggregates poses significant challenges for diameter measurements when it occludes part of the droplet boundary. The precision of the diameter measurements depends upon the image quality. Using synthetic image simulations that model the soot behavior, we developed a model based on image quality measures that assesses the measurement precision for each individual diameter measurement. Thus, diameter measurements with precision assessments were made available for follow-up scientific analysis. As a result, the algorithm's success rate for measurable runs was 98%. In cases of limited occlusion, a measurement precision of ±0.2 pixels for the FLEX-2 dataset was achieved.

42 ENGINEERING↗
DOE OSTI2021

Stochastic Gradient-Based Distributed Bayesian Estimation in Cooperative Sensor Networks

Distributed Bayesian inference provides a full quantification of uncertainty offering numerous advantages over point estimates that autonomous sensor networks are able to exploit. However, fully-decentralized Bayesian inference often requires large communication overheads and low network latency, resources that are not typically available in practical applications. In this paper, we propose a decentralized Bayesian inference approach based on stochastic gradient Langevin dynamics, which produces full posterior distributions at each of the nodes with significantly lower communication overhead. We provide analytical results on convergence of the proposed distributed algorithm to the centralized posterior, under typical network constraints. Finally, we also provide extensive simulation results to demonstrate the validity of the proposed approach.

42 ENGINEERING↗
DOE OSTI2015

Position tracking of moving liver lesion based on real-time registration between 2D ultrasound and 3D preoperative images

Purpose: Registration between 2D ultrasound (US) and 3D preoperative magnetic resonance (MR) (or computed tomography, CT) images has been studied recently for US-guided intervention. However, the existing techniques have some limits, either in the registration speed or the performance. The purpose of this work is to develop a real-time and fully automatic registration system between two intermodal images of the liver, and subsequently an indirect lesion positioning/tracking algorithm based on the registration result, for image-guided interventions. Methods: The proposed position tracking system consists of three stages. In the preoperative stage, the authors acquire several 3D preoperative MR (or CT) images at different respiratory phases. Based on the transformations obtained from nonrigid registration of the acquired 3D images, they then generate a 4D preoperative image along the respiratory phase. In the intraoperative preparatory stage, they properly attach a 3D US transducer to the patient’s body and fix its pose using a holding mechanism. They then acquire a couple of respiratory-controlled 3D US images. Via the rigid registration of these US images to the 3D preoperative images in the 4D image, the pose information of the fixed-pose 3D US transducer is determined with respect to the preoperative image coordinates. As feature(s) to use for the rigid registration, they may choose either internal liver vessels or the inferior vena cava. Since the latter is especially useful in patients with a diffuse liver disease, the authors newly propose using it. In the intraoperative real-time stage, they acquire 2D US images in real-time from the fixed-pose transducer. For each US image, they select candidates for its corresponding 2D preoperative slice from the 4D preoperative MR (or CT) image, based on the predetermined pose information of the transducer. The correct corresponding image is then found among those candidates via real-time 2D registration based on a gradient-based similarity measure. Finally, if needed, they obtain the position information of the liver lesion using the 3D preoperative image to which the registered 2D preoperative slice belongs. Results: The proposed method was applied to 23 clinical datasets and quantitative evaluations were conducted. With the exception of one clinical dataset that included US images of extremely low quality, 22 datasets of various liver status were successfully applied in the evaluation. Experimental results showed that the registration error between the anatomical features of US and preoperative MR images is less than 3 mm on average. The lesion tracking error was also found to be less than 5 mm at maximum. Conclusions: A new system has been proposed for real-time registration between 2D US and successive multiple 3D preoperative MR/CT images of the liver and was applied for indirect lesion tracking for image-guided intervention. The system is fully automatic and robust even with images that had low quality due to patient status. Through visual examinations and quantitative evaluations, it was verified that the proposed system can provide high lesion tracking accuracy as well as high registration accuracy, at performance levels which were acceptable for various clinical applications.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2018

Optimization Methods for Large-Scale Machine Learning

Here this paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications. Through case studies on text classification and the training of deep neural networks, we discuss how optimization problems arise in machine learning and what makes them challenging. A major theme of our study is that large-scale machine learning represents a distinctive setting in which the stochastic gradient (SG) method has traditionally played a central role while conventional gradient-based nonlinear optimization techniques typically falter. Based on this viewpoint, we present a comprehensive theory of a straightforward, yet versatile SG algorithm, discuss its practical behavior, and highlight opportunities for designing algorithms with improved performance. This leads to a discussion about the next generation of optimization methods for large-scale machine learning, including an investigation of two main streams of research on techniques that diminish noise in the stochastic directions and methods that make use of second-order derivative approximations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Quandary

Quandary numerically simulates and optimizes the time-evolution of open quantum systems. The underlying dynamics are modelled by Lindblad's master equation, a linear ordinary differential equation (ODE) describing quantum systems interacting with the environment. Quandary solves this ODE numerically by applying a time-stepping integration scheme, and utilizes a gradient-based optimization approach to determine optimal control pulses that drive the quantum system to a desired target state. Two optimization objectives are considered: (a) Unitary gate optimization that finds controls to realize a unitary gate transformation, and (b) optimal reset that aims to drive the quantum system to the ground states. Gradient-based optimization schemes utilizing Petsc's Tao optimization package are applied to generate control pulses that minimize the respective measure. To evaluate the gradient of the objective function, the discrete adjoint method is used while leveraging techniques from Algorithmic Differentiation to produce exact and consistent gradients. To mitigate excessive execution run times, the software can be build together with the XBraid software library which provides a parallelization strategy to distribute the time-evolution of the underlying dynamics onto multiple processor.

Petersson, NilsA.↗
DOE OSTI2023

Explaining machine-learning models for gamma-ray detection and identification

As more complex predictive models are used for gamma-ray spectral analysis, methods are needed to probe and understand their predictions and behavior. Recent work has begun to bring the latest techniques from the field of Explainable Artificial Intelligence (XAI) into the applications of gamma-ray spectroscopy, including the introduction of gradient-based methods like saliency mapping and Gradient-weighted Class Activation Mapping (Grad-CAM), and black box methods like Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP). In addition, new sources of synthetic radiological data are becoming available, and these new data sets present opportunities to train models using more data than ever before. In this work, we use a neural network model trained on synthetic NaI(Tl) urban search data to compare some of these explanation methods and identify modifications that need to be applied to adapt the methods to gamma-ray spectral data. We find that the black box methods LIME and SHAP are especially accurate in their results, and recommend SHAP since it requires little hyperparameter tuning. We also propose and demonstrate a technique for generating counterfactual explanations using orthogonal projections of LIME and SHAP explanations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Quantum Machine Learning with SQUID

In this work we present the Scaled QUantum IDentifier (SQUID), an open-source framework for exploring hybrid Quantum-Classical algorithms for classification problems. The classical infrastructure is based on PyTorch and we provide a standardized design to implement a variety of quantum models with the capability of back-propagation for efficient training. We present the structure of our framework and provide examples of using SQUID in a standard binary classification problem from the popular MNIST dataset. In particular, we highlight the implications for scalability for gradient-based optimization of quantum models on the choice of output for variational quantum models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Liquid Crystal Orientation and Shape Optimization for the Active Response of Liquid Crystal Elastomers

Liquid crystal elastomers (LCEs) are responsive materials that can undergo large reversible deformations upon exposure to external stimuli, such as electrical and thermal fields. Controlling the alignment of their liquid crystals mesogens to achieve desired shape changes unlocks a new design paradigm that is unavailable when using traditional materials. While experimental measurements can provide valuable insights into their behavior, computational analysis is essential to exploit their full potential. Accurate simulation is not, however, the end goal; rather, it is the means to achieve their optimal design. Such design optimization problems are best solved with algorithms that require gradients, i.e., sensitivities, of the cost and constraint functions with respect to the design parameters, to efficiently traverse the design space. In this work, a nonlinear LCE model and adjoint sensitivity analysis are implemented in a scalable and flexible finite element-based open source framework and integrated into a gradient-based design optimization tool. To display the versatility of the computational framework, LCE design problems that optimize both the material, i.e., liquid crystal orientation, and structural shape to reach a target actuated shapes or maximize energy absorption are solved. Multiple parameterizations, customized to address fabrication limitations, are investigated in both 2D and 3D. The case studies are followed by a discussion on the simulation and design optimization hurdles, as well as potential avenues for improving the robustness of similar computational frameworks for applications of interest.

42 ENGINEERING↗
DOE OSTI2026

Classical-Quantum Algorithm for Solving Stochastic Programs

Stochastic programming provides a rigorous mathematical framework for making decisions under uncertainty in a risk-aware manner. Two-stage stochastic programming is, perhaps, the simplest form of this framework. Here the first-stage variables represent decisions that must be made "here and now" in the face of uncertainty, while the second-stage variables are decisions made after uncertain events. However, the broad adoption of stochastic programming has been hindered by computational challenges caused by the two-stage stochastic programming formulation which requires solving an ensemble of optimization problems. Using quantum amplitude estimation (QAE), quantum computers have shown the theoretic ability to compute expectations with Monte-Carlo methods with quadratically fewer samples than classical methods. In this work, we present a quantum algorithm for computing the expectation term using QAE for given first-stage decisions. Further, we detail methods of computing gradient information from the quantum calculation enabling the application of classical gradient-based optimization techniques. The result is a classical-quantum hybrid method of solving two-stage stochastic programs. These techniques are demonstrated with computational experiments based an engineering optimization problem.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1992

Observations on computational methodologies for use in large-scale, gradient-based, multidisciplinary design incorporating advanced CFD codes

How a combination of various computational methodologies could reduce the enormous computational costs envisioned in using advanced CFD codes in gradient based optimized multidisciplinary design (MdD) procedures is briefly outlined. Implications of these MdD requirements upon advanced CFD codes are somewhat different than those imposed by a single discipline design. A means for satisfying these MdD requirements for gradient information is presented which appear to permit: (1) some leeway in the CFD solution algorithms which can be used; (2) an extension to 3-D problems; and (3) straightforward use of other computational methodologies. Many of these observations have previously been discussed as possibilities for doing parts of the problem more efficiently; the contribution here is observing how they fit together in a mutually beneficial way.

Newman, P. A.↗
NASA NTRS1992

Observations on computational methodologies for use in large-scale, gradient-based, multidisciplinary design

Various computational methodologies relevant to large-scale multidisciplinary gradient-based optimization for engineering systems design problems are examined with emphasis on the situation where one or more discipline responses required by the optimized design procedure involve the solution of a system of nonlinear partial differential equations. Such situations occur when advanced CFD codes are applied in a multidisciplinary procedure for optimizing an aerospace vehicle design. A technique for satisfying the multidisciplinary design requirements for gradient information is presented. The technique is shown to permit some leeway in the CFD algorithms which can be used, an expansion to 3D problems, and straightforward use of other computational methodologies.

Newman, P. A.↗