Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Automatic Differentiation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Coupled Time-Lapse Full-Waveform Inversion for Subsurface Flow Problems Using Intrusive Automatic Differentiation

We describe a novel framework for estimating subsurface properties, such as rock permeability and porosity, from time-lapse observed seismic data by coupling full-waveform inversion (FWI), subsurface flow processes, and rock physics models. For the inverse modeling, we handle the back propagation of gradients by an intrusive automatic differentiation strategy that offers three levels of user control: (1) At the wave physics level, we adopted the discrete adjoint method in order to use our existing high-performance FWI code; (2) at the rock physics level, we used built-in automatic differentiation operators from the TensorFlow backend; (3) at the flow physics level, we implemented customized partial differential equation (PDE) operators for the multiphase flow equations. The three-level coupled inversion strategy strikes a good balance between computational efficiency and programming efforts, and when the gradients are chained together, it constitutes a coupled inverse system. Our numerical experiments demonstrate that the three-level coupled inverse problem is superior in terms of accuracy to a traditional decoupled inversion strategy. Additionally, our method is able to simultaneously invert for parameters in empirical relationships such as the rock physics models. Our proposed inverted model can be used for reservoir performance prediction and reservoir management/optimization purposes.

54 ENVIRONMENTAL SCIENCES↗

First- and Second-Order Sensitivity Analysis of a P-Version Finite Element Equation Via Automatic Differentiation

Sensitivity analysis is a technique for determining derivatives of system responses with respect to design parameters. Among many methods available for sensitivity analysis, automatic differentiation has been proven through many applications in fluid dynamics and structural mechanics to be an accurate and easy method for obtaining derivatives. Nevertheless, the method can be computational expensive and can require a high memory space. This project will apply an automatic differentiation tool, ADIFOR, to a p-version finite element code to obtain first- and second- order then-nal derivatives, respectively. The focus of the study is on the implementation process and the performance of the ADIFOR-enhanced codes for sensitivity analysis in terms of memory requirement, computational efficiency, and accuracy.

Hou, Gene↗

Uncertainty Quantification Enabled by Automatic Differentiation for Hydrodynamic Simulation of Shock‐to‐Detonation Transition in High Explosives

Quantifying the effects of uncertainty in a reactive burn model on the run-to-detonation time in high explosives (HEs) provides a robust methodology for assessing the probability of an HE failing the IHE qualification standard. Moreover, uncertainty quantification helps evaluate whether the model calibration accurately represents data outside the calibration set. This study uses a specialized hydrodynamic simulation code for modeling detonation to determine the run-to-detonation time of the HE PBX 9502 for various impact velocities. To quickly approximate uncertainties in the model, a surrogate was constructed using a Taylor series expansion centered at the mean of the input parameters. To obtain the sensitivities required for constructing the Taylor series, HYP-percomplex Automatic Differentiation (HYPAD) was implemented. HYPAD is a methodology for infusing existing codes with automatic differentiation capabilities by augmenting variables with one or more imaginary units to compute step-size independent partial derivatives. These derivatives are accurate to machine precision with respect to the implemented numerical algorithm, meaning their accuracy reflects that of the underlying method (e.g., integration or discretization schemes). Using reduced order modeling techniques, the mean and standard deviation of the run-to-detonation time of a shock within PBX 9502 were computed for a number of initial impact velocities. A weighted least squares regression was then performed to obtain a best fit curve and prediction interval for the computed statistics. Historical data points from explosively driven wedge tests were utilized to validate the prediction interval, ensuring its reliability in predicting future outcomes. With this prediction interval and a known safety constraint curve, the most probable point of failure and the probability of failure for the HE PBX 9502 were determined.

97 MATHEMATICS AND COMPUTING↗

Q-Law for Rapid Assessment of Low Thrust Cislunar Trajectories Via Automatic Differentiation

Q-Law is a Lyapunov-based control law used to determine optimal controls for a low thrust trajectory. One major issue with its use is the difficult derivatives re-quired for calculating optimal controls at a given time. In this paper, an implementation of Q-Law with automatic differentiation via a Python package called JAX is applied. With automatic differentiation, the difficult derivatives for Q-Law’s optimal controls are calculated with ease, and derivatives of final states with respect to Q-Law’s weights are found enabling gradient-based optimization of Q-Law for the first time. Different search and optimization methods for finding optimal weights are then compared using the LEO to GEO problem, and it was found that gradient-free methods like design of experiments and genetic algorithm produced the best results, but they took the longest time to get a solution, while the gradient-based method found a locally optimal result in a much faster time. Overall, the run time for a single propagation is manageable and well-suited for a mission designer to use as an initial guess generator for trajectory optimization, or for simple orbit transfer analysis.

Nathan Steffen↗

Hypercomplex Automatic Differentiation in the Eulerian Hydrocode PAGOSA

Enabling the computation of partial derivatives or sensitivities in production hydrocodes is beneficial for design, optimization, sensitivity analysis, and uncertainty quantification. Traditional finite difference approximations of these sensitivities are inefficient since convergence studies of the step size is required for each parameter of interest. For these reasons, HYPercomplex Automatic Differentiation (HYPAD) was implemented in the Eulerian hydrocode PAGOSA. HYPAD is analogous to forward-mode automatic differentiation except hypercomplex numbers (numbers with multiple imaginary parts) are used instead of dual numbers. Accurate partial derivatives can be computed of all state variables with respect to multiple input variables in a single run. The method was implemented using operator overloading to handle hypercomplex algebra. HYPAD was demonstrated and verified on Sod’s shock tube problem to compute derivatives of the state variables with respect to a material parameter, initial conditions, and geometry.

97 MATHEMATICS AND COMPUTING↗

ZEUS: An Efficient GPU Optimization Method Integrating PSO, BFGS, and Automatic Differentiation

We introduce a novel, efficient computational method, ZEUS, for numerical optimization, and provide an open-source implementation. It has four key ingredients: (1) particle swarm optimization (PSO), (2) the use of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method, (3) automatic differentiation (AD), and (4) GPUs. Our approach addresses the computational challenges inherent in high-dimensional, non-convex optimization problems. In the first phase of the algorithm, we get a potentially good set of starting points using PSO. Thereafter, we run BFGS independently in parallel from these starting points. BFGS is one of the best-performing algorithms for numerical optimization. However, it requires the gradient of the function being optimized. ZEUS integrates automatic differentiation into BFGS thus avoiding the need for the user to calculate derivatives explicitly. The use of GPUs allows ZEUS to speed up the calculations substantially. We carry out systematic studies to explore the trade-offs between the number of PSO iterations taken, starting points, and BFGS iteration depth. We show that a handful of iterations of PSO can improve global convergence when combined with BFGS. We also present performance studies using common test functions. The source code can be found at https://github.com/fnal-numerics/global-optimizer-gpu.

Soos, Dominik [Old Dominion U.]↗

Optimum Shape Design Using Automatic Differentiation in Reverse Mode

This paper shows how to use automatic differentiation in reverse mode as a powerful tool in optimization procedures. It is also shown that for aerodynamic applications the gradients have to be as accurate as possible. In particular, the effect of having the exact gradient of he first or second order spatial discretization schemes is presented. We show that the loss of precision in the gradient affects not only the convergence, but also the final shape. Both two and three dimensional configurations of transonic and supersonic flows have been investigated. These cases involve up to several thousand control parameters.

Hafez, M.↗

OpenAD/F : a modular, open-source tool for automatic differentiation of Fortran codes.

The OpenAD/F tool allows the evaluation of derivatives of functions defined by a Fortran program. The derivative evaluation is performed by a Fortran code resulting from the analysis and transformation of the original program that defines the function of interest. OpenAD/F has been designed with a particular emphasis on modularity, flexibility, and the use of open source components. While the code transformation follows the basic principles of automatic differentiation, the tool implements new algorithmic approaches at various levels, for example, for basic block preaccumulation and call graph reversal. Unlike most other automatic differentiation tools, OpenAD/F uses components provided by the OpenAD framework, which supports a comparatively easy extension of the code transformations in a language-independent fashion. It uses code analysis results implemented in the OpenAnalysis component. The interface to the language-independent transformation engine is an XML-based format, specified through an XML schema. The implemented transformation algorithms allow efficient derivative computations utilizing locally optimized cross-country sequences of vertex, edge, and face elimination steps. Specifically, for the generation of adjoint codes, OpenAD/F supports various code reversal schemes with hierarchical checkpointing at the subroutine level. As an example from geophysical fluid dynamics a nonlinear time-dependent scalable, yet simple, barotropic ocean model is considered. OpenAD/F's reverse mode is applied to compute sensitivities of some of the model's transport properties with respect to gridded fields such as bottom topography as independent (control) variables.

54 ENVIRONMENTAL SCIENCES↗

Automatic differentiation of advanced CFD codes for multidisciplinary design

Automated multidisciplinary design of aircraft and other flight vehicles requires the optimization of complex performance objectives with respect to a number of design parameters and constraints. The effect of these independent design variables on the system performance criteria can be quantified in terms of sensitivity derivatives which must be calculated and propagated by the individual discipline simulation codes. Typical advanced CFD analysis codes do not provide such derivatives as part of a flow solution; these derivatives are very expensive to obtain by divided (finite) differences from perturbed solutions. It is shown that sensitivity derivatives can be obtained accurately and efficiently using the ADIFOR source translator for automatic differentiation. In particular, it is demonstrated that the 3-D, thin-layer Navier-Stokes, multigrid flow solver called TLNS3D is amenable to automatic differentiation in the forward mode even with its implicit iterative solution algorithm and complex turbulence modeling. It is significant that by using computational differentiation, consistent discrete nongeometric sensitivity derivatives have been obtained from an aerodynamic 3-D CFD code in a relatively short time, e.g., O(man-week) not O(man-year).

Bischof, C.↗

Qualitative and quantitative enhancement of parameter estimation for model-based diagnostics using automatic differentiation with an application to inertial fusion

Parameter estimation using observables is a fundamental concept in the experimental sciences. Mathematical models that represent the physical processes can enable reconstructions of the experimental observables and greatly assist in parameter estimation by turning it into an optimization problem which can be solved by gradient-free or gradient-based methods. In this work, the recent rise in flexible frameworks for developing differentiable scientific computing programs is leveraged in order to dramatically accelerate data analysis of a common experimental diagnostic relevant to laser–plasma and inertial fusion experiments, Thomson scattering. A differentiable Thomson-scattering data analysis tool is developed that uses reverse-mode automatic differentiation (AD) to calculate gradients. By switching from finite differencing to reverse-mode AD, three distinct outcomes are achieved. First, gradient descent is accelerated dramatically to the extent that it enables near real-time usage in laser–plasma experiments. Second, qualitatively novel quantities which require $\mathcal{O}(10^3)$ parameters can now be included in the analysis of data which enables unprecedented measurements of small-scale laser–plasma phenomena. Third, uncertainty estimation approaches that leverage the value of the Hessian become accurate and efficient because reverse-mode AD can be used for calculating the Hessian.

97 MATHEMATICS AND COMPUTING↗

Efficient GPU Implementation of Automatic Differentiation for Computational Fluid Dynamics

Many scientific and engineering applications require repeated calculations of derivatives of output functions with respect to input parameters. Automatic Differentiation (AD) is a method that automates derivative calculations and can significantly speed up code development. In Computational Fluid Dynamics (CFD), derivatives of flux functions with respect to state variables (Jacobian) are needed for efficient solutions of the nonlinear governing equations. AD of flux functions on graphics processing units (GPUs) is challenging as flux computations involve many intermediate variables that create high register pressure and require significant memory traffic because of the need to store the derivatives. This paper presents a forward-mode AD method based on multivariate dual numbers that addresses these challenges and simultaneously reduces the floating-point operation count. The dimension of the multivariate dual numbers is optimized for performance. The flux computations are restructured to minimize the number of temporary variables and reduce register pressure. For effective utilization of memory bandwidth, shared memory is used to store the local flux Jacobian. This AD implementation is compared with several other Jacobian implementations on an NVIDIA V100 GPU (V100). For three-dimensional perfect-gas compressible-flow equations implemented in a practical CFD code, the AD implementation of a flux Jacobian based on multivariate dual numbers of dimension 5 outperforms all other GPU AD implementations on V100. Its performance is comparable with the optimized hand-differentiated version. Finally, the implementation achieves 75% of the peak floating-point throughput and 61 % of the peak global device memory bandwidth usage.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A low rank approach to automatic differentiation.

This manuscript introduces a new approach for increasing the efficiency of automatic differentiation (AD) computations for estimating the first order derivatives comprising the Jacobian matrix of a complex large-scale computational model. The objective is to approximate the entire Jacobian matrix with minimized computational and storage resources. This is achieved by finding low rank approximations to a Jacobian matrix via the Efficient Subspace Method (ESM). Low rank Jacobian matrices arise in many of today's important scientific and engineering problems, e.g. nuclear reactor calculations, weather climate modeling, geophysical applications, etc. A low rank approximation replaces the original Jacobian matrix J (whose size is dictated by the size of the input and output data streams) with matrices of much smaller dimensions (determined by the numerical rank of the Jacobian matrix). This process reveals the rank of the Jacobian matrix and can be obtained by ESM via a series of r randomized matrix-vector products of the form: Jq, and J{sup T} {omega} which can be evaluated by the AD forward and reverse modes, respectively.

97 MATHEMATICS AND COMPUTING↗

Automatic differentiation for design sensitivity analysis of structural systems using multiple processors

An automatic differentiation tool (ADIFOR) is incorporated into a finite element based structural analysis program for shape and non-shape design sensitivity analysis of structural systems. The entire analysis and sensitivity procedures are parallelized and vectorized for high performance computation. Small scale examples to verify the accuracy of the proposed program and a medium scale example to demonstrate the parallel vector performance on multiple CRAY C90 processors are included.

Nguyen, Duc T.↗

Automatic differentiation of codes in nuclear engineering applications.

We discuss our experience in applying automatic differentiation (AD) to calculations in nuclear reactor applications. The document is intended as a guideline on how to apply AD to Fortran codes with significant legacy components; it is also a part of a larger research effort in uncertainty quantification using sampling methods augmented with derivative information. We provide a brief theoretical description of the concept of AD, explain the necessary changes in the code structure, and remark on possible ways to deal with non-differentiability. Numerical experiments were carried out where the derivative of a functional subset of the SAS4A/SASSYS code was computed in forward mode with several AD tools. The results are in good agreement with both the real and complex finite-difference approximations of the derivative.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Low rank approach to computing first and higher order derivatives using automatic differentiation

This manuscript outlines a new approach for increasing the efficiency of applying automatic differentiation (AD) to large scale computational models. By using the principles of the Efficient Subspace Method (ESM), low rank approximations of the derivatives for first and higher orders can be calculated using minimized computational resources. The output obtained from nuclear reactor calculations typically has a much smaller numerical rank compared to the number of inputs and outputs. This rank deficiency can be exploited to reduce the number of derivatives that need to be calculated using AD. The effective rank can be determined according to ESM by computing derivatives with AD at random inputs. Reduced or pseudo variables are then defined and new derivatives are calculated with respect to the pseudo variables. Two different AD packages are used: OpenAD and Rapsodia. OpenAD is used to determine the effective rank and the subspace that contains the derivatives. Rapsodia is then used to calculate derivatives with respect to the pseudo variables for the desired order. The overall approach is applied to two simple problems and to MATWS, a safety code for sodium cooled reactors. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Optimized finite-build stellarator coils using automatic differentiation

A new stellarator coil design code is introduced that optimizes the position and winding pack orientation of finite-build coils. The new code, called flexible optimized curves in space using automatic differentiation (AD) and finite build (FOCUSADD), performs gradient-based optimization in a high-dimensional, non-convex space. The derivatives with respect to parameters of finite-build coils are easily and efficiently computed using AD. FOCUSADD parametrizes coil positions in free space using a Fourier series and uses a multi-filament approximation to the coil winding pack. The orientation of the winding pack is parametrized with a Fourier series and can be optimized as well. Optimized finite-build coils for a Wendelstein 7-X (W7-X)-like stellarator are found, and compared with filamentary coil results. The final positions of optimized finite-build W7-X-like coils are shifted, on average, by approximately 2.5 mm relative to optimized filamentary coils. These results suggest that finite-build effects should be accounted for in the optimization of stellarators with low coil tolerances.

43 PARTICLE ACCELERATORS↗

A Framework for Distributed Approximation of Moments with Higher-Order Derivatives through Automatic Differentiation

We present a framework for the distributed approximation of moments, enabling the evaluation of the uncertainty in a dynamical system. The first and second moment, mean, and variance are computed with up to third-order Taylor series expansion. The required derivatives for the expansion are generated automatically by automatic differentiation and propagated through an implicit time stepper. The computational kernels are the accumulation of the derivatives (Jacobian, Hessian, tensor) and the covariance matrix. We apply distributed parallelism to the Hessian or third-order tensor, and the user merely has to provide a function for the differential equation, thus achieving similar ease of use as Monte Carlo-based methods. We demonstrate our approach using with benchmarks on Theta, a KNL-based system at the Argonne Leadership Computing Facility.

Automatic Differentiation↗

Automatic differentiation evaluated as a tool for rotorcraft design and optimization

This paper investigates the use of automatic differentiation (AD) as a means for generating sensitivity analyses in rotorcraft design and optimization. This technique transforms an existing computer program into a new program that performs sensitivity analysis in addition to the original analysis. The original FORTRAN program calculates a set of dependent (output) variables from a set of independent (input) variables, the new FORTRAN program calculates the partial derivatives of the dependent variables with respect to the independent variables. The AD technique is a systematic implementation of the chain rule of differentiation, this method produces derivatives to machine accuracy at a cost that is comparable with that of finite-differencing methods. For this study, an analysis code that consists of the Langley-developed hover analysis HOVT, the comprehensive rotor analysis CAMRAD/JA, and associated preprocessors is processed through the AD preprocessor ADIFOR 2.0. The resulting derivatives are compared with derivatives obtained from finite-differencing techniques. The derivatives obtained with ADIFOR 2.0 are exact within machine accuracy and do not depend on the selection of step-size, as are the derivatives obtained with finite-differencing techniques.

Walsh, Joanne L.↗