Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Automatic Differentiation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS1994

Methodology for sensitivity analysis, approximate analysis, and design optimization in CFD for multidisciplinary applications

The straightforward automatic-differentiation and the hand-differentiated incremental iterative methods are interwoven to produce a hybrid scheme that captures some of the strengths of each strategy. With this compromise, discrete aerodynamic sensitivity derivatives are calculated with the efficient incremental iterative solution algorithm of the original flow code. Moreover, the principal advantage of automatic differentiation is retained (i.e., all complicated source code for the derivative calculations is constructed quickly with accuracy). The basic equations for second-order sensitivity derivatives are presented; four methods are compared. Each scheme requires that large systems are solved first for the first-order derivatives and, in all but one method, for the first-order adjoint variables. Of these latter three schemes, two require no solutions of large systems thereafter. For the other two for which additional systems are solved, the equations and solution procedures are analogous to those for the first order derivatives. From a practical viewpoint, implementation of the second-order methods is feasible only with software tools such as automatic differentiation, because of the extreme complexity and large number of terms. First- and second-order sensitivities are calculated accurately for two airfoil problems, including a turbulent flow example; both geometric-shape and flow-condition design variables are considered. Several methods are tested; results are compared on the basis of accuracy, computational time, and computer memory. For first-order derivatives, the hybrid incremental iterative scheme obtained with automatic differentiation is competitive with the best hand-differentiated method; for six independent variables, it is at least two to four times faster than central finite differences and requires only 60 percent more memory than the original code; the performance is expected to improve further in the future.

Taylor, Arthur C., III↗
DOE OSTI2023

Reconstruction of simulated electrostatic potentials by automatic differentiation-based phase retrieval in electron microscopy imaging

Measuring interfacial electrostatic potentials is vital to understanding many fundamental materials properties. A variety of TEM methods exist for measuring electric potentials from the phase shift produced on an electron wave as it passes through a sample. However, most are either experimentally challenging or poorly suited to resolving nanoscale features. Here, we demonstrate the viability of a simple, automatic differentiation-based exit wave reconstruction from a focal series of images to accurately measure nanoscale electric potentials. Here, the analysis suggests that under optimal measurement conditions, electric potentials can be resolved to less than 0.06 V in magnitude and less than 1 nm in spatial extent.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Degrees of rate control and AutoDiff-driven direct sensitivity analysis in heterogeneous catalysis

Despite the wide application and benefits of the degree of rate control (DRC) analysis, several details remain argued, particularly about the conservation of DRCs at transient (TR) and steady-state (SS) conditions, especially for complex reaction networks. This work argues that previous proofs about the conservation properties of DRCs have been incomplete, and we provide new mathematical proofs at TR and SS conditions. In addition, we use both analytical (automatic differentiation) and numerical (finite difference) approaches to compute DRCs for the case study of ethane hydrogenolysis (EH) over Pt(111). This work confirms that at both TR and SS conditions, the sum of all DRCs, i.e., sum of the degrees of kinetic (DKRC) and thermodynamic rate control (DTRC), is conserved at zero. At SS conditions, the sum of DKRC is conserved at 1 while the sum of DTRC is conserved at −1. In corroboration of previous works, we show that the DTRC for any adsorbate at SS is equal to the product of the species coverage and a constant. In contrast, at TR conditions, the individual sums of both DTRC and DKRC are not conserved and can be any real number, with potential implications for the novel field of dynamic catalysis. Finally, we show that the conventional finite difference (FD) approach, only useful at SS, is prone to inaccuracy and very sensitive to the value of the differential change applied. The optimal differential value also varies significantly with system and rate definition. Consequently, we describe and illustrate in this work the application of the automatic differentiation (AD) approach for the more accurate determination of DRCs at both TR and SS conditions.

Automatic differentiation↗
DOE OSTI2025

JuTrack: A Julia package for auto-differentiable accelerator modeling and particle tracking

Efficient accelerator modeling and particle tracking are key for the design and configuration of modern particle accelerators. In this work, we present JuTrack, a nested accelerator modeling package developed in the Julia programming language and enhanced with compiler-level automatic differentiation (AD). With the aid of AD, JuTrack enables rapid derivative calculations in accelerator modeling, facilitating sensitivity analyses and optimization tasks. Here we demonstrate the effectiveness of AD-derived derivatives through several practical applications, including sensitivity analysis of space-charge-induced emittance growth, nonlinear beam dynamics analysis for a synchrotron light source, and lattice parameter tuning of the future Electron-Ion Collider (EIC). Through the incorporation of automatic differentiation, this package opens up new possibilities for accelerator physicists in beam physics studies and accelerator design optimization.

43 PARTICLE ACCELERATORS↗
DOE OSTI2022

Differentiable quantum chemistry with PySCF for molecules and materials at the mean-field level and beyond

We introduce an extension to the PySCF package, which makes it automatically differentiable. The implementation strategy is discussed, and example applications are presented to demonstrate the automatic differentiation framework for quantum chemistry methodology development. These include orbital optimization, properties, excited-state energies, and derivative couplings, at the mean-field level and beyond, in both molecules and solids. We also discuss some current limitations and directions for future work.

Chemistry↗
DOE OSTI2021

End-to-End Differentiable Modeling and Management of the Environment

Focal Area: (2) Data acquisition and assimilation enabled by machine learning, AI, and advanced methods including experimental/network design/optimization. We emphasize the importance of leveraging optimization techniques from AI/machine learning (ML) to solve challenging problems in Earth system modeling. Science Challenge: Automatic differentiation has had a transformative effect on ML by allowing the calculation of gradients of arbitrary functions in an incredibly large class of models. We can potentially realize similar improvements in parameter estimation and control for Earth system models (ESMs) by reimplementing them in computational frameworks from ML. Practitioners working with large (>10 7 parameters) models in ML and AI can obtain good predictive performance in a range of spatiotemporal tasks by making use of optimization via stochastic gradient descent and incorporating prior knowledge at multiple levels. We propose writing ESMs in open-source computational frameworks such as Torch, Tensorflow, and JAX to greatly expand the scope of environmental forecasting and management challenges, which can be addressed by leveraging automatic differentiation and gradient descent-like algorithms. We do not call for a wholesale replacement of physical models with data-driven surrogates, but rather advocate for interleaving physical and empirical equations in a manner that is most faithful to the extent of our scientific knowledge and observational data. Central to this topic is the merging of differentiable physical simulations with differentiable optimization layers, which are now both beginning to come to the forefront.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Hutchinson Trace Estimation for high-dimensional and high-order Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) have proven effective in solving partial differential equations (PDEs), especially when some data are available by seamlessly blending data and physics. However, extending PINNs to high-dimensional and even high-order PDEs encounters significant challenges due to the computational cost associated with automatic differentiation in the residual loss function calculation. Herein, we address the limitations of PINNs in handling high-dimensional and high-order PDEs by introducing the Hutchinson Trace Estimation (HTE) method. Starting with the second-order high-dimensional PDEs, which are ubiquitous in scientific computing, HTE is applied to transform the calculation of the entire Hessian matrix into a Hessian vector product (HVP). This approach not only alleviates the computational bottleneck via Taylor-mode automatic differentiation but also significantly reduces memory consumption from the Hessian matrix to an HVP’s scalar output. We further showcase HTE’s convergence to the original PINN loss and its unbiased behavior under specific conditions. Comparisons with the Stochastic Dimension Gradient Descent (SDGD) highlight the distinct advantages of HTE, particularly in scenarios with significant variability and variance among dimensions. We further extend the application of HTE to higher-order and higher-dimensional PDEs, specifically addressing the biharmonic equation. By employing tensor-vector products (TVP), HTE efficiently computes the colossal tensor associated with the fourth-order high-dimensional biharmonic equation, saving memory and enabling rapid computation. The effectiveness of HTE is illustrated through experimental setups, demonstrating comparable convergence rates with SDGD under memory and speed constraints. Additionally, HTE proves valuable in accelerating the Gradient-Enhanced PINN (gPINN) version as well as the Biharmonic equation. Overall, HTE opens up a new capability in scientific machine learning for tackling high-order and high-dimensional PDEs.

Curse of dimensionality↗
DOE OSTI2026

Ground and excited state gradients with end-to-end differentiable semiempirical quantum chemistry

Accurate and efficient gradients of molecular energy with respect to nuclear degrees of freedom are essential for geometry optimization and molecular dynamics, including simulations that go beyond the Born–Oppenheimer regime. A common approach involves deriving analytical formulas for new electronic structure methods, which is often conceptually difficult and requires tedious coding. Here, we implement analytical, semi-numerical, and automatic differentiation (AD)-based gradient pathways for semiempirical Hamiltonian models in the PYSEQM software package, leveraging both graphics processing unit (GPU) and central processing unit (CPU) architectures. We further extend these capabilities to excited states calculated using the configuration interaction singles and time-dependent Hartree–Fock ansätze. We benchmark wall time, peak memory usage, and accuracy across three molecular families of varying chemical complexity, including systems of up to a thousand atoms. For ground-state simulations, analytical and AD gradients achieve near-identical GPU runtimes, while semi-numerical gradients are slower on GPU but remain competitive on CPU. For excited states, both analytical and custom AD approaches using implicit differentiation show similar performance and low memory requirements, whereas gradients with full AD are memory-limited. AD gradients match analytical ones in accuracy across all tested systems, aided by a quaternion-based diatomic frame rotation for two-center quantities that ensures smooth energy surfaces. Overall, automatic differentiation emerges as a practical alternative to analytical gradients in semiempirical quantum chemistry, offering high accuracy while allowing seamless integration in AI-driven workflows and popular packages, such as PyTorch and JAX. Our results provide actionable guidance for selecting optimal gradient strategies in large-scale ground- and excited-state molecular dynamics simulations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

DPFEHM: a differentiable subsurface physics simulator

The Earth’s subsurface is a key resource that provides energy via fossil fuels and geothermal sources, stores drinking water, and is used in the fight against climate change via carbon sequestration. Simulating the physical processes that occur in the Earth’s subsurface with computers enables better use of this resource. DPFEHM is a Julia package that includes computer models with a focus on the Earth’s subsurface, especially fluid flow, which is critical for the aforementioned applications. DPFEHM is able to solve the groundwater flow equations (single phase flow), Richards equation (air/water), the advection-dispersion equation, and the 2d wave equation. One of the key features of DPFEHM is that it supports automatic differentiation, so it can be integrated into machine learning workflows using frameworks such as Flux or PyTorch. The automatic differentiation capabilities give it the same performance as adjoint methods.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

jaxhps: An elliptic PDE solver built with machine learning in mind

Elliptic partial differential equations (PDEs) can model many physical phenomena, such as electrostatics, acoustics, wave propagation, and diffusion. In scientific machine learning settings, a high-throughput PDE solver may be required to generate a training dataset, run in the inner loop of an iterative algorithm, or interface directly with a deep neural network. To provide value to machine learning users, such a PDE solver must be compatible with standard automatic differentiation frameworks, scale efficiently when run on graphics processing units (GPUs), and maintain high accuracy for a large range of input parameters. We have designed the jaxhps package with these use-cases in mind by implementing a highly efficient and accurate solver for elliptic problems with native hardware acceleration and automatic differentiation support.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

On the feasibility of using physics-informed machine learning for underground reservoir pressure management

In this work, we evaluate the feasibility of using physics-informed machine learning (PIML) for underground energy-related pressure management. To this end, we develop a PIML framework to manage underground reservoir pressures by training neural networks to determine fluid extraction rates for dedicated extraction wells during fluid injection operations given a range of reservoir conditions (e.g., transmissivity and storativity). We implement an automatically-differentiable analytical physics model of fluid flow in porous media within the PIML framework as a proxy for more complicated models. This allows us to execute a sufficient number of training scenarios to fully evaluate the feasibility of using PIML to support pressure management activities. We quantify the number of physics-model parameters required for automatic differentiation to become more efficient than finite-difference gradient calculations. We use a simple scenario with a single injector, extractor, and critical location for our feasibility analysis. We evaluate the effect of the size of the training dataset (i.e., the number of reservoir condition samples) on the accuracy and efficiency of the PIML framework. For an equivalent number of model evaluations, the larger training dataset took less time to train and produced a neural network that was able to more accurately manage reservoir pressures. We also evaluate the effect of the training dataset batch size (i.e., number of reservoir condition samples used to update the neural network coefficients during training; i.e., how the training dataset is partitioned). While training ran faster with larger batch sizes, they produced neural networks that managed pressures less accurately. We demonstrate the approach on a more complex scenario involving 10 injectors, 10 extractors, and 4 critical locations (a relatively high well density of 20 wells/km2). We provide the number of forward and adjoint model evaluations required in each case as an indication of the feasibility of using PIML for pressure management when more complicated physics models with longer execution times are used.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2000

Steady-State Computation of Constant Rotational Rate Dynamic Stability Derivatives

Dynamic stability derivatives are essential to predicting the open and closed loop performance, stability, and controllability of aircraft. Computational determination of constant-rate dynamic stability derivatives (derivatives of aircraft forces and moments with respect to constant rotational rates) is currently performed indirectly with finite differencing of multiple time-accurate computational fluid dynamics solutions. Typical time-accurate solutions require excessive amounts of computational time to complete. Formulating Navier-Stokes (N-S) equations in a rotating noninertial reference frame and applying an automatic differentiation tool to the modified code has the potential for directly computing these derivatives with a single, much faster steady-state calculation. The ability to rapidly determine static and dynamic stability derivatives by computational methods can benefit multidisciplinary design methodologies and reduce dependency on wind tunnel measurements. The CFL3D thin-layer N-S computational fluid dynamics code was modified for this study to allow calculations on complex three-dimensional configurations with constant rotation rate components in all three axes. These CFL3D modifications also have direct application to rotorcraft and turbomachinery analyses. The modified CFL3D steady-state calculation is a new capability that showed excellent agreement with results calculated by a similar formulation. The application of automatic differentiation to CFL3D allows the static stability and body-axis rate derivatives to be calculated quickly and exactly.

Park, Michael A.↗
DOE OSTI2021

Differentiable programming for online training of a neural artificial viscosity function within a staggered grid Lagrangian hydrodynamics scheme

Lagrangian methods to solve the inviscid Euler equations produce numerical oscillations near shock waves. A common approach to reducing these oscillations is to add artificial viscosity (AV) to the discrete equations. The AV term acts as a dissipative mechanism that attenuates oscillations by smearing the shock across a finite number of computational cells. However, AV introduces several control parameters that are not determined by the underlying physical model, and hence, in practice are tuned to the characteristics of a given problem. We seek to improve the standard quadratic-linear AV form by replacing it with a learned neural function that reduces oscillations relative to exact solutions of the Euler equations, resulting in a hybrid numerical-neural hydrodynamic solver. Because AV is an artificial construct that exists solely to improve the numerical properties of a hydrodynamic code, there is no offline ‘viscosity data’ against which a neural network can be trained before inserting into a numerical simulation, thus requiring online training. We achieve this via differentiable programming, i.e. end-to-end backpropagation or adjoint solution through both the neural and differential equation code, using automatic differentiation of the hybrid code in the Julia programming language to calculate the necessary loss function gradients. A novel offline pre-training step accelerates training by initializing the neural network to the default numerical AV scheme, which can be learned rapidly by space-filling sampling over the AV input space. We find that online training over early time steps of simulation is sufficient to learn a neural AV function that reduces numerical oscillations in long-term hydrodynamic shock simulations. These results offer an early proof-of-principle that online differentiable training of hybrid numerical schemes with novel neural network components can improve certain performance aspects existing in purely numerical schemes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Spatiotemporal control of laser intensity using differentiable programming

Optical techniques for spatiotemporal control can produce laser pulses with custom amplitude, phase, or polarization structure. In nonlinear optics and plasma physics, the use of structured pulses typically follows a forward design approach, in which the efficacy of a known structure is analyzed for a particular application. Inverse approaches, in contrast, enable the discovery of new structures with the potential for superior performance. Here, an implementation of the unidirectional pulse propagation equation that supports automatic differentiation is combined with gradient-based optimization to design structured pulses with features that are advantageous for a range of nonlinear optical and plasma-based applications: (1) a longitudinally uniform intensity over an extended region, (2) a superluminal intensity peak that travels many Rayleigh ranges with constant duration, spot size, and amplitude, and (3) a laser pulse that ionizes a gas to form a uniform column of plasma. In the final case, optimizing the full spatiotemporal structure improves the performance by a factor of 15 compared to optimizing only spatial or only temporal structure, highlighting the advantage of spatiotemporal control.

automatic differentiation↗