Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Automatic differentiation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Ground and excited state gradients with end-to-end differentiable semiempirical quantum chemistry

Accurate and efficient gradients of molecular energy with respect to nuclear degrees of freedom are essential for geometry optimization and molecular dynamics, including simulations that go beyond the Born–Oppenheimer regime. A common approach involves deriving analytical formulas for new electronic structure methods, which is often conceptually difficult and requires tedious coding. Here, we implement analytical, semi-numerical, and automatic differentiation (AD)-based gradient pathways for semiempirical Hamiltonian models in the PYSEQM software package, leveraging both graphics processing unit (GPU) and central processing unit (CPU) architectures. We further extend these capabilities to excited states calculated using the configuration interaction singles and time-dependent Hartree–Fock ansätze. We benchmark wall time, peak memory usage, and accuracy across three molecular families of varying chemical complexity, including systems of up to a thousand atoms. For ground-state simulations, analytical and AD gradients achieve near-identical GPU runtimes, while semi-numerical gradients are slower on GPU but remain competitive on CPU. For excited states, both analytical and custom AD approaches using implicit differentiation show similar performance and low memory requirements, whereas gradients with full AD are memory-limited. AD gradients match analytical ones in accuracy across all tested systems, aided by a quaternion-based diatomic frame rotation for two-center quantities that ensures smooth energy surfaces. Overall, automatic differentiation emerges as a practical alternative to analytical gradients in semiempirical quantum chemistry, offering high accuracy while allowing seamless integration in AI-driven workflows and popular packages, such as PyTorch and JAX. Our results provide actionable guidance for selecting optimal gradient strategies in large-scale ground- and excited-state molecular dynamics simulations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

jaxhps: An elliptic PDE solver built with machine learning in mind

Elliptic partial differential equations (PDEs) can model many physical phenomena, such as electrostatics, acoustics, wave propagation, and diffusion. In scientific machine learning settings, a high-throughput PDE solver may be required to generate a training dataset, run in the inner loop of an iterative algorithm, or interface directly with a deep neural network. To provide value to machine learning users, such a PDE solver must be compatible with standard automatic differentiation frameworks, scale efficiently when run on graphics processing units (GPUs), and maintain high accuracy for a large range of input parameters. We have designed the jaxhps package with these use-cases in mind by implementing a highly efficient and accurate solver for elliptic problems with native hardware acceleration and automatic differentiation support.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2000

Steady-State Computation of Constant Rotational Rate Dynamic Stability Derivatives

Dynamic stability derivatives are essential to predicting the open and closed loop performance, stability, and controllability of aircraft. Computational determination of constant-rate dynamic stability derivatives (derivatives of aircraft forces and moments with respect to constant rotational rates) is currently performed indirectly with finite differencing of multiple time-accurate computational fluid dynamics solutions. Typical time-accurate solutions require excessive amounts of computational time to complete. Formulating Navier-Stokes (N-S) equations in a rotating noninertial reference frame and applying an automatic differentiation tool to the modified code has the potential for directly computing these derivatives with a single, much faster steady-state calculation. The ability to rapidly determine static and dynamic stability derivatives by computational methods can benefit multidisciplinary design methodologies and reduce dependency on wind tunnel measurements. The CFL3D thin-layer N-S computational fluid dynamics code was modified for this study to allow calculations on complex three-dimensional configurations with constant rotation rate components in all three axes. These CFL3D modifications also have direct application to rotorcraft and turbomachinery analyses. The modified CFL3D steady-state calculation is a new capability that showed excellent agreement with results calculated by a similar formulation. The application of automatic differentiation to CFL3D allows the static stability and body-axis rate derivatives to be calculated quickly and exactly.

Park, Michael A.↗
DOE OSTI2026

Spatiotemporal control of laser intensity using differentiable programming

Optical techniques for spatiotemporal control can produce laser pulses with custom amplitude, phase, or polarization structure. In nonlinear optics and plasma physics, the use of structured pulses typically follows a forward design approach, in which the efficacy of a known structure is analyzed for a particular application. Inverse approaches, in contrast, enable the discovery of new structures with the potential for superior performance. Here, an implementation of the unidirectional pulse propagation equation that supports automatic differentiation is combined with gradient-based optimization to design structured pulses with features that are advantageous for a range of nonlinear optical and plasma-based applications: (1) a longitudinally uniform intensity over an extended region, (2) a superluminal intensity peak that travels many Rayleigh ranges with constant duration, spot size, and amplitude, and (3) a laser pulse that ionizes a gas to form a uniform column of plasma. In the final case, optimizing the full spatiotemporal structure improves the performance by a factor of 15 compared to optimizing only spatial or only temporal structure, highlighting the advantage of spatiotemporal control.

automatic differentiation↗
DOE OSTI2025

A comparative study of calibration techniques for finite strain elastoplasticity: Numerically-exact sensitivities for FEMU and VFM

Accurate identification of material parameters is crucial for predictive modeling in computational mechanics. Here, the two primary approaches in the experimental mechanics community for calibration from full-field digital image correlation data are known as finite element model updating (FEMU) and the virtual fields method (VFM). In VFM, the objective function is a squared mismatch between internal and external virtual work or power. In FEMU, the objective function quantifies the weighted mismatch between model predictions and corresponding experimentally measured quantities of interest. It is minimized by iteratively updating the parameters of an FE model. While FEMU is seen as more flexible, VFM is commonly used instead of FEMU due to its considerably greater computational expense. However, comparisons between the two methods usually involve approximations of gradients or sensitivities with finite difference schemes, thereby making direct assessments difficult. Hence, in this study, we compare VFM and FEMU in the context of numerically-exact sensitivities obtained through local sensitivity analyses and the application of automatic differentiation software. To this end, we conduct a series of test cases to assess both methods under practical challenges using a finite strain elastoplasticity model.

Automatic differentiation↗
DOE OSTI2025

A direct-adjoint approach for material point model calibration with application to plasticity

Here, this paper proposes a new approach for the calibration of material parameters in local elastoplastic constitutive models. The calibration is posed as a constrained optimization problem, where the constitutive model evolution equations for a single material point serve as constraints. The objective function quantifies the mismatch between the stress predicted by the model and corresponding experimental measurements. To improve calibration efficiency, a novel direct-adjoint approach is presented to compute the Hessian of the objective function, which enables the use of second-order optimization algorithms. Automatic differentiation is used for gradient and Hessian computations. Two numerical examples are employed to validate the Hessian matrices and to demonstrate that the Newton–Raphson algorithm consistently outperforms gradient-based algorithms such as L-BFGS-B.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Spectrally accurate, reverse-mode differentiable bounce-averaging algorithm and its applications

We present a fast, spectrally (exponentially) accurate, automatically differentiable bounce-averaging algorithm that is used to simplify kinetic models. Using this algorithm, implemented in the DESC stellarator optimisation suite, we can perform efficient optimisation of many objectives to improve stellarator performance, such as the effective ripple 𝜖 eff metric for the neoclassical transport coefficient in the low collisionality regime and proxies for energetic particle confinement. For the first time, we optimise a finite-beta stellarator to directly reduce neoclassical ripple transport using reverse-mode differentiation. This ensures the computational cost of differentiation is independent of the number of controllable parameters.

fusion plasma↗
NASA NTRS1997

Aerodynamic Shape Sensitivity Analysis and Design Optimization of Complex Configurations Using Unstructured Grids

A three-dimensional unstructured grid approach to aerodynamic shape sensitivity analysis and design optimization has been developed and is extended to model geometrically complex configurations. The advantage of unstructured grids (when compared with a structured-grid approach) is their inherent ability to discretize irregularly shaped domains with greater efficiency and less effort. Hence, this approach is ideally suited for geometrically complex configurations of practical interest. In this work the nonlinear Euler equations are solved using an upwind, cell-centered, finite-volume scheme. The discrete, linearized systems which result from this scheme are solved iteratively by a preconditioned conjugate-gradient-like algorithm known as GMRES for the two-dimensional geometry and a Gauss-Seidel algorithm for the three-dimensional; similar procedures are used to solve the accompanying linear aerodynamic sensitivity equations in incremental iterative form. As shown, this particular form of the sensitivity equation makes large-scale gradient-based aerodynamic optimization possible by taking advantage of memory efficient methods to construct exact Jacobian matrix-vector products. Simple parameterization techniques are utilized for demonstrative purposes. Once the surface has been deformed, the unstructured grid is adapted by considering the mesh as a system of interconnected springs. Grid sensitivities are obtained by differentiating the surface parameterization and the grid adaptation algorithms with ADIFOR (which is an advanced automatic-differentiation software tool). To demonstrate the ability of this procedure to analyze and design complex configurations of practical interest, the sensitivity analysis and shape optimization has been performed for a two-dimensional high-lift multielement airfoil and for a three-dimensional Boeing 747-200 aircraft.

Taylor, Arthur C., III↗
NASA NTRS1994

Operations for Learning with Graphical Models

This paper is a multidisciplinary review of empirical, statistical learning from a graphical model perspective. Well-known examples of graphical models include Bayesian net- works, directed graphs representing a Markov chain, and undirected networks representing a Markov field. These graphical models are extended to model data analysis and empirical learning using the notation of plates. Graphical operations for simplifying and manipulating a problem are provided including decomposition, differentiation, and the manipulation of probability models from the exponential family. These operations adapt existing techniques from statistics and automatic differentiation to graphs. Two standard algorithm schemes for learning are reviewed in a graphical framework: Gibbs sampling and the expectation maximization algorithm. Some algorithms are developed in this graphical framework including a generalized version of linear regression, techniques for feed-forward networks, and learning Gaussian and discrete Bayesian networks from data. The paper concludes by sketching some implications for data analysis and summarizing some popular algorithms that fall within the framework presented. The main original contributions here are the decomposition techniques and the demonstration that graphical models provide a framework for understanding and developing complex learning algorithms.

Buntine, Wray L.↗
NASA NTRS2001

Some Advanced Concepts in Discrete Aerodynamic Sensitivity Analysis

An efficient incremental-iterative approach for differentiating advanced flow codes is successfully demonstrated on a 2D inviscid model problem. The method employs the reverse-mode capability of the automatic- differentiation software tool ADIFOR 3.0, and is proven to yield accurate first-order aerodynamic sensitivity derivatives. A substantial reduction in CPU time and computer memory is demonstrated in comparison with results from a straight-forward, black-box reverse- mode application of ADIFOR 3.0 to the same flow code. An ADIFOR-assisted procedure for accurate second-order aerodynamic sensitivity derivatives is successfully verified on an inviscid transonic lifting airfoil example problem. The method requires that first-order derivatives are calculated first using both the forward (direct) and reverse (adjoint) procedures; then, a very efficient non-iterative calculation of all second-order derivatives can be accomplished. Accurate second derivatives (i.e., the complete Hessian matrices) of lift, wave-drag, and pitching-moment coefficients are calculated with respect to geometric- shape, angle-of-attack, and freestream Mach number

Taylor, Arthur C., III↗
DOE OSTI2024

pnnl/JAX-CanVeg

Differentiable land surface model reimplementing an existing simulator, CANOAK, in JAX—a Google-developed Python package for high-performance machine learning research using automatic differentiation. The model's purpose is to perform hybrid land surface modeling that seamlessly couples process-based components with deep neural networks

Jiang, Peishi↗
NASA NTRS2003

Some Advanced Concepts in Discrete Aerodynamic Sensitivity Analysis

An efficient incremental iterative approach for differentiating advanced flow codes is successfully demonstrated on a two-dimensional inviscid model problem. The method employs the reverse-mode capability of the automatic differentiation software tool ADIFOR 3.0 and is proven to yield accurate first-order aerodynamic sensitivity derivatives. A substantial reduction in CPU time and computer memory is demonstrated in comparison with results from a straightforward, black-box reverse-mode applicaiton of ADIFOR 3.0 to the same flow code. An ADIFOR-assisted procedure for accurate second-rder aerodynamic sensitivity derivatives is successfully verified on an inviscid transonic lifting airfoil example problem. The method requires that first-order derivatives are calculated first using both the forward (direct) and reverse (adjoinct) procedures; then, a very efficient noniterative calculation of all second-order derivatives can be accomplished. Accurate second derivatives (i.e., the complete Hesian matrices) of lift, wave drag, and pitching-moment coefficients are calculated with respect to geometric shape, angle of attack, and freestream Mach number.

Taylor, Arthur C., III↗
DOE OSTI2025

JAX-CanVeg: A Differentiable Land Surface Model

Land surface models consider the exchange of water, energy, and carbon along the soil-canopy-atmosphere continuum, which is challenging to model due to their complex interdependency and associated challenges in representing and parameterizing them. Differentiable modeling provides a new opportunity to capture these complex interactions by seamlessly hybridizing process-based models with deep neural networks (DNNs), benefiting both worlds, that is, the physical interpretation of process-based models and the learning power of DNNs. Here, we developed a differentiable land model, JAX-CanVeg. The new model builds on the legacy CanVeg by incorporating advanced functionalities through JAX in the graphic processing unit support, automatic differentiation, and integration with DNNs. We demonstrated JAX-CanVeg's hybrid modeling capability by applying the model at four flux tower sites with varying aridity. To this end, we developed a hybrid version of the Ball-Berry equation that emulates the water stress impact on stomatal closure to explore the capability of the hybrid model in (a) improving the simulations of latent heat fluxes (LE) and net ecosystem exchange (NEE), (b) improving the optimization trade-off when learning observations of both LE and NEE, and (c) benefiting a multi-layer canopy model setup. Our results show that the proposed hybrid model improved the simulations of LE and NEE at all sites, with an improved optimization trade-off over the process-based model. Additionally, the multi-layer canopy set benefited hybrid modeling at some sites. Anchored in differentiable modeling, our study provides a new avenue for modeling land-atmosphere interactions by leveraging the benefits of both data-driven learning and process-based modeling.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

The Memory Scaling of Reverse-Mode Differentiation in Particle Accelerator Simulations with Space Charge

The recent development of differentiable simulation codes for particle accelerators has enabled gradient-based workflows that promise finer control and more realistic modeling of accelerator facilities. However, when using reverse-mode automatic differentiation, the memory usage continuously increases during the simulation, and can potentially exceed the available hardware memory - especially when costly space charge computation is included. To study the memory requirements for differentiable simulations, we have implemented space charge in Cheetah, a PyTorch-based beam tracking code that supports reverse-mode differentiation. We find that the memory usage for reverse-mode differentiation grows linearly with the number of macroparticles and cells, and that it is proportional to the number of space charge kicks involved in the simulation. This general scaling can be used to evaluate whether a given differentiable simulation is feasible given hardware memory constraints.

Dhamrait, Arjun↗
NASA NTRS1997

Improving Discrete-Sensitivity-Based Approach for Practical Design Optimization

In developing the automated methodologies for simulation-based optimal shape designs, their accuracy, efficiency and practicality are the defining factors to their success. To that end, four recent improvements to the building blocks of such a methodology, intended for more practical design optimization, have been reported. First, in addition to a polynomial-based parameterization, a partial differential equation (PDE) based parameterization was shown to be a practical tool for a number of reasons. Second, an alternative has been incorporated to one of the tedious phases of developing such a methodology, namely, the automatic differentiation of the computer code for the flow analysis in order to generate the sensitivities. Third, by extending the methodology for the thin-layer Navier-Stokes (TLNS) based flow simulations, the more accurate flow physics was made available. However, the computer storage requirement for a shape optimization of a practical configuration with the -fidelity simulations (TLNS and dense-grid based simulations), required substantial computational resources. Therefore, the final improvement reported herein responded to this point by including the alternating-direct-implicit (ADI) based system solver as an alternative to the preconditioned biconjugate (PbCG) and other direct solvers.

Baysal, Oktay↗
DOE OSTI2024

When ancient numerical demons meet physics-informed machine learning: adjoint-based gradients for implicit differentiable modeling

Recent advances in differentiable modeling, a genre of physics-informed machine learning that trains neural networks (NNs) together with process-based equations, have shown promise in enhancing hydrological models' accuracy, interpretability, and knowledge-discovery potential. Current differentiable models are efficient for NN-based parameter regionalization, but the simple explicit numerical schemes paired with sequential calculations (operator splitting) can incur numerical errors whose impacts on models' representation power and learned parameters are not clear. Implicit schemes, however, cannot rely on automatic differentiation to calculate gradients due to potential issues of gradient vanishing and memory demand. Here we propose a “discretize-then-optimize” adjoint method to enable differentiable implicit numerical schemes for the first time for large-scale hydrological modeling. The adjoint model demonstrates comprehensively improved performance, with Kling–Gupta efficiency coefficients, peak-flow and low-flow metrics, and evapotranspiration that moderately surpass the already-competitive explicit model. Therefore, the previous sequential-calculation approach had a detrimental impact on the model's ability to represent hydrological dynamics. Furthermore, with a structural update that describes capillary rise, the adjoint model can better describe baseflow in arid regions and also produce low flows that outperform even pure machine learning methods such as long short-term memory networks. The adjoint model rectified some parameter distortions but did not alter spatial parameter distributions, demonstrating the robustness of regionalized parameterization. Despite higher computational expenses and modest improvements, the adjoint model's success removes the barrier for complex implicit schemes to enrich differentiable modeling in hydrology.

58 GEOSCIENCES↗
DOE OSTI2025

Jipole: A Differentiable ipole-based Code for Radiative Transfer in Curved Spacetimes

Recent imaging of supermassive black holes by the Event Horizon Telescope has relied on exhaustive parameter-space searches, matching observations to large, precomputed libraries of theoretical models. As observational data become increasingly precise, the limitations of this computationally expensive approach grow more acute, creating a pressing need for more efficient methods. In this work, we present Jipole, an automatically differentiable (AD), ipole-based code for radiative transfer in curved spacetimes, designed to compute image gradients with respect to underlying model parameters. These gradients quantify how parameter changes—such as the black hole’s spin or the observer’s inclination—affect the image, enabling more efficient parameter estimation and reducing the number of required images. We validate Jipole against ipole in two analytical tests and then compare pixelwise intensity derivatives from AD with those from finite-difference methods. We then demonstrate the utility of these gradients by performing parameter recovery for an analytical model in three increasingly complex cases for the injected image: ideal, blurred, and blurred with added noise. In most cases, high-accuracy fits are obtained in only a few optimization steps, failing only in cases with extremely low signal-to-noise ratios. These results highlight the potential of AD-based methods to accelerate robust, high-fidelity model-data comparisons in current and future black hole imaging efforts.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Application of physics-informed neural networks (PINNs) solution to coupled thermal and hydraulic processes in silty sands

Abstract The accurate modeling of water and heat transport in soils is crucial for both geo-environmental and geothermal engineering. Traditional modeling methods are problematic because they require well-defined boundaries and initial conditions. Recently, physics-informed neural networks (PINNs), which incorporate partial differential equations (PDEs) to solve forward and inverse problems, have attracted increasing attention in machine learning research. In this study, we applied PINNs to tackle hydraulic and thermal transport coupling forward problems in silty sands. A fully connected deep neural network was utilized for training. This neural network model leverages automatic differentiation to apply the governing equations as constraints, based on the mathematical approximations established by the neural network itself. We conducted forward problems and compared the solutions derived from PINNs with those from Finite Element Method (FEM) simulations. The forward problem results demonstrate the PINNs model’s capability in predicting hydraulic transport, heat transport, and thermal–hydraulic coupling in silty sands under various boundary conditions. The PINNs exhibited great performance in simulating the thermal–hydraulic coupling problem. The accuracy of the PINNs solutions shows its potential for simulation in geotechnical engineering.

Feng, Yuan↗