SEARCH · Engineering Papers
Results for “Numerical differentiation”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Data-driven linear time advance operators for the acceleration of plasma physics simulation
In this study, we demonstrate the application of data-driven linear operator construction for time advance with a goal of accelerating plasma physics simulation. We apply dynamic mode decomposition (DMD) to data produced by the nonlinear SOLPS-ITER (Scrape-off Layer Plasma Simulator - International Thermonuclear Experimental Reactor) plasma boundary code suite in order to estimate a series of linear operators and monitor their predictive accuracy via online error analysis. We find that this approach defines when these dynamics can be represented by a sequence of approximate linear operators and is essential for providing consistent projections when compared to an unconstrained application. For linear diffusion and advection–diffusion fluid test problems, we construct and apply operators within explicit and implicit time advance schemes, demonstrating that stability can be robustly guaranteed in each case. We further investigate the use of the linear time advance operators within several integration methods including forward Euler, backward Euler, and the matrix exponential. The application of this method to simulation data from SOLPS-ITER, with varying levels of Markov chain Monte Carlo numerical noise, shows that constrained DMD operators yield a capability to identify, extract, and integrate a (slow) subset of the present timescales. Example applications show that for projected speedup factors of [Formula: see text], and [Formula: see text], a mean relative error of 3%, 5%, and 8% and maximum relative error less than 20% are achievable, which appears acceptable for typical SOLPS-ITER steady-state simulations.
Neural network surrogate models for equations of state
Equation of state (EOS) data provide necessary information for accurate multiphysics modeling, which is necessary for fields such as inertial confinement fusion. Here, we suggest a neural network surrogate model of energy and entropy and use thermodynamic relationships to derive other necessary thermodynamic EOS quantities. We incorporate phase information into the model by training a phase classifier and using phase-specific regression models, which improves the modal prediction accuracy. Our model predicts energy values to 1% relative error and entropy to 3.5% relative error in a log-transformed space. Although sound speed predictions require further improvement, the derived pressure values are accurate within 10% relative error. Our results suggest that neural network models can effectively model EOS for inertial confinement fusion simulation applications.
Latent Twins
Over the past decade, scientific machine learning has transformed the development of mathematical and computational frameworks for analyzing, modeling, and predicting complex systems. From inverse problems to numerical partial differential equations (PDEs), dynamical systems, and model reduction, these advances have pushed the boundaries of what can be simulated. Yet they have often progressed in parallel, with representation learning and algorithmic solution methods evolving largely as separate pipelines. With Latent Twins, we propose a unifying mathematical framework that creates a hidden surrogate in latent space for the underlying equations. Whereas digital twins mirror physical systems in the digital world, Latent Twins mirror mathematical systems in a learned latent space governed by operators. Through this lens, classical modeling, inversion, model reduction, and operator approximation all emerge as special cases of a single principle. We establish the fundamental approximation properties of Latent Twins for both ordinary differential equations (ODEs) and PDEs and demonstrate the framework across three representative settings: (i) canonical ODEs, capturing diverse dynamical regimes; (ii) a PDE benchmark using the shallow-water equations, contrasting Latent Twin simulations with deep operator network and forecasts with a four-dimensional variational method baseline; and (iii) a challenging real-data geopotential reanalysis dataset, reconstructing and forecasting from sparse, noisy observations. Latent Twins provide a compact, interpretable surrogate for solution operators that evaluate across arbitrary time gaps in a single-shot, while remaining compatible with scientific pipelines such as assimilation, control, and uncertainty quantification. Looking forward, this framework offers scalable, theory-grounded surrogates that bridge data-driven representation learning and classical scientific modeling across disciplines.
RE-INTEGRATE EMT Simulation Software: DAE Solvers and Automation
Existing electromagnetic transient (EMT) simulation tools face challenges in accelerating EMT simulations, especially for very large-scale power networks. To tackle this issue, next generation EMT simulation tools such as RE-INTEGRATE EMT are being researched upon. Such tools should be equipped with automation capabilities and advanced numerical differential-algebraic equation (DAE) solvers. In this paper, the DAE solvers incorporated within the RE-INTEGRATE EMT simulation tool are discussed. In particular, a modified ODEINT-based DAE solver and the ARKODE solver from SUN-DIALS are leveraged within RE-INTEGRATE EMT. In addition, the automation implemented within RE-INTEGRATE EMT to automate the DAE generation (replacing the need of manual discretization and assembling DAEs) is discussed. Different use cases were implemented using the RE-INTEGRATE EMT tool and were validated with respect to baseline simulations.
Time-dependent Hamiltonian Simulation of Highly Oscillatory Dynamics and Superconvergence for Schrödinger Equation
We propose a simple quantum algorithm for simulating highly oscillatory quantum dynamics, which does not require complicated quantum control logic for handling time-ordering operators. To our knowledge, this is the first quantum algorithm that is both insensitive to the rapid changes of the time-dependent Hamiltonian and exhibits commutator scaling. Our method can be used for efficient Hamiltonian simulation in the interaction picture. In particular, we demonstrate that for the simulation of the Schrödinger equation, our method exhibits superconvergence and achieves a surprising second order convergence rate, of which the proof rests on a careful application of pseudo-differential calculus. Numerical results verify the effectiveness and the superconvergence property of our method.
First Principles Nonadiabatic Excited-State Molecular Dynamics in NWChem
Computational simulation of non-adiabatic molecular dynamics is an indispensable tool for understanding complex photoinduced processes such as internal conversion, energy transfer, charge separation, and spatial localization of excitons, to name a few. Here, we report an implementation of the fewest-switches surface hopping algorithm in the NWChem computational chemistry program. Here the surface hopping method is combined with linear-response time-dependent density functional theory calculations of adiabatic excited state potential energy surfaces. To treat quantum transitions between arbitrary electronic Born{Oppenheimer states, we have implemented both numerical and analytical differentiation schemes for derivative non-adiabatic couplings. A numerical approach for the time-derivative non-adiabatic couplings together with an analytical method for calculating non-adiabatic coupling vectors is an efficient combination for surface hopping approaches. Additionally, electronic decoherence schemes and a state reassigned unavoided crossings algorithm are also implemented to improve the accuracy of the simulated dynamics and to handle trivial unavoided crossings. We apply our code to study the ultrafast decay of photoexcited benzene, including a detailed analysis of the potential energy surface, population decay time scales, and vibrational coordinates coupled to the excitation dynamics. The development presented in this work is a baseline for future implementations of more sophisticated frameworks for simulating of non-adiabatic molecular dynamics in NWChem.
Stage-local partitioned two-step runge-kutta methods for large systems of ordinary differential equations
We introduce stage-local partitioned two-step Runge-Kutta methods are an extension of standard two-step Runge-Kutta methods, which are an alternative to the standard additive two-step Runge-Kutta methods currently existing in the literature. Furthermore, these new schemes are designed with an eye towards truly N-partitioned systems and leverage local stage approximations to make several computationally interesting approximations viable. Specifically, the focus on local stage approximations makes possible the construction of truly asynchronous schemes, in the parallel sense, possible. In addition, we show that an implicit-explicit approach to these schemes can lead to methods that require the inversion of only local nonlinear systems.
Differentiable Programming: Bridging the gap between numerical models and machine learning models
Explore the source record for details and available documents.
Numerical methods and hypoexponential approximations for gamma distributed delay differential equations
Abstract Gamma distributed delay differential equations (DDEs) arise naturally in many modelling applications. However, appropriate numerical methods for generic gamma distributed DDEs have not previously been implemented. Modellers have therefore resorted to approximating the gamma distribution with an Erlang distribution and using the linear chain technique to derive an equivalent system of ordinary differential equations (ODEs). In this work, we address the lack of appropriate numerical tools for gamma distributed DDEs in two ways. First, we develop a functional continuous Runge–Kutta (FCRK) method to numerically integrate the gamma distributed DDE without resorting to Erlang approximation. We prove the fourth-order convergence of the FCRK method and perform numerical tests to demonstrate the accuracy of the new numerical method. Nevertheless, FCRK methods for infinite delay DDEs are not widely available in existing scientific software packages. As an alternative approach to solving gamma distributed DDEs, we also derive a hypoexponential approximation of the gamma distributed DDE. This hypoexponential approach is a more accurate approximation of the true gamma distributed DDE than the common Erlang approximation but, like the Erlang approximation, can be formulated as a system of ODEs and solved numerically using standard ODE software. Using our FCRK method to provide reference solutions, we show that the common Erlang approximation may produce solutions that are qualitatively different from the underlying gamma distributed DDE. However, the proposed hypoexponential approximations do not have this limitation. Finally, we apply our hypoexponential approximations to perform statistical inference on synthetic epidemiological data to illustrate the utility of the hypoexponential approximation.
Identifying Differential Equations in Fourier Domain (FourierIdent)
We investigate identifying differential equations in the frequency domain. Fourier analysis is an important tool in theoretical analysis and numerical solvers of differential equations, yet there is limited work in exploring this connection in the identification of differential equations. This paper aims to identify the underlying differential equation in the frequency domain, from a given single realization of the differential equation perturbed by noise. Such setting imposes difficulties which are different from other identification methods where computation is carried out in the physical domain. We propose several ways to mitigate the challenges arising from noise in data and large differences in the magnitudes of frequency responses. The main takeaways are that identifying differential equations solely in the frequency domain is challenging, the method we propose is based on a form of domain partitions in the frequency domain, and this method shows benefits for complex data even with high level of noise. We introduce a Fourier feature denoising, and define the meaningful data region and the core regions of features to reduce the effect of noise in the frequency domain and to enhance the accuracy in coefficient identification. The proposed method is tested on various differential equations with linear, nonlinear, and high-order derivative feature terms, and shows advantages on complex data with many frequency modes, even under high level of noise.
How machine learning can extend electroanalytical measurements beyond analytical interpretation
Electroanalytical measurements are routinely used to estimate material properties exhibiting current and voltage signatures. Analysis of such measurements relies on analytical expressions of material properties to describe the experiments. The need for analytical expressions limits the experiments that can be used to measure properties as well as the properties that can be estimated from a given experiment. Such analytical relations are essentially solutions of the physics-based differential equations (with properties as coefficients) describing the material behavior under certain specific conditions. In recent years, a new machine learning-based approach has been gaining popularity wherein the differential equations are numerically solved to interpret the electroanalytical experiments in terms of corresponding material properties. Since the physics-based differential equations are solved, one can additionally estimate underlying fields, e.g., concentration profile, using such an approach. To exemplify the characteristics of such a machine learning assisted interpretation of electroanalytical measurements, we use data from the Hebb–Wagner test on a magnesium spinel intercalation host. In conclusion, as compared to the traditional analytical expression-based interpretation, the emerging approach decreases experimental efforts to characterize relevant material properties as well as provides field information that was previously inaccessible.
When ancient numerical demons meet physics-informed machine learning: adjoint-based gradients for implicit differentiable modeling
Recent advances in differentiable modeling, a genre of physics-informed machine learning that trains neural networks (NNs) together with process-based equations, have shown promise in enhancing hydrological models' accuracy, interpretability, and knowledge-discovery potential. Current differentiable models are efficient for NN-based parameter regionalization, but the simple explicit numerical schemes paired with sequential calculations (operator splitting) can incur numerical errors whose impacts on models' representation power and learned parameters are not clear. Implicit schemes, however, cannot rely on automatic differentiation to calculate gradients due to potential issues of gradient vanishing and memory demand. Here we propose a “discretize-then-optimize” adjoint method to enable differentiable implicit numerical schemes for the first time for large-scale hydrological modeling. The adjoint model demonstrates comprehensively improved performance, with Kling–Gupta efficiency coefficients, peak-flow and low-flow metrics, and evapotranspiration that moderately surpass the already-competitive explicit model. Therefore, the previous sequential-calculation approach had a detrimental impact on the model's ability to represent hydrological dynamics. Furthermore, with a structural update that describes capillary rise, the adjoint model can better describe baseflow in arid regions and also produce low flows that outperform even pure machine learning methods such as long short-term memory networks. The adjoint model rectified some parameter distortions but did not alter spatial parameter distributions, demonstrating the robustness of regionalized parameterization. Despite higher computational expenses and modest improvements, the adjoint model's success removes the barrier for complex implicit schemes to enrich differentiable modeling in hydrology.
Chapter 4 - Recent Advances in Identification of Differential Equations from Noisy Data: IDENT Review
Differential equations and numerical methods are extensively used to model various real-world phenomena in science and engineering. With modern developments, we aim to find the underlying differential equation from a single observation of time-dependent data. If we assume that the differential equation is a linear combination of various linear and nonlinear differential terms, then the identification problem can be formulated as solving a linear system. The goal then reduces to finding the optimal coefficient vector that best represents the time derivative of the given data. We review some recent works on the identification of differential equations. We find some common themes for the improved accuracy: (i) The formulation of linear system with proper denoising is important, (ii) how to utilize sparsity and model selection to find the correct coefficient support needs careful attention, and (iii) there are ways to improve the coefficient recovery. We present an overview and analysis of recent developments on the topic.
Broadband piezoelectric energy harvesting microgyroscopes: Design and nonlinear analysis
Small devices in remote or difficult-to-reach areas can benefit from harvesting energy from mechanical wasted energy, which reduces the requirement for a new power source. A multi-purpose energy harvesting microgyroscope system based on piezoelectric materials is suggested. The necessity of taking spatially varying electrostatic forces is examined. Considering the effects of the system's width and thickness, DC voltage, and angular speed the systems inherent frequencies are found. The partial differential equations describing the system's dynamics are numerically solved by using the differential quadrature method. Further, the numerical analysis enables to identify the optimal system design for broadband energy harvesting. The simulation results reveal that a system with a non-symmetric beam design is adequate for broadband energy harvesting. This is associated with the applied DC voltage, which may be modified to improve the broadband frequency of the system. It is concluded that the nonlinear softening effects create a broadband frequency response with high voltage output. However, DC and AC voltages need to be carefully selected in order to avoid the dynamic pull-in.
A Decision-Making Machine Learning Approach in Hermite Spectral Approximations of Partial Differential Equations
The accuracy and effectiveness of Hermite spectral methods for the numerical discretization of partial differential equations on unbounded domains are strongly affected by the amplitude of the Gaussian weight function employed to describe the approximation space. This is particularly true if the problem is under-resolved, i.e., there are no enough degrees of freedom. The issue becomes even more crucial when the equation under study is time-dependent, forcing in this way the choice of Hermite functions where the corresponding weight depends on time. In order to adapt dynamically the approximation space, it is here proposed an automatic decision-making process that relies on machine learning techniques, such as deep neural networks and support vector machines. The algorithm is numerically tested with success on a simple 1D problem, but the main goal is its exportability in the context of more serious applications. Here we also show at the end an application in the framework of plasma physics.
Ground and excited state gradients with end-to-end differentiable semiempirical quantum chemistry
Accurate and efficient gradients of molecular energy with respect to nuclear degrees of freedom are essential for geometry optimization and molecular dynamics, including simulations that go beyond the Born–Oppenheimer regime. A common approach involves deriving analytical formulas for new electronic structure methods, which is often conceptually difficult and requires tedious coding. Here, we implement analytical, semi-numerical, and automatic differentiation (AD)-based gradient pathways for semiempirical Hamiltonian models in the PYSEQM software package, leveraging both graphics processing unit (GPU) and central processing unit (CPU) architectures. We further extend these capabilities to excited states calculated using the configuration interaction singles and time-dependent Hartree–Fock ansätze. We benchmark wall time, peak memory usage, and accuracy across three molecular families of varying chemical complexity, including systems of up to a thousand atoms. For ground-state simulations, analytical and AD gradients achieve near-identical GPU runtimes, while semi-numerical gradients are slower on GPU but remain competitive on CPU. For excited states, both analytical and custom AD approaches using implicit differentiation show similar performance and low memory requirements, whereas gradients with full AD are memory-limited. AD gradients match analytical ones in accuracy across all tested systems, aided by a quaternion-based diatomic frame rotation for two-center quantities that ensures smooth energy surfaces. Overall, automatic differentiation emerges as a practical alternative to analytical gradients in semiempirical quantum chemistry, offering high accuracy while allowing seamless integration in AI-driven workflows and popular packages, such as PyTorch and JAX. Our results provide actionable guidance for selecting optimal gradient strategies in large-scale ground- and excited-state molecular dynamics simulations.
An adaptive and stability-promoting layerwise training approach for sparse deep neural network architecture
This work presents a two-stage adaptive framework for progressively developing deep neural network (DNN) architectures that generalize well for a given training data set. In the first stage, a layerwise training approach is adopted where a new layer is added each time and trained independently by freezing parameters in the previous layers. We impose desirable structures on the DNN by employing manifold regularization, sparsity regularization, and physics-informed terms. We introduce a ε – δ – stability-promoting concept as a desirable property for a learning algorithm and show that employing manifold regularization yields a ε – δ stability-promoting algorithm. Further, we also derive the necessary conditions for the trainability of a newly added layer and investigate the training saturation problem. In the second stage of the algorithm (post-processing), a sequence of shallow networks is employed to extract information from the residual produced in the first stage, thereby improving the prediction accuracy. Numerical investigations on prototype regression and classification problems demonstrate that the proposed approach can outperform fully connected DNNs of the same size. Moreover, by equipping the physics-informed neural network (PINN) with the proposed adaptive architecture strategy to solve partial differential equations, we numerically show that adaptive PINNs not only are superior to standard PINNs but also produce interpretable hidden layers with provable stability. As a result, we also apply our architecture design strategy to solve inverse problems governed by elliptic partial differential equations.