Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “algorithmic differentiation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

Karhunen–Loève deep learning method for surrogate modeling and approximate Bayesian parameter estimation

We evaluate the performance of the Karhunen-Loève Deep Neural Network (KL-DNN) framework for surrogate modeling and approximate Bayesian parameter estimation in partial differential equation models. In the surrogate model, the Karhunen-Loève (KL) expansions are used for the dimensionality reduction of the number of unknown parameters and variables, and a deep neural network is employed to relate the reduced space of parameters to that of the state variables. The KL-DNN surrogate model is used to formulate a maximum-a-posteriori-like least-squares problem, which is randomized to draw samples of the posterior distribution of the parameters. We test the proposed framework for a hypothetical unconfined aquifer via comparison with the forward MODFLOW and inverse PEST++ iterative ensemble smoother (IES) solutions as well as the state-of-the-art Fourier neural operator (FNO) and deep operator networks (DeepONets) operator learning surrogate models. Our results show that the KL-DNN surrogate model outperforms FNO and DeepONet for forward predictions. For solving inverse problems, the randomized algorithm provides the same or more accurate Bayesian predictions of the parameters than IES as evidenced by the higher log-predictive probability of both the estimated parameter field and the forecast hydraulic head. The posterior mean obtained from the randomized algorithm is closer to the reference parameter field than that obtained with FNO as the maximum a posteriori estimate.

Approximate Bayesian inference↗

Transported PDF Modeling of Compressible Turbulent Reactive Flows by using the Eulerian Monte Carlo Fields Method

Although the transported probability density function (PDF) method has been developed for decades, its application has been mainly focused on the low-Mach number flow problems. This work extends the transported PDF method to compressible flow problems. The Eulerian Monte Carlo fields (EMCF) solution method is employed to solve the transported PDF equation for compressible flow problems. A pseudo stagnation enthalpy is introduced and its stochastic partial differential equation is derived to ensure total energy conservation numerically. A new mixing model called interaction by partial exchange with mean (IPEM) is introduced to expand the available choices of mixing models for the EMCF method. The consistency of the EMCF method is examined for solving the transported PDF equation. Numerical implementation details are discussed, such as the density coupling between the compressible flow solver and the EMCF solver, discretization schemes for the mixing terms and the stochastic terms. The implemented compressible flow solver coupled with the EMCF solver is verified and validated in a series of test cases with increasing level of complexity, ranging from a statistically one-dimensional turbulent mixing layer to a self-excited resonance model rocket combustor. It is observed that in general with the increase of compressibility, there is an increase in the sensitivity of the modeling results to the different models and algorithms. This makes it necessary to develop a thorough understanding of the model sensitivity in order to develop a robust and accurate simulation solver for highly compressible turbulent reactive flows. The thermo-acoustic instability inside the model rocket combustor case is captured reasonably, which demonstrates the overall capability of the developed compressible turbulent combustion solver based on the transported PDF method.

42 ENGINEERING↗

Parallel-in-Time Simulation of Lindblad's Equation

Constructing fast quantum logic gates is critical to building a scalable quantum computer. We consider a qudit, a quantum version of a bit that can take an arbitrary number of states, coupled with a cavity. In this project, we wish to force the qudit to reach the 0-state, for any possible initial state. The coupled system changes in time according to Lindblad’s equation, an ordinary differential equation on the density matrix of the quantum system. Lindblad’s equation contains some parameters that we can control, so-called control functions. We seek control functions which force the qudit to the 0-state within 2 microseconds, which is much faster than what is currently done in practice. The search method is gradient descent, a numerical optimization method that uses gradient information to iteratively improve the control parameters. My contribution to this project is an attempt to speed up the computation of the gradient. It currently takes about 40 seconds to compute the gradient which involves solving a set of ODEs sequentially. Current supercomputers have thousands of cores, but sequential computations can only make use of 1 core at a time. We wish to divide up the work better, so that we can use many more cores at once. To this end, we have implemented the Multigrid Reduction in Time (MGRIT) algorithm. We perform a systematic parameter search on how to best apply this algorithm. Results indicate a 25 percent speed up for solving Lindblad’s equation and determining how close the final state is the 0-state.

97 MATHEMATICS AND COMPUTING↗

Beyond PID Controllers: PPO with Neuralized PID Policy for Proton Beam Intensity Control in Mu2e

We introduce a novel Proximal Policy Optimization (PPO) algorithm aimed at addressing the challenge of maintaining a uniform proton beam intensity delivery in the Muon to Electron Conversion Experiment (Mu2e) at Fermi National Accelerator Laboratory (Fermilab). Our primary objective is to regulate the spill process to ensure a consistent intensity profile, with the ultimate goal of creating an automated controller capable of providing real-time feedback and calibration of the Spill Regulation System (SRS) parameters on a millisecond timescale. We treat the Mu2e accelerator system as a Markov Decision Process suitable for Reinforcement Learning (RL), utilizing PPO to reduce bias and enhance training stability. A key innovation in our approach is the integration of a neuralized Proportional-Integral-Derivative (PID) controller into the policy function, resulting in a significant improvement in the Spill Duty Factor (SDF) by 13.6%, surpassing the performance of the current PID controller baseline by an additional 1.6%. This paper presents the preliminary offline results based on a differentiable simulator of the Mu2e accelerator. It paves the groundwork for real-time implementations and applications, representing a crucial step towards automated proton beam intensity control for the Mu2e experiment.

43 PARTICLE ACCELERATORS↗

Geometric Interpretation of the Cluster Location Problem Part II: Application to the Pahala, Hawaii, Earthquake Sequence

In the companion “Theory” article, we presented a new framing of the seismic location problem in terms of differential geometry (Harris et al., 2025). From that viewpoint, we developed a “project and correct” approach for estimating the relative locations of earthquakes. Here, in this study, we use project and correct to estimate high-precision relative locations of events from an earthquake sequence beneath the town of Pahala, Hawaii, using high-precision correlation-derived picks. The sequence was active from 2020 through 2022 and produced many highly correlated signals at Hawaii Volcano Observatory (HVO) stations on the island of Hawaii. The data we inverted consisted of 2882 events with observations at 5 HVO stations. For comparison with the travel-time image, we also produced conventional hypocenter solutions using both the Bayesloc program (Myers et al., 2007, 2009) and a purpose-built double-difference code. There were obvious structural elements in the resulting image, the resolution of which we used to test the performance of the project and the correct algorithm. For the projection step, we first produced a 3D local basis using an singular value decomposition (SVD) of the 2882 groups of times. Projection of the travel-time vectors into this basis resulted in an image with structures similar to those produced by our conventional locators, but with distortion as predicted by theory. Removing the distortion requires an inverse operator generated from the metric tensor at the geometric centroid of the events. We compared two approaches to obtaining such an inverse operator. The first uses an estimate of the geographic centroid of the event cloud from the centroid of the travel-time data. The second approach uses the centroid of the conventionally produced locations. The first approach produces a corrected image very similar to the conventional results, but with a rotation. The corrected image produced using the conventionally derived centroid is a near-exact match to the conventional locations.

Dodge, Douglas A. [Lawrence Livermore National Lab↗

Mixed Precision Fermi-Operator Expansion on Tensor Cores from a Machine Learning Perspective

Here we present a second-order recursive Fermi-operator expansion scheme using mixed precision floating point operations to perform electronic structure calculations using tensor core units. A performance of over 100 teraFLOPs is achieved for half-precision floating point operations on Nvidia’s A100 tensor core units. The second-order recursive Fermi-operator scheme is formulated in terms of a generalized, differentiable deep neural network structure, which solves the quantum mechanical electronic structure problem. We demonstrate how this network can be accelerated by optimizing the weight and bias values to substantially reduce the number of layers required for convergence. We also show how this machine learning approach can be used to optimize the coefficients of the recursive Fermi-operator expansion to accurately represent the fractional occupation numbers of the electronic states at finite temperatures.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

When and why PINNs fail to train: A neural tangent kernel perspective

Physics-informed neural networks (PINNs) have lately received great attention thanks to their flexibility in tackling a wide range of forward and inverse problems involving partial differential equations. However, despite their noticeable empirical success, little is known about how such constrained neural networks behave during their training via gradient descent. More importantly, even less is known about why such models sometimes fail to train at all. Here in this work, we aim to investigate these questions through the lens of the Neural Tangent Kernel (NTK); a kernel that captures the behavior of fully-connected neural networks in the infinite width limit during training via gradient descent. Specifically, we derive the NTK of PINNs and prove that, under appropriate conditions, it converges to a deterministic kernel that stays constant during training in the infinite-width limit. This allows us to analyze the training dynamics of PINNs through the lens of their limiting NTK and find a remarkable discrepancy in the convergence rate of the different loss components contributing to the total training error. To address this fundamental pathology, we propose a novel gradient descent algorithm that utilizes the eigenvalues of the NTK to adaptively calibrate the convergence rate of the total training error. Finally, we perform a series of numerical experiments to verify the correctness of our theory and the practical effectiveness of the proposed algorithms.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A Theory of Ultrafast Charge Transfer Relaxation with Non-Innocent Solvent Molecules

In this work we revisit the photodynamics of tetracyanoethylene-hexamethylbenzene (TCNE- HMB), the molecular complex studied by Hochstrasser et al. [J. Chem. Phys. 100, 4797–4810, 1994] that has long challenged the applicability of Marcus’s theory of elec- tron transfer for predicting photochemical reactions. Using a novel black-box electronic structure algorithm (time-dependent density functional theory with one double, TD- DFT-1D) to efficiently run molecular dynamics that can treat charge recombination, we run ab initio surface hopping molecular dynamics and confirm that, for a polar solvent, charge recombination rates can be incredibly fast (indeed faster than the sol- vent relaxation time); for non-polar solvents, the rate is much slower. We demonstrate that, although Marcus theory cannot be directly applied, these nonequilibrium (and sometimes incredibly fast) photoexcited dynamics can be effectively explained within a two-state model without any evidence of a transition through a conical intersection. Most importantly, for this paradigmatic model system, we are able to identify two nuclear coordinates of interest (rather than the single coordinate predicted by Marcus or a full set of internal quantum modes studied by Bixon and Jortner): the solvent relaxation in the first shell (that strongly modulates the energies of the charge trans- fer state and differentiates time scales for relaxation) and a nuclear displacement in the TCNE-HMB complex arising from a handful of vibrations that induces non-Born Oppenheimer motion and eventually facilitates an abrupt electronic transition to the ground state. Altogether, these findings suggest a tractable generalization of Marcus theory for future simulations of photochemistry with non-innocent solvent environ- ments in the spirit of a Hamiltonian suggested by Stuchebrukhov (J. Chem. Phys. 107, 3821, 1997).

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Deblurring for nuclei: 3D characteristics of heavy-ion collisions

Observables from nuclear and high-energy experiments can be degraded by detector performance and/or methodology in extracting the observables, such as of the final-state characteristics of heavy-ion collisions in relation to a coarsely estimated reaction-plane direction. We propose the use of deblurring methods, such as in optics, to correct for observable degradation. Our main focus is the restoration of triple-differential particle distributions in heavy-ion collisions. We demonstrate that these could be extracted from collision measurements following the Richardson-Lucy deblurring method from optics. We illustrate basic features of the restoration methodology in a schematic model assuming either ideal or more realistic particle detection. Here, the inferred three-dimensional (3D) distributions for collisions may be easier to interpret in terms of collision dynamics and sought properties of bulk matter than the currently employed Fourier coefficients, that combine information from different azimuthal angles relative to the reaction plane.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Experimental Examination of Additively Manufactured Patterns on Structural Nuclear Materials for Digital Image Correlation Strain Measurements

Abstract Background There are a limited number of commercially available sensors for monitoring the deformation of materials in-situ during harsh environment applications, such as those found in the nuclear and aerospace industries. Such sensing devices, including weldable strain gauges, extensometers, and linear variable differential transformers, can be destructive to material surfaces being investigated and typically require relatively large surface areas to attach (> 10 mm in length). Digital image correlation (DIC) is a viable, non-contact alternative to in-situ strain deformation. However, it often requires implementing artificial patterns using splattering techniques, which are difficult to reproduce. Objective Additive manufacturing capabilities offer consistent patterns using programmable fabrication methods. Methods In this work, a variety of small-scale periodic patterns with different geometries were printed directly on structural nuclear materials (i.e., stainless steel and aluminum tensile specimens) using an aerosol jet printer (AJP). Unlike other additive manufacturing techniques, AJP offers the advantage of materials selection. DIC was used to track and correlate strain to alternative measurement methods during cyclic loading, and tensile tests (up to 1100 µɛ) at room temperature. Results The results confirmed AJP has better control of pattern parameters for small fields of view and facilitate the ability of DIC algorithms to adequately process patterns with periodicity. More specifically, the printed 100 μm spaced dot and 150 μm spaced line patterns provided accurate measurements with a maximum error of less than 2% and 4% on aluminum samples when compared to an extensometer and commercially available strain gauges. Conclusion Our results highlight a new pattern fabrication technique that is form factor friendly for digital image correlation in nuclear applications.

Novich, K. A. (ORCID:0000000204466022)↗

DOME: Directional medical embedding vectors from Electronic Health Records

Motivation: The increasing availability of Electronic Health Record (EHR) systems has created enormous potential for translational research. Recent developments in representation learning techniques have led to effective large-scale representations of EHR concepts along with knowledge graphs that empower downstream EHR studies. However, most existing methods require training with patient-level data, limiting their abilities to expand the training with multi-institutional EHR data. On the other hand, scalable approaches that only require summary-level data do not incorporate temporal dependencies between concepts. Methods: We introduce a DirectiOnal Medical Embedding (DOME) algorithm to encode temporally directional relationships between medical concepts, using summary-level EHR data. Specifically, DOME first aggregates patient-level EHR data into an asymmetric co-occurrence matrix. Then it computes two Positive Pointwise Mutual Information (PPMI) matrices to correspondingly encode the pairwise prior and posterior dependencies between medical concepts. Following that, a joint matrix factorization is performed on the two PPMI matrices, which results in three vectors for each concept: a semantic embedding and two directional context embeddings. They collectively provide a comprehensive depiction of the temporal relationship between EHR concepts. Results: We highlight the advantages and translational potential of DOME through three sets of validation studies. First, DOME consistently improves existing direction-agnostic embedding vectors for disease risk prediction in several diseases, for example achieving a relative gain of 5.5% in the area under the receiver operating characteristic (AUROC) for lung cancer. Second, DOME excels in directional drug-disease relationship inference by successfully differentiating between drug side effects and indications, correspondingly achieving relative AUROC gain over the state-of-the-art methods by 10.8% and 6.6%. Finally, DOME effectively constructs directional knowledge graphs, which distinguish disease risk factors from comorbidities, thereby revealing disease progression trajectories. The source codes are provided at https://github.com/celehs/Directional-EHRembedding.

60 APPLIED LIFE SCIENCES↗

Learning Constrained Parametric Differentiable Predictive Control Policies With Guarantees

We present differentiable predictive control (DPC), a method for offline learning of constrained neural control policies for nonlinear dynamical systems with performance guarantees. We show that the sensitivities of the parametric optimal control problem can be used to obtain direct policy gradients. Specifically, we employ automatic differentiation (AD) to efficiently compute the sensitivities of the model predictive control (MPC) objective function and constraints penalties. To guarantee safety upon deployment, we derive probabilistic guarantees on closed-loop stability and constraint satisfaction based on indicator functions and Hoeffding’s inequality. We empirically demonstrate that the proposed method can learn neural control policies for various parametric optimal control tasks. In particular, we show that the proposed DPC method can stabilize systems with unstable dynamics, track time-varying references, and satisfy nonlinear state and input constraints. Our DPC method has practical time savings compared to alternative approaches for fast and memory-efficient controller design. Specifically, DPC does not depend on a supervisory controller as opposed to approximate MPC based on imitation learning. We demonstrate that, without losing performance, DPC is scalable with greatly reduced demands on memory and computation compared to implicit and explicit MPC while being more sample efficient than model-free reinforcement learning (RL) algorithms.

97 MATHEMATICS AND COMPUTING↗

MULTISTEP AND CONTINUOUS PHYSICS-INFORMED NEURAL NETWORK METHODS FOR LEARNING GOVERNING EQUATIONS AND CONSTITUTIVE RELATIONS

In this work, we investigate the applicability and relative merit of discrete and continuous versions of physics-informed neural network (PINN) methods for learning unknown governing equations or constitutive relations in a nonlinear dynamical system. In the case of unknown dynamics, entire right-hand-side (RHS) equations of the ordinary differential equations are unknown. In the case of unknown constitutive relations, however, the RHS equations are known up to the specification of constitutive relations (that may depend on the state of the system). We use a deep neural network to model unknown governing equations or constitutive relations. The discrete PINN approach combines classical multistep discretization methods for dynamical systems with neural-network-based machine learning methods. On the other hand, the continuous versions utilize deep neural networks to minimize the residual function for the continuous governing equations. We use the case of a fedbatch bioreactor system to study the effectiveness of these approaches and discuss conditions for their applicability. Our results indicate that the accuracy of the trained neural network models is much higher for the cases where we only have to learn a constitutive relation instead of all dynamics. This finding corroborates the well-known fact from scientific computing that building as much structural information as is available into an algorithm can enhance its efficiency and/or accuracy.

97 MATHEMATICS AND COMPUTING↗

Multi-Objective Adaptive Mesh Refinement Using Reinforcement Learning

Finite element methods approximate the solution to a partial differential equation (PDE) on a mesh consisting of many elements. In general, using more, smaller elements results in a lower error in the approximation. However, it is often possible to lower the error substantially by only refining, or decreasing the size of, the few elements in the mesh that have the highest error. Adaptive mesh refinement (AMR) is a process that selectively refines regions of a mesh with high error to achieve a desired accuracy in as few degrees of freedom (DOFs) as possible. AMR is favorable compared to uniform refinement, which refines all elements of the mesh equally, because it can often achieve the same accuracy without wasting extra computation time on refinement of elements that already have low error. However, it is difficult to know which elements to refine. In this report we explore ways to choose which elements to refine such that we minimize both the resulting error and the cumulative DOFs used in computation. In particular, we introduce a Pareto-front learning algorithm that trains a policy to give the optimal refinement actions to minimize the cumulative DOFs used to achieve a given target error. Such a policy is useful because it can be deployed on many different problem types where different accuracy levels are desired. Furthermore, training a single policy for a range of target errors allows us to use transfer learning to reduce the required training time.

97 MATHEMATICS AND COMPUTING↗

Numerical methods for nonlocal and fractional models

Partial differential equations (PDEs) are used with huge success to model phenomena across all scientific and engineering disciplines. However, across an equally wide swath, there exist situations in which PDEs fail to adequately model observed phenomena, or are not the best available model for that purpose. On the other hand, in many situations, nonlocal models that account for interaction occurring at a distance have been shown to more faithfully and effectively model observed phenomena that involve possible singularities and other anomalies. Here, we consider a generic nonlocal model, beginning with a short review of its definition, the properties of its solution, its mathematical analysis and of specific concrete examples. We then provide extensive discussions about numerical methods, including finite element, finite difference and spectral methods, for determining approximate solutions of the nonlocal models considered. In that discussion, we pay particular attention to a special class of nonlocal models that are the most widely studied in the literature, namely those involving fractional derivatives. The article ends with brief considerations of several modelling and algorithmic extensions, which serve to show the wide applicability of nonlocal modelling.

97 MATHEMATICS AND COMPUTING↗

Machine learning models for PDE constrained optimization

Partial differential equation (PDE)-constrained optimization problems arise in a variety of scientific and engineering applications, such as topology optimization, electrodynamics, fluid dynamics, and structural dynamics. However, these problems are often challenging and computationally expensive to solve, due to the need to solve the PDEs within the optimization loop. One approach to reducing the computational cost of these methods while providing convergence guarantees is through inexact trust region methods; this method uses lower fidelity solutions of the PDE at early stages of the optimization and adjusts the required accuracy of inexact PDE solvers as the optimization progresses. In this work, we explore the use of machine learning based surrogate models with these inexact trust region methods. We first demonstrate the potential of this approach by using Gaussian processes as the surrogate model and test this on a simple PDE-constrained optimization problem. We then document explorations into improving the computational costs of evolutional deep neural network / neural Galerkin methods, with the eventual goal of using these methods with the inexact trust region algorithms. We are able to speed up these approaches, albeit at the cost of lower accuracy.

97 MATHEMATICS AND COMPUTING↗

Numerical methods for nonlocal and fractional models

Partial differential equations (PDEs) are used, with huge success, to model phenomena arising across all scientific and engineering disciplines. However, across an equally wide swath, there exist situations in which PDE models fail to adequately model observed phenomena or are not the best available model for that purpose. On the other hand, in many situations, nonlocal models that account for interaction occurring at a distance have been shown to more faithfully and effectively model observed phenomena that involve possible singularities and other anomalies. In this article, we consider a generic nonlocal model, beginning with a short review of its definition, the properties of its solution, its mathematical analysis, and specific concrete examples. We then provide extensive discussions about numerical methods, including finite element, finite difference, and spectral methods, for determining approximate solutions of the nonlocal models considered. In that discussion, we pay particular attention to a special class of nonlocal models that are the most widely studied in the literature, namely those involving fractional derivatives. The article ends with brief considerations of several modeling and algorithmic extensions which serve to show the wide applicability of nonlocal modeling.

97 MATHEMATICS AND COMPUTING↗

Quantification of LEU Holdup using gamma ray imaging and inverse transport solver

Holdup is the residual amount of special nuclear material (SNM) remaining in a processing facility after the bulk materials have been cleaned out. In commercial uranium processing facilities, quantification of holdup is a major challenge because of the highly variable shapes and sizes of the deposits. Any method that attempts to generalize and calibrate deposit shapes in order to quantify holdup will be prone to high uncertainties. Uncertainties on the order of ±50% are typical in holdup results. In international safeguards applications, a ±50% uncertainty can result in a large amount of material unaccounted for (MUF) thereby increasing the difficulty of detecting material diversion and facility misuse. An imaging-based methodology has been developed with the objective of significantly reducing this uncertainty by using the true deposit shape, instead of relying on oversimplified geometric assumptions. The project is a collaboration between ORNL, Y-12, and the University of Tennessee, Knoxville, TN. Uranium sources of known masses were measured using the Germanium Gamma-ray Imager (GeGI), a high-resolution imaging spectrometer, creating a pixelated map for each spectral bin. Two different gamma imaging methods are employed in this work: coded aperture imaging and Compton imaging. A validated MonteCarlo model of the detector has been developed using the GEANT4 code for determining the intrinsic response of the detector, its enclosure, and the coded aperture mask. An inverse transport solver based on the Markov Chain Monte-Carlo approach known as Differential Evolution Adaptive Metropolis (DREAM) is employed to use the measurement data from the image pixels (coded aperture or Compton) to solve for the mass of 235 U in the deposit. A reliable method based on the DREAM solver has been developed to flag the infinite thickness condition of a uranium deposit. The project team is working towards improving the image reconstruction for Compton imaging so that a better localization of the source can be achieved. Besides treating the coded aperture and Compton imaging methods independently, the project is also evaluating a combined method that uses the Compton scatter data from a coded aperture measurement. GEANT4 simulations are being performed to evaluate the combined approach. The impact on the DREAM optimization as the source thickness progressively approaches infinite thickness is being evaluated. A number of uranium sources available at ORNL have been measured, and the DREAM results have been tested and validated for the coded aperture imaging. A similar effort will be carried out to validate the Compton based method once the development of algorithms for better localization are complete. The imaging based quantification is very amenable to unattended monitoring of holdup accumulation at key measurement points. A proof of concept measurement has been completed to demonstrate this capability The current work used the high energy resolution imager GeGI. However, the approach and methodologies are applicable to other imagers such as the cadmium zin telluride (CZT) based imager manufactured by H3D, Inc.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗