Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “stochastic approximation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Stochastic Trust-Region Algorithm in Random Subspaces with Convergence and Expected Complexity Analyses

Here, this work proposes a framework for large-scale stochastic derivative-free optimization (DFO) by introducing STARS, a trust-region method based on iterative minimization in random subspaces. This framework is both an algorithmic and theoretical extension of a random subspace derivative-free optimization (RSDFO) framework, and an algorithm for stochastic optimization with random models (STORM). Moreover, like RSDFO, STARS achieves scalability by minimizing interpolation models that approximate the objective in low-dimensional affine subspaces, thus significantly reducing per-iteration costs in terms of function evaluations and yielding strong performance on largescale stochastic DFO problems. The user-determined dimension of these subspaces, when the latter are defined, for example, by the columns of so-called Johnson-Lindenstrauss transforms, turns out to be independent of the dimension of the problem. For convergence purposes, inspired by the analyses of RSDFO and STORM, both a particular quality of the subspace and the accuracies of random function estimates and models are required to hold with sufficiently high, but fixed, probabilities. Using martingale theory under the latter assumptions, an almost sure global convergence of STARS to a first-order stationary point is shown, and the expected number of iterations required to reach a desired first-order accuracy is proved to be similar to that of STORM and other stochastic DFO algorithms, up to constants.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Space-Time Reduced-Order Modeling for Uncertainty Quantification

This work focuses on the space-time reduced-order modeling (ROM) method for solving large-scale uncertainty quantification (UQ) problems with multiple random coefficients. In contrast with the traditional space ROM approach, which performs dimension reduction in the spatial dimension, the space-time ROM approach performs dimension reduction on both the spatial and temporal domains, and thus enables accurate approximate solutions at a low cost. We incorporate the space-time ROM strategy with various classical stochastic UQ propagation methods such as stochastic Galerkin and Monte Carlo. Numerical results demonstrate that our methodology has significant computational advantages compared to state-of-the-art ROM approaches. By testing the approximation errors, we show that there is no obvious loss of simulation accuracy for space-time ROM given its high computational efficiency.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Laboratory Constraints on the Neutron-Spin Coupling of feV-Scale Axions

Ultralight axionlike particles can contribute to the dark matter near the Sun, leading to a distinct, stochastic signature in terrestrial experiments. We search for such particles through their neutron-spin coupling by reanalyzing approximately 40 days of data from a K- 3 He comagnetometer with a new frequency-domain likelihood-based formalism that properly accounts for stochastic effects over all axion coherence times relative to the experimental time span. Assuming that axions make up all of the dark matter in the Sun’s vicinity, we find a median 95% upper limit on the neutron-spin coupling of 2.4 × 10 −10 GeV −1 for most axion masses from 0.4 to 4 feV, which is about 5 orders of magnitude more stringent than previous laboratory bounds in that mass range. Although several peaks in the experiment’s magnetic power spectrum suggest the rejection of a white-noise null hypothesis, further analysis of their line shapes yields no positive evidence for a dark-matter axion.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2008

Energetic Consistency and Coupling of the Mean and Covariance Dynamics

The dynamical state of the ocean and atmosphere is taken to be a large dimensional random vector in a range of large-scale computational applications, including data assimilation, ensemble prediction, sensitivity analysis, and predictability studies. In each of these applications, numerical evolution of the covariance matrix of the random state plays a central role, because this matrix is used to quantify uncertainty in the state of the dynamical system. Since atmospheric and ocean dynamics are nonlinear, there is no closed evolution equation for the covariance matrix, nor for the mean state. Therefore approximate evolution equations must be used. This article studies theoretical properties of the evolution equations for the mean state and covariance matrix that arise in the second-moment closure approximation (third- and higher-order moment discard). This approximation was introduced by EPSTEIN [1969] in an early effort to introduce a stochastic element into deterministic weather forecasting, and was studied further by FLEMING [1971a,b], EPSTEIN and PITCHER [1972], and PITCHER [1977], also in the context of atmospheric predictability. It has since fallen into disuse, with a simpler one being used in current large-scale applications. The theoretical results of this article make a case that this approximation should be reconsidered for use in large-scale applications, however, because the second moment closure equations possess a property of energetic consistency that the approximate equations now in common use do not possess. A number of properties of solutions of the second-moment closure equations that result from this energetic consistency will be established.

Cohn, Stephen E.↗
NASA NTRS1989

Access of energetic particles to storm time ring current through enhanced radial 'diffusion'

It is proposed that the transport of particles with energy of at least approximately 40 keV into the geomagnetic storm time ring current can result from enhanced stochastic radial transport driven by fluctuating electric fields during a storm's main phase. The effects of such electric fields are estimated by applying radial diffusion theory, assuming a preexisting trapped-particle population as the initial condition. The feasibility of explaining observed flux increases of particles of at least approximately 40 keV at no more than approximately 4 earth radii by enhanced radial 'diffusion' is then demonstrated. It is estimated that the at least approximately 40-keV portion of the storm time ring current at approximately 3 earth radii consists of about 50 percent preexisting and about 50 percent new particles. The formation of the storm time ring current is found to be perhaps explainable via a combination of direct radial transport at energies no greater than approximately 40 keV and 'diffusive' radial transport at higher energies.

Lyons, L. R.↗
NASA NTRS1974

Approximations to and local properties of diffusions with discontinuous controls

The paper discusses several properties of control systems defined by stochastic differential equations, which are defined by the method of Girsanov, using a transformation of measures, and where the controls are discontinuous. Uniqueness of the multivariate distributions of the process is proved, and it is shown that the process is a limit, in a natural sense, of a certain discrete time approximation. Other questions, concerning the effects on the distributions of the paths, and of the cost of approximating the control by a smooth control and concerning local properties of the solution, are discussed.

Kushner, H. J.↗
DOE OSTI2024

Multistart algorithm for identifying all optima of nonconvex stochastic functions

Here, we propose a multistart algorithm to identify all local minima of a constrained, nonconvex stochastic optimization problem. The algorithm uniformly samples points in the domain and then starts a local stochastic optimization run from any point that is the "probabilistically best" point in its neighborhood. Under certain conditions, our algorithm is shown to asymptotically identify all local optima with high probability; this holds even though our algorithm is shown to almost surely start only finitely many local stochastic optimization runs. We demonstrate the performance of an implementation of our algorithm on nonconvex stochastic optimization problems, including identifying optimal variational parameters for the quantum approximate optimization algorithm.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Tensor Network Quantum Virtual Machine for Simulating Quantum Circuits at Exascale

The numerical simulation of quantum circuits is an indispensable tool for development, verification, and validation of hybrid quantum-classical algorithms intended for near-term quantum co-processors. The emergence of exascale high-performance computing (HPC) platforms presents new opportunities for pushing the boundaries of quantum circuit simulation. Here, we present a modernized version of the Tensor Network Quantum Virtual Machine (TNQVM) that serves as the quantum circuit simulation backend in the eXtreme-scale ACCelerator (XACC) framework. The new version is based on the scalable tensor network processing library ExaTN (Exascale Tensor Networks). It provides multiple configurable quantum circuit simulators that perform either an exact quantum circuit simulation via the full tensor network contraction or an approximate simulation via a suitably chosen tensor factorization scheme. Upon necessity, stochastic noise modeling from real quantum processors is incorporated into the simulations by modeling quantum channels with Kraus tensors. By combining the portable XACC quantum programming frontend and the scalable ExaTN numerical processing backend, we introduce an end-to-end virtual quantum development environment that can scale from laptops to future exascale platforms. We report initial benchmarks of our framework, which include a demonstration of the distributed execution, incorporation of quantum decoherence models, and simulation of the random quantum circuits used for the certification of quantum supremacy on Google’s Sycamore superconducting architecture.

Nguyen, Thien↗
DOE OSTI2024

Variance-Reduced Accelerated First-Order Methods: Central Limit Theorems and Confidence Statements

In this paper, we consider a strongly convex stochastic optimization problem and propose three classes of variable sample-size stochastic first-order methods: (i) the standard stochastic gradient descent method, (ii) its accelerated variant, and (iii) the stochastic heavy-ball method. In each scheme, the exact gradients are approximated by averaging across an increasing batch size of sampled gradients. We prove that when the sample size increases at a geometric rate, the generated estimates converge in mean to the optimal solution at an analogous geometric rate for schemes (i)–(iii). Based on this result, we provide central limit statements, whereby it is shown that the rescaled estimation errors converge in distribution to a normal distribution with the associated covariance matrix dependent on the Hessian matrix, the covariance of the gradient noise, and the step length. If the sample size increases at a polynomial rate, we show that the estimation errors decay at a corresponding polynomial rate and establish the associated central limit theorems (CLTs). Under certain conditions, we discuss how both the algorithms and the associated limit theorems may be extended to constrained and nonsmooth regimes. As a result, we provide an avenue to construct confidence regions for the optimal solution based on the established CLTs and test the theoretical findings on a stochastic parameter estimation problem.

Lei, Jinlong↗
DOE OSTI2025

Bias-Variance Trade-Off in Physics-Informed Neural Networks with Randomized Smoothing for High-Dimensional PDEs

Physics-Informed Neural Networks (PINNs) have triggered a paradigm shift in scientific computing, leveraging mesh-free properties and robust approximation capabilities. While proving effective for low-dimensional partial differential equations (PDEs), the computational cost of PINNs remains a hurdle in high-dimensional scenarios. This is particularly pronounced when computing high-order and high-dimensional derivatives in the physics-informed loss. Randomized Smoothing PINN (RS-PINN) introduces Gaussian noise for stochastic smoothing of the original neural net model, enabling the use of Monte Carlo methods for derivative approximation, which eliminates the need for costly automatic differentiation. Despite its computational efficiency, especially in the approximation of high-dimensional derivatives, RS-PINN introduces biases in both loss and gradients, negatively impacting convergence, especially when coupled with stochastic gradient descent (SGD) algorithms. We present a comprehensive analysis of biases in RS-PINN, attributing them to the nonlinearity of the Mean Squared Error (MSE) loss as well as the intrinsic nonlinearity of the PDE itself. We propose tailored bias correction techniques, delineating their application based on the order of PDE nonlinearity. The derivation of an unbiased RS-PINN allows for a detailed examination of its advantages and disadvantages compared to the biased version. Specifically, the biased version has a lower variance and runs faster than the unbiased version, but it is less accurate due to the bias. To optimize the bias-variance trade-off, we combine the two approaches in a hybrid method that balances the rapid convergence of the biased version with the high accuracy of the unbiased version. In addition to methodological contributions, we present an enhanced implementation of RS-PINN. Extensive experiments on diverse high-dimensional PDEs, including Fokker-Planck, Hamilton-Jacobi-Bellman (HJB), viscous Burgers’, Allen-Cahn, and Sine-Gordon equations, illustrate the bias-variance trade-off and highlight the effectiveness of the hybrid RS-PINN. Empirical guidelines are provided for selecting biased, unbiased, or hybrid versions, depending on the dimensionality and nonlinearity of the specific PDE problem.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Comprehensive uncertainty quantification (UQ) for full engineering models by solving probability density function (PDF) equation

This report details a new method for propagating parameter uncertainty (forward uncertainty quantification) in partial differential equations (PDE) based computational mechanics applications. The method provides full-field quantities of interest by solving for the joint probability density function (PDF) equations which are implied by the PDEs with uncertain parameters. Full-field uncertainty quantification enables the design of complex systems where quantities of interest, such as failure points, are not known apriori. The method, motivated by the well-known probability density function (PDF) propagation method of turbulence modeling, uses an ensemble of solutions to provide the joint PDF of desired quantities at every point in the domain. A small subset of the ensemble is computed exactly, and the remainder of the samples are computed with approximation of the driving (dynamics) term of the PDEs based on those exact solutions. Although the proposed method has commonalities with traditional interpolatory stochastic collocation methods applied directly to quantities of interest, it is distinct and exploits the parameter dependence and smoothness of the dynamics term of the governing PDEs. The efficacy of the method is demonstrated by applying it to two target problems: solid mechanics explicit dynamics with uncertain material model parameters, and reacting hypersonic fluid mechanics with uncertain chemical kinetic rate parameters. A minimally invasive implementation of the method for representative codes SPARC (reacting hypersonics) and NimbleSM (finite- element solid mechanics) and associated software details are described. For solid mechanics demonstration problems the method shows order of magnitudes improvement in accuracy over traditional stochastic collocation. For the reacting hypersonics problem, the method is implemented as a streamline integration and results show very good accuracy for the approximate sample solutions of re-entry flow past the Apollo capsule geometry at Mach 30.

42 ENGINEERING↗
DOE OSTI2023

COMPUTATIONAL MODELING OF IGNITION AND PREMIXED FLAME PROPAGATION INITIATED BY A PRE-CHAMBER TURBULENT JET

Addressing the pressing need for reduced carbon emissions, Turbulent Jet Ignition (TJI) emerges as a promising technology for ultra-lean combustion, offering enhanced thermal efficiencies and minimized cyclic variability in spark-ignited engines. To facilitate rapid testing and integration of this technology, a robust computational modeling framework is crucial. This study delves into the predictive capabilities of computational models for main-chamber ignition and premixed flame propagation using a single-cycle TJI rig measured by Biswas et al. (Applied Thermal Engineering, vol 106, 2016). Employing an open-source compressible flow simulation solver with Large Eddy Simulation (LES) for turbulence modeling, the investigation integrates the conventional Laminar Finite Rate Chemistry (LFRC) model alongside the transported Probability Density Method (PDF) for turbulence-chemistry interaction. A fully-consistent Eulerian Monte-Carlo Fields (EMCF) method is utilized to approximate the transported PDF, while Interaction by Exchange with Mean is employed to close micro-mixing terms in stochastic differential equations. A reduced chemical reaction mechanism with 21 species and 84 reactions (DRM-19) is used for solving chemical kinetics, and a double Gaussian energy deposition model is used to approximate the spark ignition in the pre-chamber. An unstructured O-grid mesh with 0.3 million cells in the prechamber and 1 million cells in the main chamber is employed. Results are divided into two phases: pre-chamber initialization and full TJI simulations. Validation of the predicted pre-chamber flame propagation and the lean ignition in the main-chamber is carried out by using available experimental data. Under quiescent conditions, both the LFRC and transported PDF methods largely underestimate the flame speed and subsequent pressure growth in the pre-chamber. A linear momentum forcing technique is applied to investigate the impact of initial turbulence in the pre-chamber, demonstrating a notable influence on flame propagation. Fine-tuning of the forcing coefficient reproduces the sudden pressure growth observed in the experiment. The experimentally validated pre-chamber simulation serves as the initial condition for the full TJI simulations. It is found that the LFRC model fails to predict lean-ignition in the main-chamber, resulting in a misfiring event. Incorporation of turbulence-chemistry interaction using the transported PDF method substantially improves the prediction of the ignition event in the main-chamber, achieving fair qualitative agreement and quantitative validation of combustion parameters within ±10% of the reported experimental data. The rich simulation results consisting of a full set of statistical description of the thermo-chemical states enable us to gain deep insights into the ignition mechanisms in the main chamber, which is limited when done experimentally. A novel dual ignition phenomenon is revealed in the TJI rig for the first time. Initially, a primary ignition kernel is formed at a downstream location which eventually detaches from the main jet. As the jet momentum decreases, a secondary ignition event follows, this time at a more upstream location which eventually combines with the primary ignition kernel to form a single connected flame front. Investigation of these ignition sequences in chemical composition space reveal distinct differences between the two. The primary ignition event in the main-chamber is followed by a large concentration of active radicals from the pre-chamber jet, accelerating the chain-branching steps, characterizing what has been referred to as flame ignition. In contrast, the secondary ignition occurs in the absence of active radicals in the pre-chamber jet, hence characterized as jet ignition. Further analysis of the effect of pre-chamber jet characteristics on lean ignition in the main-chamber is conducted by setting up cases with different initial pressure ratios (por) between the two chambers, a non-dimensional parameter, ranging from 1.2 to 3.2. As the initial pressure ratio increases, jet momentum increases, with dual ignition observed in cases above por= 2.2. Case with por= 3.2 lead to misfiring. The effect of ignition sequence on global combustion characteristics of TJI is analyzed. Dual ignition events lead to non-monotonicity in combustion characteristics such as global reaction progress variable, flame penetration, and global heat release rate. In dual ignition events, although the rate of fuel consumption and global heat release rate is initially lower, the secondary ignition leads to a sudden increase in flame surface area, resulting in a sudden jump and promoting the overall performance of the TJI system.

42 ENGINEERING↗
DOE OSTI2024

Numerical analysis of a time discretized method for nonlinear filtering problem with Lévy process observations

Abstract In this paper, we consider a nonlinear filtering model with observations driven by correlated Wiener processes and point processes. We first derive a Zakai equation whose solution is an unnormalized probability density function of the filter solution. Then, we apply a splitting-up technique to decompose the Zakai equation into three stochastic differential equations, based on which we construct a splitting-up approximate solution and prove its half-order convergence. Furthermore, we apply a finite difference method to construct a time semi-discrete approximate solution to the splitting-up system and prove its half-order convergence to the exact solution of the Zakai equation. Finally, we present some numerical experiments to demonstrate the theoretical analysis.

Mathematics↗
DOE OSTI2022

maestro

MÆSTRO stands for Multi-fidelity Adaptive Ensemble Stochastic Trust Region Optimization and it is a plug n play derivate fee stochastic optimization solver. The problem being considered in MÆSTRO involves fitting Monte Carlo simulations that describe complex phenomena to experiments. This is done by finding parameters of the resource intensive and noisy simulation that yield the least squares objective function value to the noisy experimental data. This problem is solved using a stochastic trust-region optimization algorithm where in each iteration, a local approximation of the simulation signal and of the simulation noise is constructed over data, which is obtained by running the simulation at strategically placed design points within the trust-region around the current iterate. Then the simulation components of the objective are replaced by their approximations and this analytical and closed-form optimization problem is solved to find the next iterate within the trust-region. Then the trust region is moved and the iterations continue until a satisfactory convergence criteria is met.

KRISHNAMOORTHY, MOHAN↗
NASA NTRS2003

A Eulerian-Lagrangian Model to Simulate Two-Phase/Particulate Flows

Figure 1 shows a snapshot of liquid fuel spray coming out of an injector nozzle in a realistic gas-turbine combustor. Here the spray atomization was simulated using a stochastic secondary breakup model (Apte et al. 2003a) with point-particle approximation for the droplets. Very close to the injector, it is observed that the spray density is large and the droplets cannot be treated as point-particles. The volume displaced by the liquid in this region is significant and can alter the gas-phase ow and spray evolution. In order to address this issue, one can compute the dense spray regime by an Eulerian-Lagrangian technique using advanced interface tracking/level-set methods (Sussman et al. 1994; Tryggvason et al. 2001; Herrmann 2003). This, however, is computationally intensive and may not be viable in realistic complex configurations. We therefore plan to develop a methodology based on Eulerian-Lagrangian technique which will allow us to capture the essential features of primary atomization using models to capture interactions between the fluid and droplets and which can be directly applied to the standard atomization models used in practice. The numerical scheme for unstructured grids developed by Mahesh et al. (2003) for incompressible flows is modified to take into account the droplet volume fraction. The numerical framework is directly applicable to realistic combustor geometries. Our main objectives in this work are: Develop a numerical formulation based on Eulerian-Lagrangian techniques with models for interaction terms between the fluid and particles to capture the Kelvin- Helmholtz type instabilities observed during primary atomization. Validate this technique for various two-phase and particulate flows. Assess its applicability to capture primary atomization of liquid jets in conjunction with secondary atomization models.

Apte, S. V.↗
DOE OSTI2022

VERA neutronics high-fidelity benchmark for a modern PWR core design

Nearly 20 years since the origin of the infamous 'Kord Smith Challenge', and after 10 years and hundreds of millions of dollars invested by the US Department of Energy into the development of the Virtual Environment for Reactor Applications (VERA), the capability to accurately simulate a modern pressurized water reactor (PWR) fuel cycle at the fuel rod level with high-fidelity Monte Carlo (MC) stochastic transport methods still evades us. The deterministic methods in VERA make approximations that allow it to be successfully benchmarked against measured data from hundreds of PWR fuel cycles, but none of these benchmarks provided information at the local fuel rod level or allowed quantification of these approximations' impact for modern PWR reload designs. Using MPACT to establish the estimated isotopic distributions and thermal hydraulic conditions in a recent and challenging reactor design, a 2D quarter-core benchmark problem was established and solved consistently with both MPACT, the 51-energy group deterministic transport solver in VERA, and Shift, the continuous-energy MC transport solver in VERA. Impacts of energy groups, ray spacing, and scattering treatment were evaluated. Though MPACT has been previously shown to perform well compared to MC methods for initial Cycle 1 cores with only fresh fuel, this is the first benchmark of MPACT for a challenging modern reload core with depleted fuel and burnable absorbers. This work establishes additional credibility for the methods in MPACT and highlights the efficiency of these methods relative to those in use by MC-based tools. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗