Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “stochastic gradient”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Efficient Bayesian inference with latent Hamiltonian neural networks in No-U-Turn Sampling

When sampling for Bayesian inference, one popular approach in the computational field is to use Hamiltonian Monte Carlo (HMC) and specifically the No-U-Turn Sampler (NUTS), which automatically decides the end time of the Hamiltonian trajectory. However, HMC and NUTS can require numerous numerical gradients of the target density and can prove slow in practice when relying on computationally expensive forward models. We propose Latent Hamiltonian neural networks (L-HNNs) with HMC and NUTS for solving Bayesian inference problems. Once trained, L-HNNs do not require numerical gradients of the target density during sampling, and hence numerous evaluations of the forward computational model. Moreover, L-HNNs satisfy important properties such as perfect time reversibility and Hamiltonian conservation, making them well-suited for use within HMC and NUTS because stationarity can be shown. We also propose the integration of L-HNNs in an online error monitoring scheme, in which numerical gradients of the target density are used for a few samples whenever the L-HNNs prediction errors are large. This online error monitor scheme prevents sample degeneracy in regions of low probability density and ensures robust uncertainty quantification. We demonstrate L-HNNs in NUTS with online error monitoring on several analytical examples involving complex, heavy-tailed, and high-local-curvature probability densities. We then demonstrate the applicability of L-HNNs in NUTS to two computational case studies, namely the Allen-Cahn stochastic partial differential equation and an elliptic partial differential equation with 25 and 50 inference parameters, respectively. Overall, the L-HNNs in NUTS with online error monitoring satisfactorily inferred these probability densities. In conclusion, compared to traditional NUTS, L-HNNs in NUTS with online error monitoring required 1–2 orders of magnitude fewer numerical gradients of the target density and improved the effective sample size (ESS) per gradient (which is a measure of both the sampling quality and the computational expense) by an order of magnitude.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Functional Traits Resolve Mechanisms Governing the Assembly and Distribution of Nitrogen-Cycling Microbial Communities in the Global Ocean

Microorganisms drive much of the marine nitrogen (N) cycle, which jointly controls the primary production in the global ocean. However, our understanding of the microbial communities driving the global ocean N cycle remains fragmented. Focusing on “who is doing what, where, and how?”, this study draws a clear picture describing the global biogeography of marine N-cycling microbial communities by utilizing the Tara Oceans shotgun metagenomes. The marine N-cycling communities are highly variable taxonomically but relatively even at the functional trait level, showing clear functional redundancy properties. The functional traits and taxonomic groups are shaped by the same set of geo-environmental factors, among which, depth is the major factor impacting marine N-cycling communities, differentiating mesopelagic from epipelagic communities. Latitudinal diversity gradients and distance-decay relationships are observed for taxonomic groups, but rarely or weakly for functional traits. The composition of functional traits is strongly deterministic as revealed by null model analysis, while a higher degree of stochasticity is observed for taxonomic composition. Integrating multiple lines of evidence, in addition to drawing a biogeographic picture of marine N-cycling communities, this study also demonstrated an essential microbial ecological theory—determinism governs the assembly of microbial communities performing essential biogeochemical processes; the environment selects functional traits rather than taxonomic groups; functional redundancy underlies stochastic taxonomic community assembly.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Specificity and tunability of efflux pumps: A new role for the proton gradient?

Efflux pumps that transport antibacterial drugs out of bacterial cells have broad specificity, commonly leading to broad spectrum resistance and limiting treatment strategies for infections. It remains unclear how efflux pumps can maintain this broad spectrum specificity to diverse drug molecules while limiting the efflux of other cytoplasmic content. We have investigated the origins of this broad specificity using theoretical models informed by the experimentally determined structural and kinetic properties of efflux pumps. We developed a set of mathematical models describing operation of efflux pumps as a discrete cyclic stochastic process across a network of states characterizing pump conformations and the presence/absence of bound ligands and protons. These include a minimal three-state model that lends itself to clear analytic calculations as well as a five-state model that relaxes some of the simpler model’s most strict assumptions. We found that the pump specificity is determined not solely by the drug affinity to the pump–as is commonly assumed–but it is also directly affected by the periplasmic pH and the transmembrane potential. Therefore, changes to the proton concentration gradient and voltage drop across the membrane can influence how effective the pump is at extruding a particular drug molecule. Furthermore, we found that while both the proton concentration gradient across the membrane and the transmembrane potential contribute to the thermodynamic force driving the pump, their effects on the efflux enter not strictly in a combined proton motive force. Rather, they have two distinguishable effects on the overall throughput. These results highlight the unexpected effects of thermodynamic driving forces out of equilibrium and illustrate how efflux pump structure and function are conducive to the emergence of multidrug resistance.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Temperature drives local contributions to beta diversity in mountain streams: stochastic and deterministic processes

Community variation (i.e. beta diversity) in geographical coenoclines is of substantial interest in ecology, yet the elevational patterns in beta diversity are still understudied, and no general conclusions have been reached. Here, we explored local contributions to beta diversity (LCBD) of stream bacteria, diatoms and macroinvertebrates on mountainsides. Across all taxa, we revealed a general U-shaped LCBD-elevation relationships (LCBDer), suggesting higher uniqueness of community composition at the two ends of the elevational gradients. This pattern was confirmed by coenocline simulations based on observed communities, and supported by the dominance of species replacement in total beta diversity. Across mountains and taxonomic groups, temperature was the main environmental factor underlying elevational patterns in LCBD and its metric relatives, which were also mediated by local environmental variables. The observed magnitudes of U-shaped LCBDer across mountains and taxonomic groups were specifically related to the heterogeneity of stream morphology, and the mean annual temperature on each mountain. U-shaped LCBDer were strengthened by the occurrence of generalist species with large elevational ranges, while rare species had a lesser effect on the patterns. Raup-Crick beta diversity indicated that both stochastic and deterministic processes drove the community assembly, and the latter one was positively related to the magnitudes of LCBDer. Our synthesis across mountains and taxonomic groups clearly shows that there are consistent elevational patterns in LCBD among taxonomic groups, and that these patterns are explained by similar ecological mechanisms, producing a more complete picture in understanding and bridging the spatial variation in biodiversity under changing climate.

Wang, Jianjun↗
DOE OSTI2026

A Stochastic Quasi-Newton Method in the Absence of Common Random Numbers

We present Q-SASS, a quasi-Newton method for unconstrained stochastic optimization that does not rely on common random numbers. Most existing quasi-Newton approaches leverage common random numbers to construct second-order updates. However, motivated by challenges in variational quantum algorithms—where such coordination is not possible—we consider the setting in which function values and gradients are accessible only through noisy probabilistic zeroth- and first-order oracles, and no common random numbers can be exploited. We derive high-probability tail bounds on the iteration complexity of our algorithm for nonconvex, convex, and strongly convex (more generally, those satisfying the PL condition) objective functions. Finally, we demonstrate the empirical benefits of our quasi-Newton updating scheme on both synthetic and quantum chemistry problems.

Complexity bound↗
DOE OSTI2024

Correlation Between Microscopic Current Fluctuations Observed at Ultra-Microelectrodes and Macroscopic Bulk Electrolysis Performance in Redox-Active Microemulsions

Microemulsions (μEs) have been proposed as redox flow battery (RFB) electrolytes that maximize ionic conductivity and charge capacity by synergizing two immiscible phases. However, charge transfer during electrolysis in μEs is poorly understood. Here, we show that ultramicroelectrode electrolysis of ferrocene-loaded μEs –20%, 60%, and 90% water - reveals stochastic current fluctuations. These are differentiated in the scanning electrochemical microscopy (SECM) geometry, where power spectral density analysis showed distinct changes in the frequency contributions. SECM in the substrate generation-tip collection mode showed that fluctuations arise under mass-transfer control. Significant differences in the diffusion coefficient of ferrocene species were deducted from SECM approach curves, suggesting phase transfer behavior. Using bulk electrolysis, we calculated the charge accessibility and cycling behavior in the μEs. A decrease in the stochastic behavior of the μEs seems to correlate to a higher accessibility and cycling performance, with the 90% water μE displaying the best reversibility and the 60% the lowest. Altogether, these results suggest that Marangoni-type convection driven by concentration gradients and/or μE restructuring during charge transfer play a role in the electrochemical performance of μEs. This presents opportunities for screening and diagnosing the performance of these emerging RFB electrolytes.

25 ENERGY STORAGE↗
DOE OSTI2024

Differentiable stochastic halo occupation distribution

ABSTRACT In this work, we demonstrate how differentiable stochastic sampling techniques developed in the context of deep reinforcement learning can be used to perform efficient parameter inference over stochastic, simulation-based, forward models. As a particular example, we focus on the problem of estimating parameters of halo occupation distribution (HOD) models that are used to connect galaxies with their dark matter haloes. Using a combination of continuous relaxation and gradient re-parametrization techniques, we can obtain well-defined gradients with respect to HOD parameters through discrete galaxy catalogue realizations. Having access to these gradients allows us to leverage efficient sampling schemes, such as Hamiltonian Monte Carlo, and greatly speed up parameter inference. We demonstrate our technique on a mock galaxy catalogue generated from the Bolshoi simulation using a standard HOD model and find near-identical posteriors as standard Markov chain Monte Carlo techniques with an increase of ∼8× in convergence efficiency. Our differentiable HOD model also has broad applications in full forward model approaches to cosmic structure and cosmological analysis.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Stochastic Approximation for Multi-period Simulation Optimization with Streaming Input Data

We consider a continuous-valued simulation optimization (SO) problem, where a simulator is built to optimize an expected performance measure of a real-world system while parameters of the simulator are estimated from streaming data collected periodically from the system. At each period, a new batch of data is combined with the cumulative data and the parameters are re-estimated with higher precision. The system requires the decision variable to be selected in all periods. Therefore, it is sensible for the decision-maker to update the decision variable at each period by solving a more precise SO problem with the updated parameter estimate to reduce the performance loss with respect to the target system. We define this decision-making process as the multi-period SO problem and introduce a multi-period stochastic approximation (SA) framework that generates a sequence of solutions. Two algorithms are proposed: Re-start SA (ReSA) reinitializes the stepsize sequence in each period, whereas Warm-start SA (WaSA) carefully tunes the stepsizes, taking both fewer and shorter gradient-descent steps in later periods as parameter estimates become increasingly more precise. We show that under suitable strong convexity and regularity conditions, ReSA and WaSA achieve the best possible convergence rate in expected sub-optimality either when an unbiased or a simultaneous perturbation gradient estimator is employed, while WaSA accrues significantly lower computational cost as the number of periods increases. In addition, we present the regularized ReSA, which obviates the need to know the strong convexity constant and achieves the same convergence rate at the expense of additional computation.

Computer Science↗
DOE OSTI2025

On the Sampling-Based Computation of Nash Equilibria Under Uncertainty via the Nikaido–Isoda Function

We consider the computation of an equilibrium of a stochastic Nash equilibrium problem, where the player objectives are assumed to be L 0 -Lipschitz continuous and convex, given rival decisions with convex and closed player-specific feasibility sets. To address this problem, we consider minimizing a suitably defined value function defined using the Nikaido–Isoda function. Such an avenue does not necessitate either monotonicity properties of the concatenated gradient map or potentiality requirements on the game but does require a suitable regularity requirement under which a stationary point is a Nash equilibrium. We design and analyze a sampling-enabled projected-gradient-response method, reliant on inexact resolution of a player-level best-response subproblem. Here, by deriving suitable Lipschitzian guarantees on the value function, we derive both asymptotic guarantees for the sequence of generated iterates as well as rate and complexity guarantees for computing a stationary point by appropriate choices of the sampling rate and inexactness sequence.

Nikaido-Isoda function↗
DOE OSTI2024

Coupled Thermo-Hydro-Mechanical Processes in Fractured Rocks: Some Past Scientific Highlights and Future Research Directions

Abstract Coupled thermo-hydro-mechanical (THM) processes in fractured rocks have been a topic of intense scientific research for more than 30 years. The present paper takes a look into the past and highlights some scientific advances which are of an unusual “out-of-the-box” nature, and then looks forward and discusses possible directions of future research in this interesting field of study. Concerning future research directions, we see a trend from a focus on coupled THM processes in single fractures or a few interacting fractures, to the study of coupled THM behavior in complex fracture network systems where the fractures act collectively giving rise to local stress concentration points and points of large pressure gradients. Three examples of future research directions are presented. First is an effort towards identifying characterizing parameters of a fracture network that play a direct controlling role in major coupled THM phenomena (such as induced seismicity and flow channeling), rather than parameters of stochastic distributions of fractures in the network. The second example of research direction is accounting for the heterogeneity and hierarchy of fractures in a fault or fracture zone which has been associated with major THM events in a number of geo-energy projects. The third example is at the opposite end of the first; here it is recognized that in some cases, the coupled THM processes in fractured rocks may be controlled dominantly by only a few key bridges. Identification, characterization, and evaluation of these key bridges should be one of the important research directions in the coming days.

58 GEOSCIENCES↗
DOE OSTI2022

Stochastic Learning Approach for Binary Optimization: Application to Bayesian Optimal Design of Experiments

Here, we present a novel stochastic approach to binary optimization suited for optimal experimental design (OED) for Bayesian inverse problems governed by mathematical models such as partial differential equations. The OED utility function, namely, the regularized optimality criterion, is cast into a stochastic objective function in the form of an expectation over a multivariate Bernoulli distribution. The probabilistic objective is then solved by using a stochastic optimization routine to find an optimal observational policy. This formulation (a) is generally applicable to binary optimization problems with soft constraints and is ideal for OED and sensor placement problems; (b) does not require differentiability of the original objective function (e.g., a utility function in OED applications) with respect to the design variable, and thus it enables direct employment of sparsity-enforcing penalty functions such as $\ell_0$, without needing to utilize a continuation procedure or apply a rounding technique; (c) exhibits much lower computational cost than traditional gradient-based relaxation approaches; and (d) can be applied to both linear and nonlinear OED problems with proper choice of the utility function. The proposed approach is analyzed from an optimization perspective with detailed convergence analysis of the optimization approach and is also analyzed from a machine learning perspective with correspondence to policy gradient reinforcement learning. The approach is demonstrated numerically by using an idealized two-dimensional Bayesian linear inverse problem and validated by extensive numerical experiments carried out for sensor placement in a parameter identification setup.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

An adaptive sampling augmented Lagrangian method for stochastic optimization with deterministic constraints

The primary goal of this paper is to provide an efficient solution algorithm based on the augmented Lagrangian framework for optimization problems with a stochastic objective function and deterministic constraints. Our main contribution is combining the augmented Lagrangian framework with adaptive sampling, resulting in an efficient optimization methodology validated with practical examples. To achieve the presented efficiency, here we consider inexact solutions for the augmented Lagrangian subproblems, and through an adaptive sampling mechanism, we control the variance in the gradient estimates. Furthermore, we analyze the theoretical performance of the proposed scheme by showing equivalence to a gradient descent algorithm on a Moreau envelope function, and we prove sublinear convergence for convex objectives and linear convergence for strongly convex objectives with affine equality constraints. The worst-case sample complexity of the resulting algorithm, for an arbitrary choice of penalty parameter in the augmented Lagrangian function, is $\mathscr{O}$(ϵ -3-δ ) , where ϵ > 0 is the expected error of the solution and δ > 0 is a user-defined parameter. If the penalty parameter is chosen to be $\mathscr{O}$(ϵ -1 ), we demonstrate that the result can be improved to $\mathscr{O}$(ϵ -2 ) , which is competitive with the other methods employed in the literature. Moreover, if the objective function is strongly convex with affine equality constraints, we obtain $\mathscr{O}$(ϵ -1 log(1/ϵ)) complexity. Finally, we empirically verify the performance of our adaptive sampling augmented Lagrangian framework in machine learning optimization and engineering design problems, including topology optimization of a heat sink with environmental uncertainty.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A parallel discrete dislocation dynamics/kinetic Monte Carlo method to study non-conservative plastic processes

Non-conservative processes play a fundamental role in plasticity and are behind important macroscopic phenomena such as creep, dynamic strain aging, loop raft formation, etc. In the most general case, vacancy-induced dislocation climb is the operating unit mechanism. While dislocation/vacancy interactions have been modeled in the literature using a variety of methods, the approaches developed rely on continuum descriptions of both the vacancy population and its fluxes. However, there are numerous situations in physics where point defect populations display heterogeneous concentrations and/or non-smooth kinetics. Here, a kinetic Monte Carlo (kMC) approach for modeling vacancy transport in response to arbitrary stress fields is used. Vacancies are treated as point particles and are coupled to the dislocation substructure representing a deformed material via an advection term defined by the local stress gradients. The stress fields and the dislocation substructure are evolved using a discrete dislocation dynamics (DDD) module. To extend the coupled model to the treatment of large systems, we have implemented it in the massively-parallel DDD code ParaDiS. To avoid numerical incompatibilities associated with merging deterministic (DDD) and stochastic (kMC) integration algorithms, we cast the entire elasto-plastic-diffusive problem within a single stochastic framework, taking advantage of a parallel kMC algorithm to evolve the system as a single event-driven process. The large-scale implementation enables the study of the evolution of a variety of dislocation-defect scenarios governed by non-conservative transport kinetics. After carrying out an exhaustive numerical and computational analysis of our parallel algorithm, we show results that emphasize situations where inhomogeneous vacancy dynamics are of relevance, and compare discrete kinetics to continuum solutions for several cases.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Multiscale simulation of spatially correlated microstructure via a latent space representation

When deformation gradients act on the scale of the microstructure of a part due to geometry and loading, spatial correlations and finite-size effects in simulation cells cannot be neglected. We propose a multiscale method that accounts for these effects using a variational autoencoder to encode the structure–property map of the stochastic volume elements making up the statistical description of the part. In this paradigm the autoencoder can be used to directly encode the microstructure or, alternatively, its latent space can be sampled to provide likely realizations. Furthermore, we demonstrate the method on three examples using the common additively manufactured material AlSi10Mg in: (a) a comparison with direct numerical simulation of the part microstructure, (b) a push forward of microstructural uncertainty to performance quantities of interest, and (c) a simulation of functional gradation of a part with stochastic microstructure.

Elastoplasticity↗
DOE OSTI2025

Deep-learning-enhanced assessment of wellbore barrier effectiveness in geologic storage systems with intermediate aquifers

For geologic systems where carbon dioxide (CO 2 ) is injected underground, existing wells represent potential pathways for fluid migration. Here, this study introduces a novel deep learning model to quantify the likelihood and potential magnitude of fluid migration through wellbores at sites with intermediate aquifers or thief zones between the injection units and underground drinking water sources. Synthetic datasets, generated using reservoir simulations, captured a wide range of subsurface conditions, well attributes, operational parameters, and fluid migration scenarios. Among the regression models developed to predict brine and CO 2 leakage rates and CO 2 saturations along leaky wellbores, convolutional neural network (CNN) outperformed both Light Gradient Boosting Machine and deep neural network. Additionally, a CNN-based classification model was created to predict whether brine and CO 2 would leak along a wellbore, further improving performance over regression alone. The best models were integrated into the National Risk Assessment Partnership Open-source Integrated Assessment Model for rapid, stochastic assessment of storage system containment and leakage risks. A case study demonstrated the model’s ability to simulate fluid migration through existing wells with multiple intermediate aquifers. This computationally efficient wellbore model offers value in support of site performance evaluation and risk-informed decision making by stakeholders.

CO2 leakage↗
DOE OSTI2025

Thermo-Mechanical Phase-Field Modeling of Fracture in High-Burnup UO2 Fuels Under Transient Conditions

This study presents a novel multiphysics phase-field fracture model to analyze high-burnup uranium dioxide (UO2) fuel behavior under transient reactor conditions. Fracture is treated as a stochastic phase transition, which inherently accounts for the random microstructural effects that lead to variations in the value of fracture strength. Moreover, the model takes into consideration the effects of temperature and burnup on thermal conductivity. Therefore, the model is able to predict crack initiation, propagation, and complex morphologies in response to thermal gradients and stress distributions. Several simulations were conducted to investigate the effects of operational and transient conditions on fracture behavior and the resulting cracking patterns. High-burnup fuels exhibit reduced thermal conductivity, elevating temperature gradients and resulting in extensive radial and circumferential cracks. Transient heating rates and temperatures significantly affect fracture patterns, with higher heating rates generating steeper gradients and more irregular crack trajectories. This approach provides critical insights into fuel integrity during accident scenarios and supports the safety evaluation of extended burnup limits.

Chemistry↗
DOE OSTI2023

Effect of particle size on the capture of uranium oxide colloidal particles from aqueous suspensions via high-gradient magnetic filtration

The effectiveness of High Gradient Magnetic Filtration (HGMF) in capturing uranium oxide particles from suspensions was investigated in this study. Two sets of experiments were performed to evaluate the importance of size on the capture of uranium oxide particles. The first considered two batches sieved into size bins of< 5, 5–10, 10–15, and 15–20 µm, while the second was performed using two suspensions with diameters smaller than 1.0 µm and between 1.0 and 1.5 µm. Iron oxide experiments, with particles between 0.3 and 0.8 µm, were performed for calibration purposes. In all experiments, a surfactant (Triton-X100 or sodium dodecyl sulfate) was used to prevent particle aggregation and limit the influence of non-magnetic capture mechanisms. A magnetic field of approximately 1.1 Tesla was generated using a water cooled electromagnet. HGMF was performed using tubular filters packed with ferromagnetic stainless-steel wool. Of the initial four uranium oxide particle sizes, magnetic capture was only observed for particles with a diameter of less than 5 µm, while larger particles experienced no magnetic and minimal total capture. For particles with diameters smaller than 1.0 µm and between 1.0 and 1.5 µm, capture efficiencies increased by 39 ± 9% and 34 ± 6% respectively, solely due to the magnetic field. Although the magnetic force is proportional to particle diameter, the capture efficiency decreased as diameter increased. So these results suggest that Brownian diffusion, which is influential for micron sized particles and increases with decreasing particle size, is acting in conjunction with the magnetic force to influence the efficacy of HGMF for uranium oxide. This important finding underscores the effectiveness of Brownian diffusion in increasing the rate of collision between particles and collector fibers. A stochastic trajectory model was developed to incorporate the influence of Brownian motion on particle behavior and filter removal efficiency. Modeling results are discussed and compared for uranium and iron oxide particles.

42 ENGINEERING↗