SEARCH · Engineering Papers

Results for “Stochastic modelling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,495 records · Page 83

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Kernel learning backward SDE filter for data assimilation

In this paper, we develop a kernel learning backward SDE filter method to estimate the state of a stochastic dynamical system based on its partial noisy observations. A system of forward backward stochastic differential equations is used to propagate the state of the target dynamical model, and Bayesian inference is applied to incorporate the observational information. Further, to characterize the dynamical model in the entire state space, we introduce a kernel learning method to learn a continuous global approximation for the conditional probability density function of the target state by using discrete approximated density values as training data. Numerical experiments demonstrate that the kernel learning backward SDE is highly effective.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Uncertainty Quantification and Sensitivity Analysis of Non-Nuclear Advanced Controls Testbed Reactor Mockup

The research presented in this report describes our progress in applying stochastic methods and uncertainty quantification, parametric study, and variance-based sensitivity analysis (also known as Sobol sensitivity analysis) to a full-core model of a nuclear thermal propulsion (NTP) system simulated with Griffin, with the goal of developing a reduced order (surrogate) model which can be rapidly sampled while perturbing multiple input parameters. In this NTP system, reactivity and power feedback affect the rotation of control drums, which are controlled by a hybrid proportional, integral and derivative (PID) controller, actuated by the power demand and reactivity feedback from the numerical model. This model uses reactor kinetic feedback (mean generation time and $\beta$ from a transient Griffin simulation executed with the improved quasi-static method to provide the kinetic parameters) as inputs to functions which control the CD rotation angle. Using a number of stochastic method approaches, we developed a dual purpose training-surrogate model of the NTP system using polynomial regression. The trained model can be rapidly sampled while simultaneously perturbing various input parameters of the model, such as coefficients on the PID control, or temperature (directly affect the neutron cross section). The surrogate model delivers accurate results orders-of-magnitude faster (minutes, not days) than the base model. Once the base model has been trained, distributions of the uncertain parameters can be changed at will to investigate the effects of perturbing multiple inputs and their effect on the output. For example, coefficients used in the PID control system may vary due to some physical interference, or there may be uncertainty in the temperature of the neutron cross sections in various regions of the reactor. A distribution can be placed on these parameters and operational boundaries can be determined. The goal of this work is to support development of an advanced control system to operate CDs in a functioning NTP system.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2021

Bayesian sparse learning with preconditioned stochastic gradient MCMC and its applications

Deep neural networks have been successfully employed in an extensive variety of research areas, including solving partial differential equations. Despite its significant success, there are some challenges in effectively training DNN, such as avoiding overfitting in over-parameterized DNNs and accelerating the optimization in DNNs with pathological curvature. Here, we propose a Bayesian type sparse deep learning algorithm. The algorithm utilizes a set of spike-and-slab priors for the parameters in the deep neural network. The hierarchical Bayesian mixture will be trained using an adaptive empirical method. That is, one will alternatively sample from the posterior using preconditioned stochastic gradient Langevin Dynamics (PSGLD), and optimize the latent variables via stochastic approximation. The sparsity of the network is achieved while optimizing the hyperparameters with adaptive searching and penalizing. A popular SG-MCMC approach is Stochastic gradient Langevin dynamics (SGLD). However, considering the complex geometry in the model parameter space in nonconvex learning, updating parameters using a universal step size in each component as in SGLD may cause slow mixing. To address this issue, we apply a computationally manageable preconditioner in the updating rule, which provides a step-size parameter to adapt to local geometric properties. Moreover, by smoothly optimizing the hyperparameter in the preconditioning matrix, our proposed algorithm ensures a decreasing bias, which is introduced by ignoring the correction term in the preconditioned SGLD. According to the existing theoretical framework, we show that the proposed algorithm can asymptotically converge to the correct distribution with a controllable bias under mild conditions. Numerical tests are performed on both synthetic regression problems and learning solutions of elliptic PDE, which demonstrate the accuracy and efficiency of the present work.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2020

Interpretation of borehole strain measurements using surrogate modeling-based optimization

Interpreting strain data measured during well testing requires inverting poroelastic forward models set up to represent an aquifer or reservoir. One approach is to use stochastic methods to conduct the inversion. Subsurface parameters such as elastic modulus, permeability, and geometry of heterogeneities are estimated by searching the parameter space. This is feasible, but cumbersome, requiring more than a week of computation using many hundreds of computer nodes in one of our earlier analyses. This motivated us to consider alternative methods, including an artificial neural network (ANN) as a surrogate model.

Roudini, Soheil↗

DOE OSTI2022

Posterior Regularized Bayesian Neural Network

Traditional NNs often lack the ability for uncertainty quantification. Bayesian NNs(BNNs) could help measure the confidence level by using distributions in NNs modeling. Besides, knowledge is commonly available and could improve the performance of BNNs if it can be properly incorporated. In this work, we propose a novel Posterior-Regularized BNN(PR-BNN) model by incorporating soft and hard constraints as a posterior regularization term. We also propose an augmented Lagrangian method and stochastic optimization algorithm for efficient updating via Monte Carlo sampling. The simulations and case studies for solar PV plants have shown the performance improvement of the proposed model over traditional BNNs.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Posterior Regularized Bayesian Neural Network

Traditional NNs often lack the ability for uncertainty quantification. Bayesian NNs(BNNs) could help measure the confidence level by using distributions in NNs modeling. Besides, knowledge is commonly available and could improve the performance of BNNs if it can be properly incorporated. In this work, we propose a novel Posterior-Regularized BNN(PR-BNN) model by incorporating soft and hard constraints as a posterior regularization term. We also propose an augmented Lagrangian method and stochastic optimization algorithm for efficient updating via Monte Carlo sampling. The simulations and case studies in solar energy prediction have shown the performance improvement of the proposed model over traditional BNNs.

14 SOLAR ENERGY↗

DOE OSTI2026

A new effective theory for stochastic relativistic hydrodynamics

Thermal fluctuations are a fundamental feature of dissipative systems that are essential for understanding physics near the expected critical point of QCD and in small systems. When such fluctuations are modeled naively in relativistic systems, strange features can appear such as negative self-correlation functions. We construct an effective theory for nonlinear stochastic relativistic hydrodynamics that ensure a well-posed mathematical formulation. Using Crooks fluctuation theorem, we derive a symmetry of the effective action that incorporates fluctuations through a suitable free energy functional. For divergence type theories, the action can then be fully specified using a single vector generating current. The equations of motion obtained using this procedure are guaranteed to be flux conservative and symmetric hyperbolic when the dynamics is causal. This ensures that these equations are well-posed (for suitable initial data) and are in a form that can easily be simulated, including with Metropolis techniques.

Mullins, Nicki [University of Illinois at Urbana-C↗

DOE OSTI2021

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2012

The Role of Aerosols on Precipitation Processes: Cloud Resolving Model Simulations

Cloud microphysics is inevitably affected by the smoke particle (CCN, cloud condensation nuclei) size distributions below the clouds. Therefore, size distributions parameterized as spectral bin microphysics are needed to explicitly study the effects of atmospheric aerosol concentration on cloud development, rainfall production, and rainfall rates for convective clouds. Recently, a detailed spectral-bin microphysical scheme was implemented into the Goddard Cumulus Ensemble (GCE) model. The formulation for the explicit spectral bin microphysical processes is based on solving stochastic kinetic equations for the size distribution functions of water droplets (i.e., cloud droplets and raindrops), and several types of ice particles [i.e. pristine ice crystals (columnar and plate-like), snow (dendrites and aggregates), graupel and frozen drops/hail]. Each type is described by a special size distribution function containing many categories (i.e., 33 bins). Atmospheric aerosols are also described using number density size-distribution functions. The model is tested by studying the evolution of deep cloud systems in the west Pacific warm pool region, the sub-tropics (Florida) and midlatitudes using identical thermodynamic conditions but with different concentrations of CCN: a low "clean" concentration and a high "dirty" concentration. Results indicate that the low CCN concentration case produces rainfall at the surface sooner than the high CeN case but has less cloud water mass aloft. Because the spectral-bin model explicitly calculates and allows for the examination of both the mass and number concentration of species in each size category, a detailed analysis of the instantaneous size spectrum can be obtained for these cases. It is shown that since the low (CN case produces fewer droplets, larger sizes develop due to greater condensational and collection growth, leading to a broader size spectrum in comparison to the high CCN case. Sensitivity tests were performed to identify the impact of ice processes, radiation and large-scale influence on cloud-aerosol interactive processes, especially regarding surface rainfall amounts and characteristics (i.e., heavy or convective versus light or stratiform types). In addition, an inert tracer was included to follow the vertical redistribution of aerosols by cloud processes. We will also give a brief review from observational evidence on the role of aerosol on precipitation processes.

Tao, Wei-Kuo↗

DOE OSTI2022

Contextual Active Online Model Selection with Expert Advice

How can we collect the most useful labels to learn a model selection policy, when presented with arbitrary heterogeneous data streams? In this paper, we formulate this task as a contextual active model selection problem, where at each round the learner receives an unlabeled data point along with a context. The goal is to output the best model for any given context without obtaining an excessive amount of labels. In particular, we focus on the task of selecting pre-trained classifiers, and propose a contextual active model selection algorithm (CAMS), which relies on a novel uncertainty sampling query criterion defined on a given policy class for adaptive model selection. In comparison to prior art, our algorithm does not assume a globally optimal model. We provide rigorous theoretical analysis for the regret and query complexity under both adversarial and stochastic settings. Our experiments on several benchmark classification datasets demonstrate the algorithm’s effectiveness in terms of both regret and query complexity. Notably, to achieve the same accuracy, CAMS incurs less than 10% of the label cost when compared to the best online model selection baselines on CIFAR10.

Liu, Xuefeng↗

DOE OSTI2022

A Multistage Stochastic Transmission Expansion Algorithm for Wide-Area Planning under Uncertainty

The overall objective for this project was to develop and demonstrate a set of methods for solving the transmission investment problem for a large network considering many possible scenarios of future conditions and multiple decision points when investments can be made. Project sub-objectives achieved this goal through a succession of extending the methods to apply to problems with increasing complexity or additional features, including the number of decision points, whether generation and transmission are co-optimized, and whether AC or DC power flow is used. A transmission model was developed for the Western Electric Coordinating Council (WECC) region, the high-voltage transmission system that serves the western third of the continental U.S. Using a dataset provided by WECC and by researchers from John Hopkins University, we have validated and demonstrated the model and used it to compare the new method for solving multi-stage stochastic transmission planning to several state-of-the-art techniques. The project has resulted in several key outcomes and achievements: The covariance-based method for choosing a small set of hours to represent short-term variability has superior performance in terms of accuracy to existing methods, including K-means clustering and Importance Sampling; The combined partitioning method for long-term uncertainty with the nested clustering approach for choosing representative hours for each long-term group has superior accuracy for equivalent computational effort compared with existing methods; Using the partitioning/clustering method combined with Sample Average Approximation provides both statistical bounds on the quality of the solution and at the same time, a complete investment plan for all contingencies in the full uncertainty set; no existing methods can provide both at the same time; The method is demonstrated to work well for choosing both transmission and generation investments; A variant on the method allows for both scenario selection and simultaneous correction for the error from the DC power flow approximation to provide a tractable method for AC power flow-based transmission planning under uncertainty; The method applied to the WECC case study demonstrates the additional value to the system operator and the consumer of identifying flexible investment options in the near-term decisions. In particular, the case study exhibits significant option value in postponing some transmission additions that appear useful but in some long-term system states create new congestion problems.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS2007

The Chaotic Light Curves of Accreting Black Holes

We present model light curves for accreting Black Hole Candidates (BHC) based on a recently developed model of these sources. According to this model, the observed light curves and aperiodic variability of BHC are due to a series of soft photon injections at random (Poisson) intervals and the stochastic nature of the Comptonization process in converting these soft photons to the observed high energy radiation. The additional assumption of our model is that the Comptonization process takes place in an extended but non-uniform hot plasma corona surrounding the compact object. We compute the corresponding Power Spectral Densities (PSD), autocorrelation functions, time skewness of the light curves and time lags between the light curves of the sources at different photon energies and compare our results to observation. Our model reproduces the observed light curves well, in that it provides good fits to their overall morphology (as manifest by the autocorrelation and time skewness) and also to their PSDs and time lags, by producing most of the variability power at time scales 2 a few seconds, while at the same time allowing for shots of a few msec in duration, in accordance with observation. We suggest that refinement of this type of model along with spectral and phase lag information can be used to probe the structure of this class of high energy sources.

Kazanas, Demosthenes↗

DOE OSTI2026

Is the Matrix Completion of Reduced Density Matrices Unique?

Reduced density matrices are central to describing observables in many-body quantum systems. In electronic structure theory, the two-particle reduced density matrix (2-RDM) suffices to determine the energy and other key properties. Recent work has used matrix completion, leveraging the low-rank structure of RDMs and approximate theoretical models, to reconstruct the 2-RDM from partial data and thus reduce the computational cost. However, matrix completion is, in general, an under-determined problem. Revisiting Rosina’s theorem (Rosina, M. Queen’s Papers on Pure and Applied Mathematics , 1968, No. 11, 369), we here show that the matrix completion is unique under certain conditions, identifying the subset of 2-RDM elements that enables its exact reconstruction from incomplete information. Building on this, we introduce a hybrid quantum–stochastic algorithm that achieves exact matrix completion, demonstrated through applications to the Fermi–Hubbard model.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2024

Stochastic Averaging of Radiative Transfer Coefficients for Relativistic Electrons

Abstract Synchrotron emissivities, absorptivities, and Faraday rotation and conversion coefficients are needed in modeling a variety of astrophysical sources, including Event Horizon Telescope (EHT) sources. We develop a method for estimating transfer coefficients that exploits their linear dependence on the electron distribution function, decomposing the distribution function into a sum of parts each of whose emissivity can be calculated easily. We refer to this procedure as stochastic averaging and apply it in two contexts. First, we use it to estimate the emissivity of an isotropicκdistribution function with a high-energy cutoff. The resulting coefficients can be evaluated efficiently enough to be used directly in ray-tracing calculations, and we provide an example calculation. Second, we use stochastic averaging to assess the effect of subgrid turbulence on the volume-averaged emissivity and along the way provide a prescription for a turbulent emissivity. We find that for parameters appropriate to EHT sources turbulence reduces the emissivity slightly. In the infrared, turbulence can dramatically increase the emissivity.

Astronomy & Astrophysics↗

DOE OSTI2020

Long-Term Stochasticity Combines With Short-Term Variability in Assembly Processes to Underlie Rice Paddy Sustainability

Revealing temporal patterns of community assembly processes is important for understanding how microorganisms underlie the sustainability of agroecosystem. The ancient terraced rice paddies at Longji provide an ideal platform to study temporal dynamics of agroecosystem sustainability due to their chronosequential records of soil physicochemistry and well-archived microbial information along 630-year rice cultivation. We used statistical null models to evaluate microbial assembly processes along the soil chronosequences of Longji rice paddies through time. Stochastic and deterministic assembly processes jointly governed microbial community composition within successional eras (less than 250 years), and within-era determinism was mainly driven by soil fertility and redox conditions alone or in combination. Conversely, across successional eras (i.e., over 300 years), stochasticity linearly increased with increasing duration between eras and was eventually predominant for the whole 630 years. We suggest that the impact of stochasticity vs. determinism on assembly is timescale-dependent, and we propose that the importance of stochastic assembly of microbial community at longer timescales is due to the gradual changes in soil properties under long-term rice cultivation, which in turn contribute to the sustainability of paddy ecosystem by maintaining a diverse community of microorganisms with multi-functional traits. In total, our results indicate that knowledge on the timescales at which assembly processes govern microbial community composition is key to understanding the ecological mechanisms generating agroecosystem sustainability.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2024

Multi deep learning-based stochastic microstructure reconstruction and high-fidelity micromechanics simulation of time-dependent ceramic matrix composite response

A multi deep learning-based framework is developed for efficient, automated microstructure reconstruction and generation of stochastic representative volume elements (SRVEs) with periodic boundary conditions (PBCs) for accurate modeling of ceramic matrix composite (CMC) response. The methodology comprises a convolutional neural network coupled with regression layers to act as a vanilla regression network for semantic segmentation of the microstructure, allowing accurate characterization of the phases and their distributions at the microscale. Scanning electron microscope and confocal microscope are used to obtain C/SiNC and SiC/SiNC CMCs micrographs for vanilla regression testing. Microstructure variability in terms of fiber volume fraction and porosity are quantified through the output regression layer, ensuring accurate representation of material variability in SRVE construction. Generative adversarial network (GAN) and its variants are designed to produce high-fidelity SRVE, spanning CMCs microstructure variability space. A circular padding algorithm is developed to generate SRVEs with PBCs during training of GANs. The accuracy of the generated SRVEs is established through micromechanics simulations, where an efficient formulation of the high-fidelity generalized methods of cells (HFGMC) approach is used to compute the effective mechanical properties. Furthermore, an iterative algorithm is implemented in the HFGMC solver to simulate time-dependent deformation of SiC/SiNC subjected to creep loading conditions.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Laser Wakefield Accelerator modelling with Variational Neural Networks

A machine learning model was created to predict the electron spectrum generated by a GeV-class laser wakefield accelerator. The model was constructed from variational convolutional neural networks, which mapped the results of secondary laser and plasma diagnostics to the generated electron spectrum. An ensemble of trained networks was used to predict the electron spectrum and to provide an estimation of the uncertainty of that prediction. It is anticipated that this approach will be useful for inferring the electron spectrum prior to undergoing any process that can alter or destroy the beam. In addition, the model provides insight into the scaling of electron beam properties due to stochastic fluctuations in the laser energy and plasma electron density.

43 PARTICLE ACCELERATORS↗

DOE OSTI2024

Probabilistic measures for biological adaptation and resilience

This paper introduces an approach to quantifying ecological resilience in biological systems, particularly focusing on noisy systems responding to episodic disturbances with sudden adaptations. Incorporating concepts from nonequilibrium statistical mechanics, we propose a measure termed “ecological resilience through adaptation,” specifically tailored to noisy, forced systems that undergo physiological adaptation in the face of stressful environmental changes. Randomness plays a key role, accounting for model uncertainty and the inherent variability in the dynamical response among components of biological systems. Our measure of resilience is rooted in the probabilistic description of states within these systems and is defined in terms of the dynamics of the ensemble average of a model-specific observable quantifying success or well-being. Our approach utilizes stochastic linear response theory to compute how the expected success of a system, originally in statistical equilibrium, dynamically changes in response to a environmental perturbation and a subsequent adaptation. Importantly, the resulting mathematical derivations allow for the estimation of resilience in terms of ensemble averages of simulated or experimental data. Finally, through a simple but clear conceptual example, we illustrate how our resilience measure can be interpreted and compared to other existing frameworks in the literature. The methodology is general but inspired by applications in plant systems, with the potential for broader application to complex biological processes.

60 APPLIED LIFE SCIENCES↗