Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “stochastic methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,765 records · Page 98
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

WE-H-BRA-03: Development of a Model to Include the Evolution of Resistant Tumor Subpopulations Into the Treatment Optimization Process for Schedules Involving Targeted Agents in Chemoradiation Therapy

Purpose: To develop a model that includes the process of resistance development into the treatment optimization process for schedules that include targeted therapies. Further, to validate the approach using clinical data and to apply the model to assess the optimal induction period with targeted agents before curative treatment with chemo-radiation in stage III lung cancer. Methods: Growth of the tumor and its subpopulations is modeled by Gompertzian growth dynamics, resistance induction as a stochastic process. Chemotherapy induced cell kill is modeled by log-cell kill dynamics, targeted agents similarly but restricted to the sensitive population. Radiation induced cell kill is assumed to follow the linear-quadratic model. The validation patient data consist of a cohort of lung cancer patients treated with tyrosine kinase inhibitors that had longitudinal imaging data available. Results: The resistance induction model was successfully validated using clinical trial data from 49 patients treated with targeted agents. The observed recurrence kinetics, with tumors progressing from 1.4–63 months, result in tumor growth equaling a median volume doubling time of 92 days [34–248] and a median fraction of pre-existing resistance of 0.035 [0–0.22], in agreement with previous clinical studies. The model revealed widely varying optimal time points for the use of curative therapy, reaching from ∼1m to >6m depending on the patient’s growth rate and amount of pre-existing resistance. This demonstrates the importance of patient-specific treatment schedules when targeted agents are incorporated into the treatment. Conclusion: We developed a model including evolutionary dynamics of resistant sub-populations with traditional chemotherapy and radiation cell kill models. Fitting to clinical data yielded patient specific growth rates and resistant fraction in agreement with previous studies. Further application of the model demonstrated how proper timing of chemo-radiation could minimize the probability of resistance, increasing tumor control significantly.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Acute stress reduces population-level metabolic and proteomic variation

Abstract Background Variation in omics data due to intrinsic biological stochasticity is often viewed as a challenging and undesirable feature of complex systems analyses. In fact, numerous statistical methods are utilized to minimize the variation among biological replicates. Results We demonstrate that the common statistics relative standard deviation (RSD) and coefficient of variation (CV), which are often used for quality control or part of a larger pipeline in omics analyses, can also be used as a metric of a physiological stress response. Using an approach we term Replicate Variation Analysis (RVA), we demonstrate that acute physiological stress leads to feature-wide canalization of CV profiles of metabolomes and proteomes across biological replicates. Canalization is the repression of variation between replicates, which increases phenotypic similarity. Multiple in-house mass spectrometry omics datasets in addition to publicly available data were analyzed to assess changes in CV profiles in plants, animals, and microorganisms. In addition, proteomics data sets were evaluated utilizing RVA to identify functionality of reduced CV proteins. Conclusions RVA provides a foundation for understanding omics level shifts that occur in response to cellular stress. This approach to data analysis helps characterize stress response and recovery, and could be deployed to detect populations under stress, monitor health status, and conduct environmental monitoring.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

FEDERATED LEARNING ON STOCHASTIC NEURAL NETWORKS

Federated learning is a machine learning paradigm that leverages edge computing on client devices to optimize models while maintaining user privacy by ensuring that local data remain on the device. However, since all data are collected by clients, federated learning is susceptible to latent noise in local datasets. Factors such as limited measurement capabilities or human errors may introduce inaccuracies in client data. To address this challenge, we propose the use of a stochastic neural network as the local model within the federated learning framework. Stochastic neural networks not only facilitate the estimation of the true underlying states of the data but also enable the quantification of latent noise. We refer to our federated learning approach, which incorporates stochastic neural networks as local models, as federated stochastic neural networks. In this work we will present numerical experiments demonstrating the performance and effectiveness of our method, particularly in handling nonindependent and identically distributed data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Framework for an adaptive integrated observation system using a hierarchy of machine learning approaches

Focal Area(s): 1. Data acquisition enabled by machine learning, AI, and advanced methods including experimental/network design/optimization, and hardware-related efforts involving AI. 2. Insight gleaned from complex measurements using AI, big data analytics, and other advanced methods, including explainable AI and physics- or knowledge-guided AI Science Challenge and Rationale: Atmospheric processes are stochastic, occur at scales from the micrometer to many kilometers, and are constantly changing over time. Characterizing these interactions and associated environmental conditions using traditional measurement techniques is difficult and can take years to build statistics on atmospheric phenomena that occurs episodically. Developing new innovative approaches to modify sampling strategies in real-time would enable the routine collection of targeted measurements focused on a specific set of science questions.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1973

Open-loop-feedback-optimal adaptive stochastic control of linear systems

This paper considers the suboptimal stochastic control of linear discrete-time dynamical systems with unknown or stochastically varying parameters. The suboptimal scheme is based upon the use of the open-loop-feedback-optimal method. The state and parameter estimates are generated by an extended Kalman filter algorithm. Numerical results for first-order systems are presented.

Ku, R.↗
NASA NTRS1986

Noise in remote-sensing systems - The effect on classification error

Several types of noise in remote-sensing systems are treated. The purpose is to provide enhanced understanding of the relationship of noise sources to both analysis results and sensor design. The context of optical sensors and spectral pattern recognition analysis methods is used to enable tractability for quantitative results. First, the concept of multispectral classification is reviewed. Next, stochastic models are discussed for both signals and noise, including thermal, shot and quantization noise along with atmospheric effects. A model enabling the study of the combined effect of these sources is presented, and a system performance index is defined. Theoretical results showing the interrelated effects of the noise sources on system performance are given. Results of simulations using the system model are presented for several values of system parameters, using some noise parameters of the Thematic Mapper scanner as an illustration. Results show the relative importance of each of the noise sources on system performance, including how sensor noise interacts with atmospheric effects to degrade accuracy.

Landgrebe, D. A.↗
NASA NTRS2008

Growth Control and Disease Mechanisms in Computational Embryogeny

This paper presents novel approach to applying growth control and diseases mechanisms in computational embryogeny. Our method, which mimics fundamental processes from biology, enables individuals to reach maturity in a controlled process through a stochastic environment. Three different mechanisms were implemented; disease mechanisms, gene suppression, and thermodynamic balancing. This approach was integrated as part of a structural evolutionary model. The model evolved continuum 3-D structures which support an external load. By using these mechanisms we were able to evolve individuals that reached a fixed size limit through the growth process. The growth process was an integral part of the complete development process. The size of the individuals was determined purely by the evolutionary process where different individuals matured to different sizes. Individuals which evolved with these characteristics have been found to be very robust for supporting a wide range of external loads.

Genetic algorithm↗
DOE OSTI2017

PDE-constrained Optimization under Uncertainty

Uncertainty is pervasive in all science and engineering applications. Incorporating uncertainty in physical models is therefore both natural and vital. In doing so, we often arrive at parametric systems of partial differential equations (PDEs). When passing from simulation to optimization, we obtain (typically nonconvex) infinite-dimensional optimization problems that, upon discretization, result in extremely large-scale nonlinear programs. For example, consider a linear elliptic PDE on a two- dimensional domain with a single random coeffcient. If we sampled the random input with 10,000 realizations of the coeffcient, the resulting optimization problem would have 10,000 PDE constraints. Furthermore, discretizing each PDE with piecewise linear finite elements on a 100X100 uni- form quadrilateral mesh results in 100,000,000 degrees of freedom. As a result, the critical components for ensuring mesh-independent performance of numerical optimization methods in the deterministic setting, for example, solution regularity and generalized differentiability, are even more critical in the stochastic setting.

42 ENGINEERING↗
DOE OSTI2019

Stochastic Molecular Dynamics

I can not pass up the opportunity to honor Graeme Bird, a true gentleman and a real scholar, if there ever was one. When we first met at a rarefied gas dynamics conference, it was instant mutual admiration. Furthermore, we met rarely thereafter at subsequent rarefied gas meetings since I attended them infrequently as a physicist interested in hydrodynamics, where often even a rarefied gas without deterministic collisions can serve as a simple fluid model but Graeme attended them regularly. In that rst meeting I became aware of DSMC, read Graeme paper and found to my astonishment that the only reference in that paper was to my molecular dynamics work2. Indeed DSMC is computationally faster competitor to molecular dynamics and it is this feature that I would like to emphasize in this brief note. It is for this reason that I entitled this contribution \stochastic molecular dynamics". I once told Graeme that that would have been a more descriptive name to his method than DSMC. DSMC is also a good demonstration that an algorithm developed for an entirely different purposes can also be very useful for other applications.

74 ATOMIC AND MOLECULAR PHYSICS↗
NASA NTRS1971

Application of Monte Carlo techniques to optimization of high-energy beam transport in a stochastic environment

An algorithm employing a modified sequential random perturbation, or creeping random search, was applied to the problem of optimizing the parameters of a high-energy beam transport system. The stochastic solution of the mathematical model for first-order magnetic-field expansion allows the inclusion of state-variable constraints, and the inclusion of parameter constraints allowed by the method of algorithm application eliminates the possibility of infeasible solutions. The mathematical model and the algorithm were programmed for a real-time simulation facility; thus, two important features are provided to the beam designer: (1) a strong degree of man-machine communication (even to the extent of bypassing the algorithm and applying analog-matching techniques), and (2) extensive graphics for displaying information concerning both algorithm operation and transport-system behavior. Chromatic aberration was also included in the mathematical model and in the optimization process. Results presented show this method as yielding better solutions (in terms of resolutions) to the particular problem than those of a standard analog program as well as demonstrating flexibility, in terms of elements, constraints, and chromatic aberration, allowed by user interaction with both the algorithm and the stochastic model. Example of slit usage and a limited comparison of predicted results and actual results obtained with a 600 MeV cyclotron are given.

Parrish, R. V.↗
DOE OSTI2022

Characterization of single-shot attosecond pulses with angular streaking photoelectron spectra

Most of the traditional attosecond pulse retrieval algorithms are based on a so-called attosecond streak camera technique, in which the momentum of the electron is shifted by an amount depending on the relative time delay between the attosecond pulse and the streaking infrared pulse. Thus, temporal information of the attosecond pulse is encoded in the amount of momentum shift in the streaked photoelectron momentum spectrogram S(p,τ), where p is the momentum of the electron along the polarization direction and τ is the time delay. An iterative algorithm is then employed to reconstruct the attosecond pulse from the streaking spectrogram. This method, however, cannot be applied to attosecond pulses generated from free-electron x-ray lasers where each single shot is different and stochastic in time. However, using a circularly polarized infrared laser as the streaking field, a two (or three)-dimensional angular streaking electron spectrum can be used to retrieve attosecond pulses for each shot, as well as the time delay with respect to the circularly polarized IR field. Here we show that a retrieval algorithm previously developed for the traditional streaking spectrogram can be modified to efficiently characterize single-shot attosecond pulses. The methods have been applied to retrieve 188 single shots from recent experiments. We analyze the statistical behavior of these 188 pulses in terms of pulse duration, bandwidth, pulse peak energy, and time delay with respect to the IR field. Furthermore, the retrieval algorithm is efficient and can be easily used to characterize a large number of shots in future experiments for attosecond pulses at free-electron x-ray laser facilities.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2020

Evolutionary reinforcement learning of dynamical large deviations

In this work, we show how to bound and calculate the likelihood of dynamical large deviations using evolutionary reinforcement learning. An agent, a stochastic model, propagates a continuous-time Monte Carlo trajectory and receives a reward conditioned upon the values of certain path-extensive quantities. Evolution produces progressively fitter agents, potentially allowing the calculation of a piece of a large-deviation rate function for a particular model and path-extensive quantity. For models with small state spaces, the evolutionary process acts directly on rates, and for models with large state spaces, the process acts on the weights of a neural network that parameterizes the model's rates. This approach shows how path-extensive physics problems can be considered within a framework widely used in machine learning.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2015

An adaptive multi-level simulation algorithm for stochastic biological systems

Discrete-state, continuous-time Markov models are widely used in the modeling of biochemical reaction networks. Their complexity often precludes analytic solution, and we rely on stochastic simulation algorithms (SSA) to estimate system statistics. The Gillespie algorithm is exact, but computationally costly as it simulates every single reaction. As such, approximate stochastic simulation algorithms such as the tau-leap algorithm are often used. Potentially computationally more efficient, the system statistics generated suffer from significant bias unless tau is relatively small, in which case the computational time can be comparable to that of the Gillespie algorithm. The multi-level method [Anderson and Higham, “Multi-level Monte Carlo for continuous time Markov chains, with applications in biochemical kinetics,” SIAM Multiscale Model. Simul. 10(1), 146–179 (2012)] tackles this problem. A base estimator is computed using many (cheap) sample paths at low accuracy. The bias inherent in this estimator is then reduced using a number of corrections. Each correction term is estimated using a collection of paired sample paths where one path of each pair is generated at a higher accuracy compared to the other (and so more expensive). By sharing random variables between these paired paths, the variance of each correction estimator can be reduced. This renders the multi-level method very efficient as only a relatively small number of paired paths are required to calculate each correction term. In the original multi-level method, each sample path is simulated using the tau-leap algorithm with a fixed value of τ. This approach can result in poor performance when the reaction activity of a system changes substantially over the timescale of interest. By introducing a novel adaptive time-stepping approach where τ is chosen according to the stochastic behaviour of each sample path, we extend the applicability of the multi-level method to such cases. We demonstrate the efficiency of our method using a number of examples.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2016

cDPD: A new dissipative particle dynamics method for modeling electrokinetic phenomena at the mesoscale

In this work, we develop a “charged” dissipative particle dynamics (cDPD) model for simulating mesoscopic electrokinetic phenomena governed by the stochastic Poisson-Nernst-Planck and the Navier-Stokes equations. In particular, the transport equations of ionic species are incorporated into the DPD framework by introducing extra degrees of freedom and corresponding evolution equations associated with each DPD particle. Diffusion of ionic species driven by the ionic concentration gradient, electrostatic potential gradient, and thermal fluctuations is captured accurately via pairwise fluxes between DPD particles. The electrostatic potential is obtained by solving the Poisson equation on the moving DPD particles iteratively at each time step. For charged surfaces in bounded systems, an effective boundary treatment methodology is developed for imposing both the correct hydrodynamic and electrokinetics boundary conditions in cDPD simulations. To validate the proposed cDPD model and the corresponding boundary conditions, we first study the electrostatic structure in the vicinity of a charged solid surface, i.e., we perform cDPD simulations of the electrostatic double layer and show that our results are in good agreement with the well-known mean-field theoretical solutions. Furthermore, we simulate the electrostatic structure and capacity densities between charged parallel plates in salt solutions with different salt concentrations. Moreover, we employ the proposed methodology to study the electro-osmotic and electro-osmotic/pressure-driven flows in a micro-channel. In the latter case, we simulate the dilute poly-electrolyte solution drifting by electro-osmotic flow in a micro-channel, hence demonstrating the flexibility and capability of this method in studying complex fluids with electrostatic interactions at the micro- and nano-scales.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2012

Automatic treatment of the variance estimation bias in TRIPOLI-4 criticality calculations

The central limit (CLT) theorem States conditions under which the mean of a sufficiently large number of independent random variables, each with finite mean and variance, will be approximately normally distributed. The use of Monte Carlo transport codes, such as Tripoli4, relies on those conditions. While these are verified in protection applications (the cycles provide independent measurements of fluxes and related quantities), the hypothesis of independent estimates/cycles is broken in criticality mode. Indeed the power iteration technique used in this mode couples a generation to its progeny. Often, after what is called 'source convergence' this coupling almost disappears (the solution is closed to equilibrium) but for loosely coupled systems, such as for PWR or large nuclear cores, the equilibrium is never found, or at least may take time to reach, and the variance estimation such as allowed by the CLT is under-evaluated. In this paper we first propose, by the mean of two different methods, to evaluate the typical correlation length, as measured in cycles number, and then use this information to diagnose correlation problems and to provide an improved variance estimation. Those two methods are based on Fourier spectral decomposition and on the lag k autocorrelation calculation. A theoretical modeling of the autocorrelation function, based on Gauss-Markov stochastic processes, will also be presented. Tests will be performed with Tripoli4 on a PWR pin cell. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Structure optimization with stochastic density functional theory

Linear-scaling techniques for Kohn–Sham density functional theory are essential to describe the ground state properties of extended systems. Still, these techniques often rely on the localization of the density matrix or accurate embedding approaches, limiting their applicability. In contrast, stochastic density functional theory (sDFT) achieves linear- and sub-linear scaling by statistically sampling the ground state density without relying on embedding or imposing localization. In return, ground state observables, such as the forces on the nuclei, fluctuate in sDFT, making optimizing the nuclear structure a highly non-trivial problem. In this work, we combine the most recent noise-reduction schemes for sDFT with stochastic optimization algorithms to perform structure optimization within sDFT. We compare the performance of the stochastic gradient descent approach and its variations (stochastic gradient descent with momentum) with stochastic optimization techniques that rely on the Hessian, such as the stochastic Broyden–Fletcher–Goldfarb–Shanno algorithm. In conclusion, we further provide a detailed assessment of the computational efficiency and its dependence on the optimization parameters of each method for determining the ground state structure of bulk silicon with varying supercell dimensions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1990

Construction of dynamic stochastic simulation models using knowledge-based techniques

Over the past three decades, computer-based simulation models have proven themselves to be cost-effective alternatives to the more structured deterministic methods of systems analysis. During this time, many techniques, tools and languages for constructing computer-based simulation models have been developed. More recently, advances in knowledge-based system technology have led many researchers to note the similarities between knowledge-based programming and simulation technologies and to investigate the potential application of knowledge-based programming techniques to simulation modeling. The integration of conventional simulation techniques with knowledge-based programming techniques is discussed to provide a development environment for constructing knowledge-based simulation models. A comparison of the techniques used in the construction of dynamic stochastic simulation models and those used in the construction of knowledge-based systems provides the requirements for the environment. This leads to the design and implementation of a knowledge-based simulation development environment. These techniques were used in the construction of several knowledge-based simulation models including the Advanced Launch System Model (ALSYM).

Williams, M. Douglas↗