Optimization of stochastic finite state systems.
Optimization of stochastic dynamic system with noisy output
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Optimization of stochastic dynamic system with noisy output
Nonlinear perturbation theory for optimal estimation and control of time-discrete, stochastic, dynamic systems when measurement data is noisy
Filtering and optimal control problems for discrete stochastic dynamical systems, describing various feedback controls and midcourse guidance optimization
This paper presents the formulation of a class of optimization problems dealing with selecting, at each instant of time, one measurement provided by one out of many sensors. Each measurement has an associated measurement cost. The basic problem is then to select an optimal measurement policy, during a specified observation time interval, so that a weighted combination of prediction accuracy and accumulated observation cost is optimized. The current analysis is limited to the class of linear stochastic dynamic systems and measurement subsystems. The problem of selecting the optimal measurement strategy can be transformed into a deterministic optimal control problem. It is shown that the optimal measurement policy and the associated matched Kalman-type filter can be precomputed.
Shuttle Experiment Pointing Mount System concepts and technology have been investigated and assessed with regard to payload requirements having a wide range of stability, accuracy, and control functions. Pointing systems were analyzed with end-mounted and center of gravity-mounted payloads viewing stellar and solar targets. Major error sources are identified with dynamical, stochastic, and nonlinear characteristics of structures, isolators, sensors, bearings, actuators, and controller. Results are presented which place a perspective on the potential of advanced technology to satisfy the most stringent sub-arc second pointing requirements.
The topics considered are related to linear control system design, adaptive control, failure detection, control under failure, system reliability, and large-scale systems and decentralized control. It is pointed out that the design of a linear feedback control system which regulates a process about a desirable set point or steady-state condition in the presence of disturbances is a very important problem. The linearized dynamics of the process are used for design purposes. The typical linear-quadratic design involving the solution of the optimal control problem of a linear time-invariant system with respect to a quadratic performance criterion is considered along with gain reduction theorems and the multivariable phase margin theorem. The stumbling block in many adaptive design methodologies is associated with the amount of real time computation which is necessary. Attention is also given to the desperate need to develop good theories for large-scale systems, the beginning of a microprocessor revolution, the translation of the Wiener-Hopf theory into the time domain, and advances made in dynamic team theory, dynamic stochastic games, and finite memory stochastic control.
The estimation theory of stochastic-dynamic systems is described and used in a numerical study of optimal interpolation. The general form of data assimilation methods is reviewed. The Kalman-Bucy, KB filter, and optimal interpolation (OI) filters are examined for effectiveness in performance as gain matrices using a one-dimensional form of the shallow-water equations. Control runs in the numerical analyses were performed for a ten-day forecast in concert with the OI method. The effects of optimality, initialization, and assimilation were studied. It was found that correct initialization is necessary in order to localize errors, especially near boundary points. Also, the use of small forecast error growth rates over data-sparse areas was determined to offset inaccurate modeling of correlation functions near boundaries.
A 'lagged average forecast' (LAF) model is developed for stochastic dynamic weather forecasting and used for predictions in comparison with the results of a Monte Carlo forecast (MCF). The technique involves the calculation of sample statistics from an ensemble of forecasts, with each ensemble member being an ordinary dynamical forecast (ODF). Initial conditions at a time lagging the start of the forecast period are used, with varying amounts of time for the lags. Forcing by asymmetric Newtonian heating of the lower layer is used in a two-layer, f-plane, highly truncated spectral model in a test forecasting run. Both the LAF and MCF are found to be more accurate than the ODF due to ensemble averaging with the MCF and the LAF. When a regression filter is introduced, all models become more accurate, with the LAF model giving the best results. The possibility of generating monthly or seasonal forecasts with the LAF is discussed.
The design of an actively adaptive dual controller based on an approximation of the stochastic dynamic programming equation for a multi-step horizon is presented. A dual controller that can enhance identification of the system while controlling it at the same time is derived for multi-dimensional problems. This dual controller uses sensitivity functions of the expected future cost with respect to the parameter uncertainties. A passively adaptive cautious controller and the actively adaptive dual controller are examined. In many instances, the cautious controller is seen to turn off while the latter avoids the turn-off of the control and the slow convergence of the parameter estimates, characteristic of the cautious controller. The algorithms have been applied to a multi-variable static model which represents a simplified linear version of the relationship between the vibration output and the higher harmonic control input for a helicopter. Monte Carlo comparisons based on parametric and nonparametric statistical analysis indicate the superiority of the dual controller over the baseline controller.
A general method for parallel and vector numerical solutions of stochastic dynamic programming problems is described for optimal control of general nonlinear, continuous time, multibody dynamical systems, perturbed by Poisson as well as Gaussian random white noise. Possible applications include lumped flight dynamics models for uncertain environments, such as large scale and background random atmospheric fluctuations. The numerical formulation is highly suitable for a vector multiprocessor or vectorizing supercomputer, and results exhibit high processor efficiency and numerical stability. Advanced computing techniques, data structures, and hardware help alleviate Bellman's curse of dimensionality in dynamic programming computations.
Following suggestion that substorm breakup might be caused by an interchange or ballooning instability, several magnetohydrodynamic (MHD) and gyro-kinetic stability analysis have been performed for plasma sheet magnetic field geometries. However, the stochastic ion dynamics in the highly stressed, thin high-beta near earth plasma sheet violate the locality requirements of MHD and the invarience of the magnetic moment, mu, required by gyro-kinetic theories. In this paper we develop a new linear Vlasov kinetic theory (for low frequency modes omega less than omega(sub b), omega(sub b) being the bounce frequency) which includes the dynamics of stochastic ions.
We use the Fokker-Planck-Kolmogorov equation to study the statistical aspects of stochastic dynamics of the radiation belt (RB) protons driven by nonlinear electromagnetic ion-cyclotron (EMIC) wave packets. We obtain the spectra of keV protons scattered by these waves that showsteeping near the gyroresonance, the signature of resonant wave-particle interaction that cannot be described by a simple power law. The most likely mechanism for proton precipitation events in RBs is shown to be nonlinear wave-particle interaction, namely, the scattering of RB protons into the loss cone by EMIC waves.
Characterization of non stationary and nonlinear processes is a challenge in many engineering and scientific disciplines. Climate change modeling and projection, retrieving information from Doppler measurements of hydrometeors, and modeling calibration architectures and algorithms in microwave radiometers are example applications that can benefit from improvements in the modeling and analysis of non stationary processes. Analyses of measured signals have traditionally been limited to a single measurement series. Ensemble Detection is a technique whereby mixing calibrated noise produces an ensemble measurement set. The collection of ensemble data sets enables new methods for analyzing random signals and offers powerful new approaches to studying and analyzing non stationary processes. Derived information contained in the dynamic stochastic moments of a process will enable many novel applications.
Reliability/safety analysis of stochastic dynamic systems (e.g., nuclear power plants, airplanes, chemical plants) is currently performed through a combination of Event-Tress and Fault-Trees. However, these conventional methods suffer from certain drawbacks: • Timing of events is not explicitly modeled • Ordering of events is preset by the analyst • The modeling of complex accident scenarios is driven by expert-judgment For these reasons, there is currently an increasing interest into the development of dynamic PRA methodologies since they can be used to address the deficiencies of conventional methods listed above.
Supersymmetry, relating bosons and fermions was discovered almost 40 years ago in string theory and in quantum field theory, but the seeds of its 'miraculous' properties could have been seen already in quantum mechanics - which is also where it has found some of its more important applications. This talk introduces supersymmetry via the supersymmetric anharmonic oscillator. We shall see that this seemingly trivial example is sufficiently rich, allowing us to illustrate the uses of supersymmetric concepts in a variety of fields: mathematics, elementary particle physics, critical phenomena, and stochastic dynamics.
Stochastic reaction networks that exhibit bistable behavior are common in systems biology, materials science, and catalysis. Sampling of stationary distributions is crucial for understanding and characterizing the long-time dynamics of bistable stochastic dynamical systems. However, simulations are often hindered by the insufficient sampling of rare transitions between the two metastable regions. In this paper, we apply the parallel replica method for a continuous time Markov chain in order to improve sampling of the stationary distribution in bistable stochastic reaction networks. The proposed method uses parallel computing to accelerate the sampling of rare transitions. Furthermore, it can be combined with the path-space information bounds for parametric sensitivity analysis. With the proposed methodology, we study three bistable biological networks: the Schlögl model, the genetic switch network, and the enzymatic futile cycle network. We demonstrate the algorithmic speedup achieved in these numerical benchmarks. More significant acceleration is expected when multi-core or graphics processing unit computer architectures and programming tools such as CUDA are employed
Dynamic co-operativity in monomeric enzymes is characterized in terms of a non-Michaelis-Menten kinetic behaviour. The latter is believed to be associated with mechanisms that include multiple reaction pathways due to enzymatic conformational fluctuations. Recent advances in single-molecule fluorescence spectroscopy have provided new fundamental insights on the possible mechanisms underlying reactions catalyzed by fluctuating enzymes. Here, we present a bottom-up approach to understand enzyme turnover kinetics at physiologically relevant mesoscopic concentrations informed by mechanisms extracted from single-molecule stochastic trajectories. The stochastic approach, presented here, shows the emergence of dynamic co-operativity in terms of a slowing down of the Michaelis-Menten (MM) kinetics resulting in negative co-operativity. For fewer enzymes, dynamic co-operativity emerges due to the combined effects of enzymatic conformational fluctuations and molecular discreteness. The increase in the number of enzymes, however, suppresses the effect of enzymatic conformational fluctuations such that dynamic co-operativity emerges solely due to the discrete changes in the number of reacting species. These results confirm that the turnover kinetics of fluctuating enzyme based on the parallel-pathway MM mechanism switches over to the single-pathway MM mechanism with the increase in the number of enzymes. For large enzyme numbers, convergence to the exact MM equation occurs in the limit of very high substrate concentration as the stochastic kinetics approaches the deterministic behaviour.
We report the thermal quench triggered by locked modes is known to be mainly due to open stochastic magnetic field lines connected to the wall boundary. It is essential to understand the 3D structure of open stochastic field lines since it determines the overall plasma dynamics in the system. In this study, we analyze the 3D magnetic topology for two key concepts, the connection length L c and the effective magnetic mirror ratio M eff , and present a comprehensive picture of electron and ion dynamics related to the magnetic topology. The connection length determines the 3D structure of the ambipolar potential, and a sharp potential drop across distinct L c regions induces the E × B transport and mixing across the field line. The confinement of electrons and ions along the field line is determined by the ambipolar potential and [Formula: see text] configuration. Electron and ion temperatures in magnetic hills (M eff < 1) are lower than in magnetic wells (M eff > 1) because particles in magnetic hills are more likely to escape toward the wall boundary along the field line. The mixing between the magnetic wells and hills by E × B and magnetic drift motions results in collisionless detrapping of electrons and ions, which reduces their temperature efficiently. Numerical simulations of two different magnetic configurations demonstrate the importance of the collisionless detrapping mechanism, which could be the main cause of plasma temperature drop during the thermal quench.