Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Markov process”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56

Birth/death process model

First-order Markov model developed on digital computer for population with specific characteristics. System is user interactive, self-documenting, and does not require user to have complete understanding of underlying model details. Contains thorough error-checking algorithms on input and default capabilities.

Solloway, C. B.↗

An Analysis of Post Attack Impacts and Effects of Learning Parameters on Vulnerability Assessment of Power Grid

Due to the increasing number of heterogeneous devices connected to electric power grid, the attack surface increases the threat actors. Game theory and machine learning are being used to study the power system failures caused by external manipulation. Most of existing works in the literature focus on one-shot process of attacks and fail to show the dynamic evolution of the defense strategy. In this paper, we focus on an adversarial multistage sequential game between the adversaries of the smart electric power transmission and distribution system. We study the impact of exploration rate and convergence of the attack strategies (sequences of action that creates large scale blackout based on the system capacity) based on the reinforcement learning approach. We also illustrate how the learned attack actions disrupt the normal operation of the grid by creating transmission line outages, bus voltage violations, and generation loss. This simulation studies are conducted on IEEE 9 and 39 bus systems. The results show the improvement of the defense strategy through the learning process. The results also prove the feasibility of the learned attack actions by replicating the disturbances created in simulated power system.

attack impacts↗

The Importance of Preventive Feedback: Inference from Observations of the Stellar Masses and Metallicities of Milky Way Dwarf Galaxies

Dwarf galaxies are known to have remarkably low star formation efficiency due to strong feedback. Adopting the dwarf galaxies of the Milky Way (MW) as a laboratory, we explore a flexible semi-analytic galaxy formation model to understand how the feedback processes shape the satellite galaxies of the MW. Using Markov Chain Monte Carlo, we exhaustively search a large parameter space of the model and rigorously show that the general wisdom of strong outflows as the primary feedback mechanism cannot simultaneously explain the stellar mass function and the mass–metallicity relation of the MW satellites. An extended model that assumes that a fraction of baryons is prevented from collapsing into low-mass halos in the first place can be accurately constrained to simultaneously reproduce those observations. The inference suggests that two different physical mechanisms are needed to explain the two different data sets. In particular, moderate outflows with weak halo mass dependence are needed to explain the mass–metallicity relation, and prevention of baryons falling into shallow gravitational potentials of low-mass halos (e.g., “pre-heating”) is needed to explain the low stellar mass fraction for a given subhalo mass.

79 ASTRONOMY AND ASTROPHYSICS↗

Quantum learning of classical stochastic processes: The completely positive realization problem

Among several tasks in Machine Learning, a specially important one is the problem of inferring the latent variables of a system and their causal relations with the observed behavior. A paradigmatic instance of this is the task of inferring the hidden Markov model underlying a given stochastic process. This is known as the positive realization problem (PRP), [L. Benvenuti and L. Farina, IEEE Trans. Autom. Control 49(5), 651–664 (2004)] and constitutes a central problem in machine learning. The PRP and its solutions have far-reaching consequences in many areas of systems and control theory, and is nowadays an important piece in the broad field of positive systems theory. We consider the scenario where the latent variables are quantum (i.e., quantum states of a finite-dimensional system) and the system dynamics is constrained only by physical transformations on the quantum system. The observable dynamics is then described by a quantum instrument, and the task is to determine which quantum instrument — if any — yields the process at hand by iterative application. We take as a starting point the theory of quasi-realizations, whence a description of the dynamics of the process is given in terms of linear maps on state vectors and probabilities are given by linear functionals on the state vectors. This description, despite its remarkable resemblance with the hidden Markov model, or the iterated quantum instrument, is however devoid of any stochastic or quantum mechanical interpretation, as said maps fail to satisfy any positivity conditions. The completely positive realization problem then consists in determining whether an equivalent quantum mechanical description of the same process exists. We generalize some key results of stochastic realization theory, and show that the problem has deep connections with operator systems theory, giving possible insight to the lifting problem in quotient operator systems. Our results have potential applications in quantum machine learning, device-independent characterization and reverse-engineering of stochastic processes and quantum processors, and more generally, of dynamical processes with quantum memory [M. Guţă, Phys. Rev. A 83(6), 062324 (2011); M. Guţă and N. Yamamoto, e-print http://arxiv.org/abs/1303.3771 (2013)].

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

The Metropolis Monte Carlo method with CUDA enabled Graphic Processing Units

We present a CPU–GPU system for runtime acceleration of large molecular simulations using GPU computation and memory swaps. The memory architecture of the GPU can be used both as container for simulation data stored on the graphics card and as floating-point code target, providing an effective means for the manipulation of atomistic or molecular data on the GPU. To fully take advantage of this mechanism, efficient GPU realizations of algorithms used to perform atomistic and molecular simulations are essential. Our system implements a versatile molecular engine, including inter-molecule interactions and orientational variables for performing the Metropolis Monte Carlo (MMC) algorithm, which is one type of Markov chain Monte Carlo. By combining memory objects with floating-point code fragments we have implemented an MMC parallel engine that entirely avoids the communication time of molecular data at runtime. Our runtime acceleration system is a forerunner of a new class of CPU–GPU algorithms exploiting memory concepts combined with threading for avoiding bus bandwidth and communication. The testbed molecular system used here is a condensed phase system of oligopyrrole chains. A benchmark shows a size scaling speedup of 60 for systems with 210,000 pyrrole monomers. Our implementation can easily be combined with MPI to connect in parallel several CPU–GPU duets. -- Highlights: •We parallelize the Metropolis Monte Carlo (MMC) algorithm on one CPU—GPU duet. •The Adaptive Tempering Monte Carlo employs MMC and profits from this CPU—GPU implementation. •Our benchmark shows a size scaling-up speedup of 62 for systems with 225,000 particles. •The testbed involves a polymeric system of oligopyrroles in the condensed phase. •The CPU—GPU parallelization includes dipole—dipole and Mie—Jones classic potentials.

97 MATHEMATICS AND COMPUTING↗

Semi-Markov adjunction to the Computer-Aided Markov Evaluator (CAME)

The rule-based Computer-Aided Markov Evaluator (CAME) program was expanded in its ability to incorporate the effect of fault-handling processes into the construction of a reliability model. The fault-handling processes are modeled as semi-Markov events and CAME constructs and appropriate semi-Markov model. To solve the model, the program outputs it in a form which can be directly solved with the Semi-Markov Unreliability Range Evaluator (SURE) program. As a means of evaluating the alterations made to the CAME program, the program is used to model the reliability of portions of the Integrated Airframe/Propulsion Control System Architecture (IAPSA 2) reference configuration. The reliability predictions are compared with a previous analysis. The results bear out the feasibility of utilizing CAME to generate appropriate semi-Markov models to model fault-handling processes.

Rosch, Gene↗

Stochastic differential equation model for linear growth birth and death processes with immigration and emigration

This paper discusses on linear birth and death with immigration and emigration (BIDE) process to stochastic differential equation (SDE) model. Forward Kolmogorov equation in continuous time Markov chain (CTMC) with a central-difference approximation was used to find Fokker-Planckequation corresponding to a diffusion process having the stochastic differential equation of BIDE process. The exact solution, mean and variance function of BIDE process was found.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Optimal carbon storage reservoir management through deep reinforcement learning

Model-based optimization plays a central role in energy system design and management. The complexity and high-dimensionality of many process-level models, especially those used for geosystem energy exploration and utilization, often lead to formidable computational costs when the dimension of decision space is also large. This work adopts elements of recently advanced deep learning techniques to solve a sequential decision-making problem in applied geosystem management. Specifically, a deep reinforcement learning framework was formed for optimal multiperiod planning, in which a deep Q-learning network (DQN) agent was trained to maximize rewards by learning from high-dimensional inputs and from exploitation of its past experiences. To expedite computation, deep multitask learning was used to approximate high-dimensional, multistate transition functions. Both DQN and deep multitask learning are pattern based. As a demonstration, the framework was applied to optimal carbon sequestration reservoir planning using two different types of management strategies: monitoring only and brine extraction. Both strategies are designed to mitigate potential risks due to pressure buildup. Results show that the DQN agent can identify the optimal policies to maximize the reward for given risk and cost constraints. Finally, experiments also show that knowledge the agent gained from interacting with one environment is largely preserved when deploying the same agent in other similar environments.

15 GEOTHERMAL ENERGY↗

BENCHMARK TESTS FOR MARKOV CHAIN MONTE CARLO FITTING OF EXOPLANET ECLIPSE OBSERVATIONS

Ground-based observations of exoplanet eclipses provide important clues to the planets' atmospheric physics, yet systematics in light curve analyses are not fully understood. It is unknown if measurements suggesting near-infrared flux densities brighter than models predict are real, or artifacts of the analysis processes. We created a large suite of model light curves, using both synthetic and real noise, and tested the common process of light curve modeling and parameter optimization with a Markov Chain Monte Carlo algorithm. With synthetic white noise models, we find that input eclipse signals are generally recovered within 10% accuracy for eclipse depths greater than the noise amplitude, and to smaller depths for higher sampling rates and longer baselines. Red noise models see greater discrepancies between input and measured eclipse signals, often biased in one direction. Finally, we find that in real data, systematic biases result even with a complex model to account for trends, and significant false eclipse signals may appear in a non-Gaussian distribution. To quantify the bias and validate an eclipse measurement, we compare both the planet-hosting star and several of its neighbors to a separately chosen control sample of field stars. Re-examining the Rogers et al. Ks-band measurement of CoRoT-1b finds an eclipse 3190{sup +370}{sub -440} ppm deep centered at {phi}{sub me} = 0.50418{sup +0.00197}{sub -0.00203}. Finally, we provide and recommend the use of selected data sets we generated as a benchmark test for eclipse modeling and analysis routines, and propose criteria to verify eclipse detections.

79 ASTRONOMY AND ASTROPHYSICS↗

Variational actor-critic algorithms,

We introduce a class of variational actor-critic algorithms based on a variational formulation over both the value function and the policy. The objective function of the variational formulation consists of two parts: one for maximizing the value function and the other for minimizing the Bellman residual. Besides the vanilla gradient descent with both the value function and the policy updates, we propose two variants, the clipping method and the flipping method, in order to speed up the convergence. We also prove that, when the prefactor of the Bellman residual is sufficiently large, the fixed point of the algorithm is close to the optimal policy.

97 MATHEMATICS AND COMPUTING↗

Photoproduction of the Σ + hyperon using linearly polarized photons with CLAS

Measurements of the polarization observables Σ, P, T, O x , O z for the reaction $\vec{γ}$ p → $K^0_S$Σ + using a linearly polarized photon beam of energy 1.1 to 2.1 GeV are reported. The measured data provide information on a channel that has not been studied extensively, but is required for a full coupled-channel analysis in the nucleon resonance region. Observables have been simultaneously extracted using likelihood sampling with a Markov-Chain Monte Carlo process. Angular distributions in bins of photon energy E γ are produced for each polarization observable. T, O x , and O z are first time measurements of these observables in this reaction. The extraction of Σ extends the energy range beyond a previous measurement. The measurement of P, the recoil polarization, is consistent with previous measurements. The measured data are shown to be significant enough to affect the estimation of the nucleon resonance parameters when fitted within a coupled-channels model.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Adaptive Deep Reinforcement Learning Algorithm for Distribution System Cyber Attack Defense With High Penetration of DERs

With grid modernization, smart inverters are increasingly used to execute advanced controls for distribution network reliability. However, this also increases the cyber-attack space. Here this paper focuses on the defense approaches to restore the system to normal operation circumstances in the presence of cyber-attacks. A unique deep reinforcement learning (DRL) method is developed to minimize voltage violations and reduce power losses for impacted feeders. The defense problem is reformulated as a Markov decision-making process to dynamically control DERs while minimizing load shedding. This is achieved via an improved soft actor-critic (SAC)-based DRL algorithm, which can govern DER set points and load-shedding scenarios in discrete and continuous modes via the auto-tune entropy and Gaussian policy features. Numerical comparison results on the modified IEEE 123-node system with other control approaches, such as Volt-VAR (VV), Volt-Watt (VW), and model predictive control (MPC) show that the proposed method can eliminate voltage violations and provide feasible control actions that perform complete mitigation of cyber-threats.

24 POWER TRANSMISSION AND DISTRIBUTION↗

STPMiner: A Highperformance Spatiotemporal Pattern Mining Toolbox

The volume of spatiotemporal data being generated from scientific simulations and observations from sensors is growing at an astronomical rate. This data explosion is going to pose three challenges to the existing data mining infrastructure: algorithmic, computational, and I/O. Over the years we have implemented several spatiotemporal data mining algorithms including: outliers/anomalies, colocation patterns, change patterns, clustering, classification, and prediction algorithms. In this paper we briefly discuss the core spatiotemporal pattern mining algorithms along with some of the computational and I/O challenges associated with the big data.

99 GENERAL AND MISCELLANEOUS↗

Solar data uncertainty impacts on MCMC methods for r-process nucleosynthesis

In recent work, we developed a Markov Chain Monte Carlo (MCMC) procedure to predict the ground state masses capable of forming the observed Solar r-process rare-earth abundance peak. By applying this method to nucleosynthesis calculations which make use of distinct astrophysical conditions and comparing our results to the latest precision mass measurements, we are able to shed light on the conditions/masses capable of producing a rare-earth peak which matches Solar data. Here we examine how our mass predictions change when using a few different sets of r-process Solar abundance residuals that have been reported in the literature. We explore how the differing error estimates of these Solar evaluations propagate through the Markov Chain Monte Carlo to our mass predictions. We find that Solar data which reports the rare-earth peak to have its highest abundance at mass number A = 162 can require distinctly different mass predictions from data with the peak centered at A = 164. Nevertheless, we find that two important general conclusions from past work, regarding the inconsistency of ‘cold’ astrophysical outflows with current mass measurements and the need for local stability at N = 104 in ‘hot’ scenarios, remain robust in the face of differing Solar data evaluations. Additionally, we show that the masses our procedure finds capable of producing a peak at A < 164 are not in line with the latest precision mass measurements.

79 ASTRONOMY AND ASTROPHYSICS↗

Stochastic differential equation model to Prendiville processes

The Prendiville process is another variation of the logistic model which assumes linearly decreasing population growth rate. It is a continuous time Markov chain (CTMC) taking integer values in the finite interval. The continuous time Markov chain can be approximated by stochastic differential equation (SDE). This paper discusses the stochastic differential equation of Prendiville process. The work started with the forward Kolmogorov equation in continuous time Markov chain of Prendiville process. Then it was formulated in the form of a central-difference approximation. The approximation was then used in Fokker-Planck equation in relation to the stochastic differential equation of the Prendiville process. The explicit solution of the Prendiville process was obtained from the stochastic differential equation. Therefore, the mean and variance function of the Prendiville process could be easily found from the explicit solution.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗