Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Markov Model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,135 records · Page 63

Deep Reinforcement Learning Enabled Physical-Model-Free Two-Timescale Voltage Control Method for Active Distribution Systems

Active distribution networks are being challenged by frequent and rapid voltage violations due to renewable energy integration. Conventional model-based voltage control methods rely on accurate parameters of the distribution networks, which are difficult to achieve in practice. This paper proposes a novel physical-model-free two-timescale voltage control framework for active distribution systems. To achieve fast control of PV inverters, the whole network is first partitioned into several subnetworks using voltage-reactive power sensitivity. Then, the scheduling of PV inverters in the multiple sub-networks is formulated as Markov games and solved by a multi-agent soft actor-critic (MASAC) algorithm, where each subnetwork is modeled as an intelligent agent. All agents are trained in a centralized manner to learn a coordinated strategy while being executed based on only local information for fast response. For the slower time-scale control, OLTCs and switched capacitors are coordinated by a single agent-based SAC algorithm using the global information with considering control behaviors of the inverters. Particularly, the two-level agents are trained concurrently with information exchange according to the reward signal calculated from the data-driven surrogate model. Comparative tests with different benchmark methods on IEEE 33-and 123-bus systems and 342-node low voltage distribution system demonstrate that the proposed method can effectively mitigate the fast voltage violations and achieve systematical coordination of different voltage regulation assets without the knowledge of accurate system model.

24 POWER TRANSMISSION AND DISTRIBUTION↗

A Bayesian approach for parameter estimation and prediction using a computationally intensive model

Bayesian methods have been successful in quantifying uncertainty in physics-based problems in parameter estimation and prediction. In these cases, physical measurements y are modeled as the best fit of a physics-based model eta(theta), where theta denotes the uncertain, best input setting. Hence the statistical model is of the form y = eta(theta) + c, where epsilon accounts for measurement, and possibly other, error sources. When nonlinearity is present in eta(center dot), the resulting posterior distribution for the unknown parameters in the Bayesian formulation is typically complex and nonstandard, requiring computationally demanding computational approaches such as Markov chain Monte Carlo (MCMC) to produce multivariate draws from the posterior. Although generally applicable, MCMC requires thousands (or even millions) of evaluations of the physics model eta(center dot). This requirement is problematic if the model takes hours or days to evaluate. To overcome this computational bottleneck, we present an approach adapted from Bayesian model calibration. This approach combines output from an ensemble of computational model runs with physical measurements, within a statistical formulation, to carry out inference. A key component of this approach is a statistical response surface, or emulator, estimated from the ensemble of model runs. We demonstrate this approach with a case study in estimating parameters for a density functional theory model, using experimental mass/binding energy measurements from a collection of atomic nuclei. We also demonstrate how this approach produces uncertainties in predictions for recent mass measurements obtained at Argonne National Laboratory.

Higdon, Dave↗

General squark flavour mixing: constraints, phenomenology and benchmarks

Here, we present an extensive study of non-minimal flavour violation in the squark sector in the framework of the Minimal Supersymmetric Standard Model. We investigate the effects of multiple non-vanishing flavour-violating elements in the squark mass matrices by means of a Markov Chain Monte Carlo scanning technique and identify parameter combinations that are favoured by both current data and theoretical constraints. We then detail the resulting distributions of the flavour-conserving and flavour-violating model parameters. Based on this analysis, we propose a set of benchmark scenarios relevant for future studies of non-minimal flavour violation in the Minimal Supersymmetric Standard Model.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Exploring Effective Best Management Practices in the Miyun Reservoir Watershed, China

Miyun reservoir watershed is a major source of drinking water for China’s capital, Beijing, which has a population of 21.75 million. Recently, the capacity of the Miyun reservoir to supply clean drinking water is threatened by increasing eutrophication (or algae bloom), mainly due to the discharge of wastewater and excessive fertilization application in the upstream watershed. Therefore, there is an urgent need to design effective best management practices (BMPs) to reduce upstream nutrient load and improve water quality in the Miyun reservoir. In this study, we built a watershed model (i.e. the Soil and Water Assessment Tool) in the Miyun Reservoir Watershed (MRW), and calibrated and validated it using long term sediment, nitrogen (N) and phosphorus (P) data. Further, we developed a Markov Chain based multi-objective optimization program to explore optimal BMPs with tradeoffs between economic costs and water quality responses. With the watershed model and the multi-objective optimization algorithms, we explored potential effectiveness of BMPs under two scenarios that are currently being considered: scenario 1 assumes funding of BMP implementation is from national grants and targets high water quality standard, while scenario 2 assumes funding is from farmers and targets water quality that barely meet the drinking water standard. We found substantial discrepancy between these two scenarios with respect to type and spatial configuration of BMPs and associated economic costs, highlighting the need of reconciling concerns from different stakeholders in order to arrive at a BMP plan that all parties will agree upon. In addition, we found that cross-subwatershed coordination and targeting flood season instead of year-round water quality standards could pronouncedly reduce economic cost of BMP implementation while not substantially degrade water quality. The watershed scale optimization method developed here hold the promise serve as an effective tool to explore tradeoffs between economic costs, water quality improvements, and decisions makers and stakeholders’ concerns in BMPs design, thereby informing sustainable watershed scale water resources management.

Qiu, Jiali↗

Operation Optimization using Reinforcement Learning with Integrated Artificial Reasoning Framework

In large and complex systems, operational decision-making requires a systematic analysis with a vast amount of data from both process parameters and component status monitoring. In this paper, we present an integrated artificial reasoning approach for system state transition models that can help operational decision-making with explainable and traceable reasoning. The integrated artificial reasoning framework is a physics-based approach of defining the system structure in a Bayesian network, so we leveraged it in a Markov decision process (MDP) for finding optimal operational solutions. In our proposed framework, the MDP is implemented on a dynamic Bayesian network (DBN), which represents causalities in a system. The multilevel flow modeling was utilized in order to extract these causalities in a more efficient and objective manner. Since multilevel flow modeling is based on the fundamental energy and mass conservation laws, the target system is decomposed into several mass, energy, and information structures, which serve as the basis for a DBN. The MDP consists of the processes of finding a solution for the Bellman equation, which can be derived from the conditional probability equations of the constructed DBN. System operators can capture stochastic system dynamics as multiple subsystem state transitions based on their physical relations and uncertainties coming from the component degradation process or random failures. We analyzed a simplified example system to illustrate finding an optimal operational policy with this approach.

99 GENERAL AND MISCELLANEOUS↗

EXPLORING THE ROLE OF SUB-MICRON-SIZED DUST GRAINS IN THE ATMOSPHERES OF RED L0–L6 DWARFS

We examine the hypothesis that the red near-infrared colors of some L dwarfs could be explained by a “dust haze” of small particles in their upper atmospheres. This dust haze would exist in conjunction with the clouds found in dwarfs with more typical colors. We developed a model that uses Mie theory and the Hansen particle size distributions to reproduce the extinction due to the proposed dust haze. We apply our method to 23 young L dwarfs and 23 red field L dwarfs. We constrain the properties of the dust haze including particle size distribution and column density using Markov Chain Monte Carlo methods. We find that sub-micron-range silicate grains reproduce the observed reddening. Current brown dwarf atmosphere models include large-grain (1–100 μ m) dust clouds but not sub-micron dust grains. Our results provide a strong proof of concept and motivate a combination of large and small dust grains in brown dwarf atmosphere models.

79 ASTRONOMY AND ASTROPHYSICS↗

Dark matter clumps as sources of gravitational-wave glitches in LIGO-Virgo-KAGRA data

We consider the hypothetical possibility that nonstationary glitch features in the noise of ground-based gravitational-wave detectors could be produced by small dark matter clumps that pass through the earth in the vicinity of gravitational-wave detectors. Here, we first derive the gravitational-wave strain that would be generated by the passage of such a dark matter clump. We find that the strain is primarily sourced by the Newtonian gravitational acceleration of the mirrors toward the clump and by the Shapiro time delay of the photons in the laser beams as they pass through the gravitational potential created by the dark matter clump. We also find that the Newtonian acceleration effect dominates the gravitational-wave strain for both ground and space-based interferometers. We then compare our dark matter clump, gravitational-wave strain model to 84 Koi-Fish glitches detected during the second observing run of the LIGO/Virgo/KAGRA Collaboration through a Markov chain Monte Carlo Bayesian analysis. We find that all glitches but nine can be confidently rejected as having originated from dark matter clumps. For the remaining glitches, the dark matter hypothesis cannot be excluded, and the maximum a posteriori parameters yield minimum densities of about 10 −7 g⁡/cm 3 , within the model. These results allow us to place the first direct upper limits with gravitational-wave detectors on the local over-density of dark matter in the form of clumps in the local neighborhood of Earth, namely 𝜌 DM ⁢clumps ≲ 10 −15 g⁡/cm −3 .

Astronomy and AstroPhysics↗

A stochastic approach to quantifying the blur with uncertainty estimation for high-energy X-ray imaging systems

One of the primary causes of blur in a high-energy X-ray imaging system is the shape and extent of the radiation source, or ‘spot’. It is important to be able to quantify the size of the spot as it provides a lower bound on the recoverable resolution for a radiograph, and penumbral imaging methods – which involve the analysis of blur caused by a structured aperture – can be used to obtain the spot’s spatial profile. We present a Bayesian approach for estimating the spot shape that, unlike variational methods, is robust to the initial choice of parameters. The posterior is obtained from a normal likelihood, which was constructed from a weighted least squares approximation to a Poisson noise model, and prior assumptions that enforce both smoothness and non-negativity constraints. A Markov chain Monte Carlo algorithm is used to obtain samples from the target posterior, and the reconstruction and uncertainty estimates are the computed mean and variance of the samples, respectively. Lastly, synthetic data-sets are used to demonstrate accurate reconstruction, while real data taken with high-energy X-ray imaging systems are used to demonstrate applicability and feasibility.

97 MATHEMATICS AND COMPUTING↗

Efficient uncertainty quantification methodologies for high-dimensional climate land models

In this report, we proposed, examined and implemented approaches for performing efficient uncertainty quantification (UQ) in climate land models. Specifically, we applied Bayesian compressive sensing framework to a polynomial chaos spectral expansions, enhanced it with an iterative algorithm of basis reduction, and investigated the results on test models as well as on the community land model (CLM). Furthermore, we discussed construction of efficient quadrature rules for forward propagation of uncertainties from high-dimensional, constrained input space to output quantities of interest. The work lays grounds for efficient forward UQ for high-dimensional, strongly non-linear and computationally costly climate models. Moreover, to investigate parameter inference approaches, we have applied two variants of the Markov chain Monte Carlo (MCMC) method to a soil moisture dynamics submodel of the CLM. The evaluation of these algorithms gave us a good foundation for further building out the Bayesian calibration framework towards the goal of robust component-wise calibration.

54 ENVIRONMENTAL SCIENCES↗

First Passage Moments of Finite-State Semi-Markov Processes

In this paper, we discuss the computation of first-passage moments of a regular time-homogeneous semi-Markov process (SMP) with a finite state space to certain of its states that possess the property of universal accessibility (UA). A UA state is one which is accessible from any other state of the SMP, but which may or may not connect back to one or more other states. An important characteristic of UA is that it is the state-level version of the oft-invoked process-level property of irreducibility. We adapt existing results for irreducible SMPs to the derivation of an analytical matrix expression for the first passage moments to a single UA state of the SMP. In addition, consistent point estimators for these first passage moments, together with relevant R code, are provided.

97 MATHEMATICS AND COMPUTING↗

A Note on the Effects of Linear Topology Preservation in Monte Carlo Simulations of Knotted Proteins

Monte Carlo simulations are a powerful technique and are widely used in different fields. When applied to complex molecular systems with long chains, such as those in synthetic polymers and proteins, they have the advantage of providing a fast and computationally efficient way to sample equilibrium ensembles and calculate thermodynamic and structural properties under desired conditions. Conformational Monte Carlo techniques employ a move set to perform the transitions in the simulation Markov chain. While accepted conformations must preserve the sequential bonding of the protein chain model and excluded volume among its units, the moves themselves may take the chain across itself. We call this a break in linear topology preservation. In this manuscript, we show, using simple protein models, that there is no difference in equilibrium properties calculated with a move set that preserves linear topology and one that does not. However, for complex structures, such as those of deeply knotted proteins, the preservation of linear topology provides correct equilibrium results but only after long relaxation. In any case, to analyze folding pathways, knotting mechanisms and folding kinetics, the preservation of linear topology may be an unavoidable requirement.

Especial, João N. C. (ORCID:0000000254765370)↗

Photometric Objects around Cosmic Webs (PAC) Delineated in a Spectroscopic Survey. I. Methods

We provide a method for estimating the projected density distribution ${\bar{n}}_{2}{w}_{p}({r}_{p})$ of photometric objects around spectroscopic objects in a spectroscopic survey. This quantity describes the distribution of photometric sources with certain physical properties (e.g., luminosity, mass, and color) around cosmic webs (PAC) traced by the spectroscopic objects. The method can make full use of current and future deep and wide photometric surveys to explore the formation of galaxies up to medium redshift (z s < 2) 3 with the aid of cosmological spectroscopic surveys that sample only a fairly limited species of objects (e.g., emission line galaxies). As an example, we apply the PAC method to the CMASS spectroscopic and HSC-SSP PDR2 photometric samples to explore the distribution of galaxies for a wide range of stellar masses from 10 9.0 to 10 12.0 M ⊙ around massive galaxies at z s ≈ 0.6. Using the abundance-matching method, we model ${\bar{n}}_{2}{w}_{p}({r}_{p})$ in N-body simulation using Markov chain Monte Carlo sampling, and we accurately measure the stellar–halo mass relation and stellar mass function for the whole mass range. We can also measure the conditional stellar mass function of satellites for central galaxies of different mass. The PAC method has many potential applications for studying the evolution of galaxies.

79 ASTRONOMY AND ASTROPHYSICS↗

Control theory for random systems

A survey is presented of the current knowledge available for designing and predicting the effectiveness of controllers for dynamic systems which can be modeled by ordinary differential equations. A short discussion of feedback control is followed by a description of deterministic controller design and the concept of system state. The need for more realistic disturbance models led to the use of stochastic process concepts, in particular the Gauss-Markov process. A compensator controlled system, with random forcing functions, random errors in the measurements, and random initial conditions, is treated as constituting a Gauss-Markov random process; hence the mean-square behavior of the controlled system is readily predicted. As an example, a compensator is designed for a helicopter to maintain it in hover in a gusty wind over a point on the ground.

Bryson, A. E., Jr.↗

HiRel: Hybrid Automated Reliability Predictor (HARP) integrated reliability tool system, (version 7.0). Volume 3: HARP Graphics Oriented (GO) input user's guide

The Hybrid Automated Reliability Predictor (HARP) integrated Reliability (HiRel) tool system for reliability/availability prediction offers a toolbox of integrated reliability/availability programs that can be used to customize the user's application in a workstation or nonworkstation environment. HiRel consists of interactive graphical input/output programs and four reliability/availability modeling engines that provide analytical and simulative solutions to a wide host of highly reliable fault-tolerant system architectures and is also applicable to electronic systems in general. The tool system was designed at the outset to be compatible with most computing platforms and operating systems, and some programs have been beta tested within the aerospace community for over 8 years. This document is a user's guide for the HiRel graphical preprocessor Graphics Oriented (GO) program. GO is a graphical user interface for the HARP engine that enables the drawing of reliability/availability models on a monitor. A mouse is used to select fault tree gates or Markov graphical symbols from a menu for drawing.

Bavuso, Salvatore J.↗

Probabilistic Path Planning of Montgolfier Balloons in Strong, Uncertain Wind Fields

Lighter-than-air vehicles such as hot-air balloons have been proposed for exploring Saturn s moon Titan, as well as other bodies with significant atmospheres. For these vehicles to navigate effectively, it is critical to incorporate the effects of surrounding wind fields, especially as these winds will likely be strong relative to the control authority of the vehicle. Predictive models of these wind fields are available, and previous research has considered problems of planning paths subject to these predicted forces. However, such previous work has considered the wind fields as known a priori, whereas in practical applications, the actual wind vector field is not known exactly and may deviate significantly from the wind velocities estimated by the model. A probabilistic 3D path-planning algorithm was developed for balloons to use uncertain wind models to generate time-efficient paths. The nominal goal of the algorithm is to determine what altitude and what horizontal actuation, if any is available on the vehicle, to use to reach a particular goal location in the least expected time, utilizing advantageous winds. The solution also enables one to quickly evaluate the expected time-to-goal from any other location and to avoid regions of large uncertainty. This method is designed for balloons in wind fields but may be generalized for any buoyant vehicle operating in a vector field. To prepare the planning problem, the uncertainty in the wind field is modeled. Then, the problem of reaching a particular goal location is formulated as a Markov decision process (MDP) using a discretized space approach. Solving the MDP provides a policy of what actuation option (how much buoyancy change and, if applicable, horizontal actuation) should be selected at any given location to minimize the expected time-to-goal. The results provide expected time-to-goal values from any given location on the globe in addition to the action policy. This stochastic approach can also provide insights not accessible by deterministic methods; for example, one can evaluate variability and risk associated with different scenarios, rather than only viewing the expected outcome.

Wolf, Michael↗

STPMiner: A Highperformance Spatiotemporal Pattern Mining Toolbox

The volume of spatiotemporal data being generated from scientific simulations and observations from sensors is growing at an astronomical rate. This data explosion is going to pose three challenges to the existing data mining infrastructure: algorithmic, computational, and I/O. Over the years we have implemented several spatiotemporal data mining algorithms including: outliers/anomalies, colocation patterns, change patterns, clustering, classification, and prediction algorithms. In this paper we briefly discuss the core spatiotemporal pattern mining algorithms along with some of the computational and I/O challenges associated with the big data.

99 GENERAL AND MISCELLANEOUS↗

An Evolutionary Algorithm to Personalize Stool-Based Colorectal Cancer Screening

Fecal immunochemical testing (FIT) is an established method for colorectal cancer (CRC) screening. Measured FIT-concentrations are associated with both present and future risk of CRC, and may be used for personalized screening. However, evaluation of personalized screening is computationally challenging. In this study, a broadly applicable algorithm is presented to efficiently optimize personalized screening policies that prescribe screening intervals and FIT-cutoffs, based on age and FIT-history. We present a mathematical framework for personalized screening policies and a bi-objective evolutionary algorithm that identifies policies with minimal costs and maximal health benefits. The algorithm is combined with an established microsimulation model (MISCAN-Colon), to accurately estimate the costs and benefits of generated policies, without restrictive Markov assumptions. The performance of the algorithm is demonstrated in three experiments. In Experiment 1, a relatively small benchmark problem, the optimal policies were known. The algorithm approached the maximum feasible benefits with a relative difference of 0.007%. Experiment 2 optimized both intervals and cutoffs, Experiment 3 optimized cutoffs only. Optimal policies in both experiments are unknown. Compared to policies recently evaluated for the USPSTF, personalized screening increased health benefits up to 14 and 4.3%, for Experiments 2 and 3, respectively, without adding costs. Generated policies have several features concordant with current screening recommendations. The method presented in this paper is flexible and capable of optimizing personalized screening policies evaluated with computationally-intensive but established simulation models. It can be used to inform screening policies for CRC or other diseases. For CRC, more debate is needed on what features a policy needs to exhibit to make it suitable for implementation in practice.

60 APPLIED LIFE SCIENCES↗

Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning

We present a mathematical framework and computational methods for optimally designing a finite sequence of experiments. This sequential optimal experimental design (sOED) problem is formulated as a finite-horizon partially observable Markov decision process (POMDP) under a Bayesian setting and with information-theoretic utilities. The formulation is general and may accommodate continuous random variables, non-Gaussian posteriors, and nonlinear forward models. The sOED design policy incorporates elements of feedback and lookahead simultaneously, and we show it to generalize the commonly-used batch and greedy design strategies. We solve for the sOED policy using the policy gradient (PG) method from reinforcement learning, and provide a derivation for the PG expression in the sOED context. Adopting an actor-critic approach, the policy and value functions are parameterized using deep neural networks and improved via PG estimates produced from simulated episodes of designs and observations. The new PG-sOED algorithm is first validated on a linear-Gaussian benchmark, and then compared against other design baselines on a sensor movement problem for contaminant source inversion in a convection-diffusion field. As a result, we provide explanation for the policy behaviors using knowledge of the underlying physical process.

97 MATHEMATICS AND COMPUTING↗