Bayesian modeling reveals metabolite‐dependent ultrasensitivity in the cyanobacterial circadian clock
Not Available
59 BASIC BIOLOGICAL SCIENCES↗
SEARCH · Engineering Papers
Results for “Markov Model”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Stochastic Price Generation for Evaluating Wholesale Electricity Market Bidding Strategies
This work presents a novel method for generating electricity price scenarios from statistical properties of past electricity prices using a hybrid statistical and reduced-form stochastic model. Previous work in applying stochastic differential equations (SDE) to model electricity prices has focused on daily average prices. To extend stochastic price generation methods to hourly or sub-hourly pricing, we address several weaknesses in the state-of-the-art: (1) we replace the mean-reversion component of the SDE with an ARIMA process that is better able to characterize the daily and weekly trends; (2) we extend the price-spike, or jump process to account for conditional probabilities of price spikes occurring in consecutive time steps by replacing the traditional Poisson process for modeling jumps with a generalized point process model inspired by brain neuron models; and (3) we replace the traditional method of estimating spike intensity with empirical variance with a Markov process based on observed price spike intensity transitions. The method is demonstrated with electricity prices from the US ERCOT market and a use-case example is provided for bidding an energy storage unit into the day-ahead and real-time energy markets of ERCOT using stochastic optimization methods. Results show that the the synthetic price model out performs a (naive) persistence forecast model by resulting in 24% to 47% more in profits over 168 simulated days.
29 ENERGY PLANNING, POLICY, AND ECONOMY↗
Multimodal parameter spaces of a complex multi-channel neuron model
One of the most common types of models that helps us to understand neuron behavior is based on the Hodgkin–Huxley ion channel formulation (HH model). A major challenge with inferring parameters in HH models is non-uniqueness: many different sets of ion channel parameter values produce similar outputs for the same input stimulus. Such phenomena result in an objective function that exhibits multiple modes (i.e., multiple local minima). This non-uniqueness of local optimality poses challenges for parameter estimation with many algorithmic optimization techniques. HH models additionally have severe non-linearities resulting in further challenges for inferring parameters in an algorithmic fashion. To address these challenges with a tractable method in high-dimensional parameter spaces, we propose using a particular Markov chain Monte Carlo (MCMC) algorithm, which has the advantage of inferring parameters in a Bayesian framework. The Bayesian approach is designed to be suitable for multimodal solutions to inverse problems. We introduce and demonstrate the method using a three-channel HH model. We then focus on the inference of nine parameters in an eight-channel HH model, which we analyze in detail. We explore how the MCMC algorithm can uncover complex relationships between inferred parameters using five injected current levels. The MCMC method provides as a result a nine-dimensional posterior distribution, which we analyze visually with solution maps or landscapes of the possible parameter sets. The visualized solution maps show new complex structures of the multimodal posteriors, and they allow for selection of locally and globally optimal value sets, and they visually expose parameter sensitivities and regions of higher model robustness. We envision these solution maps as enabling experimentalists to improve the design of future experiments, increase scientific productivity and improve on model structure and ideation when the MCMC algorithm is applied to experimental data.
97 MATHEMATICS AND COMPUTING↗
Commuter preferences for a first-mile/last-mile microtransit service in the United States
Transportation system models rely heavily upon value of time (VOT) estimates to predict customer behavior. Accurate VOT estimates are particularly vital for planning new services such as on-demand ride hailing or microtransit because customers’ sensitivity to wait time, walk time, and route detour time affects their likelihood of selecting these modes. If an incorrect VOT is assumed during service planning, then ridership will be depressed because of a mismatch between their preferences and how the system is designed. In this paper, we report on the measurement of VOT for microtransit, a shared first-mile/last-mile mobility service, obtained using stated preference microdata from four U.S. cities. Here, we found a median in-vehicle VOT for microtransit of $\$$18.63 (95% CI: $\$$13.39–$\$$24.46) and an access VOT of $\$$75.38 (95% CI: $\$$59.22–$\$$94.96). The former is practically equal to the VOT we found for respondents’ current modes ($\$$20.24, 95% CI: $\$$13.71–$\$$26.94). We also found that men, younger riders, the highly educated, and transit riders are more likely to be interested in microtransit. Since the disutility of time spent on microtransit is not higher than that of other modes, we believe this new service has the potential to attract riders, and particularly if the system is designed with low waiting and walking times.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
Deep Reinforcement Learning Enabled Physical-Model-Free Two-Timescale Voltage Control Method for Active Distribution Systems
Active distribution networks are being challenged by frequent and rapid voltage violations due to renewable energy integration. Conventional model-based voltage control methods rely on accurate parameters of the distribution networks, which are difficult to achieve in practice. This paper proposes a novel physical-model-free two-timescale voltage control framework for active distribution systems. To achieve fast control of PV inverters, the whole network is first partitioned into several subnetworks using voltage-reactive power sensitivity. Then, the scheduling of PV inverters in the multiple sub-networks is formulated as Markov games and solved by a multi-agent soft actor-critic (MASAC) algorithm, where each subnetwork is modeled as an intelligent agent. All agents are trained in a centralized manner to learn a coordinated strategy while being executed based on only local information for fast response. For the slower time-scale control, OLTCs and switched capacitors are coordinated by a single agent-based SAC algorithm using the global information with considering control behaviors of the inverters. Particularly, the two-level agents are trained concurrently with information exchange according to the reward signal calculated from the data-driven surrogate model. Comparative tests with different benchmark methods on IEEE 33-and 123-bus systems and 342-node low voltage distribution system demonstrate that the proposed method can effectively mitigate the fast voltage violations and achieve systematical coordination of different voltage regulation assets without the knowledge of accurate system model.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Operation Optimization using Reinforcement Learning with Integrated Artificial Reasoning Framework
In large and complex systems, operational decision-making requires a systematic analysis with a vast amount of data from both process parameters and component status monitoring. In this paper, we present an integrated artificial reasoning approach for system state transition models that can help operational decision-making with explainable and traceable reasoning. The integrated artificial reasoning framework is a physics-based approach of defining the system structure in a Bayesian network, so we leveraged it in a Markov decision process (MDP) for finding optimal operational solutions. In our proposed framework, the MDP is implemented on a dynamic Bayesian network (DBN), which represents causalities in a system. The multilevel flow modeling was utilized in order to extract these causalities in a more efficient and objective manner. Since multilevel flow modeling is based on the fundamental energy and mass conservation laws, the target system is decomposed into several mass, energy, and information structures, which serve as the basis for a DBN. The MDP consists of the processes of finding a solution for the Bellman equation, which can be derived from the conditional probability equations of the constructed DBN. System operators can capture stochastic system dynamics as multiple subsystem state transitions based on their physical relations and uncertainties coming from the component degradation process or random failures. We analyzed a simplified example system to illustrate finding an optimal operational policy with this approach.
99 GENERAL AND MISCELLANEOUS↗
Dark matter clumps as sources of gravitational-wave glitches in LIGO-Virgo-KAGRA data
We consider the hypothetical possibility that nonstationary glitch features in the noise of ground-based gravitational-wave detectors could be produced by small dark matter clumps that pass through the earth in the vicinity of gravitational-wave detectors. Here, we first derive the gravitational-wave strain that would be generated by the passage of such a dark matter clump. We find that the strain is primarily sourced by the Newtonian gravitational acceleration of the mirrors toward the clump and by the Shapiro time delay of the photons in the laser beams as they pass through the gravitational potential created by the dark matter clump. We also find that the Newtonian acceleration effect dominates the gravitational-wave strain for both ground and space-based interferometers. We then compare our dark matter clump, gravitational-wave strain model to 84 Koi-Fish glitches detected during the second observing run of the LIGO/Virgo/KAGRA Collaboration through a Markov chain Monte Carlo Bayesian analysis. We find that all glitches but nine can be confidently rejected as having originated from dark matter clumps. For the remaining glitches, the dark matter hypothesis cannot be excluded, and the maximum a posteriori parameters yield minimum densities of about 10 −7 g/cm 3 , within the model. These results allow us to place the first direct upper limits with gravitational-wave detectors on the local over-density of dark matter in the form of clumps in the local neighborhood of Earth, namely 𝜌 DM clumps ≲ 10 −15 g/cm −3 .
Astronomy and AstroPhysics↗
A Note on the Effects of Linear Topology Preservation in Monte Carlo Simulations of Knotted Proteins
Monte Carlo simulations are a powerful technique and are widely used in different fields. When applied to complex molecular systems with long chains, such as those in synthetic polymers and proteins, they have the advantage of providing a fast and computationally efficient way to sample equilibrium ensembles and calculate thermodynamic and structural properties under desired conditions. Conformational Monte Carlo techniques employ a move set to perform the transitions in the simulation Markov chain. While accepted conformations must preserve the sequential bonding of the protein chain model and excluded volume among its units, the moves themselves may take the chain across itself. We call this a break in linear topology preservation. In this manuscript, we show, using simple protein models, that there is no difference in equilibrium properties calculated with a move set that preserves linear topology and one that does not. However, for complex structures, such as those of deeply knotted proteins, the preservation of linear topology provides correct equilibrium results but only after long relaxation. In any case, to analyze folding pathways, knotting mechanisms and folding kinetics, the preservation of linear topology may be an unavoidable requirement.
Especial, João N. C. (ORCID:0000000254765370)↗
Photometric Objects around Cosmic Webs (PAC) Delineated in a Spectroscopic Survey. I. Methods
We provide a method for estimating the projected density distribution ${\bar{n}}_{2}{w}_{p}({r}_{p})$ of photometric objects around spectroscopic objects in a spectroscopic survey. This quantity describes the distribution of photometric sources with certain physical properties (e.g., luminosity, mass, and color) around cosmic webs (PAC) traced by the spectroscopic objects. The method can make full use of current and future deep and wide photometric surveys to explore the formation of galaxies up to medium redshift (z s < 2) 3 with the aid of cosmological spectroscopic surveys that sample only a fairly limited species of objects (e.g., emission line galaxies). As an example, we apply the PAC method to the CMASS spectroscopic and HSC-SSP PDR2 photometric samples to explore the distribution of galaxies for a wide range of stellar masses from 10 9.0 to 10 12.0 M ⊙ around massive galaxies at z s ≈ 0.6. Using the abundance-matching method, we model ${\bar{n}}_{2}{w}_{p}({r}_{p})$ in N-body simulation using Markov chain Monte Carlo sampling, and we accurately measure the stellar–halo mass relation and stellar mass function for the whole mass range. We can also measure the conditional stellar mass function of satellites for central galaxies of different mass. The PAC method has many potential applications for studying the evolution of galaxies.
79 ASTRONOMY AND ASTROPHYSICS↗
An Evolutionary Algorithm to Personalize Stool-Based Colorectal Cancer Screening
Fecal immunochemical testing (FIT) is an established method for colorectal cancer (CRC) screening. Measured FIT-concentrations are associated with both present and future risk of CRC, and may be used for personalized screening. However, evaluation of personalized screening is computationally challenging. In this study, a broadly applicable algorithm is presented to efficiently optimize personalized screening policies that prescribe screening intervals and FIT-cutoffs, based on age and FIT-history. We present a mathematical framework for personalized screening policies and a bi-objective evolutionary algorithm that identifies policies with minimal costs and maximal health benefits. The algorithm is combined with an established microsimulation model (MISCAN-Colon), to accurately estimate the costs and benefits of generated policies, without restrictive Markov assumptions. The performance of the algorithm is demonstrated in three experiments. In Experiment 1, a relatively small benchmark problem, the optimal policies were known. The algorithm approached the maximum feasible benefits with a relative difference of 0.007%. Experiment 2 optimized both intervals and cutoffs, Experiment 3 optimized cutoffs only. Optimal policies in both experiments are unknown. Compared to policies recently evaluated for the USPSTF, personalized screening increased health benefits up to 14 and 4.3%, for Experiments 2 and 3, respectively, without adding costs. Generated policies have several features concordant with current screening recommendations. The method presented in this paper is flexible and capable of optimizing personalized screening policies evaluated with computationally-intensive but established simulation models. It can be used to inform screening policies for CRC or other diseases. For CRC, more debate is needed on what features a policy needs to exhibit to make it suitable for implementation in practice.
60 APPLIED LIFE SCIENCES↗
Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning
We present a mathematical framework and computational methods for optimally designing a finite sequence of experiments. This sequential optimal experimental design (sOED) problem is formulated as a finite-horizon partially observable Markov decision process (POMDP) under a Bayesian setting and with information-theoretic utilities. The formulation is general and may accommodate continuous random variables, non-Gaussian posteriors, and nonlinear forward models. The sOED design policy incorporates elements of feedback and lookahead simultaneously, and we show it to generalize the commonly-used batch and greedy design strategies. We solve for the sOED policy using the policy gradient (PG) method from reinforcement learning, and provide a derivation for the PG expression in the sOED context. Adopting an actor-critic approach, the policy and value functions are parameterized using deep neural networks and improved via PG estimates produced from simulated episodes of designs and observations. The new PG-sOED algorithm is first validated on a linear-Gaussian benchmark, and then compared against other design baselines on a sensor movement problem for contaminant source inversion in a convection-diffusion field. As a result, we provide explanation for the policy behaviors using knowledge of the underlying physical process.
97 MATHEMATICS AND COMPUTING↗
A Stochastic Intracellular Model of Anthrax Infection With Spore Germination Heterogeneity
We present a stochastic mathematical model of the intracellular infection dynamics of Bacillus anthracis in macrophages. Following inhalation of B. anthracis spores, these are ingested by alveolar phagocytes. Ingested spores then begin to germinate and divide intracellularly. This can lead to the eventual death of the host cell and the extracellular release of bacterial progeny. Some macrophages successfully eliminate the intracellular bacteria and will recover. Here, a stochastic birth-and-death process with catastrophe is proposed, which includes the mechanism of spore germination and maturation of B. anthracis . The resulting model is used to explore the potential for heterogeneity in the spore germination rate, with the consideration of two extreme cases for the rate distribution: continuous Gaussian and discrete Bernoulli. We make use of approximate Bayesian computation to calibrate our model using experimental measurements from in vitro infection of murine peritoneal macrophages with spores of the Sterne 34F2 strain of B. anthracis . The calibrated stochastic model allows us to compute the probability of rupture, mean time to rupture, and rupture size distribution, of a macrophage that has been infected with one spore. We also obtain the mean spore and bacterial loads over time for a population of cells, each assumed to be initially infected with a single spore. Our results support the existence of significant heterogeneity in the germination rate, with a subset of spores expected to germinate much later than the majority. Furthermore, in agreement with experimental evidence, our results suggest that most of the spores taken up by macrophages are likely to be eliminated by the host cell, but a few germinated spores may survive phagocytosis and lead to the death of the infected cell. Finally, we discuss how this stochastic modelling approach, together with dose-response data, allows us to quantify and predict individual infection risk following exposure.
59 BASIC BIOLOGICAL SCIENCES↗
Stochastic simulation of occupant-driven energy use in a bottom-up residential building stock model
The residential buildings sector is one of the largest electricity consumers worldwide and contributes disproportionally to peak electricity demand in many regions. Strongly driven by occupant activities, household energy consumption is stochastic and heterogeneous in nature. However, most residential energy models applied by industry use homogeneous, deterministic activity schedules, which work well for predictions of annual energy consumption, but can result in unrealistic hourly or sub-hourly electric load profiles, with exaggerated or muted peaks. The increasing proportion of variable renewable energy generators means that representing the heterogeneity and stochasticity of occupant behavior is now crucial for reliable planning at both bulk-power and distribution-system scales. This work presents a novel and open-source occupancy simulation approach that can simulate a diverse set of individual occupant and household event schedules for all major electricity, fuel, and hot water end uses. To accomplish this, we evaluated three alternative occupant activity simulation approaches before selecting a hybrid combining time-inhomogeneous Markov chains and probability-sampling of event durations and magnitudes. Further, we integrated the stochastic occupancy simulation with an open-source bottom-up physics-simulation building stock model and published a set of 550,000 diverse household end-use activity schedules representing a national housing stock. The simulator was verified against time-use survey data, and simulation results were validated against measured end-use electricity data for accuracy and reliability. While we use data for the United States, our application demonstrates how similar approaches could be applied using the time-use survey data collected in many countries around the world.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
Fracton Hydrodynamics without Time-Reversal Symmetry
Here, we present an effective field theory for the nonlinear fluctuating hydrodynamics of a single conserved charge with or without time-reversal symmetry, based on the Martin-Siggia-Rose formalism. Applying this formalism to fluids with only charge and multipole conservation, and with broken time-reversal symmetry, we predict infinitely many new dynamical universality classes, including some with arbitrarily large upper critical dimensions. Using large scale simulations of classical Markov chains, we find numerical evidence for a breakdown of hydrodynamics in quadrupole-conserving models with broken time-reversal symmetry in one spatial dimension. Our framework can be applied to the hydrodynamics around stationary states of open systems, broadening the applicability of previously developed ideas and methods to a wide range of systems in driven and active matter.
75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
Generalized master equation for particle transport in binary random media with renewal statistics
Particle transport in binary stochastic mixtures is classically modeled assuming Markovian or exponential mixing statistics but in many applications material memory invalidates the Markov assumption. For non-Markovian mixing characterized by alternating renewal processes, a transport-theoretic framework is presented that provides an exact description of transport in nonscattering random binary media with general non-exponential statistics. Our approach is to Markovianize the problem by augmenting the {material type, particle flux} state space with the age or distance from the last interface. A Chapman-Kolmogorov equation is formulated for the joint probability density of the material type, particle flux, and age, and subsequently reduced to a generalized Master equation (GME) in differential form. This constitutes the primary result of this work. A state-updating Monte Carlo algorithm consistent with the GME is developed and benchmarked against analytical solutions for multiple chord-length laws. For purely absorbing renewal statistical media, the GME reproduces analytical benchmarks for the equilibrium age distribution, interior mean/variance of material-conditioned fluxes, and boundary transmittance. Simulations further demonstrate that a Markov (exponential) approximation of non-exponential statistics can introduce large errors in transmittance and interior flux profiles. Lastly, the reintroduction of memory due to scattering is briefly addressed through heuristic considerations.
Fluctuations & noise↗
Bayesian inference of structured latent spaces from neural population activity with the orthogonal stochastic linear mixing model
The brain produces diverse functions, from perceiving sounds to producing arm reaches, through the collective activity of populations of many neurons. Determining if and how the features of these exogenous variables (e.g., sound frequency, reach angle) are reflected in population neural activity is important for understanding how the brain operates. Often, high-dimensional neural population activity is confined to low-dimensional latent spaces. However, many current methods fail to extract latent spaces that are clearly structured by exogenous variables. This has contributed to a debate about whether or not brains should be thought of as dynamical systems or representational systems. Here, we developed a new latent process Bayesian regression framework, the orthogonal stochastic linear mixing model (OSLMM) which introduces an orthogonality constraint amongst time-varying mixture coefficients, and provide Markov chain Monte Carlo inference procedures. We demonstrate superior performance of OSLMM on latent trajectory recovery in synthetic experiments and show superior computational efficiency and prediction performance on several real-world benchmark data sets. We primarily focus on demonstrating the utility of OSLMM in two neural data sets: μ ECoG recordings from rat auditory cortex during presentation of pure tones and multi-single unit recordings form monkey motor cortex during complex arm reaching. We show that OSLMM achieves superior or comparable predictive accuracy of neural data and decoding of external variables (e.g., reach velocity). Most importantly, in both experimental contexts, we demonstrate that OSLMM latent trajectories directly reflect features of the sounds and reaches, demonstrating that neural dynamics are structured by neural representations. Together, these results demonstrate that OSLMM will be useful for the analysis of diverse, large-scale biological time-series datasets.
59 BASIC BIOLOGICAL SCIENCES↗
Reinforcement Learning for Intentional Islanding in Resilient Power Transmission Systems
Intentional islanding is the process of identifying and deliberately decomposing the transmission network to form self-sustained islands from an endangered network during disruptions to improve resilience and security. Most existing intentional islanding models are offline resilience decision tools and hence do not provide outage responses in a timely manner. In this paper, a reinforcement learning (RL) based model for intentional islanding is developed, which offers real-time switching control, online deployability, and adaptability to varying system conditions. The intentional islanding process is formulated as a Markov decision process, where the optimal transmission switching policy is learned using the RL approach. The control policy is learned over an environment that encompasses a Power System Simulator for Engineering (PSS/E) model of the transmission network, facilitated by an interface to the standard openAI Gym framework. The proposed RL-based methodology aims to form stable and self-sustainable islands by ensuring voltage stability while reducing the power mismatch in the formed islands. A proximal policy optimization algorithm is designed, which is suitable for controlling the on/off status of the switches with multi-layer perceptron as value and actor networks. The effectiveness of the proposed framework in the self-recovery of the grid by island formation is applied on the modified IEEE 39-bus test network and validated by dynamic simulations.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Decision-making based on Markov decision process in integrated artificial reasoning framework—Part I: Theory
This paper presents a decision-making framework based on an integrated artificial reasoning framework and Markov decision process (MDP). The integrated artificial reasoning framework provides a physics-based approach that converts system information into state transition models, and the analysis result will be represented by the transition probabilities that can be used with an MDP to find a traceable and explainable optimal pathway. A dynamic Bayesian network (DBN) is well suited for representing the structure of an MDP. The causality information among process variables (or among subsystems) is mathematically represented in a DBN by the conditional probabilities of the node’s states provided different probabilities of the parent node’s states. To define node states in a physically understandable manner, we used multilevel flow modeling (MFM). An MFM follows the fundamental energy and mass conservation laws and supports the selection of process variables that represent the system of interest so that causal relations among process variables are properly captured. An MFM-based DBN supports developing state transition models in an MDP to capture the effect of process variables of system having physical relations. The operators of the target system can capture stochastic system dynamics as multiple subsystem state transitions based on their physical relations and uncertainties coming from component degradation or random failures. We analyzed a simplified exemplary system to illustrate an optimal operational policy using the suggested approach.
Markov decision process↗