Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Markov”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Hierarchical semi-Markov models with duration-aware dynamics for activity sequences

Residential electricity demand at granular scales is driven by what people do and for how long. Accurately forecasting this demand for applications like microgrid management and demand response therefore requires generative models for activities that can produce realistic daily activity sequences, capturing both the timing and duration of human behavior. This paper develops a generative model of human activity sequences using nationally representative time-use diaries at a 10-min resolution. We use this model to quantify which demographic factors are most critical for improving predictive performance. We propose a hierarchical semi-Markov framework that addresses two key modeling challenges. First, a time-inhomogeneous Markov router learns the patterns of “which activity comes next.” Second, a semi-Markov hazard component explicitly models activity durations, capturing “how long” activities realistically last. To ensure statistical stability when data are sparse, the model pools information across related demographic groups and time blocks. The entire framework is trained and evaluated using survey design weights to ensure our findings are representative of the U.S. population. On a held-out test set, we demonstrate that explicitly modeling durations with the hazard component provides a substantial and statistically significant improvement over purely Markovian models. Furthermore, our analysis reveals a clear hierarchy of demographic factors: Sex, Day-Type, and Household Size provide the largest predictive gains, while Region and Season, though important for energy calculations, contribute little to predicting the activity sequence itself. The result is an interpretable and robust generator of synthetic activity traces, providing a high-fidelity foundation for downstream energy systems modeling.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Unfolding of the Villin Headpiece Domain: Revealing Structural Heterogeneity with Time‐Resolved X‐Ray Solution Scattering and Markov State Modeling

Understanding protein folding pathways is crucial to deciphering the principles of protein structure and function. Here, the unfolding dynamics of the 35‐residue villin headpiece (HP35) and a norleucine‐substituted variant (2F4K) using a combination of experimental and computational techniques is investigated. Time‐resolved X‐ray solution scattering coupled with equilibrium molecular dynamics simulations and Markov state modeling reveals distinct unfolding mechanisms between the two variants: HP35 and 2F4K. Specifically, HP35 exhibits a two‐state unfolding process, whereas an intermediate state is identified for the 2F4K mutant. A Markov state model constructed from simulations is used to map atomic‐level transitions to experimental observations, providing insights into the role of sequence variations in modulating folding pathways. The findings underscore the importance of integrating experimental and computational approaches to unravel protein unfolding mechanisms between heterogenous structural ensembles.

Nijhawan, Adam K. [Department of Chemistry Northwe↗
DOE OSTI2024

Spiking Markov Reward Process v.0.1

SAND2024-11150O The Spiking Markov Reward Process software is a spiking neural network that streams binary arithmetic and computes the state value function of a Markov reward process. The software will be released to the SpiNNcloud group for development of neuromorphic acceleration. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Wang, Felix↗
DOE OSTI2026

Optimal CO 2 storage management considering safety constraints in multi-stakeholder multi-site GCS projects: A Markov game perspective

Geological carbon storage (GCS) projects could involve a diverse array of stakeholders or players from public, private, and regulatory sectors, each with different objectives and responsibilities. Given the complexity, scale, and long-term nature of GCS operations, determining whether individual stakeholders can independently optimize their interests — or whether collaborative coalition agreements are needed — remains a central question for effective GCS project planning and management. To access large, high-quality storage resources, future GCS deployment may increasingly occur in geologically connected sites, where shared geological features such as pressure space and reservoir pore capacity can lead to competitive behavior among stakeholders. In this work, we propose a paradigm based on Markov games to quantitatively investigate how different coalition structures affect the goals of stakeholders. We frame this multi-stakeholder multi-site problem as a multi-agent reinforcement learning problem with safety constraints. Our approach enables agents to learn optimal strategies while complying with safety regulations. We present an example where multiple operators are injecting CO 2 into their respective project areas in a geologically connected basin. To address the high computational cost of repeated simulations of high fidelity models, a previously developed surrogate model based on the Embed-to-Control (E2C) framework is employed. Our results demonstrate the effectiveness of the proposed framework in addressing optimal management of CO 2 storage when multiple stakeholders with different objectives and goals are involved.

58 GEOSCIENCES↗
DOE OSTI2025

Accelerating multilevel Markov Chain Monte Carlo using machine learning models

Here, this work presents an efficient approach for accelerating multilevel Markov Chain Monte Carlo (MCMC) sampling for large-scale problems using low-fidelity machine learning models. While conventional techniques for large-scale Bayesian inference often substitute computationally expensive high-fidelity models with machine learning models, thereby introducing approximation errors, our approach offers a computationally efficient alternative by augmenting high-fidelity models with low-fidelity ones within a hierarchical framework. The multilevel approach utilizes the low-fidelity machine learning model (MLM) for inexpensive evaluation of proposed samples thereby improving the acceptance of samples by the high-fidelity model. The hierarchy in our multilevel algorithm is derived from geometric multigrid hierarchy. We utilize an MLM to accelerate the coarse level sampling. Training machine learning model for the coarsest level significantly reduces the computational cost associated with generating training data and training the model. We present an MCMC algorithm to accelerate the coarsest level sampling using MLM and account for the approximation error introduced. We provide theoretical proofs of detailed balance and demonstrate that our multilevel approach constitutes a consistent MCMC algorithm. Additionally, we derive the expression for cost reduction due to machine learning model to facilitate cost analysis of the hierarchical sampling algorithm. Our technique is demonstrated on a standard benchmark inference problem in groundwater flow, where we estimate the probability density of a quantity of interest using a four-level MCMC algorithm. Our proposed algorithm accelerates multilevel sampling by a factor of two while achieving similar accuracy compared to sampling using the standard multilevel algorithm.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Hierarchical Gaussian Random Field Sampling for Multilevel Markov Chain Monte Carlo: Coupling Stochastic Partial Differential Equation and the Karhunen–Loève Decomposition

This work introduces structure preserving hierarchical decompositions for sampling Gaussian random fields (GRFs) within the context of multilevel Bayesian inference in high-dimensional space. Existing scalable hierarchical sampling methods, such as those based on stochastic partial differential equations (SPDEs), often reduce the dimensionality of the sample space at the cost of accuracy of inference. Other approaches, such that those based on Karhunen-Loève (KL) expansions, offer sample space dimensionality reduction but sacrifice GRF representation accuracy and ergodicity of the Markov chain Monte Carlo (MCMC) sampler and are computationally expensive for high-dimensional problems. The proposed method integrates the dimensionality reduction capabilities of KL expansions with the scalability of SPDE-based sampling, thereby providing a robust, unified framework for high-dimensional uncertainty quantification (UQ) that is scalable and accurate, preserves ergodicity, and offers dimensionality reduction of the sample space. The hierarchy in our multilevel algorithm is derived from the geometric multigrid hierarchy. By constructing a hierarchical decomposition that maintains the covariance structure across the levels in the hierarchy, the approach enables efficient coarse-to-fine sampling while ensuring that all samples are drawn from the desired distribution. The effectiveness of the proposed method is demonstrated on a benchmark subsurface flow problem, demonstrating its effectiveness in improving computational efficiency and statistical accuracy. Furthermore, our proposed technique is more efficient and accurate and displays better convergence properties than existing methods for high-dimensional Bayesian inference problems.

Gaussian random fields↗
DOE OSTI2024

An FPGA-based hardware accelerator supporting sensitive sequence homology filtering with profile hidden Markov models

Abstract Background Sequence alignment lies at the heart of genome sequence annotation. While the BLAST suite of alignment tools has long held an important role in alignment-based sequence database search, greater sensitivity is achieved through the use of profile hidden Markov models (pHMMs). Here, we describe an FPGA hardware accelerator, called HAVAC, that targets a key bottleneck step (SSV) in the analysis pipeline of the popular pHMM alignment tool, HMMER. Results The HAVAC kernel calculates the SSV matrix at 1739 GCUPS on a $$\sim$$ ∼ $3000 Xilinx Alveo U50 FPGA accelerator card, $$\sim$$ ∼ 227× faster than the optimized SSV implementation in nhmmer . Accounting for PCI-e data transfer data processing, HAVAC is 65× faster than nhmmer’s SSV with one thread and 35× faster than nhmmer with four threads, and uses $$\sim$$ ∼ 31% the energy of a traditional high end Intel CPU. Conclusions HAVAC demonstrates the potential offered by FPGA hardware accelerators to produce dramatic speed gains in sequence annotation and related bioinformatics applications. Because these computations are performed on a co-processor, the host CPU remains free to simultaneously compute other aspects of the analysis pipeline.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Decision-making based on Markov decision process in integrated artificial reasoning framework—Part I: Theory

This paper presents a decision-making framework based on an integrated artificial reasoning framework and Markov decision process (MDP). The integrated artificial reasoning framework provides a physics-based approach that converts system information into state transition models, and the analysis result will be represented by the transition probabilities that can be used with an MDP to find a traceable and explainable optimal pathway. A dynamic Bayesian network (DBN) is well suited for representing the structure of an MDP. The causality information among process variables (or among subsystems) is mathematically represented in a DBN by the conditional probabilities of the node’s states provided different probabilities of the parent node’s states. To define node states in a physically understandable manner, we used multilevel flow modeling (MFM). An MFM follows the fundamental energy and mass conservation laws and supports the selection of process variables that represent the system of interest so that causal relations among process variables are properly captured. An MFM-based DBN supports developing state transition models in an MDP to capture the effect of process variables of system having physical relations. The operators of the target system can capture stochastic system dynamics as multiple subsystem state transitions based on their physical relations and uncertainties coming from component degradation or random failures. We analyzed a simplified exemplary system to illustrate an optimal operational policy using the suggested approach.

Markov decision process↗
DOE OSTI2025

Model Calibration with Markov Chain Monte Carlo Tutorial

The purpose of this tutorial is to demonstrate how to use Markov chain Monte Carlo (MCMC) to calibrate a model. By calibration, we mean the selection of model parameters (and, when relevant, structures). A common goal in model development and diagnostics is calibration, or the identification of model structures and parameters which are consistent with data. While models can be calibrated through hand-tuning parameters or minimizing simple error metrics such as root-mean-square-error (RMSE), these approaches can underrepresent the probabilistic nature of the data-generating process, as well as the potential for multiple model configurations to be consistent with the data. Probabilistic uncertainty quantification, which is the topic of this notebook, can address these concerns. This tutorial is presented as an appendix to the e-book: Addressing Uncertainty in MultiSector Dynamics Research.

Markov chain Monte Carlo↗
DOE OSTI2025

Particle Markov Chain Monte Carlo Approach to Inference in Transient Surface Kinetics

Here, in this work, we develop a novel Bayesian approach to study the adsorption and desorption of CO onto a Pd(111) surface, a process of great importance in natural sciences. The motivation for this work comes from the recent availability of time-resolved infrared spectroscopy data and the need for model interpretability and uncertainty quantification in chemical processes. The objective is to learn the relevant parameters that characterize the process: coverage with time, rate constants, activation energies, and pre-exponential factors. Our approach consists of three main schemes: (i) a problem design and probabilistic model for the whole system, (ii) a particle Markov chain Monte Carlo sampler to learn the hidden coverages and rate constant parameters, and (iii) two Bayesian formulations to infer the activation energies and pre-exponential factors. The flexibility of the Bayesian framework allows for uncertainty quantification where possible and integration of mathematical constraints in the model to reflect the system physically. We found that our results for the activation energies and pre-exponential factor are in agreement with those reported in the experimental literature, independently, and we provide discussions on the advantages and disadvantages as well as applicability to other systems.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Off-Lattice Markov Chain Monte Carlo Simulations of Mechanically Driven Polymers

Here, we develop off-lattice simulations of semiflexible polymer chains subjected to applied mechanical forces by using Markov Chain Monte Carlo. Our approach models the polymer as a chain of fixed length bonds, with configurations updated through adaptive nonlocal Monte Carlo moves. This proposed method enables precise calculation of a polymer’s response to a wide range of mechanical forces, which traditional on-lattice models cannot achieve. Our approach has shown excellent agreement with theoretical predictions of persistence length and end-to-end distance in quiescent states as well as stretching distances under tension. Moreover, our model eliminates the orientational bias present in on-lattice models, which significantly impacts calculations such as the scattering function, a crucial technique for revealing the polymer conformation.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Introducing a Markov chain-based time calibration procedure for multi-channel particle detectors: application to the SuperFGD and ToF detectors of the T2K experiment

Inter-channel mis-synchronisation can be a limiting factor to the time resolution of high performance timing detectors with multiple readout channels and independent electronics units. In these systems, time calibration methods employed must be able to efficiently correct for minimal mis-synchronisation between channels and achieve the best detector performance. We present an iterative time calibration method based on Markov Chains, suitable for detector systems with multiple readout channels. Starting from correlated hit pairs alone, and without requiring an external reference time measurement, the method solves for fixed per-channel offsets, with precision limited only by the intrinsic single-channel resolution. A mathematical proof that the method is able to find the correct time offsets to be assigned to each detector channel in order to achieve inter-channel synchronisation is given, and it is shown that the number of iterations to reach convergence within the desired precision is controllable with a single parameter. Numerical studies are used to confirm unbiased recovery of true offsets. Finally, the application of the calibration method to the Super Fine-Grained Detector (SuperFGD) and the Time of Flight (TOF) detector at the upgraded T2K near detector (ND280) shows good improvement in overall timing resolution, demonstrating the effectiveness in a real-world scenario and scalability.

calibration and fitting methods↗
DOE OSTI2026

Determination of proton PDF uncertainties with Markov chain Monte Carlo

We present an analysis of parton distribution functions (PDFs) of the proton using Markov chain Monte Carlo (MCMC) methods. The MCMC approach naturally implements Bayes’ theorem and, thus, provides a means to directly sample the underlying probability distribution—in this case, the probability distribution of the PDF parameters. This allows for a straightforward propagation of the resulting uncertainties into any PDF-dependent observable, preserving their simple probabilistic interpretation. In our analysis we include a broad set of deep inelastic scattering data from HERA, BCDMS and NMC experiments along with the Drell-Yan, 𝑊 and 𝑍 boson data from LHC and Tevatron experiments, which combined with theoretical calculations at next-to-next-to-leading order in QCD allow for realistic determination of PDFs. The main focus of this analysis is to explore alternative methods for PDF uncertainty estimation that are more firmly grounded in statistical principles. We show that the flexibility of the Bayes framework, allowing one, e.g., to account for non-Gaussianity or inconsistencies of datasets, is crucial to extract realistic uncertainties when such assumptions are not fulfilled. We also demonstrate that MCMC allows one to determine the Δ⁢𝜒 2 value corresponding to a given confidence level in the sample, which can, in turn, be used as a statistically well-founded tolerance criterion used in the Hessian method, thus addressing one of its main long-standing drawbacks.

Risse, Peter Clemens [Universität Münster (Germany↗
DOE OSTI2026

Determination of nuclear PDFs using Markov chain Monte Carlo methods

Global QCD analyses of nuclear parton distribution functions (nPDFs) have traditionally relied on the Hessian method for uncertainty estimation. However, the inherent Gaussian approximation and reliance on local curvature often prove insufficient for nPDF fits, which are frequently characterized by limited data constraints and non-Gaussian likelihoods. In this paper, we present the first nPDF determination based on Markov Chain Monte Carlo (MCMC) techniques, implemented within the nCTEQ framework using an adaptive Metropolis-Hastings algorithm. The MCMC approach enables a direct mapping of the posterior distribution and reveals a highly nontrivial parameter-space structure, including multiple modes and pronounced non-Gaussian behavior, particularly for the valence PDFs. We perform the first single-nucleus global analysis of lead PDFs using exclusively lead data and compare it to a multi-nuclei fit employing a standard analytic A dependence. The inclusion of lighter nuclei reduces quark uncertainties and modifies the shape of the lead PDFs, while leaving the gluon distribution largely unaffected. A complementary Hessian analysis exposes systematic limitations of the Gaussian approximation. Our results demonstrate that MCMC methods provide a more reliable framework for uncertainty quantification in nPDF determinations.

Derakhshanian, N. [Institute of Nuclear Physics Po↗
DOE OSTI2024

Markov Decision Processes for Intelligent, Risk-Informed Asset-Management Decision-Making

Advanced nuclear reactors are a promising option for aiding the world in achieving its net-zero carbon emission goals, however, there are significant challenges to attaining and maintaining economic competitiveness with other sources of electricity. To improve the economic competitiveness of advanced reactor designs, a project was initiated to explore the use of Markov Decision Processes (MDPs) to guide asset-management decision-making during advanced reactor operation. MDPs are a powerful tool for optimizing decision-making in complex environments and their application to advanced reactors can aid in planning maintenance and repair activities to minimize downtime and maximize generation. The described approach expands on previous work regarding the use of MDPs for operational decision-making through the direct incorporation of real-time plant information. The integral MDP analysis includes information from online component diagnostic tools and the plant’s real-time generation risk assessment (GRA) and probabilistic risk assessment (PRA), which evaluate plant risk from both an economic and safety perspective. The result is an asset-management optimization framework that is based on real-time data regarding plant component status and the current best-estimate of plant risk. The paper presents an overview of the theoretical framework to incorporate the different information pathways into an integral MDP analysis, along with example analyses.

Grabaskas, David↗
DOE OSTI2025

A Markov chain Monte Carlo (MCMC) Bayesian inference approach to analyze apparent activation barriers and reaction orders from microreactor data

Statistical analysis of steady-state catalytic kinetic data is often limited by data sparsity due to the slow pace at which the data is collected. Data sparsity and limitations in statistical analysis make it difficult to differentiate between mechanistic models and catalytic sites. A Bayesian inference tool is reported for catalysis researchers to estimate error in the determination of reaction orders from steady state microreactor data. The benefits of a Bayesian inference approach are discussed, as an alternative to the more common frequentist approach. The approach incorporates prior knowledge of the system and the data collected to form an error estimate on reaction orders. We investigated the effects of three distinct data treatments—individual fitting of trials, pooled analysis, and constrained regression methods—on the precision and uncertainty of reaction order determinations. To assess the robustness of our findings, we conducted sensitivity analyses to evaluate the influence of Bayesian parameters on uncertainty estimation. Additionally, we utilized synthetic data to illustrate how data quality impacts the precision of uncertainty assessments. We show Bayesian analysis can obtain a more precise estimation of error with a sparse data set than a frequentist analysis. Finally, this work provides strong evidence that the adoption of Bayesian analysis of kinetic data may help researchers make more precise arguments as to the strength of their evidence for a particular mechanistic hypothesis, or in comparing across different catalysts.

42 ENGINEERING↗