Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Markov state models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Stochastic Price Generation for Evaluating Wholesale Electricity Market Bidding Strategies

This work presents a novel method for generating electricity price scenarios from statistical properties of past electricity prices using a hybrid statistical and reduced-form stochastic model. Previous work in applying stochastic differential equations (SDE) to model electricity prices has focused on daily average prices. To extend stochastic price generation methods to hourly or sub-hourly pricing, we address several weaknesses in the state-of-the-art: (1) we replace the mean-reversion component of the SDE with an ARIMA process that is better able to characterize the daily and weekly trends; (2) we extend the price-spike, or jump process to account for conditional probabilities of price spikes occurring in consecutive time steps by replacing the traditional Poisson process for modeling jumps with a generalized point process model inspired by brain neuron models; and (3) we replace the traditional method of estimating spike intensity with empirical variance with a Markov process based on observed price spike intensity transitions. The method is demonstrated with electricity prices from the US ERCOT market and a use-case example is provided for bidding an energy storage unit into the day-ahead and real-time energy markets of ERCOT using stochastic optimization methods. Results show that the the synthetic price model out performs a (naive) persistence forecast model by resulting in 24% to 47% more in profits over 168 simulated days.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
NASA NTRS1996

On the Use of Kronecker Operators for the Solution of Generalized Stochastic Petri Nets

We discuss how to describe the Markov chain underlying a generalized stochastic Petri net using Kronecker operators on smaller matrices. We extend previous approaches by allowing both an extensive type of marking-dependent behavior for the transitions and the presence of immediate synchronizations. The derivation of the results is thoroughly formalized, including the use of Kronecker operators in the treatment of the vanishing markings and the computation of impulse-based reward measures. We use our techniques to analyze a model whose solution using conventional methods would fail because of the state-space explosion. In the conclusion, we point out ideas to parallelize our approach.

Ciardo, Gianfranco↗
DOE OSTI2026

Movement Models to Predict Low‐Altitude Flight of Soaring Birds Using Look‐Ahead Environmental Factors

Advances in fine-scale movement modeling of soaring birds can aid efforts to understand and resolve the impacts of anthropogenic activities on such birds. Soaring birds often rely on underlying terrain and low-altitude updrafts to govern their flights at rotor-swept altitudes (≤ 200 m above ground level), which puts them at risk of collision with wind turbines. We developed a data-driven Markov model at 1-s resolution that predicts the fine-scale flight behavior of golden eagles (Aquila chrysaetos) as a function of ecological covariates at the current location as well as those within an eagle's line of sight. We only considered ecological covariates that are readily available in real-time (ground elevation and wind conditions). Latent factors (age, sex, species, behavioral intent, migratory status) were intentionally left out of the model. We calibrated the model using golden eagle telemetry data collected in two different ecoregions of the United States. Given a starting location, the calibrated model simulates multiple stochastic 3D paths to produce a time-explicit and spatially explicit risk map of turbine collisions. We discovered an empirical relation between the rate of change of heading and the orographic updraft conditions within an eagle's line of sight. Our model performed most effectively when predicting predominantly-soaring flights at rotor-swept altitudes during wind conditions in which turbines are likely to be operational. The calibrated model could be used in concert with automated eagle detection and turbine curtailment technologies. Specifically, once an eagle is detected by those systems, our model could then provide accurate predictions of turbines the eagle is likely to interact with in the near term.

17 WIND ENERGY↗
DOE OSTI2022

Estimating Cosmological Constraints from Galaxy Cluster Abundance using Simulation-Based Inference

Inferring the values and uncertainties of cosmological parameters in a cosmology model is of paramount importance for modern cosmic observations. In this paper, we use the simulation-based inference (SBI) approach to estimate cosmological constraints from a simplified galaxy cluster observation analysis. Using data generated from the Quijote simulation suite and analytical models, we train a machine learning algorithm to learn the probability function between cosmological parameters and the possible galaxy cluster observables. The posterior distribution of the cosmological parameters at a given observation is then obtained by sampling the predictions from the trained algorithm. Our results show that the SBI method can successfully recover the truth values of the cosmological parameters within the 2σ limit for this simplified galaxy cluster analysis, and acquires similar posterior constraints obtained with a likelihood-based Markov Chain Monte Carlo method, the current state-of the-art method used in similar cosmological studies.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2026

mvBayes

SAND2026-16980O mvBayes implements multivariate Bayesian regression using MATLAB and decomposes a multivariate or functional response into components based on a user-specified orthogonal basis. This allows for independent modeling of each component with any chosen univariate Bayesian regression model. This tool includes methods for prediction and visualization, facilitating the evaluation of Bayesian surrogate models through the application of Bayesian theory and Markov Chain Monte Carlo (MCMC) sampling techniques. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Tucker, J. Derek [Sandia National Lab. (SNL-CA), L↗
DOE OSTI2026

Quantum-Inspired Bayesian Sampling for Uncertainty Quantification and Machine Learning (Final Technical Report)

With increasing simulation and measurement data, machine learning and artificial intelligence have been widely used in computational decision-making of complex engineering systems. The resulting tools, such as uncertainty quantification solvers, reinforcement learning, and physics-informed machine learning, have achieved great success in critical DOE tasks such as material discovery and design, energy system modeling and control, and numerical weather and climate prediction. A core topic in scientific machine learning and artificial intelligence is Bayesian inference: given an observed data set, people want to estimate the posterior distribution of a (possibly large) number of hidden parameters. Due to the flexibility and weak assumptions, Bayesian sampling has been the mainstream Bayesian inference solvers despite the rapid progress of approximate Bayesian inference. Classical Bayesian sampling methods such as Markov-chain Monte Carlo suffer from a low-acceptance rate due to the random walk nature, therefore state-of-the-art techniques use Hamiltonian Monte Carlo and its variants to efficiently draw posterior samples in a high dimension. The key idea of Hamiltonian Monte Carlo and its variants is to simulate the Hamiltonian dynamics of a classical particle with a fixed mass, and their performance significantly degrades when the posterior distribution is highly spiky or has multiple modes. Leveraging the idea of quantum physics, this project has investigated new theory, algorithms and applications of Bayesian inference (especially Bayesian sampling). The main results include: (1) novel quantum-inspired Bayesian sampling methods that can lead to better accuracy for challenging multi-modal or spiky distributions, (2) more scalable machine learning framework leveraging tensor-compressed Bayesian inference, and (3) Bayesian and sampling approaches for verifying the robustness of continuous and binary neural networks.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1972

Orbit determination in the presence of unmodeled accelerations.

In the investigation discussed here, the unmodeled acceleration is approximated as a first-order stationary Gauss-Markov process which can be separated into a time wise correlated component and a purely random component. An extended form of the Kalman-Bucy filter is used to estimate the state and the unmodeled accelerations at each observation epoch. Applications of the estimation algorithm to simulated data and to tracking data obtained during the Apollo 10 and 11 missions indicate that a more accurate estimate of the state is obtained using the algorithm and that the estimated values of the unmodeled accelerations can be correlated with physical phenomena neglected by the mathematical model.

Tapley, B. D.↗
DOE OSTI2025

Personalized and uncertainty-aware coronary hemodynamics simulations: From Bayesian estimation to improved multi-fidelity uncertainty quantification

Non-invasive simulations of coronary hemodynamics have improved clinical risk stratification and treatment outcomes for coronary artery disease, compared to relying on anatomical imaging alone. However, simulations typically use empirical approaches to distribute total coronary flow amongst the arteries in the coronary tree, which ignores patient variability, the presence of disease, and other clinical factors. Further, uncertainty in the clinical data often remains unaccounted for in the modeling pipeline. We present an end-to-end uncertainty-aware pipeline to (1) personalize coronary flow simulations by incorporating vessel-specific coronary flows as well as cardiac function; and (2) predict clinical and biomechanical quantities of interest with improved precision, while accounting for uncertainty in the clinical data. We assimilate patient-specific measurements of myocardial blood flow from clinical CT myocardial perfusion imaging to estimate branch-specific coronary artery flows. Simulated noise in the clinical data is used to estimate the joint posterior distributions of the model parameters using adaptive Markov Chain Monte Carlo sampling. Additionally, the posterior predictive distribution for the relevant quantities of interest is determined using a new approach combining multi-fidelity Monte Carlo estimation with non-linear, data-driven dimensionality reduction. This leads to improved correlations between high- and low-fidelity model outputs. Our framework accurately recapitulates clinically measured cardiac function as well as branch-specific coronary flows under measurement noise uncertainty. We observe substantial reductions in confidence intervals for estimated quantities of interest compared to single-fidelity Monte Carlo estimation and state-of-the-art multi-fidelity Monte Carlo methods. This holds especially true for quantities of interest that showed limited correlation between the low- and high-fidelity model predictions. In addition, the proposed multi-fidelity Monte Carlo estimators are significantly cheaper to compute than traditional estimators, under a specified confidence level or variance. The proposed pipeline for personalized and uncertainty-aware predictions of coronary hemodynamics is based on routine clinical measurements and recently developed techniques for CT myocardial perfusion imaging. The proposed pipeline offers significant improvements in precision and reduction in computational cost.

Bayesian parameter estimation↗
DOE OSTI2025

viral_fam_classifier

This code applies publicly available Hidden Markov Model (HMM) profiles to publicly available reference genomes of viruses. It parses the output of these searches to determine bitscore cut-offs for viral taxonomic lineages that may be represented by each HMM. Subsequently, the code will allow the user to classify novel viral sequences using these bitscore cut-offs. This code is a work-in-progress.

Kantor, Rose [Lawrence Livermore National Laborato↗
DOE OSTI2025

Lectures on statistical mechanics

Presented here is a transcription of the lecture notes from Professor Allan N. Kaufman’s graduate statistical mechanics course Physics 212A and 212B at the University of California Berkeley from the 1972–1973 academic year. 212A addressed equilibrium statistical mechanics with topics: fundamentals (micro-canonical and sub-canonical ensembles, adiabatic law and action conservation, fluctuations, pressure, and virial theorem), classical fluids and other systems (equation of state, deviations from ideality, virial coefficients and van der Waals potential, canonical ensemble and partition function, quasistatic evolution, grand-canonical ensemble and partition function, chemical potential, simple model of a phase transition, quantum virial expansion, numerical simulation of equations of state, and phase transition), chemical equilibrium (systems with multiple species and chemical reactions, law of mass action, Saha equation, chemical equilibrium including ionization and excited states), and long-range interactions (including Coulomb, dipole, and gravitational interactions, Debye–Hückel theory, and shielding). 212B addressed nonequilibrium statistical mechanics with topics: fundamentals (definitions: realizations, moments, characteristic function, and discrete variables), Brownian motion (Langevin equation, fluctuation–dissipation theorem, spatial diffusion, Boltzmann’s H-theorem), Liouville and Klimontovich equations, Landau equation (derivation, elaboration, and H-theorem, and irreversibility), Markov processes and Fokker–Planck equation (derivations of the Fokker–Planck equation and a master equation), linear response and transport theory (linear Boltzmann equation, linear response theory of Kubo and Mori, relation of entropy production to electrical conductivity, transport relations and coefficients, normal mode solutions of the transport equations, sketch of a generalized Langevin equation method for transport theory), and an introduction to nonequilibrium quantum statistical mechanics.

plasma dynamics↗
DOE OSTI2026

Probing Physics beyond the Standard Model through Combined Analyses of Next-generation Type Ia Supernova, Cosmic Microwave Background, and Baryon Acoustic Oscillation Surveys

Observations of Type Ia supernovae (SNe Ia), which probe the late Universe, together with baryon acoustic oscillations (BAO) and the cosmic microwave background (CMB), which probe the intermediate and early epochs, provide complementary constraints on the expansion history of the Universe. In this work, we forecast constraints on dark energy and other extensions to the standard cosmological model by combining the SN Ia sample expected from the Vera C. Rubin Observatory’s Legacy Survey of Space and Time (LSST), data from current and forthcoming CMB surveys, and BAO measurements from the Dark Energy Spectroscopic Instrument (DESI). For the CMB, we use temperature, polarization, and lensing power spectra (TT/EE/TE/ϕϕ) from the South Pole Telescope, the planned Advanced Simons Observatory, and a CMB-S4–like experiment. We derive constraints on ΛCDM and its extensions involving the dark energy equation-of-state parameters (w 0 , w a ) and the sum of neutrino masses ∑m ν using a Markov Chain Monte Carlo (MCMC) sampling framework. We find that the LSST Year 3 SN Ia sample can improve upon the DES Year 5 dark energy constraints by a factor of 2−2.5×, with the gains driven primarily by the significantly higher SN Ia density in the LSST sample. Similarly, DESI-DR3 shows up to a 1.8× improvement on dark energy parameters over DR2, driven largely by the substantial increase in the low-redshift sample. Combining CMB with LSST-Y3-SN Ia and DESI-DR3-BAO yields σ(w 0 ) = 0.028 and σ(w a ) = 0.11 for w 0 w a CDM cosmology with the results being largely independent of the CMB dataset. The constraints weaken by 10%–30% when freeing ∑m ν and spatial curvature. Moreover, the joint analysis of the three datasets can enable a 2σ–3σ detection of ∑m ν .

Raghunathan, Srinivasan [University of California;↗
NASA NTRS1983

Analysis of structural perturbations in systems via cost decomposition methods

It has long been common practice to analyze linear dynamic systems by decomposing the total response in terms of individual contributions which are easier to analyze. Examples of this philosophy include the expansion of transfer functions using: (1) the superposition principle, (2) residue theory and partial fraction expansions, (3) Markov parameters, Hankel matrices, and (4) regular and singular perturbations. This paper summarizes a new and different kind of expansion designed to decompose the norm of the response vector rather than the response vector itself. This is referred to as "cost-decomposition' of the system. The notable advantages of this type of decomposition are: (a) easy application to multi-input, multi-output systems, (b) natural compatibility with Linear Quadratic Gaussian Theory, (c) applicability to the analysis of more general types of structural perturbations involving inputs, outputs, states, parameters. Property (c) makes the method suitable for problems in model reduction, measurement/actuator selections, and sensitivity analysis.

Skelton, R. E.↗
NASA NTRS2014

Initial Evaluations of LoC Prediction Algorithms Using the NASA Vertical Motion Simulator

Flying near the edge of the safe operating envelope is an inherently unsafe proposition. Edge of the envelope here implies that small changes or disturbances in system state or system dynamics can take the system out of the safe envelope in a short time and could result in loss-of-control events. This study evaluated approaches to predicting loss-of-control safety margins as the aircraft gets closer to the edge of the safe operating envelope. The goal of the approach is to provide the pilot aural, visual, and tactile cues focused on maintaining the pilot's control action within predicted loss-of-control boundaries. Our predictive architecture combines quantitative loss-of-control boundaries, an adaptive prediction method to estimate in real-time Markov model parameters and associated stability margins, and a real-time data-based predictive control margins estimation algorithm. The combined architecture is applied to a nonlinear transport class aircraft. Evaluations of various feedback cues using both test and commercial pilots in the NASA Ames Vertical Motion-base Simulator (VMS) were conducted in the summer of 2013. The paper presents results of this evaluation focused on effectiveness of these approaches and the cues in preventing the pilots from entering a loss-of-control event.

Pilot Cue↗
DOE OSTI2020

Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV Based Random Access IoT Networks With NOMA

In this paper, we apply the Non-Orthogonal Multiple Access (NOMA) technique to improve the massive channel access of a wireless IoT network where solar-powered Unmanned Aerial Vehicles (UAVs) relay data from IoT devices to remote servers. Specifically, IoT devices contend for accessing the shared wireless channel using an adaptive p-persistent slotted Aloha protocol; and the solar-powered UAVs adopt Successive Interference Cancellation (SIC) to decode multiple received data from IoT devices to improve access efficiency. To enable an energy-sustainable capacity-optimal network, we study the joint problem of dynamic multi-UAV altitude control and multi-cell wireless channel access management of IoT devices as a stochastic control problem with multiple energy constraints. We first formulate this problem as a Constrained Markov Decision Process (CMDP), and propose an online model-free Constrained Deep Reinforcement Learning (CDRL) algorithm based on Lagrangian primal-dual policy optimization to solve the CMDP. Extensive simulations demonstrate that our proposed algorithm learns a cooperative policy in which the altitude of UAVs and channel access probability of IoT devices are dynamically controlled to attain the maximal long-term network capacity while ensuring energy sustainability of UAVs, outperforming baseline schemes. The proposed CDRL agent can be trained on a small network, yet the learned policy can efficiently manage networks with a massive number of IoT devices and varying initial states, which can amortize the cost of training the CDRL agent.

42 ENGINEERING↗
DOE OSTI2025

Conin

SAND2025-07645O Conin is a Python library that supports constrained analysis of probabilistic graphical models (PGMs). It enables constrained inference and learning for hidden Markov models, Bayesian networks, dynamic Bayesian networks, and Markov networks. Conin interfaces with the pgmpy library to specify general probabilistic graphical models with a variety of optimization solvers to support learning and inference. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Hart, William [Sandia National Lab. (SNL-CA), Live↗
NASA NTRS2025

Efficient GPU-Accelerated MultiSource Global Fit Pipeline for LISA Data Analysis

The large-scale analysis task of deciphering gravitational-wave signals in the LISA data stream will be difficult, requiring a large amount of computational resources and extensive development of computational methods. Its high dimensionality, multiple model types, and complicated noise profile require a global fit to all parameters and input models simultaneously. In this work, we detail our global fit algorithm, called “Erebor,” designed to accomplish this challenging task. It is capable of analyzing current state-of-the-art datasets and then growing into the future as more pieces of the pipeline are completed and added. We describe our pipeline strategy, the algorithmic setup, and the results from our analysis of the LDC2A Sangria dataset, which contains massive black hole binaries, compact galactic binaries, and a parametrized noise spectrum whose parameters are unknown to the user. The Erebor algorithm includes three unique and very useful contributions: GPU acceleration for enhanced computational efficiency; ensemble Markov Chain Monte Carlo (MCMC) sampling with multiple MCMC walkers per temperature for better mixing and parallelized sample creation; and special online updates to reversible-jump (or transdimensional) sampling distributions to ensure sampler mixing and accurate initial estimates for detectable sources in the data.We recover posterior distributions for all 15 (6) of the injected massive black hole binaries (MBHB) in the LDC2A training (hidden) dataset. We catalog ∼12000 galactic binaries (∼8000 as high confidence detections) for both the training and hidden datasets. All of the sources and their posterior distributions are provided in publicly available catalogs.

LISA↗
DOE OSTI2021

Distributional Deep Reinforcement Learning-Based Emergency Frequency Control

Emergency frequency control is one of the most critical approaches to maintain power system stability after major disturbances. With the increasing number of grid-connected renewable energy sources, existing model-based methods of frequency control are facing up with challenges of computational speed and scalability for large-scale systems. In this paper, the emergency frequency control problem is formulated as a Markov Decision Process (MDP) and solved through a novel Distributional Deep Reinforcement Learning (DDRL) method, namely the distributional soft actor critic (DSAC) method. Compared with other RL methods that only estimate the mean value, the proposed DSAC model estimates the distribution of value function over returns. This advancement can lead to more insights and knowledge for the agent, with the benefit of a much faster and more stable learning process, and the improved frequency control performance. Here, the simulation results on IEEE 39-bus and IEEE 118-bus systems demonstrate the effectiveness and robustness of proposed models, as well as the advantage compared to other state-of-the-art DRL algorithms.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Accelerating multilevel Markov Chain Monte Carlo using machine learning models

Here, this work presents an efficient approach for accelerating multilevel Markov Chain Monte Carlo (MCMC) sampling for large-scale problems using low-fidelity machine learning models. While conventional techniques for large-scale Bayesian inference often substitute computationally expensive high-fidelity models with machine learning models, thereby introducing approximation errors, our approach offers a computationally efficient alternative by augmenting high-fidelity models with low-fidelity ones within a hierarchical framework. The multilevel approach utilizes the low-fidelity machine learning model (MLM) for inexpensive evaluation of proposed samples thereby improving the acceptance of samples by the high-fidelity model. The hierarchy in our multilevel algorithm is derived from geometric multigrid hierarchy. We utilize an MLM to accelerate the coarse level sampling. Training machine learning model for the coarsest level significantly reduces the computational cost associated with generating training data and training the model. We present an MCMC algorithm to accelerate the coarsest level sampling using MLM and account for the approximation error introduced. We provide theoretical proofs of detailed balance and demonstrate that our multilevel approach constitutes a consistent MCMC algorithm. Additionally, we derive the expression for cost reduction due to machine learning model to facilitate cost analysis of the hierarchical sampling algorithm. Our technique is demonstrated on a standard benchmark inference problem in groundwater flow, where we estimate the probability density of a quantity of interest using a four-level MCMC algorithm. Our proposed algorithm accelerates multilevel sampling by a factor of two while achieving similar accuracy compared to sampling using the standard multilevel algorithm.

97 MATHEMATICS AND COMPUTING↗