Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Markov model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Deep Reinforcement Learning for Distribution System Operations: A Tutorial and Survey

Here, the rapid evolution of modern electric power distribution systems into complex networks of interconnected active devices, distributed generation (DG), and storage poses increasing difficulties for system operators. The large-scale integration of distributed energy resources (DERs) and the rapid exchange of measurement data via communication networks present major opportunities for advancing grid operations but also introduce greater uncertainty, higher data dimensionality, more complex network and device models, and challenging control and optimization problems. Deep reinforcement learning (DRL) algorithms are promising in addressing these challenges. However, they have not been effectively adapted for power systems applications, requiring extensive customization for implementation and evaluation. This has resulted in reproducibility challenges and a steep learning curve for researchers new to applying DRL algorithms to the power systems domain. To bridge these gaps, this tutorial aims to serve as a valuable resource for researchers interested in exploring learning-based algorithms to operate active power distribution networks. Specifically, this work presents a generalized process for translating sequential decision-making problems in power distribution systems into Markov decision process (MDP) formulations, illustrated through concrete grid service examples. Additionally, we introduce a simple environment design strategy to develop and evaluate example DRL algorithms for distribution system applications, complete with an included code repository to guide users through environment construction.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

GIGA-Lens: Fast Bayesian Inference for Strong Gravitational Lens Modeling

We present GIGA-Lens: a gradient-informed, GPU-accelerated Bayesian framework for modeling strong gravitational lensing systems, implemented in TensorFlow and JAX. The three components, optimization using multistart gradient descent, posterior covariance estimation with variational inference, and sampling via Hamiltonian Monte Carlo, all take advantage of gradient information through automatic differentiation and massive parallelization on graphics processing units (GPUs). We test our pipeline on a large set of simulated systems and demonstrate in detail its high level of performance. The average time to model a single system on four Nvidia A100 GPUs is 105 s. The robustness, speed, and scalability offered by this framework make it possible to model the large number of strong lenses found in current surveys and present a very promising prospect for the modeling of ${ \mathcal O }({10}^{5})$ lensing systems expected to be discovered in the era of the Vera C. Rubin Observatory, Euclid, and the Nancy Grace Roman Space Telescope.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Noise-specific beating in the higher-level Ramsey curves of a transmon qubit

We report: in the higher levels of superconducting transmon devices and more generally charge sensitive devices, T 2 * measurements made in the presence of low-frequency time-correlated 1/f charge noise and quasiparticle-induced parity flips can give an underestimation of the total dephasing time. The charge variations manifest as beating patterns observed in the overlay of several Ramsey fringe curves and are reproduced with a phenomenological Ramsey curve model, which accounts for the charge variations. T 2 * dephasing times, which more accurately represent the total dephasing time, are obtained. The phenomenological model is compared with a Lindblad master equation model. Both models are found to be in agreement with one another and the experimental data. Finally, the phenomenological formulation enables a simple method in which the power spectral density for the low-frequency noise can be inferred from the overlay of several Ramsey curves.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

GraMeR: Gra ph Me ta R einforcement learning for multi-objective influence maximization

Influence maximization (IM) is a combinatorial problem of identifying a subset of seed nodes in a network (graph), which when activated, provide a maximal spread of influence in the network for a given diffusion model and a budget for seed set size. IM has numerous applications such as viral marketing, epidemic control, sensor placement and other network-related tasks. However, its practical uses are limited due to the computational complexity of current algorithms. Recently, deep reinforcement learning has been leveraged to solve IM in order to ease the computational burden. However, there are serious limitations in current approaches, including narrow IM formulation that only consider influence via spread and ignore self-activation, low scalability to large graphs, and lack of generalizability across graph families leading to a large running time for every test network. In this work, we address these limitations through a unique approach that involves: (1) Formulating a generic IM problem as a Markov decision process that handles both intrinsic and influence activations; (2)incorporating generalizability via meta-learning across graph families. There are previous works that combine deep reinforcement learning with graph neural network, but this work solves a more realistic IM problem and incorporates generalizability across graphs via meta reinforcement learning. Extensive experiments are carried out in various standard networks to validate performance of the proposed Graph Meta Reinforcement learning (GraMeR) framework. Finally, the results indicate that GraMeR is multiple orders faster and generic than conventional approaches when applied on small to medium scale graphs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Learning infinite-horizon average-reward restless multi-action bandits via index awareness

We consider the online restless bandits with average-reward and multiple actions, where the state of each arm evolves according to a Markov decision process (MDP), and the reward of pulling an arm depends on both the current state of the corresponding MDP and the action taken. Since finding the optimal control is typically intractable for restless bandits, existing learning algorithms are often computationally expensive or with a regret bound that is exponential in the number of arms and states. In this paper, we advocate \textit{index-aware reinforcement learning} (RL) solutions to design RL algorithms operating on a much smaller dimensional subspace by exploiting the inherent structure in restless bandits. Specifically, we first propose novel index policies to address dimensionality concerns, which are provably optimal. We then leverage the indices to develop two low-complexity index-aware RL algorithms, namely, (i) GM-R2MAB, which has access to a generative model; and (ii) UC-R2MAB, which learns the model using an upper confidence style online exploitation method. We prove that both algorithms achieve a sub-linear regret that is only polynomial in the number of arms and states. A key differentiator between our algorithms and existing ones stems from the fact that our RL algorithms contain a novel exploitation that leverages our proposed provably optimal index policies for decision-makings.

Xiong, Guojun↗
DOE OSTI2020

CALPHAD Uncertainty Quantification and TDBX

CALPHAD uncertainty quantification (UQ) is the foundation of materials design with quantified confidence. We report a framework and software packages to enable CALPHAD UQ assessment and calculation using commercial CALPHAD software (Thermo-Calc). This Bayesian inference framework is coupled with a Markov chain Monte Carlo algorithm to establish uncertainty traces with a given thermodynamic database file (TDB) and corresponding experimental data points. This general framework is demonstrated with the Ni–Cr binary system. The algorithm is firstly validated on synthetic data with known ground truth. Then it is applied to real experimental data to generate posterior traces. We develop a file format named TDBX, which provides a single source of truth by combining the original TDB content and the traces for each assessed Gibbs energy parameter. CALPHAD UQ calculations are performed based on the TDBX file, from which uncertainties for phase boundaries, enthalpy curves, and solidification range are collected as examples of basic design parameters. This TDBX file with corresponding scripts are made open-source. Finally, the combination of CALPHAD UQ assessments and calculations connected by TDBX supports uncertainty-assisted modeling, enabling the integrated application of modern design with uncertainty methodologies to computational materials design.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Using PyBioNetFit to leverage qualitative and quantitative data in biological model parameterization and uncertainty quantification

Data generated in studies of cellular regulatory systems are often qualitative. For example, measurements of signaling readouts in the presence and absence of mutations may reveal a rank ordering of responses across conditions but not the precise extents of mutation-induced differences. Qualitative data are often ignored by mathematical modelers or are considered in an ad hoc manner, as in the study of Kocieniewski and Lipniacki (2013) [Phys Biol 10: 035006], which was focused on the roles of MEK isoforms in ERK activation. In this earlier study, model parameter values were tuned manually to obtain consistency with a combination of qualitative and quantitative data. This approach is not reproducible, nor does it provide insights into parametric or prediction uncertainties. Here, starting from the same data and the same ordinary differential equation (ODE) model structure, we generate formalized statements of qualitative observations, making these observations more reusable, and we improve the model parameterization procedure by applying a systematic and automated approach enabled by the software package PyBioNetFit. We also demonstrate uncertainty quantification (UQ), which was absent in the original study. Our results show that PyBioNetFit enables qualitative data to be leveraged, together with quantitative data, in parameterization of systems biology models and facilitates UQ. These capabilities are important for reliable estimation of model parameters and model analyses in studies of cellular regulatory systems and reproducibility.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Hadron spectroscopy of pseudoscalar and vector meson photoproduction using linearly polarised photons at CLAS

The field of hadron spectroscopy is informed by experiments which study excited states of the nucleon by detecting the decay products following excitation with a hadronic, leptonic or electromagnetic probe. Experiments using a polarised beam of photons provide an important contribution to the world data. This thesis describes the analysis of data from one such experiment, the CEBAF Large Acceptance Spectrometer at Jefferson Laboratory, Virginia, USA. In the experiment, an incident beam of electrons passes through a diamond radiator producing a linearly polarised photon beam of energy from 1.1 to 2.1 GeV on a liquid hydrogen target. The analysis method and results for two reaction channels are presented. In the first analysis, the polarisation observables {?, P, T, Ox, Oz} for the reaction ??? p ? K0 S ?+ are extracted simultaneously using likelihood sampling with Markov-Chain Monte-Carlo. The results are presented as angular distributions (cos ?K0S) for bins in photon energy E? for a total of 21 bins. The values extracted for T , Ox and Oz are a first measurement of these quantities for this reaction and the beam asymmetry (?) measurement supplements the one previous measurement for this quantity and extends the energy range over which it has been extracted. The measurement of the recoil polarisation, P, is consistent with previous measurements. In a second analysis, the spin density matrix elements (SDMEs) for ? p ? p ? are extracted using similar analysis techniques. The results are presented as angular distributions (cos ?? ) for bins in photon energy E? for a total of 24 bins. This work is the first to extract all nine SDMEs for the full angular range of the ? meson production. A comparison is made to the predictions of the Vector Meson Dominance (VMD) model and the results provide evidence for contributing processes other than VMD, particularly at more backward angles of the ? meson production. The data also indicate non-helicity conserving process in both the s-channel and the t-channel. The results from the first analysis have been passed to the Juelich-Bonn theory group and the effect of including the new data within their dynamical coupled channel model is described. Several nucleon resonances are affected and the impact on the pole positions, widths and photon decay amplitudes of the affected resonances is described.

Clark, Louise↗
DOE OSTI2022

Inference, Prediction, & Entropy-Rate Estimation of Continuous-Time, Discrete-Event Processes

Inferring models, predicting the future, and estimating the entropy rate of discrete-time, discrete-event processes is well-worn ground. However, a much broader class of discrete-event processes operates in continuous-time. Here, we provide new methods for inferring, predicting, and estimating them. The methods rely on an extension of Bayesian structural inference that takes advantage of neural network’s universal approximation power. Based on experiments with complex synthetic data, the methods are competitive with the state-of-the-art for prediction and entropy-rate estimation.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Self-organized criticality in multi-pulse gamma-ray bursts

The variability in multi-pulse gamma-ray bursts (GRBs) may help to reveal the mechanism of underlying processes from the central engine. To investigate whether the self-organized criticality (SOC) phenomena exist in the prompt phase of GRBs, we statistically study the properties of GRBs with more than 3 pulses in each burst by fitting the distributions of several observed physical variables with a Markov Chain Monte Carlo approach, including the isotropic energy E iso , the duration time T, and the peak count rate P of each pulse. Our sample consists of 454 pulses in 93 GRBs observed by the CGRO/BATSE satellite. The best-fitting values and uncertainties for these power-law indices of the differential frequency distributions are: α$^{d}_{E}$ = 1.54±0.09, α$^{d}_{T}$ = 1.82±$^{+0.14}_{0.15}$ and α$^{d}_{P}$ = 2.09$^{+0.18}_{–0.19}$, while the power-law indices in the cumulative frequency distributions are: α$^{c}_{E}$ = 1.44$^{+0.08}_{–0.10}$, α$^{c}_{T}$ = 1.75$^{+0.141}_{–0.13}$ and α$^{c}_{P}$ = 1.99$^{+0.16}_{–0.19}$. We find that these distributions are roughly consistent with the physical framework of a Fractal-Diffusive, Self-Organized Criticality (FD-SOC) system with the spatial dimension S = 3 and the classical diffusion β=1. The results of this study support that the jet responsible for the GRBs should be magnetically dominated and magnetic instabilities (e.g., kink model, or tearing-model instability) lead the GRB emission region into the SOC state.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Theoretical evidence of water serving as a promoter for lithium superoxide disproportionation in Li-O 2 batteries

Experimental evidence has demonstrated that the presence of water in non-aqueous electrolytes significantly affects Li-O 2 electrochemistry. Understanding the reaction mechanism for Li 2 O 2 formation in the presence of water impurities is important to understand Li-O 2 battery performance. A recent experiment has found that very small amounts of water (as low as 40 ppm) can significantly affect the product formation in Li-O 2 batteries as opposed to essentially no water (1 ppm). Although experimental as well as theoretical work has proposed mechanisms of Li 2 O 2 formation in the presence of much larger amounts of water, none of the mechanisms provide an explanation for the observations for very small amounts of water. In this work, density functional theory (DFT) was utilized to obtain a mechanistic understanding of the Li-O 2 discharge chemistry in a dimethoxyethane (DME) electrolyte containing an isolated water and no water. The reaction pathways for Li 2 O 2 formation from LiO 2 on a model system were carefully evaluated with different level of theories, i.e. PBE (PW), B3LYP/6-31G(2df,p), B3LYP/6-311++G(2df,p) and G4MP2. The results indicate that the LiO 2 disproportionation reaction to Li 2 O 2 can be promoted by the water in DME electrolyte, which explains why there is a significant difference compared to when no water is present in the experimentally observed discharge product distributions. Ab initio molecular dynamics calculations were also used to investigate the disproportionation of LiO 2 dimer in explicit DME. This work adds to the fundamental understanding of the discharge chemistry of a Li-O 2 battery.

25 ENERGY STORAGE↗
DOE OSTI2021

Supporting data for climatic clustering and longitudinal analysis with impacts on food, bioenergy, and pandemics

This data supports the conclusions found in climatic clustering and longitudinal analysis with impacts on food, bioenergy, and pandemics. Included here are (i) the binarized geolocation vectors used for exhaustive vector comparisons, (ii) the resulting climatic networks, (iii) the results of applying Markov clustering to the climatic networks, and (iv) the results of applying Correlation-of-Correlations (cor-cor) to the climatic networks. The set of binarized geolocation vectors that are used as inputs for the Combinatorial Metrics library (CoMet) are of the form comet-UUUUUxVVVVV-XXXX-YYYY.shuffled.tped where UUUUU is the number of vectors, VVVVV is the length of each vector, XXXX is the starting year, and YYYY is the ending year. Each line corresponds to a geolocation vector of binary elements A (i.e., 0) and T (i.e., 1). The set of climatic networks that are used for downstream network analysis are of the form network-U-way-XXXX-YYYY.parsed.txt where U is the order of the comparison (2-way or 3-way), XXXX is the starting year, and YYYY is the ending year. Each line corresponds to an edge linking two geolocations (defined by latitude and longitude) with its corresponding edge weight (i.e., DUO score). The set of cluster results are of the form clusters-U-way-XXXX-YYYY-thresh-VVVV-inflation-WWW.clustered.txt where U is the order of the comparison (2-way or 3-way), XXXX is the starting year, YYYY is the ending year, VVVV is the similarity threshold, and WWW is the Markov clustering inflation rate. Each line corresponds to a single cluster and is composed of a number of corresponding geolocations (defined by latitude and longitude). The set of cor-cor results are of the form corcor-U-way-XXXX-YYYY.cumulative.txt where U is the order of the comparison (2-way or 3-way), XXXX is the starting year, and YYYY is the ending year. Each line corresponds to a single geolocation with it's corresponding cor-cor value.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Parallel simulated annealing with embedded machine learning and multifidelity models for reactor core design

This paper presents extensions to a penalty-free, parallel simulated annealing (SA) algorithm for multi-constrained combinatorial optimization with the aim of embedding multi-fidelity physics models into the annealing procedure. The method uses a low-fidelity, quickly executing model for rapid design space exploration and a high-fidelity model for detailed constraint resolution and on-the-fly bias correction. Machine learning models updated within the annealing procedure were used to bridge the gap between the multi-fidelity models, which led to accurate rapid exploration and efficient detailed constraint resolution. A software implementation of the new multi-fidelity optimization methods, called ML-PSA, was demonstrated on a continuous multi-fidelity optimization problem and a constrained combinatorial PWR lattice design problem. These problems demonstrate some of the features, parallel performance characteristics, and extensible nature of the multi-fidelity SA methods. This paper shows that the developed software and procedure are a general optimization tool that can be applied to a wide variety of scientific and engineering design optimization applications. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

A warm super-Neptune around the G-dwarf star TOI-1710 revealed with TESS, SOPHIE, and HARPS-N

We report the discovery and characterization of the transiting extrasolar planet TOI-1710 b. It was first identified as a promising candidate by the Transiting Exoplanet Survey Satellite. Its planetary nature was then established with SOPHIE and HARPS-N spectroscopic observations via the radial-velocity method. The stellar parameters for the host star are derived from the spectra and a joint Markov chain Monte-Carlo adjustment of the spectral energy distribution and evolutionary tracks of TOI-1710. A joint MCMC analysis of the TESS light curve and the radial-velocity evolution allows us to determine the planetary system properties. From our analysis, TOI 1710 b is found to be a massive warm super-Neptune (M p = 28.3 ± 4.7 M ⊕ and R p = 5.34 ± 0.11 R ⊕ ) orbiting a G5V dwarf star (T eff = 5665 ± 55 K) on a nearly circular 24.3-day orbit (e = 0.16 ± 0.08). The orbital period of this planet is close to the estimated rotation period of its host star P rot = 22.5 ± 2.0 days and it has a low Keplerian semi-amplitude K = 6.4 ± 1.0 m s -1 ; we thus performed additional analyses to show the robustness of the retrieved planetary parameters. With a low bulk density of 1.03 ± 0.23 g cm -3 and orbiting a bright host star (J = 8.3, V = 9.6), TOI-1710 b is one of the best targets in this mass-radius range (near the Neptunian desert) for atmospheric characterization via transmission spectroscopy, a key measurement in constraining planet formation and evolutionary models of sub-Jovian planets.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Error field detection and correction studies towards ITER operation

In magnetic fusion devices, error field (EF) sources, spurious magnetic field perturbations, need to be identified and corrected for safe and stable (disruption-free) tokamak operation. Within Work Package Tokamak Exploitation RT04, a series of studies have been carried out to test the portability of the novel non-disruptive method, designed and tested in DIII-D (Paz-Soldan et al 2022 Nucl. Fusion62 126007), and to perform an assessment of model-based EF control strategies towards their applicability in ITER. In this paper, the lessons learned, the physical mechanism behind the magnetic island healing, which relies on enhanced viscous torque that acts against the static electro-magnetic torque, and the main control achievements are reported, together with the first design of the asynchronous EF correction current/density controller for ITER.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Closed-loop optimization of fast-charging protocols for batteries with machine learning

Simultaneously optimizing many design parameters in time-consuming experiments causes bottlenecks in a broad range of scientific and engineering disciplines. One such example is process and control optimization for lithium-ion batteries during materials selection, cell manufacturing and operation. A typical objective is to maximize battery lifetime; however, conducting even a single experiment to evaluate lifetime can take months to years. Furthermore, both large parameter spaces and high sampling variability necessitate a large number of experiments. As such, the key challenge is to reduce both the number and the duration of the experiments required. Here we develop and demonstrate a machine learning methodology to efficiently optimize a parameter space specifying the current and voltage profiles of six-step, ten-minute fast-charging protocols for maximizing battery cycle life, which can alleviate range anxiety for electric-vehicle users. We combine two key elements to reduce the optimization cost: an early-prediction model, which reduces the time per experiment by predicting the final cycle life using data from the first few cycles, and a Bayesian optimization algorithm, which reduces the number of experiments by balancing exploration and exploitation to efficiently probe the parameter space of charging protocols. Using this methodology, we rapidly identify high-cycle-life charging protocols among 224 candidates in 16 days (compared with over 500 days using exhaustive search without early prediction), and subsequently validate the accuracy and efficiency of our optimization approach. Our closed-loop methodology automatically incorporates feedback from past experiments to inform future decisions and can be generalized to other applications in battery design and, more broadly, other scientific domains that involve time-intensive experiments and multi-dimensional design spaces.

25 ENERGY STORAGE↗