Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Markov model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Trivializing Flow in 2D-O(3) model

The two dimensional O(3) sigma model, just as quantum chromodynamics, is an asymptotically free theory with a mass gap. Therefore, it is an interesting and simple toy model to investigate algorithms for Markov Chain Monte Carlo simulations of quantum chromodynamics. In this talk, we discuss the construction of a trivializing map, a field transformation from a given theory to a trivial one, through a suitably chosen gradient flow. An analytic solution for the generating functional of this trivializing flow has been obtained by a perturbative expansion in the flow time. Utilizing this solution allows for new approaches to be considered when proposing updates for a Markov Chain Monte Carlo algorithm.

Chamness, Christopher↗
DOE OSTI2021

Development of CTF modeling of interfacial drag, wall shear, and interfacial heat transfer for bubbly and annular-mist flow regimes

This milestone report discusses improvements to CTF closure models for modeling of bubbly and annular mist two-phase flow conditions, which are important for accurate modeling and simulation for boiling water reactor (BWR) conditions. Models that were improved include the two-phase wall shear, wall boiling, and phase mass and heat transfer. A new flow regime map was implemented to improve on the prediction of transition to annular flow, which was generally largely over-predicted by the legacy CTF flow regime map. The annular mist interfacial drag models were also reviewed and new models were proposed for implementation into the code. In addition to implementing new models, an activity was performed to calibrate these new models to experimental data using a Markov Chain Monte Carlo technique, thus further improving their accuracy. Finally, new experiments were implemented into the CTF validation matrix which add more two-phase pressure drop and void measurements for the assessment of CTF for modeling of BWRs. The results show that the Chisholm model, which was implemented and then calibrated, results in significantly improved comparison against the Riso two-phase pressure drop data. Furthermore, improvements in the implicit heat transfer and wall boiling models resulted in good agreement with the void measurement data.

42 ENGINEERING↗
DOE OSTI2022

Behavioral and Population Data-Driven Distribution System Load Modeling

Distribution system residential load modeling and analysis for different geographic areas within a utility or an independent system operator territory are critical for enabling small-scale, aggregated distributed energy resources to participate in grid services under Federal Energy Regulatory Commission Order No. 2222 [1]. In this study, we develop a methodology of modeling residential load profiles in different geographic areas with a focus on human behavior impact. First, we construct a behavior-based load profile model leveraging state-of-the-art appliance models. We simulate human activity and occupancy using Markov chain Monte Carlo methods calibrated with the American Time Use Survey data set. Second, we link our model with cleaned Current Population Survey data from the U.S. Census Bureau. Finally, we populate two sets of 500 households using California and Texas census data, respectively, to perform an initial analysis of the load in different geographic areas with various group features (e.g., different income levels). To distinguish the effect of population behavior differences on aggregated load, we simulate load profiles for both sets assuming fixed physical household parameters and weather data. Analysis shows that average daily load profiles vary significantly by income and income dependency varies by locality.

American Time Use Survey↗
DOE OSTI2025

Toward shell model interactions with credible uncertainties

Background: The nuclear shell model is a powerful framework for predicting nuclear structure observables, but relies on interaction matrix elements fit to experimental data as its inputs. Extending the shell model's applicability, particularly toward dripline nuclei, requires efficient fitting methods and credible uncertainty quantification. Traditional approaches face computational challenges and may underestimate uncertainties. Purpose: We develop and test a framework combining eigenvector continuation and Markov chain Monte Carlo to efficiently fit shell model interaction matrix elements and quantify their uncertainties. Methods: Eigenvector continuation is used to emulate shell model calculations, reducing computational costs. The emulator enables Markov chain Monte Carlo sampling to optimize interaction matrix elements and rigorously assess parametric uncertainties. Here, the framework is benchmarked using the USDB interaction in the 𝑠⁢𝑑 shell. Results: The emulator reproduces the USDB interaction with negligible error, validating its use in shell model fitting applications. However, we find that to obtain credible predictive intervals, the model defect of the shell model itself, rather than experimental or emulator error, must be taken into account in order to obtain credible uncertainties. Conclusions: The proposed framework provides an efficient and rigorous approach for fitting shell model interactions and quantifying uncertainties. Further, the normality assumption used in the past appears sufficient to describe the distribution of interaction matrix elements. However, it is crucial to account for model correlations to avoid underestimating uncertainties.

Nuclear forces↗
DOE OSTI2022

SDSS-IV MaStar: [α/Fe] for the MaNGA Stellar Library from synthetic model spectra

ABSTRACT We calculate the α-enhancement ratio [α/Fe] for the Mapping Nearby Galaxies at APO (MaNGA) Stellar Library (MaStar) while also fitting for the fundamental atmospheric parameters effective temperature, surface gravity, and metallicity – Teff, log g, [Fe/H]. This approach builds upon a previous catalogue of stellar parameters, whereby only the fundamental atmospheric parameters are fit with solar-scaled models. Here, we use the same Markov Chain Monte Carlo method with the additional free parameter [α/Fe]. Using the full spectral fitting code pPXF, we are able to fit multiple lines sensitive to [α/Fe] for a more robust measurement. Quality flags based on the convergence of the sampler, errors in [α/Fe] and a cut in the χ2 of the model fit are used to clean the final catalogue, returning 17 214 spectra and values in the range of −0.25 < [α/Fe] < 0.48. Comparing our calculated [α/Fe] with literature values reveals a degeneracy in cool stars with log g ≥ ∼4; this comparison is then used to create an alternative and calibrated parameter set. We also plot the final catalogue in an [Fe/H] versus [α/Fe] diagram and recover the expected result of increasing [α/Fe] with decreasing [Fe/H] for Milky Way disc-halo stars. We apply our method to a subsample of spectra of uniform resolution and higher signal to noise that finds that our results are independent of this higher signal to noise. In the context of stellar population models, we are able to cover a parameter space for the creation of intermediate to old age models at solar-scaled [α/Fe], high [Fe/H] and enhanced [α/Fe], low [Fe/H].

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Bayesian uncertainty quantification of tristructural isotropic particle fuel silver release: Decomposing model inadequacy plus experimental noise and parametric uncertainties

Tristructural isotropic (TRISO) particle fuel is one of the most promising fuel concepts enabling high temperature and high burnup reactor operation. One dominant source of radioactivity released from the TRISO particles is silver (Ag), which is subject to a high release fraction and long decay life compared to other fission products. Previous modeling efforts using the fuel performance code BISON indicated nonnegligible uncertainties in modeling the diffusion process of fission products in TRISO compared to the Advanced Gas Reactor experiments. The overall uncertainties observed when modeling the fission product diffusion can result from uncertainties in model parameters, noisy experimental measurements, and deficiencies in the developed models. The three types of underlying uncertainties have not yet been properly quantified in open literature. Here, this paper presents the Bayesian uncertainty quantification (UQ) using massively parallelizable Markov chain Monte Carlo samplers. The uncertainties due to model parameters, model inadequacy, and experimental measurement noise are quantified, with the σ term used to represent the sum of the model inadequacy and measurement noise uncertainties. It is worth noting that this is the first time the σ term is inferred for nuclear fuel experiments, as compared to using prescribed values for uncertainty quantification in previous work. The parallelizable Markov chain Monte Carlo samplers efficiently infer the model parameters and the σ term, giving insight into physical parameters like diffusion coefficients and the combined model discrepancy and measurement noise. A subsequent forward uncertainty quantification (UQ) is also performed based on the calibration results to generate more accurate predictions of the Ag release. The model inadequacy plus experimental noise is the most dominant source of uncertainty compared to the parametric uncertainty. All the UQ analyses presented in this work are based on the second series of the irradiation experiments in the Advanced Gas Reactor program.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

Robotic Planning under Uncertainty in Spatiotemporal Environments in Expeditionary Science

In the expeditionary sciences, spatiotemporally varying environments -- hydrothermal plumes, algal blooms, lava flows, or animal migrations -- are ubiquitous. Mobile robots are uniquely well-suited to study these dynamic, mesoscale natural environments. We formalize expeditionary science as a sequential decision-making problem, modeled using the language of partially-observable Markov decision processes (POMDPs). Solving the expeditionary science POMDP under real-world constraints requires efficient probabilistic modeling and decision-making in problems with complex dynamics and observational models. Previous work in informative path planning, adaptive sampling, and experimental design have shown compelling results, largely in static environments, using data-driven models and information-based rewards. However, these methodologies do not trivially extend to expeditionary science in spatiotemporal environments: they generally do not make use of scientific knowledge such as equations of state dynamics, they focus on information gathering as opposed to scientific task execution, and they make use of decision-making approaches that scale poorly to large, continuous problems with long planning horizons and real-time operational constraints. In this work, we discuss these and other challenges related to probabilistic modeling and decision-making in expeditionary science, and present some of our preliminary work that addresses these gaps. We ground our results in a real expeditionary science deployment of an autonomous underwater vehicle (AUV) in the deep ocean for hydrothermal vent discovery and characterization. Our concluding thoughts highlight remaining work to be done, and the challenges that merit consideration by the reinforcement learning and decision-making community.

Preston, Victoria↗
DOE OSTI2020

Identification of hot water end-use process of electric water heaters from energy measurements

This paper presents an algorithm for the identification of parameters for a stochastic hot water end-use process that drives a homogeneous population of thermostatically controlled electric water heaters (EWH). Usually, only metered interval consumption data (kWh) is collected and the hot water end-use process is unobservable to utility and aggregators. However, the availability of EWHs for demand response (DR) is closely coupled with the hot water end-use process. In this context, the hot water end-use process is modeled as a two-state Markov chain (Use / No use), which causes the thermostatic ON-OFF switching process to behave as a Markov renewal process (MRP). A set of first passage-time problems is developed to obtain the moments of the transition probability densities of the MRP. These problems are addressed by establishing a system of coupled partial differential equations characterizing the temperature evolution of the EWH population. A key quantity in the methodology for estimating the parameters is the total time an EWH is ON within a period of interest. It is referred to as the total busy time. Total busy time in this approach is a random variable for which analytical expressions of the moments are developed as a function of the metered window length. The latter expressions become the basis of a hot water demand model identification algorithm which is validated using agent-based simulations of EWHs.

42 ENGINEERING↗
DOE OSTI2022

SDSS-IV MaStar: Stellar parameter determination with continuum-supplemented full-spectrum fitting

We present a stellar parameter catalog built to accompany the MaStar Stellar Library, which is a comprehensive collection of empirical, medium-resolution stellar spectra. We constructed this parameter catalog by using a multicomponent χ 2 fitting approach to match MaStar spectra to models generated by interpolating the ATLAS9-based BOSZ model spectra. The total χ 2 for a given model is defined as the sum of components constructed to characterize narrow-band features of observed spectra (e.g., absorption lines) and the broadband continuum shape separately. Extinction and systematics due to flux calibration were taken into account in the fitting. The χ 2 distribution for a given region of model space was sampled using a Markov chain Monte Carlo (MCMC) algorithm, the data from which were then used to extract atmospheric parameter estimates (T eff , log g, [Fe/H], and [α/Fe]), their corresponding uncertainties, and direct extinction measurements. Two methods were used to extract parameters and uncertainties: one that accepts the MCMC’s prescribed minimum-χ 2 result, and one that uses Bayesian inference to compute a likelihood-weighted mean from the χ 2 distribution sampled by the MCMC. Results were evaluated for internal consistency using repeat observations where available and by comparing them with external data sets (e.g., APOGEE-2 and Gaia DR2). Our spectral-fitting exercise reveals possible deficiencies in current theoretical model spectra, illustrating the potential power of MaStar spectra for helping to improve the models. This paper represents an update to the parameters that were originally presented with SDSS-IV DR17. The MaStar parameter catalog containing our BestFit results is available on the SDSS-IV DR17 website as part of version 2 of the MaStar stellar parameter value-added catalog.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Differentiable stochastic halo occupation distribution

ABSTRACT In this work, we demonstrate how differentiable stochastic sampling techniques developed in the context of deep reinforcement learning can be used to perform efficient parameter inference over stochastic, simulation-based, forward models. As a particular example, we focus on the problem of estimating parameters of halo occupation distribution (HOD) models that are used to connect galaxies with their dark matter haloes. Using a combination of continuous relaxation and gradient re-parametrization techniques, we can obtain well-defined gradients with respect to HOD parameters through discrete galaxy catalogue realizations. Having access to these gradients allows us to leverage efficient sampling schemes, such as Hamiltonian Monte Carlo, and greatly speed up parameter inference. We demonstrate our technique on a mock galaxy catalogue generated from the Bolshoi simulation using a standard HOD model and find near-identical posteriors as standard Markov chain Monte Carlo techniques with an increase of ∼8× in convergence efficiency. Our differentiable HOD model also has broad applications in full forward model approaches to cosmic structure and cosmological analysis.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Variations of the physical parameters of the blazar Mrk 421 based on analysis of the spectral energy distributions

Abstract We report on the variations of the physical parameters of the jet observed in the blazar Mrk 421, and discuss the origin of X-ray flares in the jet, based on analysis of several spectral energy distributions (SEDs). The SEDs are modeled using the one-zone synchrotron self-Compton model, its parameters determined using a Markov chain Monte Carlo method. The lack of data at TeV energies means many of the parameters cannot be uniquely determined and are correlated. These are studied in detail. We find that the optimal solution can be uniquely determined only when we apply a constraint to one of four parameters: the magnetic field (B), the Doppler factor, the size of the emitting region, and the normalization factor of the electron energy distribution. We used 31 sets of SEDs from 2009 to 2014 with optical–UV data observed with UVOT/Swift and the Kanata telescope, X-ray data with XRT/Swift, and γ-ray data with the Fermi Large Area Telescope. The result of our SED analysis suggests that, in the X-ray faint state, the emission occurs in a relatively small area (∼1016 cm) with a relatively strong magnetic field (B ∼ 10−1 G). The X-ray bright state shows a tendency opposite to that of the faint state, that is, a large emitting area (∼1018 cm), probably downstream of the jet, and a weak magnetic field (B ∼ 10−3 G). The high X-ray flux was due to an increase in the maximum energy of electrons. On the other hand, the presence of two kinds of emitting areas implies that the one-zone model is unsuitable for reproducing at least part of the observed SEDs.

Yamada, Yurika↗
DOE OSTI2023

Reinforcement learning for real-time process control in high-temperature superconductor manufacturing

With high efficiency and low energy loss, high-temperature superconductors (HTS) have demonstrated their profound applications in various fields, such as medical imaging, transportation, accelerators, microwave devices, and power systems. The high-field applications of HTS tapes have raised the demand for producing cost-effective tapes with long lengths in superconductor manufacturing. However, achieving the uniform and enhanced performance of a long HTS tape is challenging due to the unstable growth conditions in the manufacturing process. Although it is confirmed that the process parameters during the advanced metal organic chemical vapor deposition (A-MOCVD) process influence the uniformity of the produced HTS tapes, the high-dimensional process parameter signals and their complicated interactions make it difficult to develop an effective control policy. In this paper, we propose a local measure for the uniformity of HTS tapes to provide instant feedback for our control policy. Then, we model the manufacturing of HTS tapes as a Markov decision process (MDP) with continuous state and action spaces to assess the instant reward in real time in our feedback control model. As our MDP involves continuous and high-dimensional state and action spaces, a neural fitted Q-iteration (NFQ) algorithm is adopted to solve the MDP with artificial neural network (ANN) function approximation. The collinearity of process parameters can restrict our capability of adjusting the process parameters, which is addressed by the principal component analysis (PCA) in our method. The control policy adjusts the PCA of process parameters using the NFQ algorithm. In conclusion, based on our case studies on real A-MOCVD dataset, the obtained control policy increases the average uniformity of tapes by 5.6% and performs especially well on sample HTS tapes with a low uniformity.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2024

Off-Lattice Markov Chain Monte Carlo Simulations of Mechanically Driven Polymers

Here, we develop off-lattice simulations of semiflexible polymer chains subjected to applied mechanical forces by using Markov Chain Monte Carlo. Our approach models the polymer as a chain of fixed length bonds, with configurations updated through adaptive nonlocal Monte Carlo moves. This proposed method enables precise calculation of a polymer’s response to a wide range of mechanical forces, which traditional on-lattice models cannot achieve. Our approach has shown excellent agreement with theoretical predictions of persistence length and end-to-end distance in quiescent states as well as stretching distances under tension. Moreover, our model eliminates the orientational bias present in on-lattice models, which significantly impacts calculations such as the scattering function, a crucial technique for revealing the polymer conformation.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

An Automatic Learning Framework for Smart Residential Communities

Predictability has been foundational to matching supply and demand in the day-to-day operation of the electric power system. Demand predictability is eroding because of the increased use of renewable energy resources and more sophisticated loads, such as electric vehicles and smart appliances. In this paper, an automatic software framework is described which can be used for load forecasting in smart communities. A time-varying clustering-based Markov chain approach is used to predict the energy consumption of residential buildings in a smart community. The training data is based on 1-minute meter data of occupied homes over one month. The data points are first clustered based on the energy consumption and the time of the day. Then, the original data is converted using the Centroids of the clusters. A time-varying Markov chain is subsequently trained to model the energy consumption behavior of residents for each home using the transformed data. The trained model is shown to successfully predict load in 5-minute intervals over a 24 hours period.

Zandi, Helia↗
DOE OSTI2024

GeoThermalCloud: Cloud Fusion of Big Data and Multi-Physics Models using Machine Learning for Discovery, Exploration, and Development of Hidden Geothermal Resources

The primary goals of this project are exploring hidden geothermal resources in the U.S.A. and designing profitable enhanced geothermal systems (EGS). Many processes and parameters control geothermal exploration and energy production from geothermal fields. Diverse datasets (e.g., geology, geochemistry, geophysics, satellite, airborne geophysics) are available to help characterize subsurface geothermal conditions. Sparse and multi-scale characteristics of these datasets prohibit properly leveraging these datasets for geothermal exploration and profitable EGS design. Recent advancements in machine learning (ML) promise to resolve these issues. The tremendous challenges and risks of geothermal exploration and production bring the demand for novel ML methods and tools that can (1) analyze large field datasets, (2) assimilate model simulations (large inputs and outputs), (3) process sparse datasets, (4) perform transfer learning (between sites with different exploratory levels), (5) extract hidden geothermal signatures in the field and simulation data, (6) label geothermal resources and processes, (7) identify high-value data acquisition targets, and (8) guide geothermal exploration and production by selecting optimal exploration, production, and drilling strategies. To address these necessities, ML-based geothermal resources exploration and enhanced geothermal systems (EGS) design tools have been developed. The exploration tool is called GeoThermalCloud and EGS design tool is called GeoDT-ML. GeoThermalCloud (https://github.com/SmartTensors/GeoThermalCloud.jl) utilizes a LANL unsupervised ML platform called SmartTensors (https://tensors.lanl.gov/) to automate data analyses and interpretations by extracting hidden signatures to identify geothermal prospects. Also, it enables the identification of critical measurements needed to identify geothermal resource signatures. Alternatively, GeoDT-ML (https://github.com/SmartTensors/GeoThermalCloud.jl/tree/master/EGS) is an ML-based alternative to GeoDT (https://github.com/GeoDesignTool/GeoDT.git), a fast, simplified multi-physics solver to evaluate EGS project designs in uncertain geologic systems. GeoDT-ML leverages recent advances in deep learning and high-performance computing. It is a faster and simpler version of GeoDT. To make this project a success, we used capabilities of LANL, PNNL, Google, Stanford, and Julia Computing. We analyzed eight datasets of the U.S.A. using GeothermalCloud and demonstrated potential highly prospective geothermal resources and identified key factors defining highly prospective sites. The first data set includes 44 locations in southwest New Mexico and 18 geological, hydrogeological, geophysical, geothermal, geochemical attributes. We defined low- and medium-temperature hydrothermal systems and discovered a new highly prospective site. The second data set analyzed 18 shallow water chemistry attributes at 14,342 locations in the Great Basin. It demarcated modestly, moderately, and highly prospective sites including key attributes for each type of prospectivity. The third data set analyzed Utah FORGE data including satellite (InSAR), geophysical (gravity, seismic), geochemical, and geothermal attributes. Here, we performed prospectivity analysis to identify future drilling locations using geological, geochemical, and geophysical attributes. Maps of temperature at depth and heat flow are constructed based on the available data. Prospectivity maps were generated, and drilling locations were proposed for future geothermal field exploration. The fourth data set analyzed 21 attributes at 120 locations in Tularosa Basin, New Mexico; data comes from past play fairway analyses in this region. ML analyses identified geothermal signatures associated with modestly, moderately, and highly hydrothermal systems. We also defined dominant attributes and spatial distribution of the geothermal signatures. The fifth, sixth, seventh, and eighth datasets include Tohatchi Springs, New Mexico, Hawaii, Brady site, Nevada, and EGS Collab, respectively. Moreover, we coupled GeothermalCloud and magnetotellurics data to pinpoint drilling locations for developing geothermal projects in the Tularosa Basin, New Mexico. GeothermalCloud found potential prospective locations for geothermal resources near White Sands Missile Range and McGregor Range at Fort Bliss. Magnetotellurics data determined the potential depth (~1800m) of geothermal prospects at McGregor Range based on apparent resistivity structures/layers in the subsurface. The McGregor Range consists of three resistivity layers and two resistivity structures. Magnetotellurics data also helps identify that the western portion of the McGregor Range has thick and low-resistivity earth materials. The low resistivity to the west is most likely for a fault system. Assuming temperature is consistent with a geothermal reservoir, the west-central part of the McGregor Range has the highest geothermal potential because of the increase in porosity and associated permeability attributed to the interpreted fault system. Also, we devised a coupling strategy between a process model and GeothermalCloud to characterize hydrogeological conditions and geothermal conditions, respectively. The process model characterizes hydrogeological and geothermal conditions on highly prospective geothermal sites provided by GeothermalCloud. We developed a physics-informed neural network (PINN) version of the Burns equation that can be easily coupled with GeothermalCloud. Furthermore, we performed an optimal design decision maximizing the economic value of an EGS power plant. This study optimized the range of well spacing between injection and production wells maximizing net present value in dollars (NPV). For this task, we used the GeoDT to simulate the Utah FORGE EGS development cycle from the initial well design to the end of production. Next, we accomplished another crucial task, which is predicting permeability of geothermal reservoirs. Predicting permeability of geothermal reservoirs is a non-trivial task because of huge computational runtime of simulation and lack of measurements. To avoid these limitations, we used easy-to-measure chemical concentrations in the subsurface as measurement data and convolutional neural network based ML model of a high-fidelity model. Next, we predicted permeability using Markov chain Monte Carlo simulation. We found that Markov chain Monte Carlo simulation predicts permeability with a high certainty if the prediction zone in the simulation area has chemical concentration data. Finally, we analyzed the DOE funded INGENIOUS and GeoDAWN projects data. For discovering hidden geothermal systems in the Great Basin, the INGENIOUS project accumulated old data, collected new data, and released them in 2022. The dataset includes a total of 24 geological, geophysical, and geochemical attributes. Data resolution and scale significantly vary prohibiting an appropriate usage. To avoid such limitations, we brought all data in the same resolution and scale by applying the inverse distance weighting interpolation technique for predicting data in unsampled locations. Subsequently, we analyzed LiDAR data of the GeoDAWN project. We received data in tiles format. The DOE’s overarching goal is to use ML on LiDAR data for finding favorable geological structures (e.g., step up faults in Brady, Nevada). To serve the purpose, we need to label favorable geologic structures that correspond to LiDAR data. We wrote an algorithm to label the LiDAR data with the favorable geologic structures.

15 GEOTHERMAL ENERGY↗
DOE OSTI2023

A Novel Multi-Agent Deep Reinforcement Learning-enabled Distributed Power Allocation Scheme for mmWave Cellular Networks

We consider the power allocation problem over shared spectrum for millimeter-Wave (mmWave) cellular downlink. Existing approaches usually find sub-optimal solutions by solving a non-convex optimization which leads to scalability issues due to centralized control. Therefore, distributed and adaptive approaches are desirable. Recently, model-free Deep Reinforcement Learning (DRL) has achieved success in such wireless resource management tasks. By modeling the radio environment as a Markov Decision Process (MDP) with the base stations (BSs) being the agents, power allocation can be automated at the agent level with comparable throughput performance to conventional centralized schemes. The multi-agent setting presents new challenges as the radio environment is impacted by the joint actions of the agents and is no longer stationary from any individual agent’s perspective. Existing literature bypasses this non-stationarity violation by ignoring it which may cause performance degradation. To tackle this issue, we propose a distributed continuous power allocation scheme based on a modified version of multi-agent Deep Deterministic Policy Gradient (MADDPG) that is tailored for the distributed multiple-agent setting. The proposed scheme employs a centralized-training distributed-execution framework where Q-functions are trained over subsets of BSs while each BS determines its transmit power based only on its own local observation. It admits constant per-BS communication and computation complexity and is thus scalable to large networks. Numerical evaluation shows that the proposed scheme adapts well to a wide range of interference conditions and can achieve comparable or better performance than several state-of-the-art non-learning approaches.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2021

A Bayesian Calibration Framework with Embedded Model Error for Model Diagnostics

We study the utility and performance of a Bayesian model error embedding construction in the context of molecular dynamics modeling of metallic alloys, where we embed model error terms in existing interatomic potential model parameters. To alleviate the computational burden of this approach, we propose a framework combining likelihood approximation and Gaussian process surrogates. Here we leverage sparse Gaussian process techniques to construct a hierarchy of increasingly accurate but more expensive surrogate models. This hierarchy is then exploited by multilevel Markov chain Monte Carlo methods to efficiently sample from the target posterior distribution. We illustrate the utility of this approach by calibrating an interatomic potential model for a family of gold-copper alloys. In particular, this case study highlights effective means for dealing with computational challenges with Bayesian model error embedding in large-scale physical models, and the utility of embedded model error for model diagnostics.

Bayesian inference↗
DOE OSTI2025

Emulator-based Bayesian calibration of a subglacial drainage model

Subglacial drainage models, often motivated by the relationship between hydrology and ice flow, sensitively depend on numerous unconstrained parameters. We explore using borehole water-pressure time series to calibrate the uncertain parameters of a popular subglacial drainage model, taking a Bayesian perspective to quantify the uncertainty in parameter estimates and in the calibrated model predictions. To reduce the computation time associated with Markov Chain Monte Carlo sampling, we construct a fast Gaussian process emulator to stand in for the subglacial drainage model. We first carry out a calibration experiment using synthetic observations consisting of model simulations with hidden parameter values as a demonstration of the method. Using real borehole water pressures measured in western Greenland, we find meaningful constraints on four of the eight model parameters and a factor-of-three reduction in uncertainty of the calibrated model predictions. These experiments illustrate Gaussian process-based Bayesian inference as a useful tool for calibration and uncertainty quantification of complex glaciological models using field data. However, significant differences between the calibrated model and the borehole data suggest that structural limitations of the model, rather than poorly constrained parameters or computational cost, remain the most important constraint on subglacial drainage modelling.

58 GEOSCIENCES↗