Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “dynamic programming”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Task allocation in geo-distributed cyber-physical systems - 256

This paper studies the task allocation algorithm for a distributed test facility (DTF), which aims to assemble geo-distributed cyber (software) and physical (hardware in the loop) components into a prototype cyber-physical system (CPS). This allows low cost testing on an early conceptual prototype (ECP) of the ultimate CPS (UCPS) to be developed. The DTF provides an instrumentation interface for carrying out reliability experiments remotely such as fault propagation analysis and in-situ testing of hardware and software components in a simulated environment. Unfortunately, the geo-distribution introduces an overhead that is not inherent to the UCPS, i.e. a significant time delay in communication that threatens the stability of the ECP and is not an appropriate representation of the behavior of the UCPS. This can be mitigated by implementing a task allocation algorithm to find a suitable configuration and assign the software components to appropriate computational locations, dynamically. This would allow the ECP to operate more efficiently with less probability of being unstable due to the delays introduced by geo-distribution. The task allocation algorithm proposed in this work uses a Monte Carlo approach along with Dynamic Programming to identify the optimal network configuration to keep the time delays to a minimum. (authors)

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Decision Analyses Related to Groundwater Remediation - 17051

Traditional decision analyses of groundwater remediation scenarios frequently fail because the probability of adverse, unanticipated events occurring is often high. This is driven by many factors including that (1) the models developed to represent the flow and transport in contaminated aquifers are typically simpler than reality and do not account for all the important governing processes, (2) probability distributions are assigned to critical performance parameters (such as model inputs or characteristics of the engineered remediation systems) even though some of these parameters might be unknown or not be very well constrained. However, there is a novel decision analysis methodology and a computational tool based on Bayesian Information-Gap Decision Theory (BIG-DT) that are designed to mitigate the shortcomings of the models and probabilistic decision analyses by leveraging a non-probabilistic decision theory -- information-gap decision theory. The new decision analysis methodology considers possible models that have not been explicitly enumerated and does not require us to commit to a particular probability distribution for model and remediation-design parameters. Both the set of possible models and the set of possible probability distributions grow as the degree of uncertainty increases. The fundamental question that BIG-DT asks is 'How large can these sets be before a particular decision results in an undesirable outcome?'. The decision that allows these sets to be the largest is considered to be the best option. In this way, BIG-DT enables robust decision-support for groundwater remediation problems. Here, we apply BIG-DT to a representative groundwater remediation scenario where different options for hydraulic containment and pump and treat are being considered. BIG-DT requires many model runs and high-performance computing resources are needed when complex models are used. We demonstrate the BIG-DT analyses on a series of synthetic problems that are designed to be consistent with real-world problems such as Los Alamos National Laboratory (LANL) contamination sites. We also discuss a general framework for how the BIG-DT analyses will be carried out at the chromium groundwater contamination site at LANL. BIG-DT is implemented in Julia (a high-level, high-performance dynamic programming language for technical computing) and is part of the MADS framework (http://mads.lanl.gov and http://madsjulia.github.io/Mads.jl). (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2018

Dynamic Investment and Counterparty Risk

We introduce a dynamic optimization framework in which collateral is used to mitigate losses arising at counterparty’s default. The investor faces two sources of risk: the default risk of the entity referencing the traded credit swap security, and counterparty risk generated from the default event of the trading counterparty. We show that the value function of the control problem coincides with the classical solution of a nonlinear dynamic programming equation. We provide an explicit characterization of the optimal investment strategy, and show that the investor does not trade if counterparty risk is sufficiently high. These findings suggest that moving credit swap trades into well-designed clearinghouses may stimulate economic activities.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

DP-Based Optimization of BESS to Substitute RICE Reserves for Improved Economic Benefits

In regions with a significant proportion of renewable energy sources, reciprocating internal combustion engines (RICE or IC) are frequently employed as grid reserve solutions. This study suggests a scheduling technique for battery energy storage systems (BESS) that utilizes dynamic programming (DP) to replace IC units, adding versatility to the total reserve capacity and resulting in higher BESS earnings compared to traditional scheduling approaches. The capacity sweeping approach to optimize BESS’s revenue sometimes leads to operation which is counter intuitive to conventional battery sizing approaches.

25 ENERGY STORAGE↗
DOE OSTI2017

Data-driven optimal charging decision making for connected and automated electric vehicles: A personal usage scenario

This study introduces an optimal charging decision making framework for connected and automated electric vehicles under a personal usage scenario. This framework aims to provide charging strategies, i.e. the choice of charging station and the amount of charged energy, by considering constraints from personal daily itineraries and existing charging infrastructure. A data-driven method is introduced to establish a stochastic energy consumption prediction model with consideration of realistic uncertainties. This is performed by analyzing a large scale electric vehicle data set. A real-time updating method is designed to construct this prediction model from new consecutive data points in an adaptive way for real-world applications. Based on this energy cost prediction framework from real electric vehicle data, multistage optimal charging decision making models are introduced, including a deterministic model for average outcome decision making and a robust model for safest charging strategies. A dynamic programming algorithm is proposed to find the optimal charging strategies. Detailed simulations and case studies demonstrate the performance of the proposed algorithms to find optimal charging strategies. They also show the potential capability of connected and automated electric vehicles to reduce the range anxiety and charging infrastructure dependency.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Incorporating Valve Switching Losses Into a Static Optimal Control Algorithm for the Hybrid Hydraulic-Electric Architecture (HHEA)

The Hybrid Hydraulic-Electric Architecture (HHEA) has, in recent years, been proposed as an energy efficient alternative to conventional load-sensing architectures in mobile machines such as excavators and wheel-loaders. HHEA leverages the advantages of hydraulic power and electric power to eliminate throttling valves while also improving the energy and control performance of the system. The architecture utilizes a set of common pressure rails to provide a majority of power and and a small electric motor driven pump to modulate this power to meet the exact demand. Previous work has developed a computationally efficient Lagrange Multiplier approach for determining the optimal pressure rail selections that minimizes the energy losses in the system. The static model used considers only the energy use for each pressure rail selection but not the losses associated with the valves during the transition. This paper presents an approach to include the switching losses in the model and in the optimization procedure. To capture the switching losses, switching events between different rails and at various input and output flow rates were simulated with consideration of valve spool dynamics. A parameterized model that summarizes the losses is then obtained, allowing switching losses to be added to the previous energy analysis. The performance of the switching loss model was compared with reference data obtained from a high-fidelity simulation model. To incorporate the switching losses into optimal control algorithm, an efficient dynamic programming approach that prevents frequent switching is adopted in place of the Lagrange multiplier approach. The overall effect of switching losses on the energy consumption and optimal control decisions is presented. In general, switching losses contribute to about 9–10% of input energy.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Data-based and secure switched cyber–physical systems

In this work, we develop a completely model-free moving target defense framework for the detection and mitigation of sensor and/or actuator attacks in cyber–physical systems with dynamics that evolve in discrete-time. We incorporate an intrusion detection mechanism based on an approximate dynamic programming technique that learns the policies for optimal regulation and optimal tracking while simultaneously defending against actuator and sensor attacks in a model-free fashion. Switching rules are leveraged to force proactive and reactive defense mechanisms as well as, guarantee the stability of the equilibrium point. Finally, as a case study, we apply the proposed moving target defense framework to a DC–DC converter that is used in electric vehicles.

42 ENGINEERING↗
DOE OSTI2015

A Characterization of the Reflected Quasipotential

Recent interest in the reflected quasipotential comes from the queueing theory literature, specifically the analysis of so-called (b,A,D) reflected Brownian motion where it is the large deviation rate function for the stationary distribution. Our purpose here is to characterize the reflected quasipotential in terms of a first-order Hamilton–Jacobi equation. Using conventional dynamic programming ideas, along with a complementarity problem formulation of the effect of the Skorokhod map on absolutely continuous paths, we will derive necessary conditions in the form of viscosity-sense boundary conditions. It turns out that even with these boundary conditions solutions are not unique. Thus a unique characterization needs to refer to some additional property of V(⋅). We establish such a characterization in two dimensions.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2019

Machine Learning-Based Anomaly Detection for Load Forecasting Under Cyberattacks

Accurate load forecasting can create both economic and reliability benefits for power system operators. However, the cyberattack on load forecasting may mislead operators to make unsuitable operational decisions for the electricity delivery. To effectively and accurately detect these cyberattacks, this paper develops a machine learning-based anomaly detection (MLAD) methodology. First, load forecasts provided by neural networks are used to reconstruct the benchmark and scaling data by using the k-means clustering. Second, the cyberattack template is estimated by the naive Bayes classification based on the cumulative distribution function and statistical features of the scaling data. Finally, the dynamic programming is utilized to calculate both the occurrence and parameter of one cyberattack on load forecasting data. Here, a widely used symbolic aggregation approximation method is compared with the developed MLAD method. Numerical simulations on the publicly load data show that the MLAD method can effectively detect cyberattacks for load forecasting data with relatively high accuracy. Also, the robustness of MLAD is verified by thousands of attack scenarios based on Monte Carlo simulation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Optimal economic dispatch policy for prosumer with energy storage considering self-consumption demand

This paper analyzed the effects of self-consumption demand on the joint economic dispatch of prosumers (energy consumers who are also producers), particularly for prosumers with both energy storage and distributed energy sources (DERs). Studies in the existing literature on the economic dispatch scheduling policy of energy storage, mostly from the perspective of electricity merchants, do not address the impacts of self-consumption demand. However, due to the intermittent and high levels of uncertainty regarding DERs generation and the dynamic demand of the prosumer, production and consumption are not always simultaneous; there are two possible scenarios in each period depending on whether DERs generation can meet prosumers' self-consumption or not. Incorporating the self-consumption demand will pose modeling challenges since these two scenarios cannot occur simultaneously in each period, and different scenarios require different decisions for prosumers. Further, this paper analyzed the two scenarios separately to find the optimal storage scheduling strategy, and the results were combined to get the optimal global solution. We focused on prosumers' economic decision-making while considering self-consumption demand and the physical constraints of a battery based on dynamic programming. Our study showed that the feasible state of charge (SOC) range of storage can be segmented into several sub-ranges by SOC reference points under the above two scenarios. As a result, a prosumer's optimal scheduling can be uniquely and conveniently selected based on the sub-ranges within which the current SOC falls. The results, therefore, provided multistage decision-making guidance for prosumers with energy storage.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2016

A Piecewise Deterministic Markov Toy Model for Traffic/Maintenance and Associated Hamilton–Jacobi Integrodifferential Systems on Networks

We study optimal control problems in infinite horizon whxen the dynamics belong to a specific class of piecewise deterministic Markov processes constrained to star-shaped networks (corresponding to a toy traffic model). We adapt the results in Soner (SIAM J Control Optim 24(6):1110–1122, 1986) to prove the regularity of the value function and the dynamic programming principle. Extending the networks and Krylov’s “shaking the coefficients” method, we prove that the value function can be seen as the solution to a linearized optimization problem set on a convenient set of probability measures. The approach relies entirely on viscosity arguments. As a by-product, the dual formulation guarantees that the value function is the pointwise supremum over regular subsolutions of the associated Hamilton–Jacobi integrodifferential system. This ensures that the value function satisfies Perron’s preconization for the (unique) candidate to viscosity solution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Optimal checkpointing for adjoint multistage time-stepping schemes

Here, we consider checkpointing strategies that minimize the number of recomputations needed when performing discrete adjoint computations using multistage time-stepping schemes that require computing several substeps within one complete time step. Specifically, we propose two algorithms that can generate optimal checkpoint-ing schedules under weak assumptions. The first is an extension of the seminal Revolve algorithm adapted to multistage schemes. The second algorithm, named CAMS, is developed based on dynamic programming, and it requires the least number of recomputations when compared with other algorithms. The CAMS algorithm is made publicly available in a library with bindings to C and Python. Numerical results show that the proposed algorithms can deliver up to two times the speedup compared with that of classical Revolve. Moreover, we discuss the utilization of the CAMS library in mature scientific computing libraries and demonstrate the ease of using it in an adjoint workflow. The proposed algorithms have been adopted by the PETSc TSAdjoint library. Their performance has been demonstrated with a large-scale PDE-constrained optimization problem on a leadership-class supercomputer. This work is a significant extension of the authors' conference paper.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Data-driven reinforcement learning–based real-time energy management system for plug-in hybrid electric vehicles

Plug-in hybrid electric vehicles (PHEVs) show great promise in reducing transportation-related fossil fuel consumption and greenhouse gas emissions. Designing an efficient energy management system (EMS) for PHEVs to achieve better fuel economy has been an active research topic for decades. Most of the advanced systems rely either on a priori knowledge of future driving conditions to achieve the optimal but not real-time solution (e.g., using a dynamic programming strategy) or on only current driving situations to achieve a real-time but nonoptimal solution (e.g., rule-based strategy). This paper proposes a reinforcement learning–based real-time EMS for PHEVs to address the trade-off between real-time performance and optimal energy savings. The proposed model can optimize the power-split control in real time while learning the optimal decisions from historical driving cycles. Here, a case study on a real-world commute trip shows that about a 12% fuel saving can be achieved without considering charging opportunities; further, an 8% fuel saving can be achieved when charging opportunities are considered, compared with the standard binary mode control strategy.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2017

Wind-Friendly Flexible Ramping Product Design in Multi-Timescale Power System Operations

With increasing wind power penetration in the electricity grid, system operators are recognizing the need for additional flexibility, and some are implementing new ramping products as a type of ancillary service. However, wind is generally thought of as causing the need for ramping services, not as being a potential source for the service. In this paper, a multi-timescale unit commitment and economic dispatch model is developed to consider the wind power ramping product (WPRP). An optimized swinging door algorithm with dynamic programming is applied to identify and forecast wind power ramps (WPRs). Designed as positive characteristics of WPRs, the WPRP is then integrated into the multi-timescale dispatch model that considers new objective functions, ramping capacity limits, active power limits, and flexible ramping requirements. Numerical simulations on the modified IEEE 118-bus system show the potential effectiveness of WPRP in increasing the economic efficiency of power system operations with high levels of wind power penetration. It is found that WPRP not only reduces the production cost by using less ramping reserves scheduled by conventional generators, but also possibly enhances the reliability of power system operations. Moreover, wind power forecasts play an important role in providing high-quality WPRP service.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2018

Hedging Strategies: Electricity Investment Decisions under Policy Uncertainty

Given uncertainty in long-term carbon reduction goals, how much non-carbon generation should be developed in the near-term? This research investigates the optimal balance between the risk of overinvesting in non-carbon sources that are ultimately not needed and the risk of underinvesting in non-carbon sources and subsequently needing to reduce carbon emissions dramatically. We employ a novel framework that incorporates a computable general equilibrium (CGE) model of the U.S. into a two-stage stochastic approximate dynamic program (ADP) focused on decisions in the electric power sector. We solve the model using an ADP algorithm that is computationally tractable while exploring the decisions and sampling the uncertain carbon limits from continuous distributions. The results of the model demonstrate that an optimal hedge is in the direction of more non-carbon investment in the near-term, in the range of 20–30% of new generation. We also demonstrate that the optimal share of non-carbon generation is increasing in the variance of the uncertainty about the long-term carbon targets, and that with greater uncertainty in the future policy regime, a balanced portfolio of non-carbon, natural gas, and coal generation is desirable.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Development and Evaluation of Velocity Predictive Optimal Energy Management Strategies in Intelligent and Connected Hybrid Electric Vehicles

In this study, a thorough and definitive evaluation of Predictive Optimal Energy Management Strategy (POEMS) applications in connected vehicles using 10 to 20 s predicted velocity is conducted for a Hybrid Electric Vehicle (HEV). The presented methodology includes synchronous datasets gathered in Fort Collins, Colorado using a test vehicle equipped with sensors to measure ego vehicle position and motion and that of surrounding objects as well as receive Infrastructure to Vehicle (I2V) information. These datasets are utilized to compare the effect of different signal categories on prediction fidelity for different prediction horizons within a POEMS framework. Multiple artificial intelligence (AI) and machine learning (ML) algorithms use the collected data to output future vehicle velocity prediction models. The effects of different combinations of signals and different models on prediction fidelity in various prediction windows are explored. All of these combinations are ultimately addressed where the rubber meets the road: fuel economy (FE) enabled from POEMS. FE optimization is performed using Model Predictive Control (MPC) with a Dynamic Programming (DP) optimizer. FE improvements from MPC control at various prediction time horizons are compared to that of full-cycle DP. All FE results are determined using high-fidelity simulations of an Autonomie 2010 Toyota Prius model. The full-cycle DP POEMS provides the theoretical upper limit on fuel economy (FE) improvement achievable with POEMS but is not currently practical for real-world implementation. Perfect prediction MPC (PP-MPC) represents the upper limit of FE improvement practically achievable with POEMS. Real-Prediction MPC (RP-MPC) can provide nearly equivalent FE improvement when used with high-fidelity predictions. Constant-Velocity MPC (CV-MPC) uses a constant speed prediction and serves as a “null” POEMS. Results showed that RP-MPC, enabled by high-fidelity ego future speed prediction, led to significant FE improvement over baseline nearly matching that of PP-MPC.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2016

B-Trees and Cache-Oblivious B-Trees with Different-Sized Atomic Keys

Most B-tree articles assume that all $\textit{N}$ keys have the same size $\textit{K}$, that $\textit{f = B/K}$ keys fit in a disk block, and therefore that the search cost is $O(\text{log}_{f+1}N) $ block transfers. When keys have variable size, B-tree operations have no nontrivial performance guarantees, however. This article provides B-tree-like performance guarantees on dictionaries that contain keys of different sizes in a model in which keys must be stored and compared as opaque objects. The resulting atomic-key dictionaries exhibit performance bounds in terms of the average key size and match the bounds when all keys are the same size. Atomic-key dictionaries can be built with minimal modification to the B-tree structure, simply by choosing the pivot keys properly. Further, this article describes both static and dynamic atomic-key dictionaries. In the static case, if there are $\textit{N}$ keys with average size $\textit{K}$, the search cost is $O(\lceil K/B\rceil \text{log}_{1 + \lceil K/B\rceil} N)$ expected transfers. It is not possible to transform these expected bounds into worst-case bounds. The cost to build the tree is $\textit{O(NK)}$ operations and $\textit{O(NK/B)}$ transfers if all keys are presented in sorted order. If not, the cost is the sorting cost. For the dynamic dictionaries, the amortized cost to insert a key $\textit{κ}$ of arbitrary length at an arbitrary rank is dominated by the cost to search for $\textit{κ}$. Specifically, the amortized cost to insert a key $\textit{κ}$ of arbitrary length and random rank is $O(\lceil K/B\rceil log_{1+ \lceil B/K\rceil } N + |κ|/B)$ transfers. A dynamic-programming algorithm is shown for constructing a search tree with minimal expected cost. This article also gives a cache-oblivious static atomic-key B-tree, which achieves the same asymptotic performance as the static B-tree dictionary, mentioned previously. A cache-oblivious data structure or algorithm is not parameterized by the block size $\textit{B}$ or memory size $\textit{M}$ in the memory hierarchy; rather, it is universal, working simultaneously for all possible values of $\textit{B}$ or $\textit{M}$. On a machine with block size $\textit{B}$, if there are $\textit{N}$ keys with average size $\textit{K}$, search operations costs $O(\lceil K/B \rceil log_{1+ \lceil B/K \rceil} N)$ block transfers in expectation. This cache-oblivious layout can be built in $\textit{O(N}\text{log}\textit{(NK))}$ processor operations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Implication of production tax credit on economic dispatch for electricity merchants with storage and wind farms

The production tax credit (PTC) promotes wind energy development, reduces power generation costs, and can affect merchants’ joint economic dispatch, particularly for electricity merchants with both energy storage and wind farms. Two common PTC policies are studied – in the first policy, a wind farm receives PTC by selling wind generation to the market and its storage can be used to store energy from the wind generation and energy purchased from the grid but the energy released from the storage cannot receive PTC; in the second policy, the energy released from the storage can also qualify for PTC but purchasing energy from the grid is not allowed. We then employ dynamic programming to study merchants' optimal decision-making while considering PTC and the physical characteristics of storage systems. We analytically show that the state of charge (SOC) range can be segmented into different regions by SOC reference points under two PTC policies. The merchant's optimal action can be conveniently and uniquely determined based on the region within which the current SOC falls. Moreover, this study illustrates that PTC could substantially alter the optimal scheduling policy structures by affecting reference points and their relationships. The results showed that the frequencies for charging and discharging storage decisions decreased with an increase in PTC subsidy. Last, we confirm that, although the first policy allows merchants to buy electricity from the market, the second policy can bring more profits when the PTC is large at the current PTC rates. Furthermore, the findings can provide multistage decision-making guidance to electricity merchants in the wholesale power market.

17 WIND ENERGY↗