Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “stochastic models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Role of Ocean and Atmosphere Variability in Scale-Dependent Thermodynamic Air-Sea Interactions

This study investigates the influence of oceanic and atmospheric processes in extratropical thermodynamic air-sea interactions resolved by satellite observations (OBS) and by two climate model simulations run with eddy-resolving high-resolution (HR) and eddy-parameterized low-resolution (LR) ocean components. Here, in this work, spectral methods are used to characterize the sea surface temperature (SST) and turbulent heat flux (THF) variability and co-variability over scales between 50 and 10,000 km and 60 days to 80 years in the Pacific Ocean. The relative roles of the ocean and atmosphere are interpreted using a stochastic upper-ocean temperature evolution model forced by noise terms representing intrinsic variability in each medium, defined using climate model data to produce realistic rather than white spectral power density distributions. The analysis of all datasets shows that the atmosphere dominates the SST and THF variability over zonal wavelengths larger than ~2,000–2,500 km. In HR and OBS, ocean processes dominate the variability of both quantities at scales smaller than the atmospheric first internal Rossby radius of deformation (R 1 , ~600–2,000 km) due to a substantial ocean forcing coinciding with a weaker atmospheric modulation of THF (and consequently of SST) than at larger scales. The ocean forcing also induces oscillations in SST and THF with periods ranging from intraseasonal to multidecadal, reflecting a red spectrum response to ocean forcing similar to that driven by atmospheric forcing. Such features are virtually absent in LR due to a weaker ocean forcing relative to HR.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Simulation and Analysis on Reactor Pressure Vessel (RPV) subjected to Pressurized Thermal Shock (PTS) under SBLOCA scenario by using Cardinal to support the fracture mechanics analyses

The structural components that comprise nuclear reactors and their supporting structures are subjected to harsh operating environments that can challenge their integrity, especially after exposure for extended durations or under accident condition. As one of the most significant components of a Reactor, the Reactor Pressure Vessel (RPV) is exposed to an aggressive environment during the operation time (e.g. more than 40 years). Ageing degradation mechanisms (e.g. thermos-fatigue) could grow initial defects up to a critical size, increasing the susceptibility to failure in the RPV. The conventional methods are mostly based on simple crack and structure geometries. Very limited studies consider the real conditions of the RPV subjected to a thermal shock due to a Loss of Coolant Accident (LOCA). During a LOCA event, the most severe conditions take place when the emergency core cooling (ECC) water is injected inside the cold legs filled initially with hotter water and/or steam. The rapid cooling of the down-comer and the internal RPV surface followed probably by re-pressurization of the RPV causes large temperature gradients and variation of pressure which induces thermal-mechanical stresses. In order to develop the model for integrity assessment of a reactor pressure vessel (RPV) subjected to pressurized thermal shock (PTS), a multi-physics simulation, which includes the thermo-hydraulic, thermo-mechanical and fracture mechanics analyses is necessary. The multi-physics simulations are performed using Cardinal, a wrapping of the GPU-oriented spectral element Computational Fluid Dynamics (CFD) code NekRS and other multi-physics sub-modules within the MOOSE framework. Cardinal now fully supports MOOSE stochastic perturbations of NekRS models with varying boundary conditions, initial conditions, material properties, and any other quantity which is defined by a kernel (such as coefficients in a momentum source model). The implementation is designed in a flexible manner so that scalar values are sent from MOOSE into a user scratch space in NekRS, which can then be applied for any purpose within the NekRS case files (both on the host and device). When modeling PTS, several factors can impact the results significantly. In this report, the impacts of the geometry of the model, Reynolds number and buoyancy effect are investigated. Two geometry, i.e., a simplified model and a realistic RPV model, with both laminar and turbulent flow condition are adopted for the PTS simulation with and without buoyancy effect. The purpose of the investigation is to understand the impact of these factors on the prediction of temperature history of RPV. The accurate prediction on the temperature evolution, which will be exported to Grizzly code for further analyses on the progression of aging mechanisms and their effects on the integrity of RPV structures, is very crucial. Based on the understanding of these factors, a more sophisticated model is built to analysis the PTS under SBLOCA scenario. A literature survey is conducted to pick the SBLOCA scenario for the multi-physics simulation. The analysis helps to explain the form and the transformation of the cold plum when the ECC is activated under SBLOCA. This model can be can be applied to study the PTS effect for different RPV configurations. The results can help to assess structural component degradation for advanced reactors.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Hybridizing Machine Learning and Physically-based Earth System Models to Improve Prediction of Multivariate Extreme Events (AI Exploration of Wildland Fire Prediction)

Focal Areas: This project responds to two focal areas identified in the DOE Call for AI4ESP White Papers: 1) Predictive modeling through the use of artificial intelligence (AI) techniques, and 2) insights gleaned from complex data using explainable AI and big data analytics. Science Challenge: Large wildland fires (hereafter wildfires) appearing as high-impact compound climate extreme events are closely related to hydroclimate and water cycle extremes that modulate surface fuel supply and combustibility. These compound events have multivariate climatic features (e.g., temperature, precipitation, relative humidity, wind, lightning) and societal drivers (e.g., forest management, land use change, human caused ignitions). Meanwhile, they induce strong feedbacks to the coupled atmosphere, biosphere, and hydrosphere by perturbing regional and global radiation budget as well as ecological, biogeochemical, and water cycles across multiple spatiotemporal scales. The nonlinear interactions between these natural and anthropogenic components of the Earth system are too complex to be completely and adequately represented in today’s Earth system models (ESMs). The inherent stochastic nature of fire activity at all scales further increases the difficulty of its prediction using ESMs that are usually developed from deterministic equations and parameterizations. Besides, concurrence of long-term (decadal to interdecadal) global climate change and fire regime shifts overlapping with short-term (intraseasonal to interannual) variations of regional fire weather and burning activity confound predictability of these compound extreme events. We propose to address the above scientific challenges by using machine learning (ML)-based data-driven modeling techniques to integrate observations and physically-based ESMs’ simulations in a computationally efficient hybrid prediction system. This prediction system is supposed to characterize the wildfire’s sensitivity to climate and exogenous drivers at high resolution (~ 0.25°) on subseasonal to seasonal (S2S) timescales providing improved predictability and explainability. We will use the system to help identify: (1) What are the computational elements of a hybrid system needed to predict compound climate extreme events such as global wildfires? (2) What are the key drivers (either natural or anthropogenic) that modulate short-term variations of multivariate fire weather and burning activity over different regions? How can one take advantage of those driver-response relationships to improve the predictability of large wildfires on S2S time scales? (3) What are the underlying physical mechanisms and sources of improved predictability? Which ML techniques are optimal in revealing and adapting these mechanisms?

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Harnessing structural stochasticity in the computational discovery and design of microstructures

This paper presents a deep generative model-based design methodology for tailoring the structural stochasticity of microstructures. Although numerous methods have been established for designing deterministic (periodic) or stochastic microstructures, a systematic design approach that allows the unified treatment of both deterministic and stochastic microstructure design domains has yet to be created. The proposed methodology resolves this issue by learning a unified feature space that embodies diverse structural patterns with continuously varying stochasticity levels. A highly diverse microstructure database is established to incorporate various types of deterministic and stochastic microstructure patterns. A property-aware deep generative model is proposed to learn a unified feature space of the structural characteristics, as well as the relationship between structure features and properties of interest. Autoencoder (AE), Variational Autoencoder (VAE), and Adversarial Autoencoder (AAE) are compared to understand their relative merits in the property-aware learning of the unified feature space. Microstructural designs with tailorable stochasticity and properties are obtained by searching the unified feature space. Multiple design cases are presented to demonstrate the capability of designing microstructures for structural stochasticity and properties. Furthermore, the proposed method is employed to create stochastically graded structures, which manipulate the mechanical behaviors by varying the local stochasticity of the structure.

36 MATERIALS SCIENCE↗
DOE OSTI2022

An indirect approach to optimize the reaction rates of thermal NO formation for diesel engines

With stringent emission regulations, it has become more important for modern diesel engine manufacturers to accurately predict engine-out nitrogen oxide (NO x ) emissions across a wide range of operating conditions. Thermal NO is the major source of engine-out NO x in modern diesel engines. For thermal NO formation, several earlier studies have recommended the forward and reverse reaction rate coefficients of the rate-limiting reaction (O + N 2 ⇌ NO + N). However, due to deficiencies in sub-models and inadequacies of reduced chemical mechanisms to represent diesel combustion, these recommended values more often than not need to be adjusted in reduced order combustion models to accurately predict engine-out NO x . Hence, in this work a systematic and computationally efficient approach has been proposed to streamline the process of determining the optimum reaction rate coefficients. Here, to develop the optimization approach, four different production diesel engines with different operating conditions in terms of speed, load, and exhaust-gas recirculation have been considered. Numerical simulations have been performed using a detailed zero-dimensional velocity-composition-frequency transported probability density function (0D-VCF-tPDF) model that uses hundreds of notional particles to capture in-cylinder stratification. Four different combinations of hydrocarbon and NO x chemical mechanisms were used to represent chemistry. It was found that for the rate-limiting reaction, the pre-exponent factors (A f1 , A r1 ) and activation energies (E A,f1 , E A,r1 ) of the forward and reverse reaction rates follow a linear band in A f1 - E A,f1 and A r1 - E A,r1 space where predicted engine-out NO x match the measured values closely. By encompassing such bands from different engines and considering constraints on activation energies, a reduced search domain of pre-exponent factors and activation energies was constructed that is expected to be applicable to any diesel engine. Eventually, computationally efficient three-line and one-line search approaches were proposed to determine the optimum values of the pre-exponent factors and activation energies that led to a minimum error between measured and predicted engine-out NO x . Finally, these three-line and one-line NO x optimization approaches were applied to a fifth production diesel engine for which the 0D-VCF-tPDF model showed a very good predictive performance in terms of predicting peak pressure, 50% burn rate, and engine-out NO x when compared to measured and 3D-CFD values.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2021

An investigation of hard-disk drive circularity accounting for socio-technical dynamics and data uncertainty

The installed data storage capacity in the U.S. will reach 2.2 Zettabytes by 2025, generating about 50 million units of end-of-life (EOL) hard-disk drives (HDDs) per year. Due to data security concerns, most EOL HDDs are currently shredded (even when still functioning), representing an economic loss. Moreover, raw material extraction linked to the increased demand for storage causes environmental impacts. Besides mitigating the threat posed by sudden restrictions of raw materials, the circular economy (CE) offers to maximize value retention in the economy and reduce the environmental impacts of human activities. Common CE strategies are reusing and recycling products. However, the reuse of hard disk drives is currently burdened by the lack of trust HDD end-users have toward other non-physical means of data removal than shredding. Here, an agent-based modeling (ABM) approach is proposed to explore how techno-economic and social factors affect end-users' decisions to adopt EOL management practices other than shredding. The proposed method also accounts for data uncertainty by applying a semi-quantitative approach. Results demonstrate how increased green procurement and more robust standards could spur end-users' trust toward data-wiping technologies. Even when accounting for uncertainty, HDDs' reuse brings better environmental and economic benefits than HDD shredding followed by material recovery. The semi-quantitative approach proposed in this study could be more universally applied in future ABM, especially given the often-stochastic nature of such models. The developed ABM is also the first to represent several HDD industry stakeholders and demonstrate how the HDD shredding lock-in situation could be resolved.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems

Constructing surrogate models for uncertainty quantification (UQ) on complex partial differential equations (PDEs) having inherently high-dimensional O(10 n ), n ≥ 2, stochastic inputs (e.g., forcing terms, boundary conditions, initial conditions) poses tremendous challenges. The “curse of dimensionality” can be addressed with suitable unsupervised learning techniques used as a pre-processing tool to encode inputs onto lower-dimensional subspaces while retaining its structural information and meaningful properties. In this work, we review and investigate thirteen dimension reduction methods including linear and nonlinear, spectral, blind source separation, convex and non-convex methods and utilize the resulting embeddings to construct a mapping to quantities of interest via polynomial chaos expansions (PCE). Here, we refer to the general proposed approach as manifold PCE (m-PCE), where manifold corresponds to the latent space resulting from any of the studied dimension reduction methods. To investigate the capabilities and limitations of these methods we conduct numerical tests for three physics-based systems (treated as black-boxes) having high-dimensional stochastic inputs of varying complexity modeled as both Gaussian and non-Gaussian random fields to investigate the effect of the intrinsic dimensionality of input data. We demonstrate both the advantages and limitations of the unsupervised learning methods and we conclude that a suitable m-PCE model provides a cost-effective approach compared to alternative algorithms proposed in the literature, including recently proposed expensive deep neural network-based surrogates and can be readily applied for high-dimensional UQ in stochastic PDEs.

42 ENGINEERING↗
DOE OSTI2020

Reshaping of Truncated Pd Nanocubes: Energetic and Kinetic Analysis Integrating Transmission Electron Microscopy with Atomistic-Level and Coarse-Grained Modeling

Stability against reshaping of metallic fcc nanocrystals synthesized with tailored far-from-equilibrium shapes is key to maintaining optimal properties for applications such as catalysis. Yet Arrhenius analysis of experimental reshaping kinetics, and appropriate theory and simulation, is lacking. As such, we use TEM to monitor the reshaping of Pd nanocubes of ~25 nm side length between 410 °C (over ~4.5 h) and 440 °C (over ~0.25 h), extracting a high effective energy barrier of E eff ≈ 4.6 eV. We also provide an analytic determination of the energy variation along the optimal pathway for reshaping that involves transfer of atoms across the nanocube surface from edges or corners to form new layers on side {100} facets. The effective barrier from this analysis is shown to increase strongly with the degree of truncation of edges and corners in the synthesized nanocube. Theory matches experiment for the appropriate degree of truncation. In addition, we perform simulations of a stochastic atomistic-level model incorporating a realistic description of diffusive hopping for undercoordinated surface atoms, thereby providing a visualization of the initial reshaping process.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Data-Driven Simulation-Based Planning for Electric Airport Shuttle Systems: A Real-World Case Study

Many airports are adopting battery electric buses in their shuttle fleets due to concerns over air quality and regulations. This study proposes a simulation-based optimization modeling framework to help airport shuttle operators effectively deploy electric buses. We evaluated a planned airport electric shuttle system with an event-driven simulator. Empirical data collected from existing systems were used to drive the simulations. We then proposed a simulation-based optimization model to determine the battery capacity, charging power, and number of chargers so that predefined objective(s) (e.g., minimizing total capital cost, minimizing emissions) are optimized. Compared to existing studies, the primary contribution of the proposed method is that it can model the real-world stochastic nature of operations in an electric bus system with much higher fidelity. To demonstrate the proposed modeling framework, we study a real-world shuttle system at the Dallas-Fort Worth International Airport, and present extensive numerical studies. When considering partial fleet electrification, the model can provide a set of Pareto optimal solutions. When considering full fleet electrification, the optimal solution requires a 50-kWh battery capacity and four 210-kW chargers, resulting in a total capital cost of $26,744,000. The results demonstrate that the proposed modeling framework can effectively optimize the planning of electric airport shuttle systems with partial or full fleet electrification.

ADVANCED PROPULSION SYSTEMS,ENERGY PLANNING, POLIC↗
DOE OSTI2021

Stochastic social behavior coupled to COVID-19 dynamics leads to waves, plateaus, and an endemic state

It is well recognized that population heterogeneity plays an important role in the spread of epidemics. While individual variations in social activity are often assumed to be persistent, that is, constant in time, here we discuss the consequences of dynamic heterogeneity. By integrating the stochastic dynamics of social activity into traditional epidemiological models, we demonstrate the emergence of a new long timescale governing the epidemic, in broad agreement with empirical data. Our stochastic social activity model captures multiple features of real-life epidemics such as COVID-19, including prolonged plateaus and multiple waves, which are transiently suppressed due to the dynamic nature of social activity. The existence of a long timescale due to the interplay between epidemic and social dynamics provides a unifying picture of how a fast-paced epidemic typically will transition to an endemic state.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2020

Improving Solution Accuracy and Convergence for Stochastic Physics Parameterizations with Colored Noise

Stochastic parameterizations are used in numerical weather prediction and climate modeling to help capture the uncertainty in the simulations and improve their statistical properties. Convergence issues can arise when time integration methods originally developed for deterministic differential equations are applied naively to stochastic problems. In previous studies, it has been demonstrated that a correction term, known in stochastic analysis as the Itô correction, can help improve solution accuracy for various deterministic numerical schemes and ensure convergence to the physically relevant solution without substantial computational overhead. The usual formulation of the Itô correction is valid only when the stochasticity is represented by white noise. In this study, a generalized formulation of the Itô correction is derived for noises of any color. The formulation is applied to a test problem described by an advection–diffusion equation forced with a spectrum of fast processes. We present numerical results for cases with both constant and spatially varying advection velocities to show that, for the same time step sizes, the introduction of the generalized Itô correction helps to substantially reduce time integration error and significantly improve the convergence rate of the numerical solutions when the forcing term in the governing equation is rough (fast varying); alternatively, for the same target accuracy, the generalized Itô correction allows for the use of significantly longer time steps and, hence, helps to reduce the computational cost of the numerical simulation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Assignment of Freight Traffic in a Large-scale Intermodal Network under Uncertainty

This paper presents a methodology for freight traffic assignment in a large-scale road-rail intermodal network under uncertainty. Network uncertainties caused by natural disasters have dramatically increased in recent years. Several of these disasters (e.g., Hurricane Sandy, Mississippi River Flooding, and Hurricane Harvey) severely disrupted the U.S. freight transportation network, and consequently, the supply chain. To account for these network uncertainties, a stochastic freight traffic assignment model is formulated. An algorithmic framework, involving the sample average approximation and gradient projection algorithm, is proposed to solve this challenging problem. The developed methodology is tested on the U.S. intermodal network with freight flow data from the Freight Analysis Framework. The experiments consider three types of natural disasters that have different risks and impacts on transportation networks: earthquakes, hurricanes, and floods. It is found that for all disaster scenarios, freight ton-miles are higher compared to the base case without uncertainty. The increase in freight ton-miles is the highest under the flooding scenario; this is because there are more states in the flood-risk areas, and they are scattered throughout the U.S.

42 ENGINEERING↗
DOE OSTI2021

Shape Stability of Truncated Octahedral fcc Metal Nanocrystals

Metallic nanocrystals (NCs) can be synthesized with tailored nonequilibrium shapes to enhance desired properties, e.g., octahedral fcc metal NCs optimize catalytic activity associated with {111} facets. However, maintenance of optimized properties requires stability against thermal reshaping. Thus, we analyze the reshaping of truncated fcc metal octahedra mediated by surface diffusion using a stochastic atomistic-level model with energetic input parameters for Pd. Furthermore, the model describes NC thermodynamics by an effective nearest-neighbor interaction and includes a realistic treatment of diffusive hopping for undercoordinated surface atoms. Kinetic Monte Carlo simulation reveals that the effective barrier, E eff , for the initial stage of reshaping is strongly tied to the degree of truncation of the vertices in the synthesized initial octahedral shapes. This feature is elucidated via exact analytic determination of the energy variation along the optimal reshaping pathway at low-temperature (T), which involves transfer of atoms from truncated {100} vertex facets to form new layers on {111} side facets. Deviations from predictions of the low-T analysis due to entropic effects are more prominent for higher T and larger NC sizes.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Endogenizing Probabilistic Resource Adequacy Risks in Deterministic Capacity Expansion Models

In this work, we demonstrate how power system capacity expansion models can understate the stochastic effects of thermal outages when considering resource availabilities on an hourly expected value basis, yielding system designs with multiple orders of magnitude more shortfall risk than stated adequacy targets. We develop a novel approximation approach to efficiently endogenize awareness of this risk in a deterministic, linear capacity expansion framework. We compare this approach to exogenous tuning of an energy reserve margin, the leading alternative method to compensate for unmodeled probabilistic shortfall risk. Empirical results from a test system show that the new endogenous method cost-effectively meets all regional reliability targets with a single optimization solve, and produces a near-identical system design as the incumbent method without the need for repeated re-optimizations to find an appropriate reserve level. The endogenous method may also use iterative re-optimizations to further improve solution quality, although these incremental benefits were modest in the system studied.

capacity expansion modeling↗
DOE OSTI2021

A Continuum Dislocation Dynamics Crystal Plasticity Approach to Irradiated Body-Centered Cubic α-Iron

Radiation-induced embrittlement of reactor pressure vessel (RPV) steels can potentially limit the operating life of nuclear power plants. Over extended exposure to radiation doses, these body-centered cubic (BCC) irons demonstrate irradiation damage. Here, we present a continuum dislocation density (CDD) crystal plasticity model to capture the interaction among dislocations and self-interstitial atom (SIA) loops in α-iron. We demonstrate the importance of modeling cross slip using a combined stochastic Monte Carlo approach and the role of slip system strength anisotropy in capturing stochastic cross slip interactions. Through these captured interactions, the CDD crystal plasticity model can capture both the stress response and the physical evolution of dislocations on different slip system planes. Single-crystal verification experiments are used to calibrate the CDD crystal plasticity model, and a set of simplified polycrystalline simulations demonstrates the model’s ability to capture the stress response from tensile experiments on α-iron.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Combined selection of the dynamic model and modeling error in nonlinear aeroelastic systems using Bayesian Inference

Here, we report a Bayesian framework for concurrent selection of physics-based models and (modeling) error models. We investigate the use of colored noise to capture the mismatch between the predictions of calibrated models and observational data that cannot be explained by measurement error alone within the context of Bayesian estimation for stochastic ordinary differential equations. Proposed models are characterized by the average data-fit, a measure of how well a model fits the measurements, and the model complexity measured using the Kullback–Leibler divergence. The use of a more complex error models increases the average data-fit but also increases the complexity of the combined model, possibly over-fitting the data. Bayesian model selection is used to find the optimal physical model as well as the optimal error model. The optimal model is defined using the evidence, where the average data-fit is balanced by the complexity of the model. The effect of colored noise process is illustrated using a nonlinear aeroelastic oscillator representing a rigid NACA0012 airfoil undergoing limit cycle oscillations due to complex fluid–structure interactions. Several quasi-steady and unsteady aerodynamic models are proposed with colored noise or white noise for the model error. The use of colored noise improves the predictive capabilities of simpler models.

42 ENGINEERING↗
DOE OSTI2022

Machine Learning Assisted HPC Workload Trace Generation for Leadership Scale Storage Systems

Monitoring and analyzing a wide range of I/O activities in an HPC cluster is important in maintaining mission-critical performance in a large-scale, multi-user, parallel storage system. Center-wide I/O traces can provide high-level information and fine-grained activities per application or per user running in the system. Studying such large-scale traces can provide helpful insights into the system. It can be used to develop predictive methods for making predictive decisions, adjusting scheduling policies, or providing decisions for the design of next-generation systems. However, sharing real-world I/O traces to expedite such research efforts leaves a few concerns; i) the cost of sharing the large traces is expensive due to this large size, and ii) privacy concern is an issue.We address such issues by building an end-to-end machine learn- ing (ML) workflow that can generate I/O traces for large-scale HPC applications. We leverage ML based feature selection and gener- ative models for I/O trace generation. The generative models are trained on I/O traces collected by the darshan I/O characterization tool over a period of one year. We present a two-step generation process consisting of two deep-learning models, called the feature generator and the trace generator. The combination of two-step generative models provides robustness by reducing the bias of the model and accounting for the stochastic nature of the I/O traces across different runs of an application. We evaluate the performance of the generative models and show that the two-step model can generate time-series I/O traces with less than 20% root mean square error.

Paul, Arnab↗
DOE OSTI2023

A Novel Deep Reinforcement Learning Approach to Traffic Signal Control with Connected Vehicles

The advent of connected vehicle (CV) technology offers new possibilities for a revolution in future transportation systems. With the availability of real-time traffic data from CVs, it is possible to more effectively optimize traffic signals to reduce congestion, increase fuel efficiency, and enhance road safety. The success of CV-based signal control depends on an accurate and computationally efficient model that accounts for the stochastic and nonlinear nature of the traffic flow. Without the necessity of prior knowledge of the traffic system’s model architecture, reinforcement learning (RL) is a promising tool to acquire the control policy through observing the transition of the traffic states. In this paper, we propose a novel data-driven traffic signal control method that leverages the latest in deep learning and reinforcement learning techniques. By incorporating a compressed representation of the traffic states, the proposed method overcomes the limitations of the existing methods in defining the action space to include more practical and flexible signal phases. The simulation results demonstrate the convergence and robust performance of the proposed method against several existing benchmark methods in terms of average vehicle speeds, queue length, wait time, and traffic density.

42 ENGINEERING↗