Liang, Yu | Engineering Papers

DOE OSTI2025

Optimizing Traffic Signal Control to Enhance Transportation Efficiency and Maximize Pedestrian Benefits in the Road Network

Increasing urban mobility requirements demand efficient transportation system strategies for both vehicular and pedestrian movement. This study enhances the Decentralized Graph-based Multi-Agent Reinforcement Learning (DGMARL) approach, originally tailored for vehicular traffic signal timing, to incorporate pedestrian traffic dynamics. The improved algorithm considers crucial metrics such as Eco_PI, assesses vehicle fuel consumption by factoring in stops and delays, and addresses pedestrian waiting time, crucial for system efficiency while acknowledging driver waiting time impact. Utilizing Digital Twin simulation along the MLK Smart Corridor in Chattanooga, Tennessee, the algorithm's performance is compared for various pedestrian control scenarios. To evaluate the effectiveness of DGMARL, this study compared DGMARL-enabled signal management with automated pedestrian traffic detection and an actuated signal management system (real-word baseline) with pedestrian recall, which predetermingly enforces a pedestrian phase every cycle. Findings indicate substantial improvements with DGMARL, showing a 28.29% enhancement in vehicle Eco_PI, a 60.55 % reduction in pedestrian waiting time, and a 55.74% decrease in driver stop delay, on average, compared to the baseline actuated signal timing plan.

Kumarasamy, Vijayalakshmi K [The University of Ten↗

DOE OSTI2024

Integration of Decentralized Graph-Based Multi-Agent Reinforcement Learning with Digital Twin for Traffic Signal Optimization

Machine learning (ML) methods, particularly Reinforcement Learning (RL), have gained widespread attention for optimizing traffic signal control in intelligent transportation systems. However, existing ML approaches often exhibit limitations in scalability and adaptability, particularly within large traffic networks. This paper introduces an innovative solution by integrating decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin to enhance traffic signal optimization, targeting the reduction of traffic congestion and network-wide fuel consumption associated with vehicle stops and stop delays. In this approach, DGMARL agents are employed to learn traffic state patterns and make informed decisions regarding traffic signal control. The integration with a Digital Twin module further facilitates this process by simulating and replicating the real-time asymmetric traffic behaviors of a complex traffic network. The evaluation of this proposed methodology utilized PTV-Vissim, a traffic simulation software, which also serves as the simulation engine for the Digital Twin. The study focused on the Martin Luther King (MLK) Smart Corridor in Chattanooga, Tennessee, USA, by considering symmetric and asymmetric road layouts and traffic conditions. Comparative analysis against an actuated signal control baseline approach revealed significant improvements. Experiment results demonstrate a remarkable 55.38% reduction in Eco_PI, a developed performance measure capturing the cumulative impact of stops and penalized stop delays on fuel consumption, over a 24 h scenario. In a PM-peak-hour scenario, the average reduction in Eco_PI reached 38.94%, indicating the substantial improvement achieved in optimizing traffic flow and reducing fuel consumption during high-demand periods. These findings underscore the effectiveness of the integrated DGMARL and Digital Twin approach in optimizing traffic signals, contributing to a more sustainable and efficient traffic management system.

42 ENGINEERING↗

DOE OSTI2024

Integration of Decentralized Graph-Based Multi-Agent Reinforcement Learning with Digital Twin for Traffic Signal Optimization

Machine learning (ML) methods, particularly Reinforcement Learning (RL), have gained widespread attention for optimizing traffic signal control in intelligent transportation systems. However, existing ML approaches often exhibit limitations in scalability and adaptability, particularly within large traffic networks. This paper introduces an innovative solution by integrating decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin to enhance traffic signal optimization, targeting the reduction of traffic congestion and network-wide fuel consumption associated with vehicle stops and stop delays. In this approach, DGMARL agents are employed to learn traffic state patterns and make informed decisions regarding traffic signal control. The integration with a Digital Twin module further facilitates this process by simulating and replicating the real-time asymmetric traffic behaviors of a complex traffic network. The evaluation of this proposed methodology utilized PTV-Vissim, a traffic simulation software, which also serves as the simulation engine for the Digital Twin. The study focused on the Martin Luther King (MLK) Smart Corridor in Chattanooga, Tennessee, USA, by considering symmetric and asymmetric road layouts and traffic conditions. Comparative analysis against an actuated signal control baseline approach revealed significant improvements. Experiment results demonstrate a remarkable 55.38% reduction in Eco_PI, a developed performance measure capturing the cumulative impact of stops and penalized stop delays on fuel consumption, over a 24 h scenario. In a PM-peak-hour scenario, the average reduction in Eco_PI reached 38.94%, indicating the substantial improvement achieved in optimizing traffic flow and reducing fuel consumption during high-demand periods. These findings underscore the effectiveness of the integrated DGMARL and Digital Twin approach in optimizing traffic signals, contributing to a more sustainable and efficient traffic management system.

actuated signal control↗

DOE OSTI2024

Pedestrian-Involved Traffic Signal Optimization Using Decentralized Graph-based Multi-Agent Reinforcement Learning

Incorporating pedestrian movements in traffic signal timing optimization is essential for ensuring smooth and safe urban transportation systems. To develop an effective signal timing plan, it is crucial to comprehend how various pedestrian accommodation strategies impact both vehicle and pedestrian flow. This study explores the application of the Decentralized Graph-based Multi-Agent Reinforcement Learning (DGMARL) method for signal timing optimization. It considers both pedestrian and vehicle traffic state and assesses the effects of fixed and adaptive pedestrian request response on traffic and signal timing. The evaluation of DGMARL encompasses vehicles Eco_PI which is fuel consumption impact related to vehicle stops and stop delay, and pedestrian waiting time to ensure overall system efficiency. The proposed approach was evaluated using a Digital Twin microscopic traffic simulation model of MLK Smart Corridor in Chattanooga, Tennessee. The evaluation outcomes of vehicles Eco_PI, pedestrian waiting time and serving time are compared with the actuated and DGMARL signal timing with fixed pedestrian recall and minimum recall to determine the most suitable approaches for various traffic conditions. The results indicated that, on average, the strategy of Signal timing optimization with adaptive pedestrian request response improved the Eco_PI by 32.43%, and Signal timing optimization with both automated pedestrian traffic demand and adaptive pedestrian request response improved the Eco_PI by 31.62% compared to the actuated and DGMARL signal timing with fixed pedestrian recall and minimum recall.

K Kumarasamy, Vijayalakshmi↗

DOE OSTI2023

Traffic Signal Optimization by Integrating Reinforcement Learning and Digital Twins

Machine learning (ML) methods, especially reinforcement learning (RL), have been widely considered for traffic signal optimization in intelligent transportation systems. Most of these ML methods are centralized, lacking in scalability and adaptability in large traffic networks. Further, it is challenging to train such ML models due to the lack of training platforms and/or the cost of deploying and training in a real traffic networks. This paper presents an approach for the integration of decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin (DT) to optimize traffic signals for the reduction of traffic congestion and network-wide fuel consumption related to stopping. Specifically, the DGMARL agents learn traffic state patterns and make decisions regarding traffic signal control with assistance from a Digital Twin module, which simulates and replicates the traffic behaviors of a real traffic network. The proposed approach was evaluated using PTV-Vissim [1], a microscopic traffic simulation platform. PTV-Vissim is also the simulation engine of the DT, enabling emulation and optimization of the traffic signals on the MLK Smart Corridor in Chattanooga, Tennessee. Compared to an actuated signal control baseline approach, experiment results show that Eco_PI, a developed performance measure capturing the impact of stops on fuel consumption, was reduced by 44.27% in a 24-hour and an average of 29.88% in a PM-peak-hour scenario.

Multi-Agent Reinforcement Learning, Digital Twin, ↗

DOE OSTI2023

Synthesis of PHI-type zeolite harmotome using barium as an inorganic structure-directing agent inspired from natural mineral compositions

Zeolites are nanoporous aluminosilicates widely used as commercial adsorbents, heterogeneous catalysts, and ion-exchange materials due to their unique porosity, acidity, and thermal stability. The vast majority of synthetically-realized zeolite structures are prepared under hydrothermal conditions in alkaline media, and often in the presence of an organic structure-directing agent that facilitates the crystallization of diverse porous networks. Due to economic and environmental disadvantages of organic-based syntheses, it is often desirable to produce zeolites in organic-free media using alkali metal ions as the most commonly employed inorganic structure-directing agents (ISDAs). Here, in this study, we were inspired by the elemental compositions of natural zeolite minerals to examine the use of an alkaline earth metal ion as the ISDA to prepare zeolites with the PHI framework type. Two natural minerals exhibiting this crystal structure are phillipsite and harmotome. Both structures contain sodium and potassium as extra-framework cations, but only harmotome contains barium cations. Here we examined how the combination of alkali and alkaline earth ISDAs promote the formation of PHI-type zeolites. Our findings reveal that each isostructure is prepared under different synthesis conditions where simply adding barium to a growth mixture does not always lead to harmotome formation. Harmotome is formed in more dilute and alkaline growth media; however, time-resolved analyses of zeolite crystallization reveal that harmotome can be synthesized at low temperatures (ca. 65 °C) with rates that are 5-times faster than syntheses of phillipsite. This seemingly indicates that barium reduces the energetic barrier for PHI-type zeolite nucleation in ways that are not fully understood. Moreover, harmotome crystals exhibit an average particle size of ca. 60 nm, which is nearly 40-fold smaller than phillipsite crystals prepared under similar conditions. Collectively, these findings identify routes to control crystallization of PHI-type zeolites and may provide a broader strategy for using the composition of natural zeolite minerals to optimize the preparation of synthetic analogues.

36 MATERIALS SCIENCE↗

NASA NTRS2015

Transverse Mode Multi-Resonant Single Crystal Transducer

A transducer is disclosed that includes a multiply resonant composite, the composite having a resonator bar of a piezoelectric single crystal configured in a d(sub 32) transverse length-extensional resonance mode having a crystallographic orientation set such that the thickness axis is in the (110) family and resonance direction is the (001) family.

Snook, Kevin A.↗

NASA NTRS2012

Wideband Single-Crystal Transducer for Bone Characterization

The microgravity conditions of space travel result in unique physiological demands on the human body. In particular, the absence of the continual mechanical stresses on the skeletal system that are present on Earth cause the bones to decalcify. Trabecular structure decreases in thickness and increases in spacing, resulting in decreased bone strength and increased risk of injury. Thus, monitoring bone health is a high priority for long-term space travel. A single probe covering all frequency bands of interest would be ideal for such measurements, and this would also minimize storage space and eliminate the complexity of integrating multiple probes. This invention is an ultrasound transducer for the structural characterization of bone. Such characterization measures features of reflected and transmitted ultrasound signals, and correlates these signals with bone structure metrics such as bone mineral density, trabecular spacing, and thickness, etc. The techniques used to determine these various metrics require measurements over a broad range of ultrasound frequencies, and therefore, complete characterization requires the use of several narrowband transducers. This is a single transducer capable of making these measurements in all the required frequency bands. The device achieves this capability through a unique combination of a broadband piezoelectric material; a design incorporating multiple resonator sizes with distinct, overlapping frequency spectra; and a micromachining process for producing the multiple-resonator pattern with common electrode surfaces between the resonators. This device consists of a pattern of resonator bars with common electrodes that is wrapped around a central mandrel such that the radiating faces of the resonators are coplanar and can be simultaneously applied to the sample to be measured. The device operates as both a source and receiver of acoustic energy. It is operated by connection to an electronic system capable of both providing an excitation signal to the transducer and amplifying the signal received from the transducer. The excitation signal may be either a wide-bandwidth signal to excite the transducer across its entire operational spectrum, or a narrow-bandwidth signal optimized for a particular measurement technique. The transducer face is applied to the skin covering the bone to be characterized, and may be operated in through transmission mode using two transducers, or in pulse-echo mode.

Liang, Yu↗

NASA NTRS2012

Wideband Single-Crystal Transducer for Bone Characterization

The microgravity conditions of space travel result in unique physiological demands on the human body. In particular, the absence of the continual mechanical stresses on the skeletal system that are present on Earth cause the bones to decalcify. Trabecular structure decreases in thickness and increases in spacing, resulting in decreased bone strength and increased risk of injury. Thus, monitoring bone health is a high priority for long-term space travel. A single probe covering all frequency bands of interest would be ideal for such measurements, and this would also minimize storage space and eliminate the complexity of integrating multiple probes. This invention is an ultrasound transducer for the structural characterization of bone. Such characterization measures features of reflected and transmitted ultrasound signals, and correlates these signals with bone structure metrics such as bone mineral density, trabecular spacing, and thickness, etc. The techniques used to determine these various metrics require measurements over a broad range of ultrasound frequencies, and therefore, complete characterization requires the use of several narrowband transducers. This is a single transducer capable of making these measurements in all the required frequency bands. The device achieves this capability through a unique combination of a broadband piezoelectric material; a design incorporating multiple resonator sizes with distinct, overlapping frequency spectra; and a micromachining process for producing the multiple-resonator pattern with common electrode surfaces between the resonators. This device consists of a pattern of resonator bars with common electrodes that is wrapped around a central mandrel such that the radiating faces of the resonators are coplanar and can be simultaneously applied to the sample to be measured. The device operates as both a source and receiver of acoustic energy. It is operated by connection to an electronic system capable of both providing an excitation signal to the transducer and amplifying the signal received from the transducer. The excitation signal may be either a wide-bandwidth signal to excite the transducer across its entire operational spectrum, or a narrow-bandwidth signal optimized for a particular measurement technique. The transducer face is applied to the skin covering the bone to be characterized, and may be operated in through-transmission mode using two transducers, or in pulse-echo mode. The transducer is a unique combination of material, design, and fabrication technique. It is based on single-crystal lead magnesium niobate lead titanate (PMN-PT) piezoelectric material. As compared to the commonly used piezoceramics, this piezocrystal has superior piezoelectric and elastic properties, which results in devices with superior bandwidth, source level, and power requirements. This design necessitates a single resonant frequency. However, by operating in a transverse length-extensional mode, with the electric field applied orthogonally to the extensional direction, resonators of different sizes can share common electrodes, resulting in a multiply-resonant structure. With carefully sized resonators, and the superior bandwidth of piezocrystal, the resonances can be made to overlap to form a smooth, wide-bandwidth characteristic.

Liang, Yu↗