Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “QUEUES”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Highly Asynchronous VisitOr Queue Graph Toolkit

HAVOQGT is a C++ framework that can be used to create highly parallel graph traversal algorithms. The framework stores the graph and algorithmic data structures on external memory that is typically mapped to high performance locally attached NAND FLASH arrays. The framework supports a vertex-centered visitor programming model. The frameworkd has been used to implement breadth first search, connected components, and single source shortest path.

Pearce, R.↗

Systems and methods for providing lockless bimodal queues for selective packet capture

In a network system, an application receiving packets can consume one or more packets in two or more stages, where the second and the later stages can selectively consume some but not all of the packets consumed by the preceding stage. Packets are transferred between two consecutive stages, called producer and consumer, via a fixed-size storage. Both the producer and the consumer can access the storage without locking it and, to facilitate selective consumption of the packets by the consumer, the consumer can transition between awake and sleep modes, where the packets are consumed in the awake mode only. The producer may also switch between awake and sleep modes. Lockless access is made possible by controlling the operation of the storage by the producer and the consumer both according to the mode of the consumer, which is communicated via a shared memory location.

Ros-Giralt, Jordi↗

Systems and methods for multiresolution priority queues

A system for storing and extracting elements according to their priority takes into account not only the priorities of the elements but also three additional parameters, namely, a priority resolution p Δ and two priority limits p min and p max . By allowing an ordering error if the difference in the priorities of elements are within the priority resolution, an improvement in performance is achieved.

Ros-Giralt, Jordi↗

A dynamic, unified design for dedicated message matching engines for collective and point-to-point communications

The Message Passing Interface (MPI) libraries use message queues to guarantee correct message ordering between communicating processes. Message queues are in the critical path of MPI communications and thus, the performance of message queue operations can have significant impact on the performance of applications. Collective communications are widely used in MPI applications and they can have considerable impact on generating long message queues. In this paper, we propose a unified message matching mechanism that improves the message queue search time by distinguishing messages coming from point-to-point and collective communications and using a distinct message queue data structure for them. For collective operations, it dynamically profiles the impact of each collective call on message queues during the application runtime and uses this information to adapt the message queue data structure for each collective dynamically. Moreover, we use a partner/non-partner message queue data structure for the messages coming from point-to-point communications. The proposed approach can successfully reduce the queue search time while maintaining scalable memory consumption. The evaluation results show that we can obtain up to 5.5x runtime speedup for applications with long list traversals. Moreover, we can gain up to 15% and 94% queue search time improvement for all elements in applications with short and medium list traversals, respectively.

97 MATHEMATICS AND COMPUTING↗

Queued Up: Characteristics of Power Plants Seeking Transmission Interconnection As of the End of 2021 [Slides]

Proposed large-scale electric generation and storage projects must apply for interconnection to the bulk power system via interconnection queues. While most projects that apply for interconnection are not subsequently built, data from these queues nonetheless provide a general indicator for mid-term trends in developer interest. Berkeley Lab compiled and analyzed data from all seven ISOs/RTOs in concert with 35 non-ISO utilities, representing an estimated 85% of all U.S. electricity load. We include all "active" projects in these generation interconnection queues through the end of 2021, as well as data on "operational" and "withdrawn" projects where those data are available. We find that the amount of new electric capacity in these queues is growing dramatically, with over 1,400 gigawatts (GW) of total generation and storage capacity now seeking connection to the grid (over 90% of which is for zero-carbon resources like solar, wind, and battery storage). Solar (676 GW) and battery storage (~420 GW) are – by far – the fastest growing resources in the queues; combined they accounted for nearly 85% of new capacity entering the queues in 2021. Substantial wind (247 GW) capacity is also seeking interconnection, 31% of which is for offshore projects (77 GW). In total, about 930 GW of zero-carbon generating capacity is currently seeking transmission access, as is 74 GW of natural gas capacity. Hybrids now comprise a large – and increasing – share of proposed projects, particularly in CAISO and the non-ISO West. 286 GW of solar hybrids (primarily solar+battery) and 19 GW of wind hybrids are currently active in the queues; nearly half of battery storage in the queues is paired with generation. However, much of this proposed capacity will be withdrawn from the queues and not built. Among a subset of queues for which data are available, only 23% of the projects seeking connection from 2000 to 2016 have subsequently been built. Completion percentages appear to be declining and are even lower for wind and solar than other resources. Additionally, wait times are on the rise: for the regions with available data, the typical duration from connection request to commercial operation increased from ~2.1 years for projects built in 2000-2010 to ~3.7 years for those built in 2011-2021.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Queued Up: Characteristics of Power Plants Seeking Transmission Interconnection As of the End of 2022 [Slides]

Proposed large-scale electric generation and storage projects must apply for interconnection to the bulk power system via interconnection queues. While most projects that apply for interconnection are not subsequently built, data from these queues nonetheless provide a general indicator for mid-term trends in developer interest. Berkeley Lab compiled and analyzed data from all seven ISOs/RTOs in concert with 35 non-ISO utilities, representing an estimated 85% of all U.S. electricity load. We include all "active" projects in these generation interconnection queues through the end of 2022, as well as data on "operational" and "withdrawn" projects where those data are available. We find that the amount of new electric capacity in these queues is growing dramatically, with over 2,000 gigawatts (GW) of total generation and storage capacity now seeking connection to the grid (over 95% of which is for zero-carbon resources like solar, wind, and battery storage). Solar (947 GW) and battery storage (~680 GW) are – by far – the fastest growing resources in the queues; combined they accounted for over 80% of new capacity entering the queues in 2022. Substantial wind (300 GW) capacity is also seeking interconnection, 38% of which is for offshore projects (113 GW). In total, about 1,250 GW of zero-carbon generating capacity is currently seeking transmission access, as is 82 GW of natural gas capacity. Hybrids projects (co-locating multiple generation and/or storage types) comprise a large – and increasing – share of proposed projects, particularly in CAISO and the non-ISO West. 457 GW of solar hybrids (primarily solar+battery) and 24 GW of wind hybrids are currently active in the queues; over half of battery storage in the queues is paired with generation. However, much of this proposed capacity will be withdrawn from the queues and not built. Among a subset of queues for which data are available, only 21% of the projects (and 14% of capacity) seeking connection from 2000 to 2017 have been built as of the end of 2022. Additionally, interconnection wait times are on the rise: The typical duration from connection request to commercial operation increased from <2 years for projects built in 2000-2007 to nearly 4 years for those built in 2018-2022 (with a median of 5 years for projects built in 2022).

24 POWER TRANSMISSION AND DISTRIBUTION↗

Batch Scheduling a Fresh Approach

The Network Queueing System (NQS) was designed to schedule jobs based on limits within queues. As systems obtain more memory, the number of queues increased to take advantage of the added memory resource. The problem now becomes too many queues. Having a large number of queues provides users with the capability to gain an unfair advantage over other users by tailoring their job to fit in an empty queue. Additionally, the large number of queues becomes confusing to the user community. The High Speed Processors group at the Numerical Aerodynamics Simulation (NAS) Facility at NASA Ames Research Center developed a new approach to batch job scheduling. This new method reduces the number of queues required by eliminating the need for queues based on resource limits. The scheduler examines each request for necessary resources before initiating the job. Also additional user limits at the complex level were added to provide a fairness to all users. Additional tools which include user job reordering are under development to work with the new scheduler. This paper discusses the objectives, design and implementation results of this new scheduler

Cardo, Nicholas P.↗

Efficient accesses of data structures using processing near memory

Systems, apparatuses, and methods for implementing efficient queues and other data structures. A queue may be shared among multiple processors and/or threads without using explicit software atomic instructions to coordinate access to the queue. System software may allocate an atomic queue and corresponding queue metadata in system memory and return, to the requesting thread, a handle referencing the queue metadata. Any number of threads may utilize the handle for accessing the atomic queue. The logic for ensuring the atomicity of accesses to the atomic queue may reside in a management unit in the memory controller coupled to the memory where the atomic queue is allocated.

97 MATHEMATICS AND COMPUTING↗

Queued Up: Characteristics of Power Plants Seeking Transmission Interconnection As of the End of 2020

Proposed large-scale electric generation and storage projects must apply for interconnection to the bulk power system via interconnection queues. While many projects that apply for interconnection are not subsequently built, data from these queues nonetheless provide a general indicator for mid-term trends in developer interest. Berkeley Lab compiled and analyzed data from all seven ISOs/RTOs in concert with 35 non-ISO utilities, representing an estimated 85% of all U.S. electricity load. We include all "active" projects in these generation interconnection queues through the end of 2020, as well as data on "completed" and "withdrawn" projects for five of the ISOs (CAISO, ISO-NE, MISO, NYISO, PJM). We find that the total capacity active in the queues is growing year-over-year, with over 750 GW of generation and an estimated 200 GW of storage capacity as of the end of 2020. Solar (462 GW) accounts for a large – and growing – share of generator capacity in the queues. Substantial wind (209 GW) capacity is also in development, 29% of which is for offshore projects (61 GW). In total, about 680 GW of zero-carbon capacity is currently seeking transmission access, as is 74 GW of natural gas capacity. Hybrids now comprise a large – and increasing – share of proposed projects, particularly in CAISO and the non-ISO West. 159 GW of solar hybrids (primarily solar+battery) and 13 GW of wind hybrids are currently active in the queues. However, much of this proposed capacity will not ultimately be built. Among a subset of queues for which data are available, only 24% of the projects seeking connection from 2000 to 2015 have subsequently been built. Completion percentages appear to be declining, and are even lower for wind and solar than other resources. Additionally, wait times are on the rise: in four ISOs, the typical duration from connection request to commercial operation increased from ~1.9 years for projects built in 2000-2009 to ~3.5 years for those built in 2010-2020. There are growing calls for queue reform to reduce cost, lead times, and speculation.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Hybrid Power Plants: Status of Operating and Proposed Plants, 2023 Edition [Slides]

Improving battery technology and the growth of variable renewable generation are driving a surge of interest in “hybrid” power plants that combine, for example, wind or solar generating capacity with co-located batteries. While most of the current interest involves pairing photovoltaic (PV) plants with batteries, other types of hybrid or co-located plants with wide-ranging configurations have been part of the U.S. electricity mix for decades. This annually updated briefing tracks and maps existing hybrid or co-located plants across the United States while also synthesizing data from power purchase agreements (PPAs) and generation interconnection queues to shed light on near- and long-term development pipelines. The scope includes “co-located hybrids” that pair two or more resources (e.g., multiple types of generation and/or generation with storage) that are operated largely independently behind a single point of interconnection, and “full hybrids” that also feature coordinated operations of the co-located resources. The focus is on plants with one megawatt (MW) or more of capacity; smaller (often behind-the-meter) projects are also increasingly common, but are not included in this data synthesis. Key findings from the latest briefing include: -At the end of 2022, there were 374 hybrid plants (>1 MW) operating across the United States (+25% compared to the end of 2021), totaling nearly 41 GW of generating capacity (+15%) and 5.4 GW/15.2 GWh of energy storage (+69%/+88%). PV+storage plants are by far the most common, dominating in terms of plant number (213), storage capacity (4.0 GW/12.5 GWh), storage:generator capacity ratio (49%), and storage duration (3.1 hours). But there are nearly twenty other hybrid plant configurations as well, including several different fossil hybrid categories (each dominated by the fossil component) as well as wind+storage, wind+PV, wind+PV+storage, geothermal+PV, and others. -Last year was another strong year for PV+storage hybrids in particular: 59 of the 62 hybrids added in 2022 were PV+storage. As of the end of 2022, there was roughly as much storage capacity operating within PV+storage hybrid plants as in standalone storage plants (~4 GW each). In storage energy terms, however, PV+storage edged out standalone storage by ~2 GWh (12.5 GWh vs. 10.4 GWh, respectively). -Interconnection queue data show continued strong developer interest in hybridization. At the close of 2022, there were 51% more hybrid plants—representing 59% more generating capacity—in interconnection queues across the United States than there were at the end of 2021. Solar dominates these proposed plants as well: at the close of 2022, there were 457 GW of solar capacity proposed as a hybrid (representing ~48% of all solar capacity in the queues), most typically pairing PV with battery storage. At the same time, there were 24 GW of wind capacity proposed as a hybrid (representing ~8% of all wind capacity in the queues), again most-often pairing wind with storage. Meanwhile, more than half of all storage in the queues is estimated to be part of a hybrid plant. While many of the plants proposed in the queues will not ultimately reach commercial operations, the depth of interest in hybrid plants—especially PV+storage—is notable, particularly in certain regions. For example, in CAISO, 97% of all solar capacity and 45% of all wind capacity in the queues is proposed as a hybrid. -The report also surveys power purchase agreement (PPA) price data from a sample of operating and proposed PV+storage plants. Though PV+storage PPA prices have fallen over time, “levelized storage adders” have recently increased somewhat to ~$\$ $7000/MW-month, ~$\$ $60/MWh-stored (assuming one full cycle per day), or ~$\$ $15/MWh-PV. Some of the recent price increase could simply reflect a trend towards higher battery:PV capacity ratios over time, which increases costs, all else being equal. The well-publicized impact of inflationary and supply chain pressures on battery prices is no doubt a contributor as well.

25 ENERGY STORAGE↗

A Mixed Integer Linear Program for Airport Departure Scheduling

Aircraft departing from an airport are subject to numerous constraints while scheduling departure times. These constraints include wake-separation constraints for successive departures, miles-in-trail separation for aircraft bound for the same departure fixes, and time-window or prioritization constraints for individual flights. Besides these, emissions as well as increased fuel consumption due to inefficient scheduling need to be included. Addressing all the above constraints in a single framework while allowing for resequencing of the aircraft using runway queues is critical to the implementation of the Next Generation Air Transport System (NextGen) concepts. Prior work on airport departure scheduling has addressed some of the above. However, existing methods use pre-determined runway queues, and schedule aircraft from these departure queues. The source of such pre-determined queues is not explicit, and could potentially be a subjective controller input. Determining runway queues and scheduling within the same framework would potentially result in better scheduling. This paper presents a mixed integer linear program (MILP) for the departure-scheduling problem. The program takes as input the incoming sequence of aircraft for departure from a runway, along with their earliest departure times and an optional prioritization scheme based on time-window of departure for each aircraft. The program then assigns these aircraft to the available departure queues and schedules departure times, explicitly considering wake separation and departure fix restrictions to minimize total delay for all aircraft. The approach is generalized and can be used in a variety of situations, and allows for aircraft prioritization based on operational as well as environmental considerations. We present the MILP in the paper, along with benefits over the first-come-first-serve (FCFS) scheme for numerous randomized problems based on real-world settings. The MILP results in substantially reduced delays as compared to FCFS, and the magnitude of the savings depends on the queue and departure fix structure. The MILP assumes deterministic aircraft arrival times at the runway queues. However, due to taxi time uncertainty, aircraft might arrive either earlier or later than these deterministic times. Thus, to incorporate this uncertainty, we present a method for using the MILP with "overlap discounted rolling planning horizon". The approach is based on valuing near-term decision results more than future ones. We develop a model of taxitime uncertainty based on real-world data, and then compare the baseline FCFS delays with delays using the above MILP in a simple rolling-horizon method and in the overlap discounted scheme.

Gupta, Gautam↗

Hybrid Power Plants: Status of Operating and Proposed Plants, 2022 Edition [Slides]

Falling battery prices and the growth of variable renewable generation are driving a surge of interest in “hybrid” power plants that combine, for example, wind or solar generating capacity with co-located batteries. While most of the current interest involves pairing photovoltaic (PV) plants with batteries, other types of hybrid or co-located plants with wide-ranging configurations have been part of the U.S. electricity mix for decades. This annually updated briefing tracks and maps existing hybrid or co-located plants across the United States while also synthesizing data mined from power purchase agreements (PPAs) and generation interconnection queues to shed light on near- and long-term development pipelines. The scope includes co-located hybrid plants that pair two or more generators and/or that pair generation with storage at a single point of interconnection, and full hybrids that feature co-location and co-control. The focus is on plants with one megawatt (MW) or more of capacity; smaller (often behind-the-meter) projects are also increasingly common, but are not included in this data synthesis. Key findings from the latest briefing include: -At the end of 2021, there were nearly 300 hybrid plants (>1 MW) operating across the United States, totaling nearly 36 gigawatts (GW) of generating capacity and 3.2 GW/8.1 GWh of energy storage. PV+storage plants are by far the most common, dominating in terms of plant number (140), storage capacity (2.2 GW/7.0 GWh), storage:generator ratio (53%), and storage duration (3.2 hours). But there are nearly twenty other hybrid plant configurations as well, including several different fossil hybrid categories (each dominated by the fossil component) as well as wind+storage, wind+PV, wind+PV+storage, geothermal+PV, and others. -Last year was a breakout year for PV+storage hybrids in particular: 67 of the 74 hybrids added in 2021 were PV+storage. By the end of 2021, there were more GW of battery capacity installed in PV+storage hybrids (2.2 GW) than as standalone storage plants (1.8 GW). The difference is even starker in energy terms, with PV+storage plants hosting twice as much battery capacity as standalone storage plants (7 GWh vs. 3.5 GWh, respectively). Much of the battery capacity added in hybrid form in 2021 was a battery retrofit to a pre-existing PV plant. -Data on plants under development from the interconnection queues of all seven ISOs/RTOs plus 35 individual utilities suggest that these hybridization trends are likely to continue. At the close of 2021, there were more than 670 GW of solar plants in the nation’s queues; 285 GW (~42%) of this capacity was proposed as a hybrid, most typically pairing PV with battery storage (PV+storage represented nearly 90% of all hybrid capacity in the queues). For wind, 247 GW of capacity sat in the queues, with 19 GW (~8%) proposed as a hybrid, again most-often pairing wind with storage (wind+storage represented ~4% of all hybrid capacity in the queues). Meanwhile, nearly half of all storage in the queues is estimated to be part of a hybrid plant. While many of these proposed plants will not ultimately reach commercial operations, the depth of interest in hybrid plants—especially PV+storage—is notable.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Control of Networked Traffic Flow Distribution - A Stochastic Distribution System Perspective

Networked traffic flow is a common scenario for urban transportation, where the distribution of vehicle queues either at controlled intersections or highway segments reflect the smoothness of the traffic flow in the network. At signalized intersections, the traffic queues are controlled by traffic signal control settings and effective traffic lights control would realize both smooth traffic flow and minimize fuel consumption. Funded by the Energy Efficient Mobility Systems (EEMS) program of the Vehicle Technologies Office of the US Department of Energy, we performed a preliminary investigation on the modelling and control framework in context of urban network of signalized intersections. In specific, we developed a recursive input-output traffic queueing models. The queue formation can be modeled as a stochastic process where the number of vehicles entering each intersection is a random number. Further, we proposed a preliminary B-Spline stochastic model for a one-way single-lane corridor traffic system based on theory of stochastic distribution control.. It has been shown that the developed stochastic model would provide the optimal probability density function (PDF) of the traffic queueing length as a dynamic function of the traffic signal setting parameters. Based upon such a stochastic distribution model, we have proposed a preliminary closed loop framework on stochastic distribution control for the traffic queueing system to make the traffic queueing length PDF follow a target PDF that potentially realizes the smooth traffic flow distribution in a concerned corridor.

Wang, Hong↗

Tandem Predictions for HPC Jobs

At the core of the predictive analytics applied to High Performance Computing (HPC), the most prominent tasks are the prediction of job runtimes and the prediction of job queue times, both of which have the potential for informing HPC users during their every-day decision making. Accurate runtime predictions can help users better choose so-called wallclock times at job submission, decreasing the odds of their jobs waiting in queues longer than necessary. The accurate and timely queue time predictions offered for the available partitions can inform the favorable selection of partitions for running jobs. This potential is well understood as we see in the abundance of research studies that propose solutions for these tasks, including the work published in the last several years. These tasks are seemingly receptive to the Machine Learning (ML) solutions, considering that there is no shortage of training data where HPC centers over time run millions and millions of jobs. However, we study the existing research literature, as well as look for examples in the toolchains supported on the exemplar HPC facilities, and, surprisingly, do not find any practical solutions that are ready to be adopted. We interpret this as a manifestation of the shortage of UX/UI efforts that support HPC analytics and also as a sign that the research has not come to the consensus on solving these tasks. In this study, we aim to shed new light on the long-running task of job queue time prediction by exploring the utility of runtime predictions in improving prediction accuracy and, actually, predicting these two metrics together, in tandem. In other words, we show how runtime predictions become valuable input in the queue time modeling. We challenge the existing approaches to feature engineering for the queue time prediction and describe promising results we obtained for a large dataset of HPC jobs from a supercomputer at the National Renewable Energy Laboratory.

HPC↗

Tandem Predictions for HPC Jobs: Preprint

At the core of the predictive analytics applied to High Performance Computing (HPC), the most prominent tasks are the prediction of job runtimes and the prediction of job queue times, both of which have the potential for informing HPC users during their every-day decision making. Accurate runtime predictions can help users better choose so-called wallclock times at job submission, decreasing the odds of their jobs waiting in queues longer than necessary. The accurate and timely queue time predictions offered for the available partitions can inform the favorable selection of partitions for running jobs. This potential is well understood as we see in the abundance of research studies that propose solutions for these tasks, including the work published in the last several years. These tasks are seemingly receptive to the Machine Learning (ML) solutions, considering that there is no shortage of training data where HPC centers over time run millions and millions of jobs. However, we study the existing research literature, as well as look for examples in the toolchains supported on the exemplar HPC facilities, and, surprisingly, do not find any practical solutions that are ready to be adopted. We interpret this as a manifestation of the shortage of UX/UI efforts that support HPC analytics and also as a sign that the research has not come to the consensus on solving these tasks. In this study, we aim to shed new light on the long-running task of job queue time prediction by exploring the utility of runtime predictions in improving prediction accuracy and, actually, predicting these two metrics together, in tandem. In other words, we show how runtime predictions become valuable input in the queue time modeling. We challenge the existing approaches to feature engineering for the queue time prediction and describe promising results we obtained for a large dataset of HPC jobs from a supercomputer at the National Renewable Energy Laboratory.

97 MATHEMATICS AND COMPUTING↗