Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “QoS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Can I/O Variability Be Reduced on QoS-Less HPC Storage Systems?

For a production high-performance computing (HPC) system, where storage devices are shared between multiple applications and managed in a best effort manner, I/O contention is often a major problem. In this paper, we propose a balanced messaging-based re-routing in conjunction with throttling at the middleware level. This work tackles two key challenges that have not been fully resolved in the past: whether I/O variability can be reduced on a QoS-less HPC storage system, and how to design a runtime scheduling system that can scale up to a large amount of cores. The proposed scheme uses a two-level messaging system to re-route I/O requests to a less congested storage location so that write performance is improved, while limiting the impact on read by throttling re-routing. An analytical model is derived to guide the setup of optimal throttling factor. We thoroughly analyze the virtual messaging layer overhead and explore whether the in-transit buffering is effective in managing I/O variability. Contrary to the intuition, in-transit buffer cannot completely solve the problem. It can reduce the absolute variability but not the relative variability. The proposed scheme is verified against a synthetic benchmark as well as being used by production applications.

Huang, Dan↗

QoS aware balancing in data centers

One embodiment of the present invention provides a system for balancing loads between two physical resources. During operation, for each physical resource, the system establishes resource-usage models for a plurality of jobs that share the physical resource, identifies quality-of-service (QoS) levels associated with the jobs, and computes a QoS balance point for the physical resource based on the resource-usage models and the QoS ratings. The QoS balance point indicates a rating of QoS, above which substantial resource provisioning is required. In response to detecting a difference between a first QoS balance point associated with a first physical resource and a second QoS balance point associated with a second physical resource, the system identifies one or more jobs to be moved between the first and second physical resources such that the difference between the first and the second QoS balance points is reduced.

Greene, Daniel H.↗

QoS support for end users of I/O-intensive applications using shared storage systems

I/O-intensive applications are becoming increasingly common on today's high-performance computing systems. While performance of compute-bound applications can be effectively guaranteed with techniques such as space sharing or QoS-aware process scheduling, it remains a challenge to meet QoS requirements for end users of I/O-intensive applications using shared storage systems because it is difficult to differentiate I/O services for different applications with individual quality requirements. Furthermore, it is difficult for end users to accurately specify performance goals to the storage system using I/O-related metrics such as request latency or throughput. As access patterns, request rates, and the system workload change in time, a fixed I/O performance goal, such as bounds on throughput or latency, can be expensive to achieve and may not lead to a meaningful performance guarantees such as bounded program execution time. We propose a scheme supporting end-users QoS goals, specified in terms of program execution time, in shared storage environments. We automatically translate the users performance goals into instantaneous I/O throughput bounds using a machine learning technique, and use dynamically determined service time windows to efficiently meet the throughput bounds. We have implemented this scheme in the PVFS2 parallel file system and have conducted an extensive evaluation. Our results show that this scheme can satisfy realistic end-user QoS requirements by making highly efficient use of the I/O resources. The scheme seeks to balance programs attainment of QoS requirements, and saves as much of the remaining I/O capacity as possible for best-effort programs.

97 MATHEMATICS AND COMPUTING↗

QoS-aware edge AI placement and scheduling with multiple implementations in FaaS-based edge computing

Resource constraints on the computing continuum require that we make smart decisions for serving AI-based services at the network edge. AI-based services typically have multiple implementations (e.g., image classification implementations include SqueezeNet, DenseNet, and others) with varying trade-offs (e.g., latency and accuracy). The question then is how should AI-based services be placed across Function-as-a-Service (FaaS) based edge computing systems in order to maximize total Quality-of-Service (QoS). To address this question, we propose a problem that jointly aims to solve (i) edge AI service placement and (ii) request scheduling. These are done across two time-scales (one for placement and one for scheduling). Here we first cast the problem as an integer linear program. We then decompose the problem into separate placement and scheduling subproblems and prove that both are NP-hard. We then propose a novel placement algorithm that places services while considering device-to-device communication across edge clouds to offload requests to one another. Our results show that the proposed placement algorithm is able to outperform a state-of-the-art placement algorithm for AI-based services, and other baseline heuristics, with regard to maximizing total QoS. Additionally, we present a federated learning-based framework, FLIES, to predict the future incoming service requests and their QoS requirements. Our results also show that our FLIES algorithm is able to outperform a standard decentralized learning baseline for predicting incoming requests and show comparable predictive performance when compared to centralized training.

97 MATHEMATICS AND COMPUTING↗

Asynchronous Object Storage with QoS for Scientific and Commercial Big Data

This paper presents our design for an asynchronous object storage system intended for use in scientific and commercial big data workloads. Use cases from the target workload do- mains are used to motivate the key abstractions used in the application programming interface (API). The architecture of the Scalable Object Store (SOS), a prototype object stor- age system that supports the API s facilities, is presented. The SOS serves as a vehicle for future research into scalable and resilient big data object storage. We briefly review our research into providing efficient storage servers capable of providing quality of service (QoS) contracts relevant for big data use cases.

Cloud Storage↗

Adaptive QoS for Data Transfers using Software-Defined Networking

Software-defined networking (SDN) has demonstrated tremendous potential for altering the networking landscape in both academia andindustry. While traffic engineering as network intelligence in classic technologies have received considerable attention from the researchcommunity, traffic engineering in the SDN space, particularly in regards to quality of service (QoS), remains relatively unexplored.While recent efforts have begun to show how SDN might be leverage to meet the complex traffic-shaping needs of modern networks, there is still much that can be done to illustrate its potential, and in particular how it could prevent degradation of service for all QoStraffic. We propose methods to allow other flows to use bandwidth unused by high-priority transfers, while ensuring that bandwidthassigned to those high-priority transfers is available when needed. We present fine-tuned control over bandwidth allocation that achieves an efficient combination of both the differentiated service (DiffServ)-style classification and the integrated service (IntServ)-style perflowconsiderations. We also show how this method may be used over a wide area network (WAN) to optimally adjust bandwidth allocations when high-priority data transfers are not using their allocated bandwidth.

Boley, Josh↗

QoS-Enabled, High-performance Storage Systems for Data-Intensive Scientific Computing (Final Report)

This project's work resulted in the following research projects: (1) BORG - Block-reORGanization for Self-optimizing Storage Systems; (2) ABLE - Active Block Layer Extensions; (3) EXCES - EXternal Caching in Energy-Saving Storage Systems; and (4) GRIO - Guaranteed-Rate I/O Scheduler. These projects together help in substantially advancing the over-arching project goal of developing "QoS-Enabled, High-Performance Storage Systems".

99 GENERAL AND MISCELLANEOUS↗

End-to-End Network QoS via Scheduling of Flexible Resource Reservation Requests

Modern data-intensive applications move vast amounts of data between multiple locations around the world. To enable predictable and reliable data transfer, next generation networks allow such applications to reserve network resources for exclusive use. In this paper, we solve an important problem (called SMR3) to accommodate multiple and concurrent network reservation requests between a pair of end-sites. Given the varying availability of bandwidth within the network, our goal is to accommodate as many reservation requests as possible while minimizing the total time needed to complete the data transfers. We first prove that SMR3 is an NP-hard problem. Then we solve it by developing a polynomial-time heuristic, called RRA. The RRA algorithm hinges on an efficient mechanism to accommodate large number of requests by minimizing the bandwidth wastage. Finally, via numerical results, we show that RRA constructs schedules that accommodate significantly larger number of requests compared to other, seemingly efficient, heuristics.

97 MATHEMATICS AND COMPUTING↗

An Autonomic Middleware Solution for Coordinating Multiple QoS Controls

Adaptive self-managing applications can adapt their behavior through components that monitor the application behavior and provide feedback controls. This paper outlines an autonomic approach to coordinate multiple controls for managing service quality using executable control models. In this approach, controls are modeled as process models. Moreover, controls with cross-cutting concerns are provisioned by a dedicated process model. The flexibility of this approach allows composing new controls from existing control components. The coordination of their dependencies is performed within a unified architecture framework for modeling, deploying and executing these models. We integrate the process modeling and execution techniques into a middleware architecture to deliver such features. To demonstrate the practical utilization of this approach, we employ it to manage fail-over and over-loading controls for a service oriented loan brokering application. The empirical results further validate that this solution is not only sensitive to resolving cross-cutting interests of multiple controls, but also lightweight as it incurs low computational overhead.

99 GENERAL AND MISCELLANEOUS↗

Systems and methods for quality of service (QoS) based management of bottlenecks and flows in networks

Techniques based on the Theory of Bottleneck Ordering can reveal the bottleneck structure of a network, and the Theory of Flow ordering can take advantage of the revealed bottleneck structure to manage and configure network flows so as to improve the overall network performance. These two techniques provide insights into the inherent topological properties of a network at least in three areas: (1) identification of the regions of influence of each bottleneck; (2) the order in which bottlenecks (and flows traversing them) may converge to their steady state transmission rates in distributed congestion control algorithms; and (3) the design of optimized traffic engineering policies.

97 MATHEMATICS AND COMPUTING↗

Large oscillatory thermal hall effect in kagome metals

The thermal Hall effect recently provided intriguing probes to the ground state of exotic quantum matters. These observations of transverse thermal Hall signals lead to the debate on the fermionic versus bosonic origins of these phenomena. The recent report of quantum oscillations (QOs) in Kitaev spin liquid points to a possible resolution. The Landau level quantization would most likely capture only the fermionic thermal transport effect. However, the QOs in the thermal Hall effect are generally hard to detect. In this work, we report the observation of a large oscillatory thermal Hall effect of correlated Kagome metals. We detect a 180-degree phase change of the oscillation and demonstrate the phase flip as an essential feature for QOs in the thermal transport properties. More importantly, the QOs in the thermal Hall channel are more profound than those in the electrical Hall channel, which strongly violates the Wiedemann–Franz (WF) law for QOs. This result presents the oscillatory thermal Hall effect as a powerful probe to the correlated quantum materials.

36 MATERIALS SCIENCE↗

dCache: from Resilience to Quality of Service

A major goal of future dCache development will be to allow users to define file Quality of Service (QoS) in a more flexible way than currently available. This will mean implementing what might be called a QoS rule engine responsible for registering and managing time-bound QoS transitions for files or storage units. In anticipation of this extension to existing dCache capabilities, the Resilience service, which maintains on-disk replica state, needs to undergo both structural modification and generalization. This paper describes ongoing work to transform Resilience into the new architecture which will eventually support a more broadly defined file QoS.

97 MATHEMATICS AND COMPUTING↗

Interpretable Models for Workflow Differentiation in High-Performance Scientific Networks

Scientific workflows in high-performance networks spawn hundreds of interdependent flows that must be managed collectively—yet existing network classifiers treat each flow in isolation, leading to fragmented QoS decisions and missed interflow patterns. We present a novel traffic classification solution that operates at the workflow level, distinguishing entire filetransfer operations from streaming analytics by capturing how concurrent flows interact and burst together. We introduce a workflow identification window (WIW) that ingests raw packet headers from parallel flows into unified tensors, preserving the spatial-temporal patterns that differentiate scientific workflows. This approach achieves 98.7% accuracy using CNN, LSTM, and hybrid architectures, while maintaining 84% accuracy on production traffic collected a week later—demonstrating robustness to temporal drift. By integrating SHAP and GradCAM explainability, we reveal that early-packet timing patterns and cross-flow correlations drive classification decisions, providing operators with interpretable insights. Our system enables coherent workflow-level QoS enforcement and dynamic bandwidth allocation in scientific networks, eliminating manual per-flow configuration while maintaining classification latency at millisecond level.

Giannakou, Anna [LBL, Berkeley]↗