Engineering Papers⌕ Search

Engineering topics

Ros-Giralt, Jordi

Publications and source records attributed to Ros-Giralt, Jordi.

Systems and methods for detecting large network flows

In a system for efficiently detecting large/elephant flows in a network, the rate at which the received packets are sampled is adjusted according to the measured heavy tailedness of the arriving traffic, such that the measured heavy tailedness reaches a specified target level. The heavy tailedness is measured using the estimated sizes of different flows associated with the arriving packets. When the measured heavy tailedness reaches and remains at the specified target level, the flows having the largest estimated sizes are likely to be the largest/elephant flows in the network.

Ros-Giralt, Jordi↗

Systems and methods for multiresolution priority queues

A system for storing and extracting elements according to their priority takes into account not only the priorities of the elements but also three additional parameters, namely, a priority resolution p Δ and two priority limits p min and p max . By allowing an ordering error if the difference in the priorities of elements are within the priority resolution, an improvement in performance is achieved.

Ros-Giralt, Jordi↗

Systems and methods for providing lockless bimodal queues for selective packet capture

In a network system, an application receiving packets can consume one or more packets in two or more stages, where the second and the later stages can selectively consume some but not all of the packets consumed by the preceding stage. Packets are transferred between two consecutive stages, called producer and consumer, via a fixed-size storage. Both the producer and the consumer can access the storage without locking it and, to facilitate selective consumption of the packets by the consumer, the consumer can transition between awake and sleep modes, where the packets are consumed in the awake mode only. The producer may also switch between awake and sleep modes. Lockless access is made possible by controlling the operation of the storage by the producer and the consumer both according to the mode of the consumer, which is communicated via a shared memory location.

Ros-Giralt, Jordi↗

Manipulating bottlenecks in systems modeled as networks

A technique is described for quantifying the effect of a perturbation on a property of a network link or flow, and to add a new flow, reroute an existing flow, or to reshape a flow, based on a quantitative estimation of the perturbation.

Ros-Giralt, Jordi↗

Systems and methods for quality of service (QoS) based management of bottlenecks and flows in networks

Techniques based on the Theory of Bottleneck Ordering can reveal the bottleneck structure of a network, and the Theory of Flow ordering can take advantage of the revealed bottleneck structure to manage and configure network flows so as to improve the overall network performance. These two techniques provide insights into the inherent topological properties of a network at least in three areas: (1) identification of the regions of influence of each bottleneck; (2) the order in which bottlenecks (and flows traversing them) may converge to their steady state transmission rates in distributed congestion control algorithms; and (3) the design of optimized traffic engineering policies.

97 MATHEMATICS AND COMPUTING↗

GradientGraph

Under this SBIR Phase II, Reservoir Labs has developed G2 Analytics, a new technology that allows network operators to analyze bottleneck and flow performance with high precision. G2 delivers a new analytical approach and framework to resolve a variety of key problems found in modern communication networks, including: traffic engineering, routing, flow scheduling, network design, capacity planning, resiliency analysis, network slicing, or service level agreement (SLA) management, among others. G2 leverages the bottleneck structure of congestion-controlled communication networks, a recent mathematical discovery by the Reservoir team [RL19b, RL20a, RL20b, RL21a]. Bottleneck structures reveal how perturbations on flows and links propagate through the network, providing an analytical framework to measure (qualitatively and quantitatively) the ripple effects induced as they traverse the network. Leveraging the mathematics of bottleneck structures, Reservoir Labs is developing the G2 technology to provide network operators with a framework to design, optimize and troubleshoot network performance. This delivery includes the G2 software stack.

Yellamraju, Sruthi↗

Systems and methods for fast detection of elephant flows in network traffic

In a system for efficiently detecting large/elephant flows in a network, the rate at which the received packets are sampled is adjusted according to a top flow detection likelihood computed for a cache of flows identified in the arriving network traffic. After observing packets sampled from the network, Dirichlet-Categorical inference is employed to calculate a posterior distribution that captures uncertainty about the sizes of each flow, yielding a top flow detection likelihood. The posterior distribution is used to find the most likely subset of elephant flows. The technique rapidly converges to the optimal sampling rate at a speed O(1/n), where n is the number of packet samples received, and the only hyperparameter required is the targeted detection likelihood.

Gudibanda, Aditya↗

Computing Bottleneck Structures at Scale for High-Precision Network Performance Analysis

The Theory of Bottleneck Structures is a recently-developed framework for studying the performance of data networks. It describes how local perturbations in one part of the network propagate and interact with others. This framework is a powerful analytical tool that allows network operators to make accurate predictions about network behavior and thereby optimize performance. Previous work implemented a software package for bottleneck structure analysis, but applied it only to toy examples. In this work, we introduce the first software package capable of scaling bottleneck structure analysis to production-size networks. Here, we benchmark our system using logs from ESnet, the Department of Energy's high-performance data network that connects research institutions in the U.S. Using the previously published tool as a baseline, we demonstrate that our system achieves vastly improved performance, constructing the bottleneck structure graphs in 0.21 s and calculating link derivatives in 0.09 s on average. We also study the asymptotic complexity of our core algorithms, demonstrating good scaling properties and strong agreement with theoretical bounds. These results indicate that our new software package can maintain its fast performance when applied to even larger networks. They also show that our software is efficient enough to analyze rapidly changing networks in real time. Overall, we demonstrate the feasibility of applying bottleneck structure analysis to solve practical problems in large, real-world data networks.

benchmark↗

Algorithms and data structures to accelerate network analysis

As the sheer amount of computer generated data continues to grow exponentially, new bottlenecks are unveiled that require rethinking our traditional software and hardware architectures. Here in this paper we present five algorithms and data structures (long queue emulation, lockless bimodal queues, tail early dropping, LFN tables, and multiresolution priority queues) designed to optimize the process of analyzing network traffic. We integrated these optimizations on R-Scope, a high performance network appliance that runs the Bro network analyzer, and present benchmarks showcasing performance speed ups of 5X at traffic rates of 10 Gbps.

97 MATHEMATICS AND COMPUTING↗

Efficient packet forwarding using cyber-security aware policies

For balancing load, a forwarder can selectively direct data from the forwarder to a processor according to a loading parameter. The selective direction includes forwarding the data to the processor for processing, transforming and/or forwarding the data to another node, and dropping the data. The forwarder can also adjust the loading parameter based on, at least in part, feedback received from the processor. One or more processing elements can store values associated with one or more flows into a structure without locking the structure. The stored values can be used to determine how to direct the flows, e.g., whether to process a flow or to drop it. The structure can be used within an information channel providing feedback to a processor.

97 MATHEMATICS AND COMPUTING↗

Efficient packet forwarding using cyber-security aware policies

For balancing load, a forwarder can selectively direct data from the forwarder to a processor according to a loading parameter. The selective direction includes forwarding the data to the processor for processing, transforming and/or forwarding the data to another node, and dropping the data. The forwarder can also adjust the loading parameter based on, at least in part, feedback received from the processor. One or more processing elements can store values associated with one or more flows into a structure without locking the structure. The stored values can be used to determine how to direct the flows, e.g., whether to process a flow or to drop it. The structure can be used within an information channel providing feedback to a processor.

97 MATHEMATICS AND COMPUTING↗

System, apparatus and methods to implement high-speed network analyzers

Systems, apparatus and methods for the implementation of high-speed network analyzers are provided. A set of high-level specifications is used to define the behavior of the network analyzer emitted by a compiler. An optimized inline workflow to process regular expressions is presented without sacrificing the semantic capabilities of the processing engine. An optimized packet dispatcher implements a subset of the functions implemented by the network analyzer, providing a fast and slow path workflow used to accelerate specific processing units. Such dispatcher facility can also be used as a cache of policies, wherein if a policy is found, then packet manipulations associated with the policy can be quickly performed. An optimized method of generating DFA specifications for network signatures is also presented. The method accepts several optimization criteria, such as min-max allocations or optimal allocations based on the probability of occurrence of each signature input bit.

42 ENGINEERING↗