Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “BUFFER MANAGER”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Buffer management for sequential decoding

Sequential decoding has been found to be an efficient means of communicating at low undetected error rates from deep space probes, but erasure or computational overflow remains a significant problem. Erasure of a block occurs when the decoder has not finished decoding that block at the time that it must be output. By drawing upon analogies in computer time sharing, this paper develops a buffer-management strategy which reduces the decoder idle time to a negligible level, and therefore improves the erasure probability of a sequential decoder. For a decoder with a speed advantage of ten and a buffer size of ten blocks, operating at an erasure rate of .01, use of this buffer-management strategy reduces the erasure rate to less than .0001.

Layland, J. W.↗

Buffer Management Simulation in ATM Networks

This paper presents a simulation of a new dynamic buffer allocation management scheme in ATM networks. To achieve this objective, an algorithm that detects congestion and updates the dynamic buffer allocation scheme was developed for the OPNET simulation package via the creation of a new ATM module.

simulations Asynchronous Transfer Mode ATM Switch ↗

Buffered freepointer management memory system

A system and method of buffered freepointer management to handle burst traffic to fixed size structures in an external memory system. A circular queue stores implicitly linked free memory locations, along with an explicitly linked list in memory. The queue is updated at the head with newly released locations, and new locations from memory are added at the tail. When a freed location in the queue is reused, external memory need not be updated. When the queue is full, the system attempts to release some of the freepointers such as by dropping them if they are already linked, updating the linked list in memory only if those dropped are not already linked. Latency can be further reduced by loading new locations from memory when the queue is nearly empty, rather than waiting for empty condition, and by writing unlinked locations to memory when the queue is nearly full.

Jacob, Philip↗

An incremental database access method for autonomous interoperable databases

We investigated a number of design and performance issues of interoperable database management systems (DBMS's). The major results of our investigation were obtained in the areas of client-server database architectures for heterogeneous DBMS's, incremental computation models, buffer management techniques, and query optimization. We finished a prototype of an advanced client-server workstation-based DBMS which allows access to multiple heterogeneous commercial DBMS's. Experiments and simulations were then run to compare its performance with the standard client-server architectures. The focus of this research was on adaptive optimization methods of heterogeneous database systems. Adaptive buffer management accounts for the random and object-oriented access methods for which no known characterization of the access patterns exists. Adaptive query optimization means that value distributions and selectives, which play the most significant role in query plan evaluation, are continuously refined to reflect the actual values as opposed to static ones that are computed off-line. Query feedback is a concept that was first introduced to the literature by our group. We employed query feedback for both adaptive buffer management and for computing value distributions and selectivities. For adaptive buffer management, we use the page faults of prior executions to achieve more 'informed' management decisions. For the estimation of the distributions of the selectivities, we use curve-fitting techniques, such as least squares and splines, for regressing on these values.

Roussopoulos, Nicholas↗

Quantifying the Variability of “Fixed-Width” Buffers on Harvested Lands in Western Oregon and Washington

In contemporary forest management, buffers of unharvested trees are left along streams to protect riparian and aquatic ecosystems. Buffer regulations often focus on specific minimum width requirements, which aid in straightforward regulation and application, but minimum widths also suggest buffered edges are uniform and contain little variability. Conceptual papers suggest that alternative buffer configurations may offer greater flexibility in landscape-level protection, increase forest complexity, and enhance aquatic and riparian biodiversity. However, before considering alternatives to fixed-width buffers, it is necessary to quantify the inherent variability in current buffer practices present on the landscape. In this study, we used aerial imagery to quantify variability of buffer widths in two hundred randomly selected recently harvested units on managed land in Oregon and Washington with both fish-bearing and non-fish-bearing sections of stream. Wider buffers on larger streams had a greater magnitude of variability, but when normalized by stream size, variation was greater in smaller streams, and overall, variation ranged from 25% to 50% of the mean width. Despite local variability, buffer widths rarely fell below 9.14 m (30 ft). The variation quantified here provides an initial measure of variability to inform future management, given emerging interest in variable retention buffers.

58 GEOSCIENCES↗

Runtime extension for neural network training with heterogeneous memory

Systems, apparatuses, and methods for managing buffers in a neural network implementation with heterogeneous memory are disclosed. A system includes a neural network coupled to a first memory and a second memory. The first memory is a relatively low-capacity, high-bandwidth memory while the second memory is a relatively high-capacity, low-bandwidth memory. During a forward propagation pass of the neural network, a run-time manager monitors the usage of the buffers for the various layers of the neural network. During a backward propagation pass of the neural network, the run-time manager determines how to move the buffers between the first and second memories based on the monitored buffer usage during the forward propagation pass. As a result, the run-time manager is able to reduce memory access latency for the layers of the neural network during the backward propagation pass.

Mappouras, Georgios↗

Recovery Act: Energy Efficiency of Data Networks through Rate Adaptation (EEDNRA) - Final Technical Report

This Concept Definition Study focused on developing a scientific understanding of methods to reduce energy consumption in data networks using rate adaptation. Rate adaptation is a collection of techniques that reduce energy consumption when traffic is light, and only require full energy when traffic is at full provisioned capacity. Rate adaptation is a very promising technique for saving energy: modern data networks are typically operated at average rates well below capacity, but network equipment has not yet been designed to incorporate rate adaptation. The Study concerns packet-switching equipment, routers and switches; such equipment forms the backbone of the modern Internet. The focus of the study is on algorithms and protocols that can be implemented in software or firmware to exploit hardware power-control mechanisms. Hardware power-control mechanisms are widely used in the computer industry, and are beginning to be available for networking equipment as well. Network equipment has different performance requirements than computer equipment because of the very fast rate of packet arrival; hence novel power-control algorithms are required for networking. This study resulted in five published papers, one internal report, and two patent applications, documented below. The specific technical accomplishments are the following: • A model for the power consumption of switching equipment used in service-provider telecommunication networks as a function of operating state, and measured power-consumption values for typical current equipment. • An algorithm for use in a router that adapts packet processing rate and hence power consumption to traffic load while maintaining performance guarantees on delay and throughput. • An algorithm that performs network-wide traffic routing with the objective of minimizing energy consumption, assuming that routers have less-than-ideal rate adaptivity. • An estimate of the potential energy savings in service-provider networks using feasibly-implementable rate adaptivity. • A buffer-management algorithm that is designed to reduce the size of router buffers, and hence energy consumed. • A packet-scheduling algorithm designed to minimize packet-processing energy requirements. Additional research is recommended in at least two areas: further exploration of rate-adaptation in network switching equipment, including incorporation of rate-adaptation in actual hardware, allowing experimentation in operational networks; and development of control protocols that allow parts of networks to be shut down while minimizing disruption to traffic flow in the network. The research is an integral part of a large effort within Bell Laboratories, Alcatel-Lucent, aimed at dramatic improvements in the energy efficiency of telecommunication networks. This Study did not explicitly consider any commercialization opportunities.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

FTP 100: An Ultra-High Speed Data Transfer Service Over Next Generation 100 Gigabit Per Second Network

Data-intensive applications, including high energy and nuclear physics, astrophysics, climate modeling, nano-scale materials science, genomics, and financing, are expected to generate exabytes of data over the coming years, which must be transferred, visualized, and analyzed by geographically distributed teams of users. High-performance network capabilities must be available to these users at the application level in a transparent, virtualized manner. Moreover, the application users must have the capability to move large datasets from local and remote locations across network environments to their home institutions. To solve these challenges, the main goal of our project is to design and evaluate high-performance data transfer software to support various data-intensive applications. First, we have designed a middleware software that provides access to Remote Direct Memory Access (RDMA) functionalities. This middleware integrates network access, memory management and multitasking in its core design. We address a number of issues related to its efficient implementation, for instance, explicit buffer management and memory registration, and parallelization of RDMA operations, which are vital to delivering the benefit of RDMA to the applications. Built on top of this middleware, an implementation and experimental evaluation of the RDMA-based FTP software, RFTP, is described and evaluated. This application has been implemented by our team to exploit the full capabilities of advanced RDMA mechanisms for ultra-high speed bulk data transfer applications on Energy Sciences Network (ESnet). Second, we designed our data transfer software to optimize TCP/IP based data transfer performance such that RFTP can be fully compatible with today’s Internet. Our kernel optimization techniques with Linux system calls sendfile and splice, can reduce data copy cost. In this report, we summarize the technical challenges of our project, the primary software design methods, the major project milestones achieved, as well as the testbed evaluation work and demonstrations during our project life time.

97 MATHEMATICS AND COMPUTING↗

System and Method for Network Bandwidth, Buffers and Timing Management Using Hybrid Scheduling of Traffic with Different Priorities and Guarantees

Systems and methods for network bandwidth, buffers and timing management using hybrid scheduling of traffic with different priorities and guarantees are provided. In certain embodiments, a method of managing network scheduling and configuration comprises, for each transmitting end station, reserving one exclusive buffer for each virtual link to be transmitted from the transmitting end station; for each receiving end station, reserving exclusive buffers for each virtual link to be received at the receiving end station; and for each switch, reserving a exclusive buffer for each virtual link to be received at an input port of the switch. The method further comprises determining if each respective transmitting end station, receiving end station, and switch has sufficient capability to support the reserved buffers; and reporting buffer infeasibility if each respective transmitting end station, receiving end station, and switch does not have sufficient capability to support the reserved buffers.

Varadarajan, Srivatsan↗

Quality of Service for Real-Time Applications Over Next Generation Data Networks

This project, which started on January 1, 2000, was funded by NASA Glenn Research Center for duration of one year. The deliverables of the project included the following tasks: Study of QoS mapping between the edge and core networks envisioned in the Next Generation networks will provide us with the QoS guarantees that can be obtained from next generation networks. Buffer management techniques to provide strict guarantees to real-time end-to-end applications through preferential treatment to packets belonging to real-time applications. In particular, use of ECN to help reduce the loss on high bandwidth-delay product satellite networks needs to be studied. Effect of Prioritized Packet Discard to increase goodput of the network and reduce the buffering requirements in the ATM switches. Provision of new IP circuit emulation services over Satellite IP backbones using MPLS will be studied. Determine the architecture and requirements for internetworking ATN and the Next Generation Internet for real-time applications.

Ivancic, William↗

Quality of Service for Real-Time Applications Over Next Generation Data Networks

This project, which started on January 1, 2000, was funded by the NASA Glenn Research Center for duration of one year. The deliverables of the project included the following tasks: (1) Study of QoS mapping between the edge and core networks envisioned in the Next Generation networks will provide us with the QoS guarantees that can be obtained from next generation networks; (2) Buffer management techniques to provide strict guarantees to real-time end-to-end applications through preferential treatment to packets belonging to real-time applications. In particular, use of ECN to help reduce the loss on high bandwidth-delay product satellite networks needs to be studied; (3) Effect of Prioritized Packet Discard to increase goodput of the network and reduce the buffering requirements in the ATM switches; (4) Provision of new IP circuit emulation services over Satellite IP backbones using MPLS will be studied; and (5) Determine the architecture and requirements for internetworking ATN and the Next Generation Internet for real-time applications. The project has been completed on time. All the objectives and deliverables of the project have been completed. Research results obtained from this project have been published in a number of papers in journals, conferences, and technical reports, included in this document.

Atiquzzaman, Mohammed↗

Accelerating k-NN Algorithm with Hybrid MPI and OpenSHMEM

Machine Learning algorithms are benefiting from the continuous improvement of programming models, including MPI, MapReduce and PGAS. k-Nearest Neighbors (k-NN) algorithm is a widely used machine learning algorithm, applied to supervised learning tasks such as classification. Several parallel implementations of k-NN have been proposed in the literature and practice. However, on high-performance computing systems with high-speed interconnects, it is important to further accelerate existing designs of the k-NN algorithm through taking advantage of scalable programming models. To improve the performance of k-NN on large-scale environment with InfiniBand network, this paper proposes several alternative hybrid MPI+OpenSHMEM designs and performs a systemic evaluation and analysis on typical workloads. The hybrid designs leverage the one-sided memory access to better overlap communication with computation than the existing pure MPI design, and propose better schemes for efficient buffer management. The implementation based on k-NN program from MaTEx with MVAPICH2-X (Unified MPI+PGAS Communication Runtime over InfiniBand) shows up to 9.0% time reduction for training KDD Cup 2010 workload over 512 cores, and 27.6% time reduction for small workload with balanced communication and computation. Experiments of running with varied number of cores show that our design can maintain good scalability.

Lin, Jian↗

Design and implementation of interface units for high speed fiber optics local area networks and broadband integrated services digital networks

The design and implementation of interface units for high speed Fiber Optic Local Area Networks and Broadband Integrated Services Digital Networks are discussed. During the last years, a number of network adapters that are designed to support high speed communications have emerged. This approach to the design of a high speed network interface unit was to implement package processing functions in hardware, using VLSI technology. The VLSI hardware implementation of a buffer management unit, which is required in such architectures, is described.

Tobagi, Fouad A.↗

The TurboLAN project. Phase 1: Protocol choices for high speed local area networks. Phase 2: TurboLAN Intelligent Network Adapter Card, (TINAC) architecture

The hardware and the software architecture of the TurboLAN Intelligent Network Adapter Card (TINAC) are described. A high level as well as detailed treatment of the workings of various components of the TINAC are presented. The TINAC is divided into the following four major functional units: (1) the network access unit (NAU); (2) the buffer management unit; (3) the host interface unit; and (4) the node processor unit.

Alkhatib, Hasan S.↗

System and method for implementing periodic early discard in on-chip buffer memories of network elements

An advance is made over the prior art in accordance with the principles of the present invention that is directed to a new approach for a system and method for a buffer management scheme called Periodic Early Discard (PED). The invention builds on the observation that, in presence of TCP traffic, the length of a queue can be stabilized by selection of an appropriate frequency for packet dropping. For any combination of number of TCP connections and distribution of the respective RTT values, there exists an ideal packet drop frequency that prevents the queue from over-flowing or under-flowing. While the value of the ideal packet drop frequency may quickly change over time and is sensitive to the series of TCP connections affected by past packet losses, and most of all is impossible to compute inline, it is possible to approximate it with a margin of error that allows keeping the queue occupancy within a pre-defined range for extended periods of time. The PED scheme aims at tracking the (unknown) ideal packet drop frequency, adjusting the approximated value based on the evolution of the queue occupancy, with corrections of the approximated packet drop frequency that occur at a timescale that is comparable to the aggregate time constant of the set of TCP connections that traverse the queue.

97 MATHEMATICS AND COMPUTING↗

End-System Aware Peak Link Utilization Transport for Big Data Transfer over Dedicated Connections

High-performance networks provisioning 10/40-Gbps dedicated channels have been rapidly developed andincreasingly deployed to support big data transfers on the order of terabytes or petabytes in large-scale network-intensiveapplications. However, end users have not seen corresponding increase in application throughput mainly because traditional end-to-end transport methods are not optimized for such connections. New congestion or flow control mechanisms are desirable to meet the challenges brought to transport protocol design by dedicated connections. The advent and proliferation of multi-core processors make it now possible to improve application throughput by providing multiple processing and network resources to a single data transfer. Based on the existing PLUT method, we proposed a new dynamic buffer management scheme and an automatic parallelism tuning mechanism to achieve the maximal bottleneck rate. The performance superiority of PLUT is justified by the experimental results collected over a local 10Gbps dedicated connection.

Rao, Nageswara↗

Toward Managing HPC Burst Buffers Effectively: Draining Strategy to Regulate Bursty I/O Behavior

HPC (high-performance computing) applications usually show bursty I/O behaviors. In order to expedite the applications, permanent storage systems are usually provisioned to serve such I/O bursts. Approaching the era of exascale computing, non-volatile RAM is introduced as burst buffers, to absorb the bursty bulk data and relax the I/O provisioning requirement of the permanent storage systems. However, without judiciously draining the burst buffers, I/O bursts are passed down to the underlying storage systems, which causes severe I/O contention issues.In order to minimize the I/O provisioning requirement and resolve the issues caused by I/O bursts, we propose a proactive draining scheme to manage the draining process of distributed node-local burst buffers. In addition, we develop an I/O provisioning model to predict the minimized I/O provisioning requirement for permanent storage systems. Evaluation results show that applying the proactive draining scheme largely relaxes the I/O provisioning requirement while preserving the I/O performance of underlying storage systems.

97 MATHEMATICS AND COMPUTING↗