Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “bandwidth reservation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

42 records · Page 3

GPU Direct I/O with HDF5

Exascale HPC systems are being designed with accelerators, such as GPUs, to accelerate parts of applications. In machine learning workloads as well as large-scale simulations that use GPUs as accelerators, the CPU (or host) memory is currently used as a buffer for data transfers between GPU (or device) memory and the file system. If the CPU does not need to operate on the data, then this is sub-optimal because it wastes host memory by reserving space for duplicated data. Furthermore, this “bounce buffer” approach wastes CPU cycles spent on transferring data. A new technique, NVIDIA GPUDirect Storage (GDS), can eliminate the need to use the host memory as a bounce buffer. Thereby, it becomes possible to transfer data directly between the device memory and the file system. This direct data path shortens latency by omitting the extra copy and enables higher-bandwidth. To take full advantage of GDS in existing applications, it is necessary to provide support with existing I/O libraries, such as HDF5 and MPI-IO, which are heavily used in applications. In this paper, we describe our effort of integrating GDS with HDF5, the top I/O library at NERSC and at DOE leadership computing facilities. We design and implement this integration using a HDF5 Virtual File Driver (VFD). The GDS VFD provides a file system abstraction to the application that allows HDF5 applications to perform I/O without the need to move data between CPUs and GPUs explicitly. We compare performance of the HDF5 GDS VFD with explicit data movement approaches and demonstrate superior performance with the GDS method.

Ravi, J↗

TURBO: Terabits/s Using Reconfigurable Bandwidth Optics (Final Report)

Large scale, big science applications are generating petabytes of data that need to be shared among multiple locations. Moving a petabyte of data at 100 Gb/s takes 22 hours. Reducing this to single digit hours or below is clearly attractive and with future applications anticipating exabyte scale transfers, multi-terabit per second capacities are needed. This scale of capacity is available using optical systems. Core optical systems have aggregate per fiber capacities on the order of 10 Tb/s and more than 100 Tb/s system experiments have been realized in the lab. But large optical capacities are not just reserved for the core as similar system capacities are available in metro networks right up to the enterprise, campus, or data center. In general, the same technology is used today in the metro area as in the long-haul networks—reconfigurable optical add drop multiplexing (ROADM) node-based wavelength division multiplexing (WDM) systems. Data centers already support massive internal capacities on the scale of 100’s of Tb/s. As new photonic integrated technologies mature, for example through initiatives such as the Integrated Photonics Innovative Manufacturing Institute (IP-IMI), the cost of optical interfaces is expected to decrease, and higher capacity and higher performance interfaces will become more affordable. This set of circumstances creates the potential that Tb/s capacities will be available at the enterprise and campus level to support large scale science network applications.

42 ENGINEERING↗

An Internal ALD-Based High Voltage Divider and Signal Circuit for MCP-based Photodetectors

We describe a pin-less design for the high voltage (HV) resistive divider of the all-glass LAPPD (TM) 8 in,square thin photodetector module. The divider, which distributes high voltage applied to the photocathode to the two micro-channel plates (MCPs) that constitute the amplification stage, is comprised of the two MCPs and three glass mechanical spacers, each of which is coated with a resistive layer using atomic layer deposition (ALD). The three glass grid spacers and the two MCPs form a continuous resistive path between cathode and anode, with the voltages across the MCPs and the spacers determined by the resistance of each. High voltage is applied on an external tab on the top glass window that connects to the photocathode through the metal seal. The DC ground is supplied by microstrips on the bottom glass plate that form the high-bandwidth anode. The microstrips exit the package through the glass-frit seal of the anode base-plate and the package sidewall. The divider is thus completely internal, with no HV pins penetrating the low-profile flat glass package. Measurements of the performance of the divider are presented for the 8 in.-square MCP and spacer package in a custom test fixture and for an assembled externally pumped LAPPD (TM) prototype with an aluminum photocathode. (C) 2015 Elsevier B.V. All rights reserved.

Adams, Bernhard W.↗

LambdaStation: Exploiting Advance Networks In Data Intensive High Energy Physics Applications

Lambda Station software implements selective, dynamic, secure path control between local storage & analysis facilities, and high bandwidth, wide-area networks (WANs). It is intended to facilitate use of desirable, alternate wide area network paths which may only be intermittently available, or subject to policies that restrict usage to specified traffic. Lambda Station clients gain awareness of potential alternate network paths via Clarens-based web services, including path characteristics such as bandwidth and availability. If alternate path setup is requested and granted, Lambda Station will configure the local network infrastructure to properly forward designated data flows via the alternate path. A fully functional implementation of Lambda Station, capable of dynamic alternate WAN path setup and teardown, has been successfully developed. A limited Lambda Station-awareness capability within the Storage Resource Manager (SRM) product has been developed. Lambda Station has been successfully tested in a number of venues, including Super Computing 2008. LambdaStation software, developed by the Fermilab team, enables dynamic allocation of alternate network paths for high impact traffic and to forward designated flows across LAN. It negotiates with reservation and provisioning systems of WAN control planes, be it based on SONET channels, demand tunnels, or dynamic circuit networks. It creates End-To-End circuit between single hosts, computer farms or networks with predictable performance characteristics, preserving QoS if supported in LAN and WAN and tied security policy allowing only specific traffic to be forwarded or received through created path. Lambda Station project also explores Network Awareness capabilities.

97 MATHEMATICS AND COMPUTING↗

Analysis of adaptive algorithms for an integrated communication network

Techniques were examined that trade communication bandwidth for decreased transmission delays. When the network is lightly used, these schemes attempt to use additional network resources to decrease communication delays. As the network utilization rises, the schemes degrade gracefully, still providing service but with minimal use of the network. Because the schemes use a combination of circuit and packet switching, they should respond to variations in the types and amounts of network traffic. Also, a combination of circuit and packet switching to support the widely varying traffic demands imposed on an integrated network was investigated. The packet switched component is best suited to bursty traffic where some delays in delivery are acceptable. The circuit switched component is reserved for traffic that must meet real time constraints. Selected packet routing algorithms that might be used in an integrated network were simulated. An integrated traffic places widely varying workload demands on a network. Adaptive algorithms were identified, ones that respond to both the transient and evolutionary changes that arise in integrated networks. A new algorithm was developed, hybrid weighted routing, that adapts to workload changes.

Reed, Daniel A.↗

On-demand Overlay Networks for Large Scientific Data Transfers

Large scale scientific data transfers are central to scientific processes. Data from large experimental facilities have to be moved to local institutions for analysis or often data needs to be moved between local clusters and large supercomputing centers. In this paper, we propose and evaluate a network overlay architecture to enable highthroughput, on-demand, coordinated data transfers over wide-area networks. Our work leverages Phoebus and On-demand Secure Circuits and AdvanceReservation System (OSCARS) to provide high performance wide-area network connections. OSCARS enables dynamic provisioning of network paths with guaranteed bandwidth and Phoebus enables the coordination and effective utilization of the OSCARS network paths. Our evaluation shows that this approach leads to improved end-to-end data transfer throughput with minimal overheads. The achievedthroughput using our overlay was limited only by the ability of the end hosts to sink the data.

97 MATHEMATICS AND COMPUTING↗