Using End-to-End Bandwidth Estimates for Anomaly Detection beyond Enterprise Boundaries
No abstract prepared.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
No abstract prepared.
We are witnessing a new era that offers new opportunities to conduct scientific research with the help of recent advancements in computational and storage technologies. Computational intensive science spans multiple scientific domains, such as particle physics, climate modeling, and bio-informatics simulations. These large-scale applications necessitate collaborators to access very large data sets resulting from simulations performed in geographically distributed institutions. Furthermore, often scientific experimental facilities generate massive data sets that need to be transferred to validate the simulation data in remote collaborating sites. A major component needed to support these needs is the communication infrastructure which enables high performance visualization, large volume data analysis, and also provides access to computational resources. In order to provide high-speed on-demand data access between collaborating institutions, national governments support next generation research networks such as Internet 2 and ESnet (Energy Sciences Network). Delivering network-as-a-service that provides predictable performance, efficient resource utilization and better coordination between compute and storage resources is highly desirable. In this paper, we study network provisioning and advanced bandwidth reservation in ESnet for on-demand high performance data transfers. We present a novel approach for path finding in time-dependent transport networks with bandwidth guarantees. We plan to improve the current ESnet advance network reservation system, OSCARS [3], by presenting to the clients, the possible reservation options and alternatives for earliest completion time and shortest transfer duration. The Energy Sciences Network (ESnet) provides high bandwidth connections between research laboratories and academic institutions for data sharing and video/voice communication. The ESnet On-Demand Secure Circuits and Advance Reservation System (OSCARS) establishes guaranteed bandwidth of secure virtual circuits at a certain time, for a certain bandwidth and length of time. Though OSCARS operates within the ESnet, it also supplies end-to-end provisioning between multiple autonomous network domains. OSCARS gets reservation requests through a standard web service interface, and conducts a Quality-of-service (QoS) path for bandwidth guarantees. Multi-protocol Label Switching (MPLS) and the Resource Reservation Protocol (RSVP) enable to create a virtual circuit using Label Switched Paths (LSP's). It contains three main components: a reservation manager, a bandwidth scheduler, and a path setup subsystem. The bandwidth scheduler needs to have information about the current and future states of the network topology in order to accomplish end-to-end bandwidth guaranteed paths.
The high-performance networking requirements for next generation large-scale applications belong to two broad classes: (a) high bandwidths, typically multiples of 10Gbps, to support bulk data transfers, and (b) stable bandwidths, typically at much lower bandwidths, to support computational steering, remote visualization, and remote control of instrumentation. Current Internet technologies, however, are severely limited in meeting these demands because such bulk bandwidths are available only in the backbone, and stable control channels are hard to realize over shared connections. The UltraScience Net (USN) facilitates the development of such technologies by providing dynamic, cross-country dedicated 10Gbps channels for large data transfers, and 150 Mbps channels for interactive and control operations. Contributions of the USN project are two-fold: (a) Infrastructure Technologies for Network Experimental Facility: USN developed and/or demonstrated a number of infrastructure technologies needed for a national-scale network experimental facility. Compared to Internet, USN's data-plane is different in that it can be partitioned into isolated layer-1 or layer-2 connections, and its control-plane is different in the ability of users and applications to setup and tear down channels as needed. Its design required several new components including a Virtual Private Network infrastructure, a bandwidth and channel scheduler, and a dynamic signaling daemon. The control-plane employs a centralized scheduler to compute the channel allocations and a signaling daemon to generate configuration signals to switches. In a nutshell, USN demonstrated the ability to build and operate a stable national-scale switched network. (b) Structured Network Research Experiments: A number of network research experiments have been conducted on USN that cannot be easily supported over existing network facilities, including test-beds and production networks. It settled an open matter by demonstrating that the performance of switched connections and Multiple Protocol Label Switching tunnels over routed networks are comparable. Furthermore, such connections can be easily peered, and the performance of the resultant hybrid connections is still comparable to the constituent pure connections. USN experiments demonstrated that Infiniband transport can be effectively extended to wide-area connections of thousands of miles, which opens up new opportunities for efficient bulk data transport. USN provided dedicated connections to Cray X1 supercomputer and helped diagnose TCP performance problems which might have been otherwise incorrectly attributed to traffic on shared connections. USN contributed to the development of transport methods for dedicated connections to other traffic. Recently, experiments were conducted to assess the performance of application acceleration devices that employ flow optimization and data compression methods to improve TCP performance.
In this article, we propose a new protection scheme for backbone networks to guarantee high service availability. The presented scheme does not require any reconfiguration immediately after the failure (i.e., it is proactive). At the same time, it does not require any reserved backup network resources either. To achieve these seemingly contradictory goals, we utilize the recent advancements in Machine Learning (ML) to implement a network intelligence that periodically re-allocates the unused capacity as protection bandwidth to meet the service availability requirements of each connection. Our goal is achieved by two components (1) predicting the traffic for the next period on each link, and (2) intelligently selecting the best fit dedicated protection scheme for the next period depending on the estimated unused (spare) bandwidth and the previous service availability violations. Note that re-allocating protection bandwidth affects neither the operational connections nor the current best practice of operators to over-provision network bandwidth to support elephant flows. Finally, we provide a case study on the real traffic from Energy Sciences Network (ESnet), a high-speed, international scientific backbone network. The key benefit of our framework is that adaptively utilizing the over-provisioned bandwidth for spare capacity is sufficient to improve the availability from three-nines to five-nines (in ESnet for the 30 examined connections). The drawback is negligible bandwidth limitations; the user perceives a minor and very temporal bandwidth limitation in less than 0.1% of the time.
The overheads of classical decoding for quantum error correction in cryogenic quantum systems grow rapidly with the number of logical qubits and their correction code distance. Decoding at room temperature is bottlenecked by refrigerator I/O bandwidth while cryogenic on-chip decoding is limited by area/power/thermal budget. To overcome these overheads, we are motivated by the observation that in the common case (over 90% of the time), error correction 'syndromes' are fairly trivial with high redundancy / sparsity, since the error correction codes are over-provisioned to be able to correct for uncommon worst-case complex scenarios (to ensure substantially low logical error rates). If suitably exploited, these trivial scenarios can be handled with insignificant overhead, thereby alleviating any bottlenecks towards handling the worst-case scenarios by state-of-the-art means. Here, we propose Better Than Worst-Case Decoding for Quantum Error Correction, targeting cryogenic quantum systems and Surface Code, consisting of: On-chip Clique Decoder: An extremely lightweight decoder for correcting trivial common-case errors, designed for the cryogenic domain. The decoder is implemented and evaluated for SFQ logic. Statistical Off-chip Bandwidth Allocation: A statistical confidence-based technique for allocation of off-chip decoding bandwidth, to efficiently handle the rare complex decodes that are not covered by the Clique Decoder. Decode-Overflow Execution Stalling: A method to stall circuit execution, for the worst-case scenarios in which the provisioned off-chip bandwidth is insufficient to complete all requested off-chip decodes. In all, BTWC decoding achieves 70-99+% off-chip bandwidth elimination across a range of logical and physical error rates, without significantly sacrificing the accuracy of a state-of-the-art off-chip decoder. Further, it achieves 10-1000x bandwidth reduction over prior bandwidth reduction techniques, as well as 15-37x resource overhead reduction compared to prior on-chip decoding.
This research discusses development of a software-controlled laboratory instrument based spread spectrum time domain reflectometry system (SSTDR). This constitutes one task within PNNL’s Light Water Sustainability Program (LWRS) whose mission includes advancing nondestructive examination (NDE) techniques for off-line and on-line in-situ cable condition monitoring. In 2022, PNNL evaluated SSTDR for detection and characterization of a number of cable anomalies (Glass et al. 2022). The review included comparison of SSTDR to Frequency Domain Reflectometry (FDR) techniques which have enjoyed encouraging feedback and are starting to be used in nuclear power plants for periodic cable condition monitoring of cable systems as part of the plant’s overall cable aging management program. The FDR test introduces a broad-band chirp onto the cable at the cable end then listens for any reflection from a change of impedance along the cable caused by a damaged conductor or insulation, splices, contact with moisture, or other cable anomalies. The signal is captured in the frequency domain then transformed back to the time domain using an inverse Fourier transform (IFT). Based on the velocity of propagation, the impedance response signal is plotted against distance along the cable. Peak locations along the X-axis indicate the distance along the cable where a portion of the signal has been reflected back to the instrument as a result of a cable anomaly. The FDR test is considered the gold standard of reflectometry however it does require the cable to be de-energized to perform the test. The LIVEWIRE commercial SSTDR produces a similar plot to the FDR however all processing is in the time domain. A pseudo-random noise code (PN code) is input onto the cable conductor and the instrument listens for any reflected response from cable anomalies. The SSTDR processes the signal as an autocorrelation comparing the input PN code to any reflected signal detected. The autocorrelation analysis for thermal aging, water and water ingress detection, ground fault and phase-to-phase fault detection at various locations along the cable and with the cable attached and detached from a motor load, and on both energized and un-energized conditions were performed. These results were contrasted to Frequency Domain Reflectometry (FDR) measurements of the un-energized cable. Results were encouraging but indicated more work was warranted – particularly with the SSTDR, it seemed that the insulation damage would likely be better evaluated with multiple bandwidth cable tests particularly including larger bandwidths than were possible with the current commercial instrument. The commercial instrument’s bandwidth was set at 6, 12, 24, and 48MHz but note that SSTDR and FDR definitions of bandwidth trend similarly but are not the same. The FDR response could be more broadly adjusted, and the bandwidth of 100 to 500 MHz produced the best responses. FDR responses to anomalies were clearer than SSTDR responses and indications were that a broader bandwidth SSTDR may lead to improved SSTDR detection capability. This project used a laboratory instrument based SSTDR (primarily using an Arbitrary Waveform Generator (AWG) and a digital oscilloscope plus Python in-house software) that allowed software adjustment of the SSTDR bandwidth, window functions applied to the exciting Pseudo-random Noise (PN) code plus and other aspects of the SSTDR signal processing. Hereafter, this will be referred to as the PNNL SSTDR. Evaluating specific performance of the PNNL SSTDR is left to a separate report. This report documents hardware and software development to produce the SSTDR cable test system.
The present disclosure pertains to systems and methods for assessing reliability of communication links in a software defined network (SDN). In one embodiment, a system may include an SDN architecture subsystem configured to generate SDN architecture information and a bandwidth subsystem configured to generate bandwidth information. A latency subsystem may receive the bandwidth information from the bandwidth subsystem and may be configured to generate latency information using the bandwidth information. A failover subsystem may receive the SDN architecture information, the bandwidth information, and the latency information and generate a failover assessment and a failover route for the communication link. A reliability assessment subsystem may receive the SDN architecture information, the bandwidth information, and the latency information, and the failover assessment and generate a reliability assessment. A traffic routing subsystem may receive the reliability assessment and route data traffic to the failover route based on the reliability assessment.
Summary Disaggregated memory is a promising approach that addresses the limitations of traditional memory architectures by enabling memory to be decoupled from compute nodes and shared across a data center. Cloud platforms have deployed such systems to improve overall system memory utilization, but performance can vary across workloads. High‐performance computing (HPC) is crucial in scientific and engineering applications, where HPC machines also face the issue of underutilized memory. As a result, improving system memory utilization while understanding workload performance is essential for HPC operators. Therefore, learning the potential of a disaggregated memory system before deployment is a critical step. This paper proposes a methodology for exploring the design space of a disaggregated memory system. It incorporates key metrics that affect performance on disaggregated memory systems: memory capacity, local and remote memory access ratio, injection bandwidth, and bisection bandwidth, providing an intuitive approach to guide machine configurations based on technology trends and workload characteristics. We apply our methodology to analyze thirteen diverse workloads, including AI training, data analysis, genomics, protein, fusion, atomic nuclei, and traditional HPC bookends. Our methodology demonstrates the ability to comprehend the potential and pitfalls of a disaggregated memory system and provides motivation for machine configurations. Our results show that eleven of our thirteen applications can leverage injection bandwidth disaggregated memory without affecting performance, while one pays a rack bisection bandwidth penalty and two pay the system‐wide bisection bandwidth penalty. In addition, we also show that intra‐rack memory disaggregation would meet the application's memory requirement and provide enough remote memory bandwidth.
Abstract Manipulating the frequency and bandwidth of nonclassical light is essential for implementing frequency-encoded/multiplexed quantum computation, communication, and networking protocols, and for bridging spectral mismatch among various quantum systems. However, quantum spectral control requires a strong nonlinearity mediated by light, microwave, or acoustics, which is challenging to realize with high efficiency, low noise, and on an integrated chip. Here, we demonstrate both frequency shifting and bandwidth compression of heralded single-photon pulses using an integrated thin-film lithium niobate (TFLN) phase modulator. We achieve record-high electro-optic frequency shearing of telecom single photons over terahertz range (±641 GHz or ±5.2 nm), enabling high visibility quantum interference between frequency-nondegenerate photon pairs. We further operate the modulator as a time lens and demonstrate over eighteen-fold (6.55 nm to 0.35 nm) bandwidth compression of single photons. Our results showcase the viability and promise of on-chip quantum spectral control for scalable photonic quantum information processing.
We report that broad bandwidth, infrared light sources have the potential to revolutionize inertial confinement fusion (ICF) by suppressing laser-plasma instabilities. There is, however, a tradeoff: The broad bandwidth precludes high efficiency conversion from the infrared to the ultraviolet, where laser-plasma interactions are weaker. Operation in the infrared could intensify the role of resonance absorption, an effect long suspected to be the shortcoming of early ICF experiments. Here we present simulations exploring the effect of bandwidth on resonance absorption. In the linear regime, bandwidth has little effect on resonance absorption; in the nonlinear regime, bandwidth suppresses enhanced absorption resulting from the electromagnetic decay instability. Lastly, these findings evince that regardless of bandwidth, an ICF implosion will confront at least linear levels of resonance absorption.
We report how quantum information encoded into the spectral degree of freedom of a single-photon state may be teleported using a finite spectrally entangled biphoton state. We further demonstrate how the bandwidth of the teleported wave form can be controllably and coherently dilated using a spread-spectral variant of teleportation. We calculate analytical expressions for the fidelities of spectral and spread-spectral teleportation when complex-valued Gaussian states are transferred using a proposed experimental approach. Finally, we discuss the utility of these techniques for integrating broad-bandwidth photonic qubits with narrow-bandwidth receivers in quantum communication systems.
The third-generation South Pole Telescope camera (SPT-3G) improves upon its predecessor (SPTpol) by an order of magnitude increase in detectors on the focal plane. The technology used to read out and control these detectors, digital frequency-domain multiplexing (DfMUX), is conceptually the same as used for SPTpol, but extended to accommodate more detectors. A nearly 5× expansion in the readout operating bandwidth has enabled the use of this large focal plane, and SPT-3G performance meets the forecasting targets relevant to its science objectives. However, the electrical dynamics of the higher-bandwidth readout differ from predictions based on models of the SPTpol system due to the higher frequencies used and parasitic impedances associated with new cryogenic electronic architecture. To address this, we present an updated derivation for electrical crosstalk in higher-bandwidth DfMUX systems and identify two previously uncharacterized contributions to readout noise, which become dominant at high bias frequency. The updated crosstalk and noise models successfully describe the measured crosstalk and readout noise performance of SPT-3G. These results also suggest specific changes to warm electronics component values, wire-harness properties, and SQUID parameters, to improve the readout system for future experiments using DfMUX, such as the LiteBIRD space telescope.
Optical isolators are an essential component of photonic systems. Current integrated optical isolators have limited bandwidths due to stringent phase-matching conditions, resonant structures, or material absorption. Here, we demonstrate a wideband integrated optical isolator in thin-film lithium niobate photonics. We use dynamic standing-wave modulation in a tandem configuration to break Lorentz reciprocity and achieve isolation. We measure an isolation ratio of 15 dB and insertion loss below 0.5 dB for a continuous wave laser input at 1550 nm. In addition, we experimentally show that this isolator can simultaneously operate at visible and telecom wavelengths with comparable performance. Isolation bandwidths up to ∼100 nm can be achieved simultaneously at both visible and telecom wavelengths, limited only by the modulation bandwidth. Our device’s dual-band isolation, high flexibility, and real-time tunability can enable novel non-reciprocal functionality on integrated photonic platforms.
Currently, the fast orbit feedback (FOFB) system correct the transverse orbit in both planes via air coil magnets installed over low conductivity sections of vacuum chamber (stainless steel bellows) with a fast acquisition (FA) that samples at 10kHz. Bandwidth of a few kHz has been achieved (power-supply + corrector + vacuum chamber) for the purpose that the FOFB bandwidth limitation is not anymore limited by the correctors. The bandwidths of the FOFB system in both planes have been experimentally characterized in the open-loop condition via sinwave excitation method. They are significantly lower than the measurements (2.5kHz in x) done by Weixing Chen et al when the control system was largely bypassed, only from PSI to fast correctors, final to BPM. The estimated dI/dt as a function of the frequency from the beam noise spectrum has been compared to the slew rate of the power supply. The most recent study evidences: after the optimization of fast corrector compensation circuit, the power supply system along shows the 3dB bandwidth of 5.6kHz in the horizontal plane and 3.5kHz in the vertical plane. Preliminary beam studies indicate the potential increase of the system bandwidth and the gain within the FOFB bandwidth. Finally, the plan for combing the FOFB and SOFB systems to correct down to DC has been discussed.
Wireless links are the most popular form of communications with an ever-growing demand for speed. This has resulted in an overpopulated Radio Frequency (RF) spectrum with not enough bandwidth to meet demand. The two main tactics utilized by the wireless community to overcome this problem is to 1) put more data in a given bandwidth, i.e. higher spectral efficiency and 2) use the available bandwidth more efficiently by tuning the user’s channel (i.e. carrier frequency) to an unused part of spectrum. Higher spectral efficiency is achieved by using complex signals that carry information on both the in-phase (I) carrier and quadrature-phase (Q) carrier, known as Quadrature Amplitude Modulation (QAM), and increasing the number of levels on each carrier, known as the constellation size (4QAM, 16QAM etc.). However higher constellation sizes and higher bandwidth require higher fidelity RF signals. This project looked at a photonic-based high-fidelity RF amplitude and phase modulator which modulates the amplitude and phase of femtosecond laser pulses to generate relatively high-power complex microwave waveforms at carrier frequencies up to 70GHz. The goal of the feasibility study was to build a proof-of-concept system in a laboratory setting that demonstrates the photonic generation of QAM signals at electronically tunable RF carrier frequencies. Our lab prototype achieved a signal-to-noise-and-distortion (SINAD) ratio of 39dB over a 500MHz bandwidth. The signal-to-noise ratio (SNR) of the system was measured at 42dB over a 500MHz bandwidth which is on par with state-of-the-art. The main limitation to the SINAD was the inadequate pre-compensation of the electronic Digital-to-Analog Converter (eDAC) and amplifier frequency response as well as the noise of the measurement equipment. This was backed up by simulations which showed that with perfect pre compensation and using ideal measurement equipment, a noise limited SINAD of 47dB could be achieved. We also showed, through simulations, that multiple DACs could be “optically stacked” to achieve 54dB SINAD.
In the race to PetaFLOP-speed supercomputing systems, the increase in computational capability has been accompanied by corresponding increases in CPU count, total RAM, and storage capacity. However, a proportional increase in storage bandwidth has lagged behind. In order to improve system reliability and to reduce maintenance effort for modern large-scale systems, system designers have opted to remove node-local storage from the compute nodes. Today's multi-TeraFLOP supercomputers are typically attached to parallel file systems that provide only tens of GBs/s of I/O bandwidth. As a result, such machines have access to much less than 1GB/s of I/O bandwidth per TeraFLOP of compute power, which is below the generally accepted limit required for a well-balanced system. In a many ways, the current I/O bottleneck limits the capabilities of modern supercomputers, specifically in terms of limiting their working sets and restricting fault tolerance techniques, which become critical on systems consisting of tens of thousands of components. This paper resolves the dilemma between high performance and high reliability by presenting an alternative system design which makes use of node-local storage to improve aggregate system I/O bandwidth. In this work, we focus on the checkpointing use-case and present an experimental evaluation of the Scalable Checkpoint/Restart (SCR) library, a new adaptive checkpointing library that uses node-local storage to significantly improve the checkpointing performance of large-scale supercomputers. Experiments show that SCR achieves unprecedented write speeds, reaching a measured 700GB/s of aggregate bandwidth on 8,752 processors and an estimated 1TB/s for a similarly structured machine of 12,500 processors. This corresponds to a speedup of over 70x compared to the bandwidth provided by the 10GB/s parallel file system the cluster uses. Further, SCR can adapt to an environment in which there is wide variation in performance or capacity among the individual node-local storage elements.
Big data transfer in large-scale scientific and business applications is increasingly carried out over connections with guaranteed bandwidth provisioned in High-performance Networks (HPNs) via advance bandwidth reservation. Provisioning agents need to carefully schedule data transfer requests, compute network paths, and allocate appropriate bandwidths. Such reserved bandwidths, if not fully utilized, could be simply wasted due to the exclusive access during the approved time window, and cause extra overhead and complexity for resource management. This calls for accurate performance prediction to reserve bandwidths that match actual needs and avoid over-provisioning. We employ machine learning algorithms to predict big data transfer performance based on extensive performance measurements collected in the past several years from data transfer tests using different protocols and toolkits between various end sites on several real-life physical or emulated testbeds. We first analyze the performance patterns in response to a comprehensive list of parameters in end-host systems, network connections, and data transfer applications, which motivate the use of machine learning and also help us identify the effects of latent factors. We then propose threshold- and clustering-based methods to eliminate negative effects of latent factors in data preprocessing and build a robust performance predictor based on customized domain-oriented loss functions. The performance of the proposed methods is verified by extensive experiments using SVR and RFR as well as theoretical analysis of the general performance bound.
It has long been recognized that broadband laser light has the potential to control parametric instabilities in inertial-confinement-fusion (ICF) plasmas. Here, we use results from laser-plasma-interaction simulations to estimate the bandwidth requirements for mitigating the three predominant classes of instabilities in direct-drive ICF implosions: cross-beam energy transfer (CBET), two-plasmon decay (TPD), and stimulated Raman scattering (SRS). We find that for frequency-tripled, Nd:glass laser light, a bandwidth of 8.5 THz can significantly increase laser absorption by suppressing CBET, while ∼13 THz is needed to mitigate absolute TPD and SRS on an ignition-scale platform. None of the glass lasers used in contemporary ICF experiments, however, possess a bandwidth greater than 1 THz and reaching larger values requires the use of an auxiliary broadening technique such as optical parametric amplification or stimulated-rotational-Raman scattering. An arguably superior approach is the adoption of an argon-fluoride (ArF) laser as an ICF driver. Besides having a broad bandwidth of ∼10 THz, the ArF laser also possesses the shortest wavelength (193 nm) that can scale to the high energy/power required for ICF—a feature that helps to mitigate parametric instabilities even further. We show that these native properties of ArF laser light are sufficient to eliminate nearly all CBET scattering in a direct-drive target and also raise absolute TPD and SRS thresholds well above those for broadband glass lasers. The effective control of parametric instabilities with broad bandwidth is potentially a “game changer” in ICF because it would enable higher laser intensities and ablation pressures in future target designs.