Erasure Coding on File Transfer Appliance for Nearline Storage With High Degree Sharding.
Abstract not provided.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Abstract not provided.
The nature of the optical/microwave interface aboard the relay satellite is considered. To allow for the maximum system flexibility, without overburdening either the optical or RF channel, demodulating the optical on board the relay satellite but leaving the optical channel decoding to be performed at the ground station is examined. The occurrence of erasures in the optical channel is treated. A hard decision on the erasure (i.e., the relay selecting a symbol at random in case of erasure occurrence) seriously degrades the performance of the overall system. Coding the erasure occurrences at the relay and transmitting this information via an extra bit to the ground station where it can be used by the decoder is suggested. Many examples with varying bit/photon energy efficiency and for the noisy and noiseless optical channel are considered. It is shown that coding the erasure occurrences dramatically improves the performance of the cascaded channel relative to the case of hard decision on the erasure by the relay.
Sequential decoding has been found to be an efficient means of communicating at low undetected error rates from deep space probes, but erasure or computational overflow remains a significant problem. Erasure of a block occurs when the decoder has not finished decoding that block at the time that it must be output. By drawing upon analogies in computer time sharing, this paper develops a buffer-management strategy which reduces the decoder idle time to a negligible level, and therefore improves the erasure probability of a sequential decoder. For a decoder with a speed advantage of ten and a buffer size of ten blocks, operating at an erasure rate of .01, use of this buffer-management strategy reduces the erasure rate to less than .0001.
The use of Reed-Solomon (RS) block codes over the pulse position modulated (PPM) frames to obtain the largest degree of error correction is considered. Since RS codes can correct both symbol errors and symbol erasures, a question arises as to the best way to demodulate the PPM laser fields in order to generate the input symbols for the RS decoder. The method selected for demodulating (converting the received laser field to digital symbols) defines the erasure and transmitted symbols of the laser link, and therefore determines the work error probabilities of the system. Several demodulating schemes are considered, and the effect of each on RS decoding performance computed. This computation was carried out for various optical receiver models. It is shown that simple threshold decisioning of pulse slots produces performance that degrades as the background noise increases. This is caused by the generation of too many erasures for the RS decoder to handle. A decision scheme, delta-max demodulation which offers improvement over threshold decisioning by redefining the generation of an erasure is proposed.
A coding system under consideration for the Galileo S-band low-gain antenna mission is a concatenated system using a variable redundancy Reed-Solomon outer code and a (14,1/4) convolutional inner code. The 8-bit Reed-Solomon symbols are interleaved to depth 8, and the eight 255-symbol codewords in each interleaved block have redundancies 64, 20, 20, 20, 64, 20, 20, and 20, respectively (or equivalently, the codewords have 191, 235, 235, 235, 191, 235, 235, and 235 8-bit information symbols, respectively). This concatenated code is to be decoded by an enhanced decoder that utilizes a maximum likelihood (Viterbi) convolutional decoder; a Reed Solomon decoder capable of processing erasures; an algorithm for declaring erasures in undecoded codewords based on known erroneous symbols in neighboring decodable words; a second Viterbi decoding operation (redecoding) constrained to follow only paths consistent with the known symbols from previously decodable Reed-Solomon codewords; and a second Reed-Solomon decoding operation using the output from the Viterbi redecoder and additional erasure declarations to the extent possible. It is estimated that this code and decoder can achieve a decoded bit error rate of 1 x 10(exp 7) at a concatenated code signal-to-noise ratio of 0.76 dB. By comparison, a threshold of 1.17 dB is required for a baseline coding system consisting of the same (14,1/4) convolutional code, a (255,223) Reed-Solomon code with constant redundancy 32 also interleaved to depth 8, a one-pass Viterbi decoder, and a Reed Solomon decoder incapable of declaring or utilizing erasures. The relative gain of the enhanced system is thus 0.41 dB. It is predicted from analysis based on an assumption of infinite interleaving that the coding gain could be further improved by approximately 0.2 dB if four stages of Viterbi decoding and four levels of Reed-Solomon redundancy are permitted. Confirmation of this effect and specification of the optimum four-level redundancy profile for depth-8 interleaving is currently being done.
The dramatic increase in the scale of current and planned high-end HPC systems is leading new challenges, such as the growing costs of data movement and IO, and the reduced mean time between failures (MTBF) of system components. In-situ workflows, i.e., executing the entire application workflows on the HPC system, have emerged as an attractive approach to address data-related challenges by moving computations closer to the data, and staging-based frameworks have been effectively used to support in-situ workflows at scale. However, the resilience of these staging-based solutions has not been addressed, and they remain susceptible to expensive data failures. Furthermore, naive use of data resilience techniques such as n-way replication and erasure codes can impact latency and/or result in significant storage overheads. In this article, we present CoREC, a scalable and resilient in-memory data staging runtime for large-scale in-situ workflows. CoREC uses a novel hybrid approach that combines dynamic replication with erasure coding based on data access patterns. It also leverages multiple levels of replications and erasure coding to support diverse data resiliency requirements. Furthermore, the article presents optimizations for load balancing and conflict-avoiding encoding, and a low overhead, lazy data recovery scheme. We have implemented the CoREC runtime and have deployed with the DataSpaces staging service on leadership class computing machines and present an experimental evaluation in the article. Here, the experiments demonstrate that CoREC can tolerate in-memory data failures while maintaining low latency and sustaining high overall storage efficiency at large scales.
Abstract The entanglement wedge reconstruction paradigm in AdS/CFT states that for a bulk qudit within the entanglement wedge of a boundary subregion$$ \overline{A} $$ A ¯ , operators acting on the bulk qudit can be reconstructed as CFT operators on$$ \overline{A} $$ A ¯ . This naturally fits within the framework of quantum error correction, with the CFT states containing the bulk qudit forming a code protected against the erasure of the boundary subregionA. In this paper, we set up and study a framework for relational bulk reconstruction in holography: given two code subspaces both protected against erasure of the boundary regionA, the goal is to relate the operator reconstructions between the two spaces. To accomplish this, we assume that the two code subspaces are smoothly connected by a one-parameter family of codes all protected against the erasure ofA, and that the maximally-entangled states on these codes are all full-rank. We argue that such code subspaces can naturally be constructed in holography in a “measurement-based” setting. In this setting, we derive a flow equation for the operator reconstruction of a fixed code subspace operator using modular theory which can, in principle, be integrated to relate the reconstructed operators all along the flow. We observe a striking resemblance between our formulas for relational bulk reconstruction and the infinite-time limit of Connes cocycle flow, and take some steps towards making this connection more rigorous. We also provide alternative derivations of our reconstruction formulas in terms of a canonical reconstruction map we call the modular reflection operator.
Client-side data caching serves as an excellent mechanism to store and analyze the rapidly growing amount of scientific data. In our previous work, we built a distributed local cache on unreliable desktop storage contributions. This offers several desirable properties, such as performance impedance matching, improved space utilization, and high parallel I/O bandwidth. Such a low-cost, best-effort cache, however, is faced with the vagaries of storage node availability: these donated machines may be significantly less reliable than dedicated systems and cannot be controlled centrally. In this paper, we address %the tradeoffs between techniques that favor %availability or performance when it comes to cache management. the performance impact of data availability in the distributed scientific data cache setting. We then present a novel approach to storage cache management, {\em remote partial data recovery (RPDR)}. We compare our approach to two standard techniques, namely replication and erasure coding, both extended to the target caching environment. Our evaluation uses a trace-driven simulation parameterized with benchmarking results from our distributed cache prototype. The results with multiple real-world traces indicate that RPDR significantly outperforms both replication and erasure coding in many cases and overall the combination of RPDR and erasure coding yields the best performance.
Previous work has demonstrated that fluorophosphate (FP) glasses doped with trivalent samarium (Sm{sup 3+}) can be used as a dosimetric detector in microbeam radiation therapy (MRT) to measure high radiation doses and large dose variations with a resolution in the micrometer range. The present work addresses the use of intense optical radiation at 405 nm to erase the recorded dose information in Sm{sup 3+}-doped FP glass plates and examines the underlying physics. We have evaluated both the conversion and optical erasure of Sm{sup 3+}-doped FP glasses using synchrotron-generated high-dose x-rays at the Canadian Light Source. The Sm-ion valency conversion is accompanied by the appearance of x-ray induced optical absorbance due to the trapping of holes and electrons into phosphorus-oxygen hole (POHC) and electron (POEC) capture centers. Nearly complete Sm{sup 2+} to Sm{sup 3+} reconversion (erasure) may be achieved by intense optical illumination. Combined analysis of absorbance and electron spin resonance measurements indicates that the optical illumination causes partial disappearance of the POHC and the appearance of new POEC. The suggested model for the observed phenomena is based on the release of electrons during the Sm{sup 2+} to Sm{sup 3+} reconversion process, the capture of these electrons by POHC (and hence their disappearance), or by PO groups, with the appearance of new and/or additional POEC. Optical erasure may be used as a practical means to erase the recorded data and permits the reuse of these Sm-doped FP glasses in monitoring dose in MRT.
In modern science, the growing complexity of large-scale scientific projects has led to an increasing reliance on cross-facility scientific workflows, where resources and expertise from multiple institutions and geographic locations are leveraged to accelerate scientific discovery. These workflows often require transmitting huge amounts of scientific data through wide-area networks. Although high-speed networks like ESnet and transfer services such as Globus have improved data mobility, several challenges remain. The sheer volume of data can overwhelm network bandwidth, widely used transport protocols such as TCP suffer from inefficiencies due to retransmissions triggered by packet loss, and existing fault-tolerance mechanisms like erasure coding introduce substantial overhead. In this paper, we propose Janus, a resilient and adaptable data transmission approach designed for cross-facility scientific workflows. Unlike traditional TCP-based methods, Janus leverages UDP, integrates erasure coding for fault tolerance, and combines it with error-bounded lossy compression to reduce overhead. This novel design allows users to balance data transmission time and accuracy, optimizing transfer performance based on specific scientific requirements. Additionally, Janus dynamically adjusts erasure coding parameters in response to real-time network conditions, ensuring efficient data transfers even in fluctuating environments. We develop optimization models for determining ideal configurations and implement adaptive data transfer protocols to enhance reliability. Through extensive simulations and real-network experiments, we demonstrate that Janus significantly improves transfer efficiency while maintaining data fidelity.
In a simple multiple-access communication system, two geographically separated users attempt to communicate binary data to two data sinks over a common channel called a multiple access channel. User one sends codewords from a block code C sub one, while user two sends codewords from a block code C sub two. The two users occupy the same frequency slot, transmit at the same time, and use the same type of modulation. Block codes which are uniquely decodable and capable of correcting errors are constructed for two multiple-access channel models. The first model is referred to as a noiseless multiple-access binary erasure channel. If the two transmitted bits from the two users are zeros, a zero is transmitted over the channel to the receiver; if the two transmitted bits are ones, a one is transmitted to the receiver; if the two bits are different, an erasure symbol is transmitted to the receiver. The second model is also a multiple-access binary erasure channel but with noise introduced.
It is well known that the Euclidean algorithm or its equivalent, continued fractions, can be used to find the error locator polynomial needed to decode a Reed-Solomon (RS) code. It is shown that this algorithm can be used for both time and transform domain decoding by replacing its initial conditions with the Forney syndromes and the erasure locator polynomial. By this means both the errata locator polynomial and the errate evaluator polynomial can be obtained with the Euclidean algorithm. With these ideas, both time and transform domain Reed-Solomon decoders for correcting errors and erasures are simplified and compared. As a consequence, the architectures of Reed-Solomon decoders for correcting both errors and erasures can be made more modular, regular, simple, and naturally suitable for VLSI implementation.
We analyze the transient response of quantum dot thermal machines that can be driven by hyperfine interaction acting as a source of classical information. Our setup comprises a quantum dot coupled to two contacts that drive heat flow while coupled to a nuclear spin bath. The quantum dot thermal machines operate both as batteries and as engines, depending on the parameter range. The electrons in the quantum dot interact with the nuclear spins via hyperfine spin-flip processes as typically seen in solid state systems such as GaAs quantum dots. The hyperfine interaction in such systems, which is often treated as a deterrent for quantum information processing, can favorably be regarded as a driving agent for classical information flow into a heat engine setup. We relate this information flow to Landauer’s erasure of the nuclear spin bath, leading to a battery operation. We further demonstrate that the setup can perform as a transient power source even under a voltage bias across the dot. Focusing on the transient thermoelectric operation, our analysis clearly indicates the role of Landauer’s erasure to deliver a higher output power than a conventional quantum dot thermoelectric setup and an efficiency greater than that of an identical Carnot cycle in steady state, which is consistent with recently proposed bounds on efficiency for systems subject to a feedback controller. The role of nuclear spin relaxation processes on these aspects is also studied. Finally, we introduce the Coulomb interaction in the dot and analyze the transient thermoelectric response of the system. Our results elaborate on the effective use of somewhat undesirable scattering processes as a non-equilibrium source of Shannon information flow in thermal machines and the possibilities that may arise from the use of a quantum information source.
Quantum measurement is sometimes more effective when its result is not definite. Partial measurement turns the initial superposition not into a certain state but to a greater probability for it, enabling probing the quantum state in cases where complete measurement makes the noncommuting variables inaccessible. It also enables full quantum erasure that, unlike prevailing method, can be carried out even on recorded results. Aharonov's weak measurement is another method of imprecisely measuring quantum variables, outsmarting the uncertainty principle in even subtler ways. Happily, the two methods complement and corroborate one another in several interesting ways. We gedankenly apply these measurements to the EPR case. A pair of entangled particles undergoes more than one pair of partial and weak measurements, which, unlike complete measurements, leave them partially correlated. Their erasure is then shown to be as nonlocal as measurement itself. Surprisingly, the temporal relations between such measurements in the EPR setting do not follow the temporal sequence perceived by an external observer. For each particle, the measurements performed on the other operate as if they occurred (with signs reversed) in its own past, and in reversed order. This fully accords with Cramer's transactional interpretation and Aharonov's two state-vector formalism.
In magnetic recording media, the grains or clusters reverse their magnetization over a range of reversal field, resulting in a switching field distribution. In order to achieve high areal densities, it is desirable to understand and minimize such a distribution. Clusters of grains which contain stacking faults (SF) or fcc phase have lower anisotropy, an order lower than those without them. It is believed that such low anisotropy regions reverse their magnetization at a much lower reversal field than the rest of the material with a larger anisotropy. Such clusters/grains cause recording performance deterioration, such as adjacent track erasure and dc noise. Therefore, the observation of clusters that reverse at very low reversal fields (nucleation sites, NS) could give information on the noise and the adjacent track erasure. Potentially, the observed clusters could also provide information on the SF. In this paper, we study the reversal of nucleation sites in granular perpendicular media based on a magnetic force microscope (MFM) methodology and validate the observations with high resolution cross-section transmission electron microscopy (HRTEM) measurements. Samples, wherein a high anisotropy CoPt layer was introduced to control the NS or SF in a systematic way, were evaluated by MFM, TEM, and magnetometry. The magnetic properties indicated that the thickness of the CoPt layer results in an increase of nucleation sites. TEM measurements indicated a correlation between the thickness of CoPt layer and the stacking fault density. A clear correlation was also observed between the MFM results, TEM observations, and the coercivity and nucleation field of the samples, validating the effectiveness of the proposed method in evaluating the nucleation sites which potentially arise from stacking faults.
The design of quantum hardware that reduces and mitigates errors is essential for practical quantum error correction (QEC) and useful quantum computation. To this end, we introduce the circuit-Quantum Electrodynamics (QED) dual-rail qubit in which our physical qubit is encoded in the single-photon subspace, { | 01 〉 , | 10 〉 } , of two superconducting microwave cavities. The dominant photon loss errors can be detected and converted into erasure errors, which are in general much easier to correct. In contrast to linear optics, a circuit-QED implementation of the dual-rail code offers unique capabilities. Using just one additional transmon ancilla per dual-rail qubit, we describe how to perform a gate-based set of universal operations that includes state preparation, logical readout, and parametrizable single and two-qubit gates. Moreover, first-order hardware errors in the cavities and the transmon can be detected and converted to erasure errors in all operations, leaving background Pauli errors that are orders of magnitude smaller. Hence, the dual-rail cavity qubit exhibits a favorable hierarchy of error rates and is expected to perform well below the relevant QEC thresholds with today’s coherence times.
Interferometers are widely used in imaging technologies to achieve enhanced spatial resolution, but require that the incoming photons be indistinguishable. In previous work, we built and analyzed color erasure detectors which expand the scope of intensity interferometry to accommodate sources of different colors. Here we demonstrate experimentally how color erasure detectors can achieve improved spatial resolution in an imaging task, well beyond the diffraction limit. Utilizing two 10.9 mm-aperture telescopes and a 0.8 m baseline, we measure the distance between a 1063.6 nm source and a 1064.4 nm source separated by 4.2 mm at a distance of 1.43 km, which surpasses the diffraction limit of a single telescope by about 40 times. Moreover, chromatic intensity interferometry allows us to recover the phase of the Fourier transform of the imaged objects – a quantity that is, in the presence of modest noise, inaccessible to conventional intensity interferometry.