Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data corruption”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

Exploring versioned distributed arrays for resilience in scientific applications: Global view resilience

Exascale studies project reliability challenges for future HPC systems. We present the Global View Resilience (GVR) system, a library for portable resilience. GVR begins with a subset of the Global Arrays interface, and adds new capabilities to create versions, name versions, and compute on version data. Applications can focus versioning where and when it is most productive, and customize for each application structure independently. This control is portable, and its embedding in application source makes it natural to express and easy to maintain. The ability to name multiple versions and “partially materialize” them efficiently makes ambitious forward-recovery based on “data slices” across versions or data structures both easy to express and efficient. Using several large applications (OpenMC, preconditioned conjugate gradient (PCG) solver, ddcMD, and Chombo), we evaluate the programming effort to add resilience. The required changes are small (< 2% lines of code (LOC)), localized and machine-independent, and perhaps most important, require no software architecture changes. We also measure the overhead of adding GVR versioning and show that overheads < 2% are generally achieved. This overhead suggests that GVR can be implemented in large-scale codes and support portable error recovery with modest investment and runtime impact. Our results are drawn from both IBM BG/Q and Cray XC30 experiments, demonstrating portability. We also present two case studies of flexible error recovery, illustrating how GVR can be used for multi-version rollback recovery, and several different forward-recovery schemes. GVR’s multi-version enables applications to survive latent errors (silent data corruption) with significant detection latency, and forward recovery can make that recovery extremely efficient. Lastly, our results suggest that GVR is scalable, portable, and efficient. GVR interfaces are flexible, supporting a variety of recovery schemes, and altogether GVR embodies a gentle-slope path to tolerate growing error rates in future extreme-scale systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

FORTÉ Machine Learning

Of the non-corrupted data collected by the Orbiting Experiment (forté) satellite’s Photo-Diode Detector during the year 2001, I estimate that 7.9% of 914 894 signals are noise. My result differs dramatically from Guillen’s estimate of 96%. To arrive at this estimate, I used Gaussian mixture model (GMM) clustering–unsupervised machine learning–to aggregate the wave forms into groups based on the absolute value of the lowest 25 positive frequency discrete Fourier transform coefficients. Then, I marked several of the groups as noise by inspecting a random sampling of wave forms from each group. Marking groups as either noise or non-noise is a supervised binary classification operation. After removing the signals in noise groups from further consideration, I clustered the remaining signals into families. Again, I used a GMM, but for the familial clustering I used a Non-Negative Matrix Factorization feature vector transform. The result was 9 distinct families of lightning signals, as well as a second stage of noise filtering. To efficiently represent the entirety of the signal space, I broke each family into deciles based on their distance from the family mean. In this case, distance means the log-likelihood based on the GMM. Signals in lower deciles are more similar in shape and amplitude to their family average. I took the top 200 samples from each decile of each group, resulting in 18 000 signals. These signal approximately represent the entirety of the forté observations. To represent outliers, I also kept a zoo of the 1000 signals furthest from any family’s average. All told, the resulting data set represents the forté data with a reduction of about 51:1. To allow synthesis of an arbitrarily large number of test signals, I also captured each family’s average signal and the time-sample covariance matrix over the signals in each family. Using these two pieces of information, I can synthesize new waveforms by using a Gaussian random realization from the family average and covariance matrix. I wrote a program to test the synthesis quality. The program shows me two signals on the screen, one synthesized and one randomly drawn from the data. I attempted to identify the synthesized signal. Although the synthesis is imperfect, in an A/B comparison I only correctly chose the synthesized signal 36% of the time.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Whole-Program Adaptive Error Detection and Mitigation. Final Report

Errors in application state resulting from faults in hardware are an increasing concern on extreme-scale computing systems. Errors that escape detection and lead to silent data corruption are particularly problematic. Detecting errors is an important first step toward fault tolerant program execution. The multi-institutional project addressed a comprehensive approach to error detection and mitigation for scientific applications that combined configurable error detectors, a unified reliability specification, and whole-program detector composition.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Radiation Hardening – Test Capabilities (LANSCE) [Slides]

The Los Alamos Neutron Science Center (LANSCE) has exceptional facilities for research and testing of radiation effects in electronics. We have the best terrestrial neutron source for sea-level and airplane-altitude environments. We have a thermal neutron test capability at the Lujan Center and a proton beam test capability at the “Blue Room”. LANL has the programmatic mission to develop this radiation-effects program, namely space and satellite environments, high-performance computing (silent data corruption), and weapons delivery systems. We have a strong connection to industry which continues to grow.

42 ENGINEERING↗
NASA NTRS1978

Bursty channel errors and the Viterbi decoder

Recent applications have developed for spread spectrum communications, hardware data transfer, high rate digital systems, etc. that use channels for which errors tend to occur in short bursts in addition to those at random, i.e., compound channels. Viterbi decoding algorithms are generally very good for random error channels but are not as efficient for burst errors or for compound channels. This paper presents the results of a computer simulation study of the performance of various Viterbi decoders when receiving data corrupted with burst and random errors on the same channel. Simulations were performed using hard-decision CPSK.

Ingels, F.↗
NASA NTRS1987

A model-free method for mass spectrometer response correction

A new method for correction of mass spectrometer output signals is described. Response-time distortion is reduced independently of any model of mass spectrometer behavior. The delay of the system is found first from the cross-correlation function of a step change and its response. A two-sided time-domain digital correction filter (deconvolution filter) is generated next from the same step response data using a regression procedure. Other data are corrected using the filter and delay. The mean squared error between a step response and a step is reduced considerably more after the use of a deconvolution filter than after the application of a second-order model correction. O2 consumption and CO2 production values calculated from data corrupted by a simulated dynamic process return to near the uncorrupted values after correction. Although a clean step response or the ensemble average of several responses contaminated with noise is needed for the generation of the filter, random noise of magnitude not above 0.5 percent added to the response to be corrected does not impair the correction severely.

Shykoff, Barbara E.↗
NASA NTRS1993

Measurement methods and algorithms for comparison of local and remote clocks

Several methods for characterizing the performance of clocks with special emphasis on using calibration information that is acquired via an unreliable or noisy channel is discussed. Time-domain variance estimators and frequency-domain techniques such as cross-spectral analysis are discussed. Each of these methods has advantages and limitations that will be illustrated using data obtained via GPS, ACTS, and other methods. No one technique will be optimum for all of these analyses, and some of these problems cannot be completely characterized by any of the techniques discussed. The inverse problem of communicating frequency and time corrections to a real-time steered clock are also discussed. Methods were developed to mitigate the disastrous problems of data corruption and loss of computer control.

Levine, Judah↗
NASA NTRS1995

Estimating the Cramer-Rao bound for restored astronomical observations

This work addresses the problem of assigning confidence intervals to estimated photometry data obtained from astronomical observations. The proposed solution is to estimate the Cramer-Rao bound, which is an analytical expression that describes the minimum obtainable mean square error associated with a given estimate of a parameter. This Letter presents a compact and simple form for the bound associated with a linear estimator such as a Wiener filter estimator. A prescription for estimating the variance associated with each element in a restored object was developed using an analytical model for observed data corrupted by either Poisson or Gaussian noise. Both one- and two-dimensional examples are presented.

Zaccheo, T. S.↗
NASA NTRS1998

A new frame-based registration algorithm

This paper presents a new algorithm for frame registration. Our algorithm requires only that the frame be comprised of straight rods, as opposed to the N structures or an accurate frame model required by existing algorithms. The algorithm utilizes the full 3D information in the frame as well as a least squares weighting scheme to achieve highly accurate registration. We use simulated CT data to assess the accuracy of our algorithm. We compare the performance of the proposed algorithm to two commonly used algorithms. Simulation results show that the proposed algorithm is comparable to the best existing techniques with knowledge of the exact mathematical frame model. For CT data corrupted with an unknown in-plane rotation or translation, the proposed technique is also comparable to the best existing techniques. However, in situations where there is a discrepancy of more than 2 mm (0.7% of the frame dimension) between the frame and the mathematical model, the proposed technique is significantly better (p < or = 0.05) than the existing techniques. The proposed algorithm can be applied to any existing frame without modification. It provides better registration accuracy and is robust against model mis-match. It allows greater flexibility on the frame structure. Lastly, it reduces the frame construction cost as adherence to a concise model is not required.

NASA Center ARC↗
NASA NTRS2005

Designing and Implementing a Distributed System Architecture for the Mars Rover Mission Planning Software (Maestro)

Distributed systems allow scientists from around the world to plan missions concurrently, while being updated on the revisions of their colleagues in real time. However, permitting multiple clients to simultaneously modify a single data repository can quickly lead to data corruption or inconsistent states between users. Since our message broker, the Java Message Service, does not ensure that messages will be received in the order they were published, we must implement our own numbering scheme to guarantee that changes to mission plans are performed in the correct sequence. Furthermore, distributed architectures must ensure that as new users connect to the system, they synchronize with the database without missing any messages or falling into an inconsistent state. Robust systems must also guarantee that all clients will remain synchronized with the database even in the case of multiple client failure, which can occur at any time due to lost network connections or a user's own system instability. The final design for the distributed system behind the Mars rover mission planning software fulfills all of these requirements and upon completion will be deployed to MER at the end of 2005 as well as Phoenix (2007) and MSL (2009).

Goldgof, Gregory M.↗
NASA NTRS2004

Using software security analysis to verify the secure socket layer (SSL) protocol

nal Aeronautics and Space Administration (NASA) have tens of thousands of networked computer systems and applications. Software Security vulnerabilities present risks such as lost or corrupted data, information the3, and unavailability of critical systems. These risks represent potentially enormous costs to NASA. The NASA Code Q research initiative 'Reducing Software Security Risk (RSSR) Trough an Integrated Approach '' offers, among its capabilities, formal verification of software security properties, through the use of model based verification (MBV) to address software security risks. [1,2,3,4,5,6] MBV is a formal approach to software assurance that combines analysis of software, via abstract models, with technology, such as model checkers, that provide automation of the mechanical portions of the analysis process. This paper will discuss: The need for formal analysis to assure software systems with respect to software and why testing alone cannot provide it. The means by which MBV with a Flexible Modeling Framework (FMF) accomplishes the necessary analysis task. An example of FMF style MBV in the verification of properties over the Secure Socket Layer (SSL) communication protocol as a demonstration.

software↗
NASA NTRS2004

Model based verification of the Secure Socket Layer (SSL) Protocol for NASA systems

The National Aeronautics and Space Administration (NASA) has tens of thousands of networked computer systems and applications. Software Security vulnerabilities present risks such as lost or corrupted data, information theft, and unavailability of critical systems. These risks represent potentially enormous costs to NASA. The NASA Code Q research initiative 'Reducing Software Security Risk (RSSR) Trough an Integrated Approach' offers formal verification of information technology (IT), through the creation of a Software Security Assessment Instrument (SSAI), to address software security risks.

software security↗
NASA NTRS2013

Behavior Prediction Tools Strengthen Nanoelectronics

Several years ago, NASA started making plans to send robots to explore the deep, dark craters on the Moon. As part of these plans, NASA needed modeling tools to help engineer unique electronics to withstand extremely cold temperatures. According to Jonathan Pellish, a flight systems test engineer at Goddard Space Flight Center, "An instrument sitting in a shadowed crater on one of the Moon s poles would hover around 43 K", that is, 43 kelvin, equivalent to -382 F. Such frigid temperatures are one of the main factors that make the extreme space environments encountered on the Moon and elsewhere so extreme. Radiation is another main concern. "Radiation is always present in the space environment," says Pellish. "Small to moderate solar energetic particle events happen regularly and extreme events happen less than a handful of times throughout the 7 active years of the 11-year solar cycle." Radiation can corrupt data, propagate to other systems, require component power cycling, and cause a host of other harmful effects. In order to explore places like the Moon, Jupiter, Saturn, Venus, and Mars, NASA must use electronic communication devices like transmitters and receivers and data collection devices like infrared cameras that can resist the effects of extreme temperature and radiation; otherwise, the electronics would not be reliable for the duration of the mission.

Source record↗
NASA NTRS2012

CASSIUS: The Cassini Uplink Scheduler

The Cassini Uplink Scheduler (CASSIUS) is cross-platform software used to generate a radiation sequence plan for commands being sent to the Cassini spacecraft. Because signals must travel through varying amounts of Earth's atmosphere, several different modes of constant telemetry rates have been devised. These modes guarantee that the spacecraft and the Deep Space Network agree with respect to the data transmission rate. However, the memory readout of a command will be lost if it occurs on a telemetry mode boundary. Given a list of spacecraft message files as well as the available telemetry modes, CASSIUS can find an uplink sequence that ensures safe transmission of each file. In addition, it can predict when the two on-board solid state recorders will swap. CASSIUS prevents data corruption by making sure that commands are not planned for memory readout during telemetry rate changes or a solid state recorder swap.

Cassini mission↗
DOE OSTI2020

Analyzing the Effects of Cyberattacks on Distribution System State Estimation: Preprint

Key components of power systems—such as energy management systems, automatic generation control, and state estimation—are under serious vulnerability from cyber attacks. Cyber threats in electric grids have increased significantly because of the increased interconnectivity of supervisory control and data acquisition systems and public network infrastructure. As the penetration level of distributed energy resources increases, it is imperative to employ system-monitoring techniques such as state estimation for the reliable operation of distribution systems. Recently, multiple methods have been developed that exploit the low rank property of distribution system state matrix and are robust to bad data, such as matrix completion. This paper analyzes the impact of various realistic cyber attack scenarios on matrix completion. Realistic cyber attack scenarios are converted into data corruption models that are used in an extensive simulation of a custom IEEE 123-bus system.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2019

BonVoision: Leveraging Spatial Data Smoothness For Recovery From Memory Soft Errors

The increasing soft error rates in memory systems raise an emerging concern for modern computing systems. As a result, detectable but uncorrectable errors (DUEs) become potentially more frequent and affect HPC applications. Today, upon encountering a DUE, HPC applications crash, incurring significant performance, storage, and energy overheads. In this paper, we propose a technique to continue application execution past a DUE through the repair of the corrupted memory data by leveraging spatial data smoothness. We present BonVoision, a run-time system that intercepts DUE events, analyzes the binary to identify data elements in the structural neighborhood of the event, and fixes the corrupted data elements by interpolating from the values in their neighborhood. Our evaluation demonstrates that BonVoision incurs negligible overhead and outperforms other recovery strategies by a factor of 2×, on average. We demonstrate that BonVoision also improves the efficiency of existing checkpointing/restart schemes by approximately increasing the optimal checkpoint interval by 23%.

Fang, Bo↗