Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data corruption”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Whole-Program Adaptive Error Detection and Mitigation. Final Report

Errors in application state resulting from faults in hardware are an increasing concern on extreme-scale computing systems. Errors that escape detection and lead to silent data corruption are particularly problematic. Detecting errors is an important first step toward fault tolerant program execution. The multi-institutional project addressed a comprehensive approach to error detection and mitigation for scientific applications that combined configurable error detectors, a unified reliability specification, and whole-program detector composition.

97 MATHEMATICS AND COMPUTING↗

Radiation Hardening – Test Capabilities (LANSCE) [Slides]

The Los Alamos Neutron Science Center (LANSCE) has exceptional facilities for research and testing of radiation effects in electronics. We have the best terrestrial neutron source for sea-level and airplane-altitude environments. We have a thermal neutron test capability at the Lujan Center and a proton beam test capability at the “Blue Room”. LANL has the programmatic mission to develop this radiation-effects program, namely space and satellite environments, high-performance computing (silent data corruption), and weapons delivery systems. We have a strong connection to industry which continues to grow.

42 ENGINEERING↗

On Undecidability Aspects of Resilient Computations and Implications to Exascale

Future Exascale computing systems with a large number of processors, memory elements and interconnection links, are expected to experience multiple, complex faults, which affect both applications and operating-runtime systems. A variety of algorithms, frameworks and tools are being proposed to realize and/or verify the resilience properties of computations that guarantee correct results on failure-prone computing systems. We analytically show that certain resilient computation problems in presence of general classes of faults are undecidable, that is, no algorithms exist for solving them. We first show that the membership verification in a generic set of resilient computations is undecidable. We describe classes of faults that can create infinite loops or non-halting computations, whose detection in general is undecidable. We then show certain resilient computation problems to be undecidable by using reductions from the loop detection and halting problems under two formulations, namely, an abstract programming language and Turing machines, respectively. These two reductions highlight different failure effects: the former represents program and data corruption, and the latter illustrates incorrect program execution. These results call for broad-based, well-characterized resilience approaches that complement purely computational solutions using methods such as hardware monitors, co-designs, and system- and application-specific diagnosis codes.

Exascale systems↗

Analyzing the Effects of Cyberattacks on Distribution System State Estimation: Preprint

Key components of power systems—such as energy management systems, automatic generation control, and state estimation—are under serious vulnerability from cyber attacks. Cyber threats in electric grids have increased significantly because of the increased interconnectivity of supervisory control and data acquisition systems and public network infrastructure. As the penetration level of distributed energy resources increases, it is imperative to employ system-monitoring techniques such as state estimation for the reliable operation of distribution systems. Recently, multiple methods have been developed that exploit the low rank property of distribution system state matrix and are robust to bad data, such as matrix completion. This paper analyzes the impact of various realistic cyber attack scenarios on matrix completion. Realistic cyber attack scenarios are converted into data corruption models that are used in an extensive simulation of a custom IEEE 123-bus system.

41 EE - Solar Energy Technologies Office (EE-4S)↗

BonVoision: Leveraging Spatial Data Smoothness For Recovery From Memory Soft Errors

The increasing soft error rates in memory systems raise an emerging concern for modern computing systems. As a result, detectable but uncorrectable errors (DUEs) become potentially more frequent and affect HPC applications. Today, upon encountering a DUE, HPC applications crash, incurring significant performance, storage, and energy overheads. In this paper, we propose a technique to continue application execution past a DUE through the repair of the corrupted memory data by leveraging spatial data smoothness. We present BonVoision, a run-time system that intercepts DUE events, analyzes the binary to identify data elements in the structural neighborhood of the event, and fixes the corrupted data elements by interpolating from the values in their neighborhood. Our evaluation demonstrates that BonVoision incurs negligible overhead and outperforms other recovery strategies by a factor of 2×, on average. We demonstrate that BonVoision also improves the efficiency of existing checkpointing/restart schemes by approximately increasing the optimal checkpoint interval by 23%.

Fang, Bo↗

Prior-based artifact correction (PBAC) in computed tomography

Purpose: Image quality in computed tomography (CT) often suffers from artifacts which may reduce the diagnostic value of the image. In many cases, these artifacts result from missing or corrupt regions in the projection data, e.g., in the case of metal, truncation, and limited angle artifacts. The authors propose a generalized correction method for different kinds of artifacts resulting from missing or corrupt data by making use of available prior knowledge to perform data completion. Methods: The proposed prior-based artifact correction (PBAC) method requires prior knowledge in form of a planning CT of the same patient or in form of a CT scan of a different patient showing the same body region. In both cases, the prior image is registered to the patient image using a deformable transformation. The registered prior is forward projected and data completion of the patient projections is performed using smooth sinogram inpainting. The obtained projection data are used to reconstruct the corrected image. Results: The authors investigate metal and truncation artifacts in patient data sets acquired with a clinical CT and limited angle artifacts in an anthropomorphic head phantom data set acquired with a gantry-based flat detector CT device. In all cases, the corrected images obtained by PBAC are nearly artifact-free. Compared to conventional correction methods, PBAC achieves better artifact suppression while preserving the patient-specific anatomy at the same time. Further, the authors show that prominent anatomical details in the prior image seem to have only minor impact on the correction result. Conclusions: The results show that PBAC has the potential to effectively correct for metal, truncation, and limited angle artifacts if adequate prior data are available. Since the proposed method makes use of a generalized algorithm, PBAC may also be applicable to other artifacts resulting from missing or corrupt data.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Resilient Cooperative Voltage Control for Distribution Network with High Penetration Distributed Energy Resources

Here, this paper considers the problem of designing a resilient distributed voltage control algorithm for distribution systems with high penetration of distributed energy resources in the presence of an unknown cyber-attack. The purpose of the attack is to force the system to violate the operating voltage limit by intercepting its communication channels and inserting exogenous signals to perturb and/or modify the information being exchanged. We first review the cooperative voltage control proposed in our previous work and provide a new stability analysis for it. Next, we present a resilient cooperative voltage control algorithm by introducing a virtual system interconnected with the original system such that the voltage can be maintained within the operational limit under unknown attacks. The resiliency of the proposed algorithm is demonstrated via simulations on the IEEE 8500-node system when subjected to an attack which consists of corrupting the data being exchanged in the communication network between two generation units.

cooperative control↗

Final scientific report for DOE award title: Improving the Representation of Ice Sedimentation Rates in Global Climate Models

It is well known that cirrus clouds play a major role in regulating the earth’s climate, but the details of how this works are just beginning to be understood. This project targeted the main property of cirrus clouds that influence climate processes; the ice fall speed. That is, this project improves the representation of the mass-weighted ice particle fall velocity, V m , in climate models, used to predict future climate on global and regional scales. Prior to 2007, the dominant sizes of ice particles in cirrus clouds were poorly understood, making it virtually impossible to predict how cirrus clouds interact with sunlight and thermal radiation. Due to several studies investigating the performance of optical probes used to measure the ice particle size distribution (PSD), as well as the remote sensing results from our last ARM project, it is now well established that the anomalously high concentrations of small ice crystals often reported prior to 2007 were measurement artifacts. Advances in the design and data processing of optical probes have greatly reduced these ice artifacts that resulted from the shattering of ice particles on the probe tips and/or inlet tube, and PSD measurements from one of these improved probes (the 2-dimensional Stereo or 2D-S probe) are utilized in this project to parameterize V m for climate models. Our original plan in the proposal was to parameterize the ice PSD (in terms of temperature and ice water content) and ice particle mass and projected area (in terms of mass- and area-dimensional power laws or m-D/A-D expressions) since these are the microphysical properties that determine V m , and then proceed to calculate V m from these parameterized properties. But the 2D-S probe directly measures ice particle projected area and indirectly estimates ice particle mass for each size bin. It soon became apparent that the original plan would introduce more uncertainty in the V m calculations than simply using the 2D-S measurements to directly calculate V m . By calculating V m directly from the measured PSD, ice particle projected area and estimated mass, more accurate estimates of V m are obtained. These V m values were then parameterized for climate models by relating them to (1) sampling temperature and ice water content (IWC) and (2) the effective diameter (D e ) of the ice PSD. Parameterization (1) is appropriate for climate models having single-moment microphysical schemes whereas (2) is appropriate for double-moment microphysical schemes and yields more accurate V m estimates. These parameterizations were developed for tropical cirrus clouds, Arctic cirrus, mid-latitude synoptic cirrus and mid-latitude anvil cirrus clouds based on field campaigns in these regions. An important but unexpected result of this research was the discovery of microphysical evidence indicating the mechanisms by which ice crystals are produced in cirrus clouds. This evidence, derived from PSD measurements, indicates that homogeneous freezing ice nucleation dominates in mid-latitude synoptic cirrus clouds, whereas heterogeneous ice nucleation processes dominate in mid-latitude anvil cirrus. Based on these findings, D e was parameterized in terms of temperature (T) for conditions dominated by (1) homo- and (2) heterogeneous ice nucleation. From this, an experiment was designed for global climate models (GCMs). The net radiative forcing from cirrus clouds may be affected by the means ice is produced (homo- or heterogeneously), and this net forcing contributes to climate sensitivity (i.e. the change in mean global surface temperature resulting from a doubling of CO 2 ). The objective of this GCM experiment was to determine how a change in ice nucleation mode affects the predicted global radiation balance. In the first simulation (Run 1), the D e -T relationship for homogeneous nucleation is used at all latitudes, while in the second simulation (Run 2), the D e -T relationship for heterogeneous nucleation is used at all latitudes. For both runs, V m is calculated from D e . Two GCMs were used; the Community Atmosphere Model version 5 (CAM5) and a European GCM known as ECHAM5 (thanks to our European colleagues who collaborated with us). Similar results were obtained from both GCMs in the Northern Hemisphere mid-latitudes, with a net cooling of ~ 1.0 W m -2 due to heterogeneous nucleation, relative to Run 1. The mean global net cooling was 2.4 W m -2 for the ECHAM5 GCM while CAM5 produced a mean global net cooling of about 0.8 W m -2 . This dependence of the radiation balance on nucleation mode is substantial when one considers the direct radiative forcing from a CO 2 doubling is 4 W m -2 . The differences between GCMs in mean global net cooling estimates may demonstrate a need for improving the representation of cirrus clouds in GCMs, including the coupling between microphysical and radiative properties. Unfortunately, after completing this GCM experiment, we learned from the company that provided the 2D-S microphysical data that the data was corrupted due to a computer program coding problem. Therefore the microphysical data had to be reprocessed and reanalyzed, and the GCM experiments were redone under our current ASR project but using an improved experimental design.

54 ENVIRONMENTAL SCIENCES↗

Online real-time learning of dynamical systems from noisy streaming data

Abstract Recent advancements in sensing and communication facilitate obtaining high-frequency real-time data from various physical systems like power networks, climate systems, biological networks, etc. However, since the data are recorded by physical sensors, it is natural that the obtained data is corrupted by measurement noise. In this paper, we present a novel algorithm for online real-time learning of dynamical systems from noisy time-series data, which employs the Robust Koopman operator framework to mitigate the effect of measurement noise. The proposed algorithm has three main advantages: (a) it allows for online real-time monitoring of a dynamical system; (b) it obtains a linear representation of the underlying dynamical system, thus enabling the user to use linear systems theory for analysis and control of the system; (c) it is computationally fast and less intensive than the popular extended dynamic mode decomposition (EDMD) algorithm. We illustrate the efficiency of the proposed algorithm by applying it to identify the Van der Pol oscillator, the chaotic attractor of the Henon map, the IEEE 68 bus system, and a ring network of Van der Pol oscillators.

97 MATHEMATICS AND COMPUTING↗

e RPCA : Robust Principal Component Analysis for Exponential Family Distributions

Abstract Robust principal component analysis (RPCA) is a widely used method for recovering low‐rank structure from data matrices corrupted by significant and sparse outliers. These corruptions may arise from occlusions, malicious tampering, or other causes for anomalies, and the joint identification of such corruptions with low‐rank background is critical for process monitoring and diagnosis. However, existing RPCA methods and their extensions largely do not account for the underlying probabilistic distribution for the data matrices, which in many applications are known and can be highly non‐Gaussian. We thus propose a new method called RPCA for exponential family distributions (), which can perform the desired decomposition into low‐rank and sparse matrices when such a distribution falls within the exponential family. We present a novel alternating direction method of multiplier optimization algorithm for efficient decomposition, under either its natural or canonical parametrization. The effectiveness of is then demonstrated in two applications: the first for steel sheet defect detection and the second for crime activity monitoring in the Atlanta metropolitan area.

Zheng, Xiaojun↗

A Visual Comparison of Silent Error Propagation

High-performance computing (HPC) systems play a critical role in facilitating scientific discoveries. Their scale and complexity (e.g., the number of computational units and software stack) continue to grow as new systems are expected to process increasingly more data and reduce computing time. However, with more processing elements, the probability that these systems will experience a random bit-flip error that corrupts a program's output also increases, which is often recognized as silent data corruption. Analyzing the resiliency of HPC applications in extreme-scale computing to silent data corruption is crucial but difficult. An HPC application often contains a large number of computation units that need to be tested, and error propagation caused by error corruption is complex and difficult to interpret. Here, to accommodate this challenge, we propose an interactive visualization system that helps HPC researchers understand the resiliency of HPC applications and compare their error propagation. Our system models an application's error propagation to study a program's resiliency by constructing and visualizing its fault tolerance boundary. Coordinating with multiple interactive designs, our system enables domain experts to efficiently explore the complicated spatial and temporal correlation between error propagations. At the end, the system integrated a nonmonotonic error propagation analysis with an adjustable graph propagation visualization to help domain experts examine the details of error propagation and answer such questions as why an error is mitigated or amplified by program execution.

97 MATHEMATICS AND COMPUTING↗

MRI-Based Computed Tomography Metal Artifact Correction Method for Improving Proton Range Calculation Accuracy

Purpose: Computed tomography (CT) artifacts can severely degrade dose calculation accuracy in proton therapy. Prompted by the recently increased popularity of magnetic resonance imaging (MRI) in the radiation therapy clinic, we developed an MRI-based CT artifact correction method for improving the accuracy of proton range calculations. Methods and Materials: The proposed method replaces corrupted CT data by mapping CT Hounsfield units (HU number) from a nearby artifact-free slice, using a coregistered MRI. MRI and CT volumetric images were registered with use of 3-dimensional (3D) deformable image registration (DIR). The registration was fine-tuned on a slice-by-slice basis by using 2D DIR. Based on the intensity of paired MRI pixel values and HU from an artifact-free slice, we performed a comprehensive analysis to predict the correct HU for the corrupted region. For a proof-of-concept validation, metal artifacts were simulated on a reference data set. Proton range was calculated using reference, artifactual, and corrected images to quantify the reduction in proton range error. The correction method was applied to 4 unique clinical cases. Results: The correction method resulted in substantial artifact reduction, both quantitatively and qualitatively. On respective simulated brain and head and neck CT images, the mean error was reduced from 495 and 370 HU to 108 and 92 HU after correction. Correspondingly, the absolute mean proton range errors of 2.4 cm and 1.7 cm were reduced to less than 2 mm in both cases. Conclusions: Our MRI-based CT artifact correction method can improve CT image quality and proton range calculation accuracy for patients with severe CT artifacts.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

TH-CD-206-12: Image-Based Motion Estimation for Plaque Visualization in Coronary Computed Tomography Angiography

Purpose: Visualization and quantification of coronary artery calcification and atherosclerotic plaque benefits from coronary artery motion (CAM) artifact elimination. This work applies a rigid linear motion model to a Volume of Interest (VoI) for estimating motion estimation and compensation of image degradation in Coronary Computed Tomography Angiography (CCTA). Methods: In both simulation and testbench experiments, translational CAM was generated by displacement of the imaging object (i.e. simulated coronary artery and explanted human heart) by ∼8 mm, approximating the motion of a main coronary branch. Rotation was assumed to be negligible. A motion degraded region containing a calcification was selected as the VoI. Local residual motion was assumed to be rigid and linear over the acquisition window, simulating motion observed during diastasis. The (negative) magnitude of the image gradient of the reconstructed VoI was chosen as the motion estimation objective and was minimized with Covariance Matrix Adaptation Evolution Strategy (CMAES). Results: Reconstruction incorporated the estimated CAM yielded signification recovery of fine calcification structures as well as reduced motion artifacts within the selected local region. The compensated reconstruction was further evaluated using two image similarity metrics, the structural similarity index (SSIM) and Root Mean Square Error (RMSE). At the calcification site, the compensated data achieved a 3% increase in SSIM and a 91.2% decrease in RMSE in comparison with the uncompensated reconstruction. Conclusion: Results demonstrate the feasibility of our image-based motion estimation method exploiting a local rigid linear model for CAM compensation. The method shows promising preliminary results for the application of such estimation in CCTA. Further work will involve motion estimation of complex motion corrupted patient data acquired from clinical CT scanner.

60 APPLIED LIFE SCIENCES↗

WeakIdent: Weak formulation for identifying differential equation using narrow-fit and trimming

Data-driven identification of differential equations is an interesting but challenging problem, especially when the given data are corrupted by noise. When the governing differential equation is a linear combination of various differential terms, the identification problem can be formulated as solving a linear system, with the feature matrix consisting of linear and nonlinear terms multiplied by a coefficient vector. This product is equal to the time derivative term, and thus generates dynamical behaviors. The goal is to identify the correct terms that form the equation to capture the dynamics of the given data. We propose a general and robust framework to recover differential equations using a weak formulation with two new mechanisms, narrow-fit and trimming, for both ordinary and partial differential equations (ODEs and PDEs). The weak formulation facilitates an efficient and robust way to handle noise, and two new mechanisms, narrow-fit and trimming, improve the coefficient support and value recoveries respectively. For each sparsity level, Subspace Pursuit is utilized to find an initial set of support from the large dictionary. Then, we focus on highly dynamic regions (rows of the feature matrix), and error normalize the feature matrix in the narrow-fit step. The support is further updated via trimming the terms that contribute the least. Finally, the support set of features with the smallest Cross-Validation error is chosen as the result. A comprehensive set of numerical experiments are presented for both systems of ODEs and PDEs with various noise levels. The proposed method gives a robust recovery of the coefficients, and a significant denoising effect which can handle up to 100% noise-to-signal ratio for some equations. We compare the proposed method with several state-of-the-art algorithms for the recovery of differential equations.

97 MATHEMATICS AND COMPUTING↗

PARIS: Predicting application resilience using machine learning

The traditional method to study application resilience to errors in HPC applications uses fault injection (FI), a time-consuming approach. Furthermore, while analytical models have been built to overcome the inefficiencies of FI, they lack accuracy. In this paper, we present PARIS, a machine-learning method to predict application resilience that avoids the time-consuming process of random FI and provides higher prediction accuracy than analytical models. PARIS captures the implicit relationship between application characteristics and application resilience, which is difficult to capture using most analytical models. We overcome many technical challenges for feature construction, extraction, and selection to use machine learning in our prediction approach. Our evaluation on 16 HPC benchmarks shows that PARIS achieves high prediction accuracy. PARIS is up to 450x faster than random FI (49x on average). Compared to the state-of-the-art analytical model, PARIS is at least 63% better in terms of accuracy and has comparable execution time on average.

97 MATHEMATICS AND COMPUTING↗

Spontaneous Raman–LIF–CO–OH measurements of species concentration in turbulent spray flames

This paper presents new measurements of species concentrations, temperature and mixture fraction in selected regions of a turbulent ethanol spray flame. The line-Raman–LIF–COsingle bondOH setup developed at the Sandia's Combustion Research Facility is utilised to probe regions of a spray flame where laser breakdown of liquid droplets is avoided and the remaining interferences can be corrected. The spray flame is stabilised on the piloted Sydney needle spray burner, where axial translation of the liquid injecting needle in the air-blast stream can transition the spray from dilute to dense. The solution to obtaining successful measurements is found to be multifaceted and includes: the appropriate selection of flame conditions; high sensitivity of the Raman detection system permitting reduced laser energies; development of a pre-processing algorithm to reject strong droplet interferences; and application of the hybrid matrix inversion method combined with wavelet denoising to account for interference corrections and noise at the very low signal levels obtained. Unique and necessary for the successful measurements reported in this paper, a pre-processing algorithm is outlined that removes data points corrupted with strong interferences from droplets. These interferences arise from a range of sources, but the most intense are due to the laser interaction with surrounding mist or liquid fragments, such that measurements near the jet centreline are corrupted and hence discarded. Reliable measurements of mixture fraction, temperature obtained from the sum of the species number densities, and species mole fractions are reported for regions in the flames sufficiently far from the centreline. The paper demonstrates the feasibility of the judicious use of Raman scattering in turbulent spray flames, the results of which will be extremely useful for validating numerical simulations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Holographic phase retrieval and reference design

Here, a general mathematical framework and recovery algorithm is introduced for the holographic phase retrieval problem. In this problem, which arises in holographic coherent diffraction imaging, a 'reference' portion of the signal to be recovered via phase retrieval is a priori known from experimental design. A generic formula is additionally derived for the expected recovery error when the measurement data is corrupted by Poisson shot noise. This facilitates an optimization perspective towards reference design and analysis. We employ this optimization perspective towards quantifying the performance of various reference choices.

97 MATHEMATICS AND COMPUTING↗

Algorithm-Based Fault Tolerance for Convolutional Neural Networks

Convolutional neural networks (CNNs) are becoming more and more important for solving challenging and critical problems in many fields. CNN inference applications have been deployed in safety-critical systems, which may suffer from soft errors caused by high-energy particles, high temperature, or abnormal voltage. Of critical importance is ensuring the stability of the CNN inference process against soft errors. Traditional fault tolerance methods are not suitable for CNN inference because error-correcting code is unable to protect computational components, instruction duplication techniques incur high overhead, and existing algorithm-based fault tolerance (ABFT) techniques cannot protect all convolution implementations. In this paper, we focus on how to protect the CNN inference process against soft errors as efficiently as possible, with the following three contributions. (1) We propose several systematic ABFT schemes based on checksum techniques and analyze their fault protection ability and runtime thoroughly. Unlike traditional ABFT based on matrix-matrix multiplication, our schemes support any convolution implementations. (2) We design a novel workflow integrating all the proposed schemes to obtain a high detection/correction ability with limited total runtime overhead. (3) We perform our evaluation using ImageNet with well-known CNN models including AlexNet, VGG-19, ResNet-18, and YOLOv2. Here, experimental results demonstrate that our implementation can handle soft errors with very limited runtime overhead (4%~8% in both error-free and error-injected situations).

97 MATHEMATICS AND COMPUTING↗