Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data corruption”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Dynamically Reconfigurable Data Readout of Pixel Detectors for Automatic Synchronization with Data Acquisition Systems

Reconfigurable detectors with dynamically selectable sensing and readout modes are highly desirable for implementing edge computing as well as enabling advanced imaging techniques such as foveation. The concept of a camera system capable of simultaneous passive imaging and dynamic ranging in different regions of the detector is presented. Such an adaptive-autonomous detector with both spatial and temporal control requires programmable window of exposure (time frames), ability to switch between readout modes such as full-frame imaging and zero-suppressed data, modification of the number of pixel data bits and independent programmability for distinct detector regions. In this work, a method is presented for seamlessly changing time frames and readout modes without data corruption while still ensuring that the data acquisition system (DAQ) does not need to stop and resynchronize at each change of setting, thus avoiding significant dead time. Data throughput is maximized by using a minimum unique data format, rather than lengthy frame headers, to differentiate between consecutive frames. A data control and transmitter (DCT) synchronizes data transfer from the pixel to the periphery, reconfigures the data to transmit it serially off-chip, while providing optimized decision support based on a DAQ definable mode. Measurements on a test structure demonstrate that the DCT can operate at 1 GHz in a 65 nm LP CMOS process.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Nuclear Data Management and Analysis System Plan

The United States Department of Energy Advanced Reactor Technologies Program was formed in Fiscal Year 2015 and encompasses the Next Generation Nuclear Plant Project and Very High Temperature Reactor (VHTR) Program as they were known previously. The VHTR Program was created to support design and licensing of the first VHTR nuclear plant. Data created for and used by the program must be qualified for use, stored in a readily accessible electronic form, categorized to assure the correct data are used, and controlled to prevent data corruption or inadvertent changes. The Nuclear Data Management and Analysis System was designed to support the data needs of the VHTR Program, at the time and now the Advanced Reactor Technologies Program. Since its inception, use of the Nuclear Data Management and Analysis System has expanded to support additional projects and programs with similar requirements for control, analysis, and availability of large data sets.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2016

Exploring the Interplay of Resilience and Energy Consumption for a Task-Based Partial Differential Equations Preconditioner

We discuss algorithm-based resilience to silent data corruption (SDC) in a task- based domain-decomposition preconditioner for partial differential equations (PDEs). The algorithm exploits a reformulation of the PDE as a sampling problem, followed by a solution update through data manipulation that is resilient to SDC. The implementation is based on a server-client model where all state information is held by the servers, while clients are designed solely as computational units. Scalability tests run up to ~ 51 K cores show a parallel efficiency greater than 90%. We use a 2D elliptic PDE and a fault model based on random single bit-flip to demonstrate the resilience of the application to synthetically injected SDC. We discuss two fault scenarios: one based on the corruption of all data of a target task, and the other involving the corruption of a single data point. We show that for our application, given the test problem considered, a four-fold increase in the number of faults only yields a 2% change in the overhead to overcome their presence, from 7% to 9%. We then discuss potential savings in energy consumption via dynamics voltage/frequency scaling, and its interplay with fault-rates, and application overhead.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Online Detection of Power Grid Anomalies via Federated Learning

Data from sensors is critical for advanced applica- tions that support efficient, reliable, and resilient electric grid operations. Historically, data from phasor measurement units (PMU) has been utilized to develop a wide variety of wide area control and protection applications suitable for power grid control centers. However, until now, most of these could not be deployed for automated operations due to a set of data corruption challenges and uncertainty in the incoming data pipeline. In this paper, we address the problem of detecting different variety of anomalies that are evident in different high- speed power grid measurements. The paper discusses a workflow for handling problems with data acquisition and highlights some of the key findings suitable for anomaly detection in a centralized and distributed environment. The effectiveness of the proposed method was demonstrated with results utilizing realistic PMU datasets

Shinkle, Matthew W.↗
DOE OSTI2022

Data recovery via covert cognizance for unattended operational resilience

One of the important premises of unattended operation, a highly promoted characteristic of fission batteries and advanced microreactors, is the ability to automate the analysis of sensors data used in support of operational monitoring and control. Here, to meet this vision, this work proposes a new monitoring and data recovery paradigm to ensure resilience against data corruption which may be the result of malicious intrusion into the reactor operational network. This is paramount to ensure 100% availability under contingency scenarios such as cyberattacks. In support of this vision, earlier work has presented the concept of covert cognizance and demonstrated its mathematical ability to identify and embed cognizance parameters under the noise-dominated null space of the sensors data. This work extends this concept and applies it in real-time to demonstrate three key characteristics: zero-impact, zero-observability, and data recovery, where the first characteristic is to ensure no impact on operation, the second is immunity to discovery by pattern recognition techniques, and the third is to allow recovery of corrupt or falsified data. Recognizing that fission batteries are designed to operate under steady state most of the time, we elect to employ a small modular reactor model under transient operational conditions to demonstrate the operational resilience enabled by the covert cognizance paradigm. Specifically, the PI controller is augmented with the covert cognizance modules to develop self-awareness and enable automatic data recovery. The developed modules are expected to be equally applicable to a wide range of advanced reactor technologies relying on full or partial unattended control.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
NASA NTRS2020

Data Accountability and Uncertainty Analysis for the Mars Science Laboratory

This paper presents machine learning-based approaches to automate and optimize the detection of volume loss for the downlink process of telemetry data from the Mars Curiosity Rover. The Curiosity observes volume loss and data corruption, requiring re-transmits from the rover and Ground Data System Analysts (GDSA) to monitor the data flow. To resolve this issue, we created a data pipeline to accumulate data from various data sources in the downlink process and detect where the data is missed. In this paper, we benchmarked different methodologies based on the accuracy and excitability of them to identify whether a downlink data that is received to the ground system is complete or incomplete. Our results show that machine learning methods can improve the performance of the GDSA by 55% while the user can diagnose why data is missed and provide an explanation for the data accountability problem.

Chowdhury, Ameera↗
DOE OSTI2021

Organic contamination detection for isotopic analysis of water by laser spectroscopy

Hydrogen and oxygen stable isotope ratios (δ 2 H, δ 17 O, and δ 18 O values) are commonly used tracers of water. These ratios can be measured by isotope ratio infrared spectroscopy (IRIS). However, IRIS approaches are prone to errors induced by organic compounds present in plant, soil, and natural water samples. A novel approach using 17 O-excess values has shown promise for flagging spectrally contaminated plant samples during IRIS analysis. A systematic assessment of this flagging system is needed to prove it useful. Errors induced by methanol and ethanol water mixtures on measured IRIS and isotope ratio mass spectrometry (IRMS) results were evaluated. For IRIS analyses both liquid- and vapour-mode (via direct vapour equilibration) methods are used. The δ 2 H, δ 17 O, and δ 18 O values were measured and compared with known reference values to determine the errors induced by methanol and ethanol contamination. In addition, the 17 O-excess contamination detection approach was tested. This is a post-processing detection tool for both liquid and vapour IRIS triple-isotope analyses, utilizing calculated 17 O-excess values to flag contaminated samples. Organic contamination induced significant errors in IRIS results, not seen in IRMS results. Methanol caused larger errors than ethanol. Results from vapour-IRIS analyses had larger errors than those from liquid-IRIS analyses. The 17 O-excess approach identified methanol driven error in liquid- and vapour-mode IRIS samples at levels where isotope results became unacceptably erroneous. For ethanol contaminated samples, a mix of erroneous and correct flagging occurred with the 17 O-excess method. Here our results indicate that methanol is the more problematic contaminant for data corruption. The 17 O-excess method was therefore useful for data quality control. Organic contamination caused significant errors in IRIS stable isotope results. These errors were larger during vapour analyses than during liquid IRIS analyses, and larger for methanol than ethanol contamination. The 17 O-excess method is highly sensitive for detecting narrowband (methanol) contamination error in vapour and liquid analysis modes in IRIS.

60 APPLIED LIFE SCIENCES↗
NASA NTRS2012

NAND flash screening and qualification guideline for space application

All space missions have a need for nonvolatile memory (NVM), which maintains data integrity when unpowered. Types of NVM include PROM, EEPROM, NOR Flash, and NAND Flash. PROMs, NOR Flash, and EEPROMs are good choices for storing smaller file size data such as boot code or FPGA configurations. These products have excellent data retention characteristics and can reliably store data for years, unpowered, without any data corruption. They can also be read many times without disturbing the data. Another application of NVM is storage of science and engineering data, which requires large of amounts of memory. The highest density memories available today are SDRAM and NAND Flash. However, the power required to operate and store data in NAND Flash is far less than SDRAM. Wherever high density and low power is required, NAND Flash is very attractive.

Heidecker, Jason↗
DOE OSTI2023

Spatio-Temporal Denoising Graph Autoencoders with Data Augmentation for Missing Photovoltaic Data Imputation

The integration of the global Photovoltaic (PV) market with real time data-loggers has enabled large scale PV data analytical pipelines for power forecasting and long-term reliability assessment of PV fleets. Nevertheless, the performance of PV data analysis heavily depends on the quality of PV timeseries data. This paper proposes a novel Spatio-Temporal Denoising Graph Autoencoder (STD-GAE) framework to impute missing PV Power Data. STDGAE exploits temporal correlation, spatial coherence, and value dependencies from domain knowledge to recover missing data. It is empowered by two modules. (1) To cope with sparse yet various scenarios of missing data, STD-GAE incorporates a domain-knowledge aware data augmentation module that creates plausible variations of missing data patterns. This generalizes STD-GAE to robust imputation over different seasons and environment. (2) STD-GAE nontrivially integrates spatiotemporal graph convolution layers (to recover local missing data by observed “neighboring” PV plants) and denoising autoencoder (to recover corrupted data from augmented counterpart) to improve the accuracy of imputation accuracy at PV fleet level. We have evaluated our proposed model on two realworld PV datasets. Experimental results show that STD-GAE can achieve a gain of 43.14% in imputation accuracy and remains less sensitive to missing rate, different seasons, and missing scenarios, compared with state-of-the-art data imputation methods such as MIDA and LRTC-TNN.

Fan, Yangxin↗
NASA NTRS1993

Magellan Recorder Data Recovery Algorithms

This paper describes algorithms implemented by the Magellan High Rate Processor to recover radar data corrupted by the failure of an onboard tape recorder that dropped bits. For data with error correction coding, an algorithm was developed that decodes data in the presence of bit errors and missing bits.

Error↗
DOE OSTI2020

MFNets: Multifidelity data-driven networks for Bayesian learning and prediction

This paper presents a multifidelity uncertainty quantification framework called MFNets. We seek to address three existing challenges that arise when experimental and simulation data from different sources are used to enhance statistical estimation and prediction with quantified uncertainty. Specifically, we demonstrate that MFNets can (1) fuse heterogeneous data sources arising from simulations with different parameterizations, e.g simulation models with different uncertain parameters or data sets collected under different environmental conditions; (2) encode known relationships among data sources to reduce data requirements; and (3) improve the robustness of existing multi-fidelity approaches to corrupted data. MFNets construct a network of latent variables (LVs) to facilitate the fusion of data from an ensemble of sources of varying credibility and cost. These LVs are posited as explanatory variables that provide the source of correlation in the observed data. Furthermore, MFNets provide a way to encode prior physical knowledge to enable efficient estimation of statistics and/or construction of surrogates via conditional independence relations on the LVs. We highlight the utility of our framework with a number of theoretical results which assess the quality of the posterior mean as a frequentist estimator and compare it to standard sampling approaches that use single fidelity, multilevel, and control variate Monte Carlo estimators. We also use the proposed framework to derive the Monte Carlo-based control variate estimator entirely from the use of Bayes rule and linear-Gaussian models -- to our knowledge the first such derivation. Finally, we demonstrate the ability to work with different uncertain parameters across different models.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1982

Reconstruction of atmospheric pollutant concentrations from remote sensing data - An application of distributed parameter observer theory

The reconstruction of a concentration distribution from spatially averaged and noise-corrupted data is a central problem in processing atmospheric remote sensing data. Distributed parameter observer theory is used to develop reconstructibility conditions for distributed parameter systems having measurements typical of those in remote sensing. The relation of the reconstructibility condition to the stability of the distributed parameter observer is demonstrated. The theory is applied to a variety of remote sensing situations, and it is found that those in which concentrations are measured as a function of altitude satisfy the conditions of distributed state reconstructibility.

Koda, M.↗
DOE OSTI2018

LADR: low-cost application-level detector for reducing silent output corruptions

Applications running on future high performance computing (HPC) systems are more likely to experience transient faults due to technology scaling trends with respect to higher circuit density, smaller transistor size and near-threshold voltage (NTV) operations. A transient fault could corrupt application state without warning, possibly leading to incorrect application output. Such errors are called silent data corruptions (SDCs). In this paper, we present LADR, a low-cost application-level SDC detector for scientific applications. LADR protects scientific applications from SDCs by watching for data anomalies in their state variables (those of scientific interest). It employs compile-time data-flow analysis to minimize the number of monitored variables, thereby reducing runtime and memory overheads while maintaining a high level of fault coverage with low false positive rates. We evaluated LADR with 4 scientific workloads and results show that LADR achieved < 80% fault coverage with only ∼ 3% runtime overheads and ∼ 1% memory overheads. As compared to prior state-of-the-art anomaly-based detection methods, SDC achieved comparable or improved fault coverage, but reduced runtime overheads by 21% ∼ 75%, and memory overheads by 35% ∼ 55% for the evaluated workloads. We believe that such an approach with low memory and runtime overheads coupled with attractive detection precision makes LADR a viable approach for assuring the correct output from large-scale high performance simulations.

Chen, Chao↗
NASA NTRS2004

Software Implements a Space-Mission File-Transfer Protocol

CFDP is a computer program that implements the CCSDS (Consultative Committee for Space Data Systems) File Delivery Protocol, which is an international standard for automatic, reliable transfers of files of data between locations on Earth and in outer space. CFDP administers concurrent file transfers in both directions, delivery of data out of transmission order, reliable and unreliable transmission modes, and automatic retransmission of lost or corrupted data by use of one or more of several lost-segment-detection modes. The program also implements several data-integrity measures, including file checksums and optional cyclic redundancy checks for each protocol data unit. The metadata accompanying each file can include messages to users application programs and commands for operating on remote file systems.

Rundstrom, Kathleen↗
DOE OSTI2022

Physics-informed machine learning modeling for predictive control using noisy data

Due to the occurrence of over-fitting at the learning phase, the modeling of chemical processes via artificial neural networks (ANN) by using corrupted data (i.e., noisy data) is an ongoing challenge. Therefore, this work investigates the effect of both Gaussian and non-Gaussian noise on the performance of process-structure based recurrent neural networks (RNN) models, which take the form of partially-connected RNN models in this work, that are used to approximate a class of multi-input-multi-outputs nonlinear systems. Furthermore, two different techniques, specifically Monte Carlo dropout and co-teaching, are utilized in the development of partially-connected RNN models. Here, these two techniques are employed to reduce the over-fitting in ANNs when noisy data is used in the training process and, hence, to improve the open-loop accuracy as well as the closed-loop performance under a Lyapunov-based model predictive controller (MPC). Aspen Plus Dynamics, a well-known high-fidelity process simulator, is used to simulate a large-scale chemical process application in order to demonstrate the anticipated improvements in both open-loop approximation and closed-loop controller performance in the presence of Gaussian and non-Gaussian noise in the data set using physics-informed RNNs.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1993

Neuromorphic Learning From Noisy Data

Two reports present numerical study of performance of feedforward neural network trained by back-propagation algorithm in learning continuous-valued mappings from data corrupted by noise. Two types of noise considered: plant noise which affects dynamics of controlled process and data-processing noise, which occurs during analog processing and digital sampling of signals. Study performed with view toward use of neural networks as neurocontrollers to substitute for, or enhance, performances of human experts in controlling mechanical devices in presence of sensor and actuator noise and to enhance performances of more-conventional digital feedback electronic process controllers in noisy environments.

Merrill, Walter C.↗
NASA NTRS1991

Computed Flow About The Integrated Space Shuttle

Report discusses numerical simulations of flow of air about integrated Space Shuttle in ascent. Goal: to improve understanding of, and ability to predict, how integrated Space Shuttle will perform during both nominal and aborted ascent under various conditions. These and other numerical simulations intended both to supplement wind-tunnel data, corrupted to some extent by scaling and wall-interference effects, and to compensate for scarcity of valid flight data.

Buning, P. G.↗
NASA NTRS1980

Development of advanced techniques for rotorcraft state estimation and parameter identification

An integrated methodology for rotorcraft system identification consists of rotorcraft mathematical modeling, three distinct data processing steps, and a technique for designing inputs to improve the identifiability of the data. These elements are as follows: (1) a Kalman filter smoother algorithm which estimates states and sensor errors from error corrupted data. Gust time histories and statistics may also be estimated; (2) a model structure estimation algorithm for isolating a model which adequately explains the data; (3) a maximum likelihood algorithm for estimating the parameters and estimates for the variance of these estimates; and (4) an input design algorithm, based on a maximum likelihood approach, which provides inputs to improve the accuracy of parameter estimates. Each step is discussed with examples to both flight and simulated data cases.

Hall, W. E., Jr.↗