SEARCH · Engineering Papers
Results for “Bad data”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Bad data packet capture device
An apparatus and method for capturing data packets for analysis on a network computing system includes a sending node and a receiving node connected by a bi-directional communication link. The sending node sends a data transmission to the receiving node on the bi-directional communication link, and the receiving node receives the data transmission and verifies the data transmission to determine valid data and invalid data and verify retransmissions of invalid data as corresponding valid data. A memory device communicates with the receiving node for storing the invalid data and the corresponding valid data. A computing node communicates with the memory device and receives and performs an analysis of the invalid data and the corresponding valid data received from the memory device.
Robust Matrix Completion State Estimation in Distribution Systems
Due to the insufficient measurements in the distribution system state estimation (DSSE), full observability and redundant measurements are difficult to achieve without using the pseudo measurements. The matrix completion state estimation (MCSE) combines the matrix completion and power system model to estimate voltage by exploring the low-rank characteristics of the matrix. This paper proposes a robust matrix completion state estimation (RMCSE) to estimate the voltage in a distribution system under a low-observability condition. Tradition state estimation weighted least squares (WLS) method requires full observability to calculate the states and needs redundant measurements to proceed a bad data detection. The proposed method improves the robustness of the MCSE to bad data by minimizing the rank of the matrix and measurements residual with different weights. It can estimate the system state in a low-observability system and has robust estimates without the bad data detection process in the face of multiple bad data. The method is numerically evaluated on the IEEE 33-node radial distribution system. The estimation performance and robustness of RMCSE are compared with the WLS with the largest normalized residual bad data identification (WLS-LNR), and the MCSE.
Robust Matrix Completion State Estimation in Distribution Systems: Preprint
Due to the insufficient measurements in the distribution system state estimation (DSSE), full observability and redundant measurements are difficult to achieve without using the pseudo measurements. The matrix completion state estimation (MCSE) combines the matrix completion and power system model to estimate voltage by exploring the low-rank characteristics of the matrix. This paper proposes a robust matrix completion state estimation (RMCSE) to estimate the voltage in a distribution system under a low-observability condition. Tradition state estimation weighted least squares (WLS) method requires full observability to calculate the states and needs redundant measurements to proceed a bad data detection. The proposed method improves the robustness of the MCSE to bad data by minimizing the rank of the matrix and measurements residual with different weights. It can estimate the system state in a low-observability system and has robust estimates without the bad data detection process in the face of multiple bad data. The method is numerically evaluated on the IEEE 33-node radial distribution system. The estimation performance and robustness of RMCSE are compared with the WLS with the largest normalized residual bad data identification (WLS-LNR), and the MCSE.
LANL Meteorological Program: 2020 Data Completeness/Quality Report
Los Alamos National Laboratory (LANL) operates four mesa-top meteorology towers: Technical Area (TA) 06, TA-49, TA-53, and TA-54. An additional tower is located in Mortandad Canyon (TA-5 MDCN), and a rain gauge at North Community (NCOM). A description of the meteorology monitoring network is found in Dewart and Boggs (2014). Mesa-top towers are instrumented at 1.2 meters (m), 11.5 m, 23 m, and 46 m. In addition, TA-06 is instrumented at 92 m. The TA-5 MDCN tower is 10 m in height and is instrumented at 1.2 m and 10 m. Data are collected every 15 minutes. Range checking is done on each measurement every 15 minutes; data that are beyond normal ranges are eliminated from the data set and replaced by a code for missing data. In addition, data are reviewed weekly by meteorologists to identify bad data not identified by range checking. The data steward eliminates these data from the data set and replaces them with a code for missing data. The instrument technicians also review that data and schedule instrument replacement as required. Data completeness is determined by the number of total 15-minute records available versus the total number of possible measurements for the entire year. As a rule, the meteorologists do not attempt to estimate data that are eliminated as bad data. Original datalogger records, containing bad data, can be recalled from program archival storage.
LANL Meteorological Program: 2022 Data Completeness/Quality Report
Los Alamos National Laboratory (LANL) operates seven mesa-top instrumented meteorology towers: Technical Area (TA) 6, TA-49, TA-53, TA-54, TA-63, TA-54B, and TA-16B. An additional instrumented tower is located in Mortandad Canyon (TA-5 MDCN), and there is a rain gauge at North Community (NCOM), located within the town of Los Alamos. The 10 meter (m) towers at TA-63, TA- 54B, and TA-16B were installed in 2021, and will be included in the 2023 data completeness/quality report. A description of the meteorology monitoring network, prior to the installation of the TA-63, TA-54B, and TA-16B is found in Dewart and Boggs (2014). Four of the mesa-top towers (e.g., TA-6, TA-49, TA-53, and TA-54) are instrumented at the 1.2 m, 11.5 m, 23 m, and 46 m levels. In addition, the TA-6 tower is instrumented at the 92 m level. The TA-5 MDCN tower is 10 m in height and is instrumented at 1.2 m and 10 m. Data are collected and averaged every 15 minutes. Range checking is performed on each measurement every 15 minutes; data that are beyond normal ranges are eliminated from the data set and replaced by a code for missing data. In addition, data are reviewed weekly by qualified meteorologists to identify bad data not identified by the range checking technique. The data steward eliminates these data from the data set and replaces them with a code for missing data. The instrument technicians also review that data and schedule instrument replacement, as required. All instruments are calibrated at a frequency that meets the criteria identified in ANSI/ANS-3.11-2015. Data completeness is determined by the number of total 15-minute records available versus the number of possible measurements for the entire year. As a rule, the meteorologists do not attempt to estimate data that are eliminated as bad data. Original datalogger records, including bad data, can be recalled from program archival storage.
LANL Meteorological Program: 2023 Data Completeness/Quality Report
Los Alamos National Laboratory (LANL) operates seven mesa-top instrumented meteorology towers: Technical Area (TA) 6, TA-49, TA-53, TA-54, TA-63, TA-54B, and TA-16B. An additional instrumented tower is located in Mortandad Canyon (TA-5 MDCN), and there is a rain gauge at North Community (NCOM), located within the town of Los Alamos. The 10 meter (m) towers at TA-63, TA-54B, and TA-16B have been in testing since they were installed in 2021, and will be included in a future data completeness report. A description of the meteorology monitoring network, prior to the installation of the TA-63, TA-54B, and TA-16B is found in Dewart and Boggs (2014). Four of the mesa-top towers (e.g., TA-6, TA-49, TA-53, and TA-54) are instrumented at the 1.2 m, 11.5 m, 23 m, and 46 m levels. In addition, the TA-6 tower is instrumented at the 92 m level. The TA-5 MDCN tower is 10 m in height and is instrumented at 1.2 m and 10 m. Data are collected and averaged every 15 minutes. Range checking is performed on each measurement every 15 minutes; data that are beyond normal ranges are eliminated from the data set and replaced by a code for missing data. In addition, data are reviewed weekly by qualified meteorologists to identify bad data not identified by the range checking technique. The data steward eliminates these data from the data set and replaces them with a code for missing data. The instrument technicians also review that data and schedule instrument replacement, as required. All instruments are calibrated at a frequency that meets the criteria identified in ANSI/ANS-3.11-2015. Data completeness is determined by the number of total 15-minute records available versus the number of possible measurements for the entire year. As a rule, the meteorologists do not attempt to estimate data that are eliminated as bad data. Original datalogger records, including bad data, can be recalled from program archival storage.
Correlation-Aided Robust Decentralized Dynamic State Estimation of Power Systems with Unknown Control Inputs
This paper proposes a correlation-aided robust adaptive unscented Kalman filter for power system decentralized dynamic state estimation with unknown inputs, termed as robust AUKF-UI. The temporal and spatial correlations among the unknown inputs are used to derive a vector auto-regressive (VAR) model in an adaptive manner. This VAR model is further integrated together with state transition and measurement models for joint state and unknown inputs estimation. This allows taking into account the implicit cross-correlations between the states and the unknown inputs. As a result, the rank requirement for unknown input vector estimation is relaxed and the local generator frequency measurement is not required. The temporal correlations of time series innovation vectors, predicted state and input vectors are also leveraged by the robust AUKFUI to detect, identify and process bad data. Without these correlations, it is very challenging to address bad data with unknown inputs. Simulation results carried out on the IEEE 39-bus system demonstrate that the proposed robust AUKF-UI achieves much better results than other methods in the presence of low measurement redundancy, strong nonlinearity, and bad data.
A Robust State Estimation Framework Considering Measurement Correlations and Imperfect Synchronization
This paper develops a robust power system state estimation framework with the consideration of measurement correlations and imperfect synchronization. In the framework, correlations of SCADA and PMU measurements are calculated separately through unscented transformation and a vector autoregression (VAR) model. In particular, PMU measurements during the waiting period of two SCADA measurement scans are buffered to develop the VAR model. The latter takes into account the temporal and spatial correlations of PMU measurements and provides redundant measurements to suppress bad data and mitigate imperfect synchronization. In case of synchronization issues between SCADA and PMU measurements, either forecasted PMU measurements or prior SCADA measurements from the last estimation run are leveraged to restore system observability. After that, robust generalized maximum-likelihood (GM)-estimator is developed to integrate full measurement correlations and handle bad data in SCADA and PMU measurements. Thanks to the robustness and flexibility of the proposed robust estimator, bad data as well as imperfect measurement synchronization are mitigated, yielding high statistical efficiency. Comprehensive comparison results with other alternatives under various conditions demonstrate the benefits of the proposed framework.
Stationary table CT dosimetry and anomalous scanner-reported values of CTDI{sub vol}
Purpose: Anomalous, scanner-reported values of CTDI{sub vol} for stationary phantom/table protocols (having elevated values of CTDI{sub vol} over 300% higher than the actual dose to the phantom) have been observed; which are well-beyond the typical accuracy expected of CTDI{sub vol} as a phantom dose. Recognition of these outliers as “bad data” is important to users of CT dose index tracking systems (e.g., ACR DIR), and a method for recognition and correction is provided. Methods: Rigorous methods and equations are presented which describe the dose distributions for stationary-table CT. A comparison with formulae for scanner-reported values of CTDI{sub vol} clearly identifies the source of these anomalies. Results: For the stationary table, use of the CTDI{sub 100} formula (applicable to a moving phantom only) overestimates the dose due to extra scatter and also includes an overbeaming correction, both of which are nonexistent when the phantom (or patient) is held stationary. The reported DLP remains robust for the stationary phantom. Conclusions: The CTDI-paradigm does not apply in the case of a stationary phantom and simpler nonintegral equations suffice. A method of correction of the currently reported CTDI{sub vol} using the approach-to-equilibrium formula H(a) and an overbeaming correction factor serves to scale the reported CTDI{sub vol} values to more accurate levels for stationary-table CT, as well as serving as an indicator in the detection of “bad data.”.
AI & Physics-Based Bad Command/Data Detection in Large Power Electronics Systems: Multi-Port Autonomous Reconfigurable Solar Power Plant (MARS)
Detection of bad data from measurement sensors and bad commands from control centers need to be carried out to avoid instabilities within large power electronics systems. Towards the same, in this paper, model and data driven methods are proposed to identify anomalies in measured data and commands received by large power electronics systems. The large power electronics system considered in this paper is a multi-port autonomous reconfigurable solar power plant (MARS), which consists of photovoltaic (PV) and energy storage systems (ESSs) that connect to high-voltage direct current (HVdc) system and transmission ac power grid. The proposed algorithms in the MARS power plant to detect bad data from measurements and bad commands from control centers are evaluated in simulations and hardware-in-the-loop (HIL) tests. Furthermore, it has been observed that the proposed algorithms are able to detect bad measurements and commands in all the use cases evaluated.
Experimental Setup for Grid Control Device Software Updates in Supply Chain Cyber-Security
Supply chain cyberattacks that exploit insecure third-party software are a growing concern for the security of the electric power grid. These attacks seek to deploy malicious software in grid control devices during the fabrication, shipment, installation, and maintenance stages, or as part of routine software updates. Malicious software on grid control devices may inject bad data or execute bad commands, which can cause blackouts and damage power equipment. This paper describes an experimental setup to simulate the software update process of a commercial power relay as part of a hardware-in-the-loop simulation for grid supply chain cyber-security assessment. The laboratory setup was successfully utilized to study three supply chain cyber-security use cases.
Explainable multi-fidelity Bayesian neural network for distribution system state estimation
Distribution System State Estimation (DSSE) is frequently constrained by limited real-time measurements, the uncertainties introduced by distributed energy resources, and the presence of bad data. To address them, this paper proposes an enhanced Multi-Fidelity Bayesian Neural Network (MFBNN) DSSE approach. A low-fidelity layer based on a Deep Neural Network (DNN) is first pre-trained on pseudo-measurement data to learn fundamental state features. Subsequently, a high-fidelity Bayesian Neural Network (BNN) layer leverages limited but high-quality real-time measurements to refine these features, thereby achieving accurate DSSE. Additionally, the deep SHapley Additive exPlanation (SHAP) is developed to quantify the influence of measurement data on DSSE through dual perspectives of global feature importance and local nodal contributions, establishing a hierarchical explainability framework for machine learning-based DSSE. Comparative studies conducted on the IEEE 13-bus system and a real-world 2135-node system from Dominion Energy demonstrate that the proposed method excels in estimation accuracy, even under situations of high noise levels, bad data, and missing data. Further comparisons with Weighted Least Squares (WLS) and other machine learning-based DSSE approaches verify that the proposed framework offers higher accuracy, improved interpretability, and enhanced robustness.
Taylor-Expansion-Based Robust Power Flow in Unbalanced Distribution Systems: A Hybrid Data-Aided Method
Traditional power flow methods often adopt certain assumptions designed for passive balanced distribution systems, thus lacking practicality for unbalanced operation. moreover, their computation accuracy and efficiency are heavily subject to unknown errors and bad data in measurements or prediction data of distributed energy resources (ders). to address these issues, this paper proposes a hybrid data-aided robust power flow algorithm in unbalanced distribution systems, which combines taylor series expansion knowledge with a data-driven regression technique. the proposed method initiates a linearization power flow model to derive an explicitly analytical solution by modified taylor expansion. to mitigate the approximation loss that surges due to the der integration and bad data, we further develop a data-aided robust support vector regression approach to estimate the errors efficiently. comparative analysis in the 13-bus and 123-bus ieee unbalanced feeders shows that the proposed hybrid algorithm achieves superior computational efficiency, with guaranteed accuracy and robustness against outliers.