Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bad data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Data Assimilation Cycling for Weather Analysis

This software package runs the atmospheric model MM5 in data assimilation cycling mode to produce an optimized weather analysis, including the ability to insert or adjust a hurricane vortex. The program runs MM5 through a cycle of short forecasts every three hours where the vortex is adjusted to match the observed hurricane location and storm intensity. This technique adjusts the surrounding environment so that the proper steering current and environmental shear are achieved. MM5cycle uses a Cressman analysis to blend observation into model fields to get a more accurate weather analysis. Quality control of observations is also done in every cycle to remove bad data that may contaminate the analysis. This technique can assimilate and propagate data in time from intermittent and infrequent observations while maintaining the atmospheric field in a dynamically balanced state. The software consists of a C-shell script (MM5cycle.driver) and three FORTRAN programs (splitMM5files.F, comRegrid.F, and insert_vortex.F), and are contained in the pre-processor component of MM5 called "Regridder." The model is first initialized with data from a global model such as the Global Forecast System (GFS), which also provides lateral boundary conditions. These data are separated into single-time files using splitMM5.F. The hurricane vortex is then bogussed in the correct location and with the correct wind field using insert_vortex.F. The modified initial and boundary conditions are then recombined into the model fields using comRegrid.F. The model then makes a three-hour forecast. The three-hour forecast data from MM5 now become the analysis for the next short forecast run, where the vortex will again be adjusted. The process repeats itself until the desired time of analysis is achieved. This code can also assimilate observations if desired.

Tran, Nam↗

Protorheology in practice: Avoiding misinterpretation

Protorheology is the paradigm that any observed flow or deformation is a chance to infer quantitative rheological properties. While this creates many opportunities for insight, there is significant risk of misunderstanding the physics involved, e.g. misinterpreting a liquid as a solid or mistaking viscous flow time as viscoelastic relaxation time. We describe these and other potential mistakes, use case studies to show how serious the problems can be, and contrast misinterpretations with correct approaches and interpretations. Some issues are especially important with materials involving colloidal particles and flows involving surface tension. Whether the reader is making inference from a tilted vial, time-lapse gravity-driven flow, a bounce test, die swell, or any other protorheology observation, the examples here serve as a guide for avoiding bad data in protorheology.

42 ENGINEERING↗

Specifying angular reference for robust three-phase state estimation

While the phase angle of any of the bus voltages can be chosen as the angular reference in the state estimation formulation of positive sequence networks, the same approach does not readily extend to three-phase network state estimation problem. It is commonly assumed that there is at least one bus where the bus voltages are perfectly balanced with phase angles displaced ±120° and these balanced three phase voltages are used as the three-phase reference in solving the three-phase state estimation problem. This assumption may be quite realistic in transmission networks, and for distribution networks with a strong transmission system connection. However, it might not be realistic to assume existence of a perfectly balanced reference bus in today’s distribution systems with ever increasing penetration of renewable sources or for isolated operation of microgrids. In this paper, a novel state estimation formulation will be presented which facilitates correct solution irrespective of the existence of buses with perfectly balanced voltages. The new formulation is general, and lends itself to bad data processing. It yields accurate results in any three-phase power system irrespective of its operating conditions (balanced or highly unbalanced), configuration (isolated microgrid, connected to transmission system, etc.) and whether or not it contains any synchronous generators. The performance of the method is validated using the IEEE 123 bus three-phase system.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Mask effects on cosmological studies with weak-lensing peak statistics

With numerical simulations, we analyze in detail how the bad data removal, i.e., the mask effect, can influence the peak statistics of the weak-lensing convergence field reconstructed from the shear measurement of background galaxies. It is found that high peak fractions are systematically enhanced because of the presence of masks; the larger the masked area is, the higher the enhancement is. In the case where the total masked area is about 13% of the survey area, the fraction of peaks with signal-to-noise ratio ν ≥ 3 is ∼11% of the total number of peaks, compared with ∼7% of the mask-free case in our considered cosmological model. This can have significant effects on cosmological studies with weak-lensing convergence peak statistics, inducing a large bias in the parameter constraints if the effects are not taken into account properly. Even for a survey area of 9 deg{sup 2}, the bias in (Ω {sub m}, σ{sub 8}) is already intolerably large and close to 3σ. It is noted that most of the affected peaks are close to the masked regions. Therefore, excluding peaks in those regions in the peak statistics can reduce the bias effect but at the expense of losing usable survey areas. Further investigations find that the enhancement of the number of high peaks around the masked regions can be largely attributed to the smaller number of galaxies usable in the weak-lensing convergence reconstruction, leading to higher noise than that of the areas away from the masks. We thus develop a model in which we exclude only those very large masks with radius larger than 3' but keep all the other masked regions in peak counting statistics. For the remaining part, we treat the areas close to and away from the masked regions separately with different noise levels. It is shown that this two-noise-level model can account for the mask effect on peak statistics very well, and the bias in cosmological parameters is significantly reduced if this model is applied in the parameter fitting.

79 ASTRONOMY AND ASTROPHYSICS↗

Intelligent Prediction of States in Multi-port Autonomous Reconfigurable Solar power plant (MARS)

In power electronics, prediction of states may be used for identification of faults, determination of aging of components, identification of bad data measurements, among others. Prediction of states in power electronics have broadly been based on: (a) physics-based models, (b) data-driven models, and (c) hybrid models. In this paper, data-driven approaches are presented for intelligent prediction of states in multi-port autonomous reconfigurable solar power plant (MARS) and compared. The data-set needed to train the data-driven models based on artificial intelligence (AI) algorithms has been identified and the trained models are evaluated under different extrapolated normal and abnormal operating conditions. The AI algorithms include nonlinear auto-regressive exogenous model (NARX), spiking neural networks (SNN), and decision tree. The models are compared and contrasted. The best model (NARX) is evaluated under different normal and abnormal operating conditions that have indicated accurate prediction.

Debnath, Suman↗

Net Load Redistribution Attacks on Nodal Voltage Magnitude Estimation in AC Distribution Networks

A high penetration level of smart devices and communication networks increases the threat of cyber-attacks in the distribution system. In this paper, we model a hidden, coordinated, net load redistribution attack (NLRA) in an AC distribution system. Based on local information of an attack region, the attacker’s goal is to create violations in nodal voltage magnitude estimation. Acting as a system operator equipped with global AC state estimation and bad data detection, we validate the stealthiness of the hidden NLRA in multiple attack cases. Simulation results on a modified PG&E 69-node distribution system show the validity of the proposed NLRA. The influence of NLRA on the distribution system is assessed and the impact of attack regions, attack timing, and system observability is also revealed.

Zhang, Hang↗

2020 IEEE PES Innovative Smart Grid Technologies Europe (ISGT-Europe)

Recent proliferation of distributed energy sources in distribution or sub-transmission systems necessitates close monitoring of these three-phase power grids which typically operate under unbalanced loading conditions. Unlike the transmission systems where the network equations are commonly based on the positive sequence component models, a detailed three phase model will have to be used in implementing network applications for these systems. In the specific case of the state estimator, where measurement and parameter errors may bias the solution, bad data and parameter error detection algorithms should also be incorporated. Implementing the state estimator and error detection algorithms for three-phase systems impose additional computational burden and modifications to the state estimation code. This paper proposes a practical solution to avoid these issues by using synchronized phasor measurements and modal decoupling. The previously developed parameter error detection algorithm based on the normalized Lagrange multipliers (NLM) test is applied to the measurements independently in each mode in parallel, not only saving CPU time but also avoiding new code development for a three-phase estimator. Different parameter error scenarios are created and tested to verify the effectiveness of the proposed error detection approach.

Khalili, Ramtin↗

Prediction of Power Measurements Using Adaptive Filters

With the advent of smart grid concept, Internet of Things (IoT) and the deployment of smart meters, the cyberattack threats on power networks have increased due to the use of communication systems that can be accessed by adversaries. Attackers will have the ability to manipulate the outcomes of smart meters which in turn influence the core application of Energy Management System 9EMS): State Estimation (SE). Bad data analytic tools may fail to detect some attacks into measurements. Meanwhile, Machine Learning (ML) solutions have been proposed for detecting False Data Injection (FDI) attacks. However, there is a lack of ML time-series solutions presented in the state-of-the-art that is yet to be complex. In signal processing, time-series solutions do not only consider the signal, but also the statistics of the signal over time. Therefore, in this paper, a machine learning for time-series solutions is presented as an application to model the measurements of the power grid that are used in SE. The presented model takes into account adaptive linear and non-linear filters: Finite Impulse Response (FIR), and Infinite Impulse Response (IIR). The presented models are implemented and performed on the IEEE-118 bus system. The results indicate the advantage of applying those filters over the state-of-the-art machine learning solutions.

Hamad, Khaled↗

Anomaly Detection and Mitigation in FACTS-based Wide-Area Voltage Control Systems using Machine Learning

With the increasing deployment of Flexible AC Transmission System (FACTS) devices in wide-area voltage control systems (WAVCS) for achieving improved voltage stability of bulk power systems, the possibility for cyber attacks on these systems is also increasing. Successful stealthy cyber attacks that are difficult to detect by traditional informational technology (IT)-based cybersecurity solutions or threshold-based bad data detectors can lead to a voltage collapse in power grid. This paper presents the testbed-based attacks implementation and real-time evaluation of machine learning (ML) algorithm for detecting and mitigating stealthy cyber attacks on FACTS-based WAVCS on a hardware-in-the-loop (HIL) testbed. Initially, we discuss the implementation of a fuzzy logic controller (FLC) that controls a Static VAR Compensator (SVC) device deployed in a two-area four-machine Kundur power system for improving transient voltage stability. Later, the ML-based Anomaly Detection and Mitigation (ADM) system is implemented on the cyber-physical HIL testbed to detect and mitigate various stealthy cyber attacks, which are injected in real-time over the wide-area network (WAN). The experimental results show accurate and effective performance of ADM system in detecting and mitigating anomalies while keeping the grid stable and within the system operating limits, as defined by the North America Electric Reliability Corporation (NERC).

24 POWER TRANSMISSION AND DISTRIBUTION↗

Towards Smart Grids Enhanced Situation Awareness: A Bi-Level Quasi-Static State Estimation Model

Smart Grid situational awareness is provided by Energy Management Systems. A core process of these systems is State Estimation. The great majority of state estimators model the Smart Grid through a set of nonlinear algebraic equations, named the measurement model. Problem formulation considers the Gauss solution. Several model improvements have been presented regarding the Gauss solution, aiming between others to provide measurement noise robustness to the state estimation process. While considerable effort has been focused on such developments, state estimation is still constrained by the implicit modelling error, and thus inevitably vulnerable to cyber-threats. In this work, a state estimation bi-level formal model is presented towards Smart Grids enhanced situational awareness considering the concepts of synthetic measurements and innovation. Comparative test results with the state-of-the-art on the IEEE 14-bus system are presented highlighting improved situational awareness to bad data. Easy-to-implement model, without hard-to-derive parameters, built-on the classic weighted least squares solution, highlight potential aspects for real-life implementation.

cyber security↗

Distribution System State Estimation Using a Multiple Iteration Extended Kalman Filter Approach

To support the operation of modern distribution systems, operators require real-time visibility into system states. Due to a lack of measurements and unbalanced operation, the state estimation in distribution systems is challenging as compared to transmission systems. This paper proposes the utilization of a Multiple Iteration - Extended Kalman Filter based approach for the distribution system state estimation. This modified version of the baseline extended Kalman filter iterates over the update step multiple times thereby reducing the estimation error. The proposed algorithm along with the auxiliary algorithms such as bad data detection is integrated into a co-simulation environment. Case studies show that the proposed state estimation method can result in a lesser estimation error as compared to the baseline approach.

Bhatti, Bilal Ahmad↗

QA/QC-ed BCI meteorological drivers

Time series raw datasets of Temperature, Solar Radiation, Relative Humidity, Rainfall, Barometric Pressure, Wind Speed, and Wind Direction for the Barro Colorado (PA-BCI) site for the period from 2003-01-01 to 2016-12-31 were downloaded from the STRI website. The QA/QC protocol of raw datasets included the detection and removal of NAs, outliers and bad data points, imputation of missing data, and creation of the time series with equidistant time stamps. The time series for different meteorological parameters were aligned into a single database to be used for modeling. Statistical QA/QC analysis was performed using a series of R libraries in Rstudio. The changes in the dataset were flagged. The file BCI_met_drivers_2003-2016_QAQC_summary_report.docx provides the details of the QA/QC data analysis.

54 ENVIRONMENTAL SCIENCES↗

Evaluation of Fiber Bragg Grating and Distributed Optical Fiber Temperature Sensors

Fiber optic temperature sensors were evaluated in the High Temperature Test Lab (HTTL) to determine the accuracy of the measurements at various temperatures. A distributed temperature sensor was evaluated up to 550C and a fiber Bragg grating sensor was evaluated up to 750C. HTTL measurements indicate that there is a drift in fiber Bragg sensor over time of approximately -10C with higher accuracy at temperatures above 300C. The distributed sensor produced some bad data points at and above 500C but produced measurements with less than 2% error at increasing temperatures up to 400C

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

WISP: Watching grid Infrastructure Stealthily through Proxies (Final Technical Report)

The complex interdependencies of cyber systems (sensors and communications), physical grids and associated electricity market operations make protecting electric power grids a significant challenge. The energy sector is constantly under new, targeted, advanced and dangerous cyber-attacks that have the potential to result in the loss of human life. These threats are further exacerbated by our need to modernize the grid. One focus of cyber security research in smart grids is the securing of the SCADA system through advanced intrusion detection systems (IDS) and bad data detection algorithms in state estimation. These methods either require full knowledge of the system topology and parameters or fail to understand the physical behaviors under attack. WISP (Watching grid Infrastructure Stealthily through Proxies) is designed to provide additional protection to the power grid using only publicly available data. In particular, WISP exploits the spatio-temporal nature of the real time locational marginal prices (LMPs), in conjunction with other information such as bids, weather, outages and load data to analyze anomalous power pricing behaviors and then correlate those observations to localize regions of interest and identify potential cyber events. WISP is non-intrusive as the tool is deployed as a service in the Cloud or on premise and provides reliable information to system operators for enhanced situational awareness, without impeding energy delivery functions. The WISP technology comprises three modules: the data-driven anomaly detection core, the vulnerability and risk analysis and the root cause analysis. The data-driven anomaly detection core performs the tasks of feature selection, anomaly detection and attack region localization. The vulnerability and risk analysis module provides system level information of the vulnerable variables and times, assisting the operators in selecting monitoring and protection nodes. The root cause analysis module takes the detection results and identifies potential operational conditions that contribute to the detected anomalies. In Phase I, we have demonstrated the feasibility and effectiveness of WISP. We developed a realistic electricity market simulator capable of generating normal and attack market data under various operational conditions. We developed a series of cyber-attack detection and analysis algorithms and evaluated them under multiple data sources. Finally, we integrated all modules into an end-to-end software, providing functions for data management, data analytics and visualization. Specifically, we have achieved: (i) real-time data acceptance from external utility interfaces with >99% acceptance rate; (ii) high performance anomaly detection algorithms with >98% detection accuracy and <0.1% false alarm rate; and (iii) ultra-low computing delay <50 milliseconds. Additionally, our team developed algorithms to identify the vulnerable variables in electricity market operations and root cause analysis functions to identify major contributors to the price spikes. These ancillary modules are necessary when deploying WISP in real world industry environment. In Phase II, we have demonstrated the effectiveness of WISP software on realistic largescale power systems. We performed red team testing for the Phase I WISP software and identified software vulnerabilities and implemented corresponding mitigation solutions. We adapted the electricity market simulator for the Texas synthetic 2000-bus system and generated datasets for the false data injection attacks. We created database and visualization interfaces for the Texas system and the ISO New England system. We performed software optimization in terms of operation efficiency, computing speed and detection accuracy. Finally, we tested the software on the Texas system and the ISO New England system and evaluated the detection performance. Overall, we achieved above 89% detection rate, below 3% false alarm rate and below 37 seconds of end-to-end detection delay.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Anomaly Detection in Power System State Estimation: Review and New Directions

Foundational and state-of-the-art anomaly-detection methods through power system state estimation are reviewed. Traditional components for bad data detection, such as chi-square testing, residual-based methods, and hypothesis testing, are discussed to explain the motivations for recent anomaly-detection methods given the increasing complexity of power grids, energy management systems, and cyber-threats. In particular, state estimation anomaly detection based on data-driven quickest-change detection and artificial intelligence are discussed, and directions for research are suggested with particular emphasis on considerations of the future smart grid.

42 ENGINEERING↗

Onboard navigation of the Space Shuttle

This paper discusses the Kalman filter used in the navigation software of the Space Shuttle. The form of the filter will be discussed: the standard Kalman filter versus the square-root version of this filter. Both of these filters have severe nonlinearity problems. Two successful solutions of the nonlinearity problem are presented. A real-time program must have a good method of editing bad data. The data editing scheme is discussed. Those elements of the Kalman filter state vector which are random variables will be discussed.

Lear, W. M.↗

The High Resolution Spectrograph for the Space Telescope

The High Resoltuion Spectrograph (HRS) in conjunction with the Space Telescope (ST), extends ultraviolet astronomical spectroscopy to higher spectra, spatial, and time resolutions than previously achieved, as well as to fainter and more distant celestial objects. Other significant advances inherent in the instrument are high photometric accuracy and efficient operation via exposure meter control and real time rejection of bad data. These capabilities are provided to accomplish the scientific programs of the HRS investigation definition team, which concern the interstellar medium, stellar winds, and evolutionary aspects of stellar atmosphere studies; the determination of chemical abundances relevant to stellar evolution; the investigation of quasars and Seyfert galaxy nuclei; and the analysis of the atmospheres of solar system objects, including comets.

Brandt, J. C.↗

On the impact of the FGGE on tropical forecasts

It has become evident from operational experience that a good forecast requires, in general, a good analysis. This is as true in the tropics as it is in the extratropics. One can find many examples of bad data being accepted by the analysis scheme and significantly degrading the subsequent forecast. Equally, a lack of data may result in a forecast completely missing the early development of, for example, a midlatitude cyclone or tropical depression. Also, single level data may be misinterpreted by an analysis scheme. These problems are compounded in the tropics by our relative ignorance of the dynamics, the necessarily crude descriptions of many important physical processes, and fundamental in analysis. The impact of the FGGE on tropical analysis/forecasting is reviewed.

Heckley, W. A.↗