SEARCH · Engineering Papers

Results for “Missing data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

All sources NASA NTRS DOE OSTI

DOE OSTI2022

QA/QC of the East River, Colorado, discharge and geochemical time series datasets (Almont, BCC, and Pump House) to be used for modeling of hydrogeochemical balance

The following datasets were QA/QC-ed (Quality Assurance/Quality Control): 1. Brush Creek Confluence (BCC) discharge data (from Helen Malenda, USGS, Colorado School of Mines), which were calculated using the pressure transducer data and rating curves. The original 15 min time series data were presented as mean daily discharge. 2. Almont discharge data from United States Geological Survey (USGS). The original data were in 15 min time intervals, and were averaged to mean daily discharge time series. 3. Pump House discharge data as mean daily discharge (downloaded from the SFA portal). 4. BCC and Pump House chemistry data from SFA data portal and/or original spreadsheets provided by Roelof Versteeg. The following challenging QA/QC problems of the datasets were resolved: Missing data with the duration of gaps up to >1 month; Duplicated dates; Anomalies and outliers of discharge and concentrations; Time stamps of measurements of the discharge and concentrations are not aligned (hydrogeochemical balance calculations require the timestamps to be aligned). All QA/QC-ed datasets are given as csv files. The csv files were prepared using the xts files with multiple worksheets, which are also included in the data packages. Figures of the QA/QC-ed datasets are given in the jpeg and pdf formats. The QA/QC-ed datasets have been used to quantify discharge and chemical concentrations in river water in order to understand riverine exports of water and dissolved constituents in the East River watershed. These datasets served as a basis in the presentation given by P. Fox et al. at the 2021 Goldschmidt Conference.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2020

AUTONOMIE VID

Autonomie Vehicle Information Database (VID) offers a comprehensive list of vehicle specifications since 1990. The database details more than 65,000 vehicles with hundreds of attributes. The database is the result of the development of a general automated data collection framework as well as the development of building blocks for processing, cleaning, integrating and analyzing complex data. The data has undergone several layers of outlier detections processes, machine learning based imputations methods have been used to deal with missing data problems, and new fields have been created according to the rules of feature engineering. Thanks to this streamlined data pipelines, the resulting processed aggregated data should deliver a unique level of information to the user in which the content can be efficiently maintained and updated.

Moswd, Ayman↗

DOE OSTI2016

Online mapping and forecasting of epidemics using open-source indicators

Open-source indicators have been proposed as a way of tracking and forecasting disease outbreaks. Some, such are meteorological data, are readily available as reanalysis products. Others, such as those derived from our online behavior (web searches, media article etc.) are gathered easily and are more timely than public health reporting. In this study we investigate how these datastreams may be combined to provide useful epidemiological information. The investigation is performed by building data assimilation systems to track influenza in California and dengue in India. The first does not suffer from incomplete data and was chosen to explore disease modeling needs. The second explores the case when observational data is sparse and disease modeling complexities are beside the point. The two test cases are for opposite ends of the disease tracking spectrum. We find that data assimilation systems that produce disease activity maps can be constructed. Further, being able to combine multiple open-source datastreams is a necessity as any one individually is not very informative. The data assimilation systems have very little in common except that they contain disease models, calibration algorithms and some ability to impute missing data. Thus while the data assimilation systems share the goal for accurate forecasting, they are practically designed to compensate for the shortcomings of the datastreams. Thus we expect them to be disease and location-specific.

42 ENGINEERING↗

DOE OSTI2015

Optimizing Geothermal Drilling: Oil and Gas Technology Transfer

There is a significant amount of financial risk associated with geothermal drilling; however, there are opportunities to improve upon current practices and technologies used. The scope of this drilling operational study included 21 geothermal wells and 21 oil and gas wells. The goal was to determine a 'perfect well' using historical data to compare the best oil and gas well to the best geothermal well. Unfortunately, limitations encountered in the study included missing data (bit records, mud information, etc.), poor data collection, and difficult to ascertain handwriting. An online software database was used to format drilling data to IADC coded daily drilling reports and generate analysis figures. Six major issues have been found in geothermal drilling operations. These problems include lost circulation, rig/equipment selection, cementing, penetration rate, drilling program, and time management. As a result of these issues, geothermal drilling averages 56.4 days longer than drilling comparable oil and gas wells in the wells in this study. Roughly $13.9 million would be lost due to non-productive time in the 21 geothermal wells and only $1.3 million in the oil and gas wells, assuming a cost of $50,000 per day. Comparable events such as drilling the same sized hole, tripping in/out, cementing, and running the same size casing took substantially less time in the oil and gas wells. Geothermal wells were drilled using older and/or less advanced technology to depths less than 10,000 feet, while oil and gas wells reached 12,500 feet faster with purpose built rigs. A new approach is now underway that will optimize drilling programs throughout the drilling industry. It is the use of Mechanical Specific Energy (MSE) as a tool to realize efficient drilling processes. However, a work-flow must also be established in order for there to be an efficient drilling program. Potential improvements for current geothermal operations are: the use of electronic records, real time services, and official glossary terms to describe rig operations, and advanced drilling rigs/technology.

15 GEOTHERMAL ENERGY↗

DOE OSTI2015

Optimizing Geothermal Drilling: Oil and Gas Technology Transfer

There is a significant amount of financial risk associated with geothermal drilling. This study of drilling operations seeks opportunities to improve upon current practices and technologies. The scope of this study included analyzing 21 geothermal wells and 21 oil and gas wells. The goal was to determine a 'Perfect Well' using historical data to compare the best oil and gas well to the best geothermal well. Unfortunately, limitations encountered in the study included missing data (bit records, mud information, etc.) and poor data collection practices An online software database was used to format drilling data to IADC coded daily drilling reports and generate figures for analysis. Six major issues have been found in geothermal drilling operations. These problems include lost circulation, rig/ equipment selection, cementing, penetration rate, drilling program, and time management. As a result of these issues, geothermal drilling averaged 56.4 days longer than drilling comparable oil and gas wells in the wells in this study. Roughly $13.9 million was spent on non-productive time in the 21 geothermal wells, compared with only $1.3 million in the oil and gas wells, assuming a cost of $50,000 per day. Comparable events such as drilling the same sized hole, tripping in/out, cementing, and running the same size casing took substantially less time in the oil and gas wells. Geothermal wells were drilled using older and/or less advanced technology to depths less than 10,000 feet, while oil and gas wells reached 12,500 feet faster with purpose built rigs. A new approach is now underway that will optimize drilling programs throughout the drilling industry using Mechanical Specific Energy (MSE) as a tool to realize efficient drilling processes. Potential improvements for current geothermal operations are: the use of electronic records, real time services, and official glossary terms to describe rig operations, and advanced drilling rigs/technology.

15 GEOTHERMAL ENERGY↗

DOE OSTI2024

An iterative bidirectional gradient boosting approach for CVR baseline estimation

Here this paper presents a novel Iterative Bidirectional Gradient Boosting Model (IBi-GBM) for estimating the baseline of Conservation Voltage Reduction (CVR) programs. In contrast to many existing methods, we treat CVR baseline estimation as a missing data retrieval problem. The approach involves dividing the load and its corresponding temperature profiles into three periods: pre-CVR, CVR, and post-CVR. To restore the missing load profile during the CVR period, the method employs a three-step process. First, a forward-pass GBM is executed using data from the pre-CVR period as inputs. Subsequently, a backward-pass GBM is applied using data from the post-CVR period. The two restored load profiles are reconciled, considering pre-calculated weights derived from forecasting accuracy, and only the leftmost and rightmost points are retained. The newly restored points are then included as inputs for the subsequent iteration. This iterative procedure continues until the original load data in the CVR period is fully restored. We develop IBi-GBM using actual smart meter and Supervisory Control and Data Acquisition (SCADA) data. Our results demonstrate that IBi-GBM exhibits robust performance across various data resolutions and in different seasons and outperforms existing methods by achieving a 1-2% reduction in normalized Root Mean Square Error (nRMSE).

42 ENGINEERING↗

DOE OSTI2026

Imputation of urban environmental sensor data using gated attention bidirectional long short-term memory (GA-BiLSTM): methods, performance, and implications

Urban environmental monitoring networks frequently encounter significant data gaps due to sensor malfunctions, environmental disturbances, and communication failures. Reliable approaches to address these gaps are essential for ensuring the continuity and quality of environmental data streams. In this study, we developed a gated attention bidirectional long short-term memory (GA-BiLSTM) model to impute missing data in a dense urban monitoring network. Using observations from the CROCUS network in Chicago, we evaluated GA-BiLSTM against widely used approaches (XGBoost and K-nearest neighbors) under scenarios of both short-term intermittent gaps and prolonged outages. GA-BiLSTM consistently outperformed comparative methods, particularly during extended outages of up to ten days, demonstrating its ability to capture spatiotemporal dependencies across sensor nodes. Beyond performance metrics, feature importance and spatial network analyses highlighted the unexpected but critical predictive role of peripheral rural nodes, underlining their strategic value for maintaining robust urban monitoring systems. These results emphasize that advanced imputation methods can substantially improve the reliability of environmental monitoring networks and support more resilient data infrastructures for urban sustainability.

Data imputation↗

DOE OSTI2024

Soil Water Percolation Flux, April 2017 to March 2019, BR-Ma2, Manaus

Soil water percolation flux measurements derived from six passive wick flux meters across three topographic positions (valley, slope and plateau). Data has been processed to compute the mean for each topographic position where two flux meters were installed at each topographic position. Missing data represented a few percent of the total data set and were filled using two approaches. In the event one of the flux meters malfunctioned, we used data from the other sensor from the same topographic position without averaging. When both sensors were not working, such as during the period from September to October 2017 for the slope and valley and July 2018 for the plateau, data was filled using a nonlinear regression model that was based on local precipitation data and coefficients that were empirically derived for each topographic position. The metadata tab included in the .csv has additional information on locations where the sensors were installed, and other installation/maintenance details. Contact ksolander@lanl.gov if you need to use this dataset for additional information.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Frost Tables, NGEE Areas B, C and D, Utqiagvik (Barrow), Alaska, 2012-2014

This dataset represents spatially intensive thaw depth surveys with individual point measurements spaced ~0.5 m apart. The three ~10x10m grids cover an ice wedge and a portion of its two neighboring polygons. The data contain thaw depth, frost table elevation, ground surface elevation, active layer depth and surface water inundation across three seasons (2012, 2013 and 2014) at Utqiagvik (Barrow) NGEE Areas B, C and D. There are a total of 69 extracted files separated by month, year, and area. The data were originally provided in excel workbooks, these files have been preserved within the zip "Original_Files." The files were transformed for archiving with images pulled out into JPEGs, data preserved as CSV files and some methodological information preserved as TXT files. Some functions, features, and formatting were lost during the transformation. Missing data were defined with blank cells. Additional documentation can be found in files labelled with "information.txt" file endings.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).[2024-07-15] Renamed BEO_Thaw_Depth_Frost_Table_Area_*_*_*.csv, BEO_Thaw_Depth_Frost_Table_Area_*_*_*_Plot_1.JPEG, and BEO_Thaw_Depth_Frost_Table_Area_*_*_*_Plot_2.JPEG names that had E or L to be Early or Late. Renamed BEO_Thaw_Depth_Frost_Table_Area_*_*_*_Plot_1.JPEG, BEO_Thaw_Depth_Frost_Table_Area_*_*_*_Plot_2.JPEG, and BEO_Thaw_Depth_Frost_Table_Area_*_information_Plot_1.JPEG to BEO_Thaw_Depth_Frost_Table_Area_*_*_*_Gridpoints.JPEG, BEO_Thaw_Depth_Frost_Table_Area_*_*_*_3Dgraphic.JPEG, and BEO_Thaw_Depth_Frost_Table_Area_*_information_LiDAR_DEM.JPEG, respectively. Updated BEO_Thaw_Depth_Frost_Table_flmd.csv file to reflect above updates.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2016

Ground Water Levels for NGEE Areas A, B, C and D, Utqiagvik (Barrow), Alaska, 2012-2014

Ice wedge polygonal tundra water levels were measured at a total of 45 locations representing polygon centers and troughs during three summers. Early season water levels, which were still affected by ice and snow, are represented by manual measurements only. Continuous (less than hourly) measurements followed through early fall (~mid-Sep). The data set contains inundation depth (cm), absolute water level and local ground surface elevation (masl). There are a total of 209 files in this dataset. The data were originally provided in Excel workbooks and these files have been preserved within the zip "Original_Files." The workbooks were also transformed and separated by well ID, year, and/or area into 205 extracted files. The Excel files were transformed for archiving with images pulled out into JPEGs, data preserved as CSV files and some methodological information preserved as TXT files. Some functions, features, and formatting were lost during the transformation. Missing data were defined with blank cells. Additional documentation can be found in files labelled with "information.txt" file endings.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2020

Benchmark probabilistic solar forecasts: Characteristics and recommendations

We illustrate and compare commonly used benchmark, or reference, methods for probabilistic solar forecasting that researchers use to measure the performance of their proposed techniques. A thorough review of the literature indicates wide variation in the benchmarks implemented in probabilistic solar forecast studies. To promote consistent and sensible methodological comparisons, we implement and compare ten variants from six common benchmark classes at two temporal scales: intra-hourly forecasts and hourly resolution forecasts. Using open-source Surface Radiation Budget Network (SURFRAD) data from 2018, these benchmark methods are compared using proper probabilistic metrics and common diagnostic tools. Practical implementation issues, such as the impact of missing data and applicability for operational forecasting, are also discussed. Furthermore, we make recommendations for practitioners on the appropriate selection of benchmark methods to properly showcase state-of-the-art improvements in forecast reliability and sharpness. All code and open-source data are available on Github for reproducibility and for other researchers to apply the same benchmark methods to their own data.

14 SOLAR ENERGY↗

DOE OSTI2022

Deep learning-based spatio-temporal estimate of greenhouse gas emissions using satellite data

Accurate estimation of greenhouse gases (GHGs) emissions is very important for developing mitigation strategies to climate change by controlling and reducing GHG emissions. This project aims to develop multiple deep learning approaches to estimate anthropogenic greenhouse gas emissions using multiple types of satellite data. NO2 concentration is chosen as an example of GHGs to evaluate the proposed approach. Two sentinel satellites (sentinel-2 and sentinel-5P) provide multiscale observations of GHGs from 10-60m resolution (sentinel-2) to ~kilometer scale resolution (sentinel-5P). Among multiple deep learning (DL) architectures evaluated, two best DL models demonstrate that key features of spatio-temporal satellite data and additional information (e.g., observation times and/or coordinates of ground stations) can be extracted using convolutional neural networks and feed forward neural networks, respectively. In particular, irregular time series data from different NO 2 observation stations limit the flexibility of long short-term memory architecture, requiring zero-padding to fill in missing data. However, deep neural operator (DNO) architecture can stack time-series data as input, providing the flexibility of input structure without zero-padding. As a result, the DNO outperformed other deep learning architectures to account for time-varying features. Overall, temporal patterns with smooth seasonal variations were predicted very well, while frequent fluctuation patterns were not predicted well. In addition, uncertainty quantification using conformal inference method is performed to account for prediction ranges. Overall, this research will lead to a new groundwork for estimating greenhouse gas concentrations using multiple satellite data to enhance our capability of tracking the cause of climate change and developing mitigation strategies.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2020

Characterizing Sub-Cohorts via Data Normalization and Representation Learning

The process of identifying a cohort of interest is a very challenging task. It requires manually inspecting many patient records of complex structure that might include medical coding errors and missing data. This paper presents a computational pipeline for refining the process of cohort selection based on medical concepts recorded in the electronic health records (EHRs). The pipeline extracts EHR data for a given cohort and normalizes this data using standard vocabularies. Then a stacked denoising autoencoder is used to embed the normalized patient vectors in a low dimensional space, where the patients are subsequently clustered into sub-cohorts. The goal is to represent the cohort in a standard format and abstract variants of sub-populations. As a use-case, we applied the pipeline to 1.8 million Veterans diagnosed with major depressive disorder (MDD), and identified four meaningful sub-cohorts using the features learned by the autoencoder. Then, each sub-cohort was explored using a set of keywords for interpretation.

Rush III, Everett↗

DOE OSTI2016

Automated Dynamic Demand Response Implementation on a Micro-grid

In this paper, we describe a system for real-time automated Dynamic and Sustainable Demand Response with sparse data consumption prediction implemented on the University of Southern California campus microgrid. Supply side approaches to resolving energy supply-load imbalance do not work at high levels of renewable energy penetration. Dynamic Demand Response (D 2 R) is a widely used demand-side technique to dynamically adjust electricity consumption during peak load periods. Our D 2 R system consists of accurate machine learning based energy consumption forecasting models that work with sparse data coupled with fast and sustainable load curtailment optimization algorithms that provide the ability to dynamically adapt to changing supply-load imbalances in near real-time. Our Sustainable DR (SDR) algorithms attempt to distribute customer curtailment evenly across sub-intervals during a DR event and avoid expensive demand peaks during a few sub-intervals. It also ensures that each customer is penalized fairly in order to achieve the targeted curtailment. We develop near linear-time constant-factor approximation algorithms along with Polynomial Time Approximation Schemes (PTAS) for SDR curtailment that minimizes the curtailment error defined as the difference between the target and achieved curtailment values. Our SDR curtailment problem is formulated as an Integer Linear Program that optimally matches customers to curtailment strategies during a DR event while also explicitly accounting for customer strategy switching overhead as a constraint. We demonstrate the results of our D 2 R system using real data from experiments performed on the USC smartgrid and show that 1) our prediction algorithms can very accurately predict energy consumption even with noisy or missing data and 2) our curtailment algorithms deliver DR with extremely low curtailment errors in the 0.01-0.05 kWh range.

Kuppannagari, Sanmukh R.↗

DOE OSTI2024

COMPASS-FME Synoptic Sites Level 1 Sensor Data v1-2

This is the version 1-2 Level 1 (L1) data release for COMPASS-FME environmental sensors located at our synoptic field sites. COMPASS-FME is studying sites in two distinct regions, the Chesapeake Bay and the Western Lake Erie Basin. We established the network at seven "synoptic" (observational) sites along the Chesapeake Bay and Lake Erie coastlines, collectively generating over three million observations per month, to track and comprehend environmental changes where land and water intersect. Additionally, the two regions provide an interesting contrast of saltwater and freshwater coasts that allow us to differentiate the impacts of inundation and coastal water chemistries in two nationally important coastal systems.L1 data are close to raw, but are units-transformed and have out-of-instrument-bounds and out-of-service flags added. Duplicates and missing data are removed but otherwise these data are not filtered, and have not been subject to any additional algorithmic or human QA/QC. Any scientific analyses of L1 data should be performed with care. **This dataset will be updated quarterly with new data for the duration of the project**This dataset includes:- An overall dataset README file that describes the current version, gives citation and contact information, etc.- Site- and year-specific folders, each holding up to 12 CSV (comma separated value) data files for each site and plot in that year.- Metadata files within each site-year folder provide full information on data units, expected ranges, contact information, as well as a general description of the site.- Environmental sensor types that appear in the data files include weather (ClimaVUE50, CS, RM Young, and LI instruments in the graphs below); soil conditions (TEROS12); soil redox state (Redox); groundwater variables (AquaTROLL200 and AquaTROLL600); open water sondes (Exo); tree sap velocity (Sapflow); and system voltage and state (Datalogger). Data are normally logged every 15 minutes. Please see v1-2 Synoptic L1 Sensor Package Quick Start.pdf for detailed information on data package structure, temporal coverage, and versioning.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

COMPASS-FME Terrestrial Ecosystem Manipulation to Probe the Effects of Storm Treatments (TEMPEST) Experiment Level 1 Sensor Data v1-2

This is the version 1-2 Level 1 (L1) data release for COMPASS-FME environmental sensors located at our Terrestrial Ecosystem Manipulation to Probe the Effects of Storm Treatments (TEMPEST) experimental site. This manipulative, ecosystem-scale TEMPEST experiment addresses the potential for freshwater and estuarine-water disturbance events to alter tree function, species composition, and ecosystem processes in a deciduous coastal forest in MD, USA. The experiment uses a large-unit (2000 m2), un-replicated experimental design, with three 50 m × 40 m plots serving as control, freshwater, and estuarine-water treatments.L1 data are close to raw, but are units-transformed and have out-of-instrument-bounds and out-of-service flags added. Duplicates and missing data are removed but otherwise these data are not filtered, and have not been subject to any additional algorithmic or human QA/QC. Any scientific analyses of L1 data should be performed with care. **This dataset will be updated quarterly with new data for the duration of the project**This dataset includes:- An overall dataset README file that describes the current version, gives citation and contact information, etc.- Site- and year-specific folders, each holding up to 12 CSV (comma separated value) data files for each site and plot in that year.- Metadata files within each site-year folder provide full information on data units, expected ranges, contact information, detailed flood times, as well as a general description of the site.- Environmental sensor types that appear in the data files include weather (ClimaVUE50, CS, RM Young, and LI instruments in the graphs below); soil conditions (TEROS12); soil redox state (Redox); groundwater variables (AquaTROLL200 and AquaTROLL600); open water sondes (Exo); tree sap velocity (Sapflow); and system voltage and state (Datalogger). Data are normally logged every 15 minutes.Please see v1-2 TEMPEST L1 Sensor Package Quick Start.pdf for detailed information on data package structure, temporal coverage, and versioning.The TEMPEST flood events occurred on the following dates. They lasted for ~10 hours each day and delivered ~80,000 gallons to each plot; many data streams are available at 1 or 5 minute frequency during these periods.* Tests: Aug 25 (fresh plot) and Sep 9 (salt plot), 2021* TEMPEST 1: June 22, 2022* TEMPEST 2: June 6-7, 2023* TEMPEST 3: June 11-13, 2024

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2016

Inferring Netflow Data

Netflow data is a specific format for looking at network traffic. With this format, many security tests are easy to run. Cloud Service Providers produce logs that do not contain all of the content that Netflow data contains. There is a possibility that the fields that are in Netflow data but absent in Cloud Service Provider logs can be inferred from the fields that are present in the Cloud Service Provider logs. By inferring the missing data fields, the same test that can be run with Netflow data would be possible to run on the logs produced by Cloud Service Providers. Cloud Service Providers include Amazon Web Services and Box.com. There are multiple different types of Cloud Services, and each provider handles them differently and produces different logs. There are IaaS (meaning infrastructure as a service), Saas (meaning software as a service), and Paas (meaning platform as a service). Each provides a different use for the user.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Phase Identification in Real Distribution Networks with High PV Penetration Using Advanced Metering Infrastructure Data

Many distribution network monitoring and control applications - including state estimation, volt/VAR optimization, and network reconfiguration - rely on accurate network models; however, the network models maintained by utilities can become outdated because of restoration activities, network reconfiguration, and missing data. With the widespread deployment of advanced metering infrastructure (AMI), abundant measurement data from low-voltage secondary networks are available. The AMI measurement data can be used for phase identification to improve the network models. Although the existing phase identification techniques work well in passive distribution feeders that do not have photovoltaic (PV) generation, they can fail to accurately identify the phases in the presence of PV. This paper proposes a robust phase identification algorithm based on supervised machine learning that accurately identifies the AMI meter phase connectivity in the presence of significant PV generation. The proposed algorithm does not require network topology information or feeder head measurement data. The algorithm is validated using the AMI measurement data collected in the field and the field-validated phase connectivity database on two real distribution feeders from San Diego Gas & Electric Company that have significant PV generation.

advanced metering infrastructure↗