A Model-free Approach for Estimating Service Transformer Capacity Using Residential Smart Meter Data
Not Available
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Not Available
The load profile at a feeder-head is usually known to utility engineers while the nodal load profiles are not. However, the nodal load profiles are increasingly important for conducting time-series analysis in distribution systems. Therefore, in this paper, we present a pivot-point based, two-stage feeder load disaggregation algorithm using smart meter data. The two stages are load profile selection (LPS) and load profile allocation (LPA). In the LPS stage, a random load profile selection process is first executed to meet the load diversity requirement. Then, a few pairs of pivot points are selected as the matching targets. After that, a matching algorithm will run repetitively to select one load profile at a time for matching the reference load profile at the pivot points. In the LPA stage, the LPS selected load profiles are allocated to each load node on the feeder considering distribution transformer loading limits, load composition, and square-footage. The proposed method is validated using actual data collected in a North Carolina service area. Finally, simulation results show that the proposed method can generate a unique load shape for each load node while match the shape of their aggregated profile with the actual feeder head load profile.
The lack of consistent, accurate information on evapotranspiration (ET) and consumptive use of water by irrigated agriculture is one of the most important data gaps for water managers in the western United States (U.S.) and other arid agricultural regions globally. The ability to easily access information on ET is central to improving water budgets across the West, advancing the use of data-driven irrigation management strategies, and expanding incentive-driven conservation programs. Recent advances in remote sensing of ET have led to the development of multiple approaches for field-scale ET mapping that have been used for local and regional water resource management applications by U.S. state and federal agencies. The OpenET project is a community-driven effort that is building upon these advances to develop an operational system for generating and distributing ET data at a field scale using an ensemble of six well-established satellite-based approaches for mapping ET. Key objectives of OpenET include: Increasing access to remotely sensed ET data through a web-based data explorer and data services; supporting the use of ET data for a range of water resource management applications; and development of use cases and training resources for agricultural producers and water resource managers. Here we describe the OpenET framework, including the models used in the ensemble, the satellite, meteorological, and ancillary data inputs to the system, and the OpenET data visualization and access tools. We also summarize an extensive intercomparison and accuracy assessment conducted using ground measurements of ET from 139 flux tower sites instrumented with open path eddy covariance systems. Results calculated for 24 cropland sites from Phase I of the intercomparison and accuracy assessment demonstrate strong agreement between the satellite-driven ET models and the flux tower ET data. For the six models that have been evaluated to date (ALEXI/DisALEXI, eeMETRIC, geeSEBAL, PT-JPL, SIMS, and SSEBop) and the ensemble mean, the weighted average mean absolute error (MAE) values across all sites range from 13.6 to 21.6 mm/month at a monthly timestep, and 0.74 to 1.07 mm/day at a daily timestep. At seasonal time scales, for all but one of the models the weighted mean total ET is within ±8% of both the ensemble mean and the weighted mean total ET calculated from the flux tower data. Overall, the ensemble mean performs as well as any individual model across nearly all accuracy statistics for croplands, though some individual models may perform better for specific sites and regions. We conclude with three brief use cases to illustrate current applications and benefits of increased access to ET data, and discuss key lessons learned from the development of OpenET.
The intelligent distributed dispatch and scheduling (iDDS) service is a versatile workflow orchestration system designed for large-scale, distributed scientific computing. iDDS extends traditional workload and data management by integrating data-aware execution, conditional logic, and programmable workflows, enabling automation of complex and dynamic processing pipelines. Originally developed for the ATLAS experiment at the large hadron collider, iDDS has evolved into an experiment-agnostic platform that supports both template-driven workflows and a Function-as-a-Task model for Python-based orchestration. This paper presents the architecture and core components of iDDS, highlighting its scalability, modular message-driven design, and integration with systems such as PanDA and Rucio. We demonstrate its versatility through real-world use cases: fine-grained tape resource optimization for ATLAS, orchestration of large Directed Acyclic Graph (DAG) workflows for the Rubin Observatory, distributed hyperparameter optimization for machine learning applications, active learning for physics analyses, and AI-assisted detector design at the electron–ion collider. By unifying workload scheduling, data movement, and adaptive decision-making, iDDS reduces operational overhead and enables reproducible, high-throughput workflows across heterogeneous infrastructures. We conclude with current challenges and future directions, including interactive, cloud-native, and serverless workflow support.
Deploying environmental monitoring instruments in the marine environment can be challenging, facing challenges around device survivability, biofouling and corrosion, and consistent data collection. This project explores the use of data fusion – the process of integrating multiple data sources to produce more consistent, accurate, and useful information – to build a consistent long-term monitoring system at the Marine and Coastal Research Laboratory (MCRL) in Sequim, Washington. Unused instruments that had been acquired from past projects were inventoried and deployments planned on the MCRL pier and floating dock. A total of 8 instruments were deployed including a tide gauge, hydrophone, acoustic Doppler current profiler (ADCP), photosynthetically active radiation (PAR) sensors, meteorological station, and three water quality sensors. Deployments were planned to be well-protected around the pier structure and a maintenance schedule was created for cleaning and recalibration. An automated data pipeline was created to aggregate data on edge computers that push data to Amazon Web Services (AWS) cloud storage every 15 minutes, performing automated quality control and data transformations using the Time Series Data Analytical Toolkit (TSDAT). Continued efforts are underway to maintain this system into the future, take a data-driven approach to maintenance scheduling, improve the reliability of the system, and share the data with a variety of end-users.
This report was compiled to provide recommendations on the availability of public background data from the U.S. Federal life cycle assessment (LCA) Data Commons to be conformant with the Association for Life Cycle Assessment (ACLCA) 2022 Product Category Rule (PCR) Open Standard to build technical tools that can assist industry in creating more comparable Type II Environmental Product Declarations (EPDs) for Federal Buy Clean and sustainability initiatives. The Federal LCA Commons is not only a public data source but also a consistently structured, self-referencing mega-repository for data developed by federal agency experts (in agency repositories) and by academia, nonprofit organizations, and industry (via the US Life Cycle Inventory Database). The Federal LCA Commons Technical Working Group is continuously improving the standardization of data documentation, formatting, and nomenclature to ensure lossless data loading and accurate data representation. This report and appendixes include the following: 1) An introduction to data-driven Buy Clean and decarbonization initiatives at the federal level; 2) The current status and associated challenges with LCA data and EPD standards and comparability; 3) Opportunities for the Federal LCA Commons to support conformance with the ACLCA 2022 PCR Open Standard and provide resources to implement the Federal Sustainability Plan, Buy Clean Program, and Inflation Reduction Act (IRA) sustainability goals and objectives. To date, the Federal LCA Commons is the result of coordinated work by National Renewable Energy Laboratory (NREL), the U.S. Department of Agriculture (USDA), the Environmental Protection Agency (EPA), the National Energy Technology Laboratory (NETL), the Argonne National Laboratory (ANL), the U.S. Army Corps of Engineers (USACE), the Federal Highway Administration (FHWA), the U.S. Forest Service (USFS), the Federal Aviation Administration (FAA), the Department of Defense (DoD) and the National Institute of Standards and Technologies (NIST). The Federal LCA Commons will continue to combine databases from the collaborating agencies while remaining a public resource. There are several initiatives among the collaborating agencies to expand the Federal LCA Commons and dedicated federal funding and resources could accelerate and strengthen these initiatives.
We have developed a long short-term memory stacked ensemble (LSTM-SE) surrogate modeling approach that can provide rapid predictions of microstructural evolution and the resultant mechanical properties of American Iron and Steel Institute (AISI) 316L series stainless steel (316LSS) fuel cladding under conditions of varying temperature and radiation dose rate. To acquire training data, we developed and implemented a kinetic Monte Carlo (KMC) model to simulate precipitation kinetics of M 23 C 6 , γ', and G phases within SS316L cladding. Experimentally reported precipitation kinetics of SS316L in literature were linked to the kinetic parameters of the simulated precipitation in our KMC model. The model was then used to simulate microstructure evolution under synthetically generated treatments of varying temperature and radiation dose rate, for periods of up to 3000 hours. Changes in volume fraction, number density, and particle size of precipitates were recorded, and particle area fractions were correlated using statistical methods to develop the surrogate model. Simultaneously, the mechanical properties of the simulated microstructures were evaluated using microstructure-based finite element method (FEM) analysis to determine the elastic modulus, yield stress, ultimate tensile strength, and elongation to failure of the aged microstructures. Using this approach, our surrogate model can predict precipitation behavior within 0.25% volume fraction and mechanical properties within 6% relative error from the values predicted by the KMC and FEM models using 50 training simulations as input. The trained recurrent neural network-based model can return estimations of precipitation kinetics and mechanical properties ~1000 times faster than the physics-based codes. This work demonstrates, as a proof of concept, that reactor material service lifetimes under variable service conditions can be predicted for a statistics-based model from a practicably obtainable dataset.
The objectives of this project are as follows: • The population behavior of fielded modules in various conditions of use • Predictions of materials in specific climatic zones • Understanding of a module’s local environment in the field on its degradation It aims to understand how and what backsheet materials of photovoltaic modules degrade in the real world field. • Field Survey Protocol This project is started from the protocol, because all the data, information, domain knowledge are from experience of the real world field surveys, which is based on the protocol. The Protocol is explored from the experience of the field survey observations. With the increasing of the field surveys, it is refined for three versions, which are Task 1.0 (Section 3.1), Task 6.0 (Section 3.6), Task 10.0 (Section 3.10) respectively. It includes a document and a training video, which is able to direct other teams to follow the same procedure with the sites surveyed during this project. The documents include the detailed information but is not limited to the terminology definition, instruments SOP, preparation items for the surveys, form for the data collections, the order of the information collection. The final version of the protocol can be found at Appendix A, see Section 5. Additional, It can also be found at Open Science Framework (OSF), see Section 3.18 for detail. • Written Waiver Request Before staring the field surveys, the request of the waiver for international surveys is completed, because the limitation of climate zone in the United States, see Appendix B in Section 6 for the request documents. Unfortunately, only 1 international site from Taiwan, China can be finished, due to the COVID-19. • Field Survey According to the protocol we built in Section 3.1, 3.6 and 3.10, 41 sites have been surveyed across seven different climate zones (Cfa, Csa, Csb, BSk, Dfa, Dfb, Am). A variety of materials, including Polyethylene Naphthalate (PEN), Polyethylene Terephthalate (PET), Polyvinyl Fluoride (PVF), Polyvinylidene Fluoride (PVDF), Acrylic PVDF, Fluoroethylene Vinyl Ether (FEVE), and Glass, were identified. These sites are located in various states including California, South Carolina, New Mexico, Maryland, Ohio, Tennessee, Florida, Massachusetts, Illinois, Minnesota, Oregon, Colorado, and Taiwan, Republic of China. The ages of the sites ranged from 2 - 38 years in service and the field size varied from 1 MW - 25 MW. All requirements for the modeling have been satisfied. Some observations like ’Edge Effect’ for the rows and Junction box heating will also be a useful knowledge to build the model. Section 3.7 provides detailed information on the sites visited during this reporting period.
Power system failures or outages due to short-circuits or “faults” can result in long service interruptions leading to significant socio-economic consequences. It is critical for electrical utilities to quickly ascertain fault characteristics, including location, type, and duration, to reduce the service time of an outage. Existing fault detection mechanisms (relays and digital fault recorders) are slow to communicate the fault characteristics upstream to the substations and control centers for action to be taken quickly. Fortunately, due to availability of high-resolution phasor measurement units (PMUs), more event-driven solutions can be captured in real time. In this paper, we propose a data-driven approach for determining fault characteristics using samples of fault trajectories. A random forest regressor (RFR)-based model is used to detect real-time fault location and its duration simultaneously. This model is based on combining multiple uncorrelated trees with state-of-the-art boosting and aggregating techniques in order to obtain robust generalizations and greater accuracy without overfitting or underfitting. Four cases were studied to evaluate the performance of RFR: 1. Detecting fault location (case 1), 2. Predicting fault duration (case 2), 3. Handling missing data (case 3), and 4. Identifying fault location and length in a real-time streaming environment (case 4). A comparative analysis was conducted between the RFR algorithm and state-of-the-art models, including deep neural network, Hoeffding tree, neural network, support vector machine, decision tree, naive Bayesian, and K-nearest neighborhood. Experiments revealed that RFR consistently outperformed the other models in detection accuracy, prediction error, and processing time.
Driven by the nature of the research questions now most critical to further progress in heliophysics science, data-driven research has evolved from a model once centered on individual instrument Principal investigator groups and a circle of immediate collaborators into a more inclusive and open environment where data gathered ay great public cost must then be findable and useable throughout the broad national and international research community. In this paper and as an introduction to this special session, we will draw a picture of existing and evolving resources throughout the heliophyscs community, the capabilities and data now available to end users, and the relationships and complementarity of different elements in the environment today. We will cite the relative roles of mission and instrument data centers and resident archives, multi-mission data centers, and the growing importance of virtual discipline observatories and cross-cutting services including the evolution of a common data dictionary. We will briefly summarize our view of the most important challenges still faced by users and providers, and our vision in ow the efforts today can evolve into a more and more enabling data framework for the global research community to tap the widest range of existing missions and their data to address a full range of critical science questions from the scale of microphysics to the heliospheric system as a whole.
The Simple, Scalable, Script-based Science Processing (S4P) Archive (S4PA) is a disk-based archival system for remote sensing data. It is based on the data-driven framework of S4P and is used for data transfer, data preprocessing, metadata generation, data archive, and data distribution. New data are automatically detected by the system. S4P provides services such as data access control, data subscription, metadata publication, data replication, and data recovery. It comprises scripts that control the data flow. The system detects the availability of data on an FTP (file transfer protocol) server, initiates data transfer, preprocesses data if necessary, and archives it on readily available disk drives with FTP and HTTP (Hypertext Transfer Protocol) access, allowing instantaneous data access. There are options for plug-ins for data preprocessing before storage. Publication of metadata to external applications such as the Earth Observing System Clearinghouse (ECHO) is also supported. S4PA includes a graphical user interface for monitoring the system operation and a tool for deploying the system. To ensure reliability, S4P continuously checks stored data for integrity, Further reliability is provided by tape backups of disks made once a disk partition is full and closed. The system is designed for low maintenance, requiring minimal operator oversight.
The Lower Mekong region is one of the most landslide-prone areas of the world. Despite the need for dynamic characterization of landslide hazard zones within the region, it is largely understudied for several reasons. Dynamic and integrated understanding of landslide processes requires landslide inventories across the region, which have not been available previously. Computational limitations also hamper regional landslide hazard assessment, including accessing and processing remotely sensed information. Finally, open-source software and modelling packages are required to address regional landslide hazard analysis. Leveraging an open-source data-driven global Landslide Hazard Assessment for Situational Awareness model framework, this study develops a region-specific dynamic landslide hazard system leveraging satellite-based Earth observation data to assess landslide hazards across the lower Mekong region. A set of landslide inventories were prepared from high-resolution optical imagery using advanced image-processing techniques. Several static and dynamic explanatory variables (i.e., rainfall, soil moisture, slope, relief, distance to roads, distance to faults, distance to rivers) were considered during the model development phase. An extreme gradient boosting decision tree model was trained for the monsoon period of 2015–2019 and the model was evaluated with independent inventory information for the 2020 monsoon period. The model performance demonstrated considerable skill using receiver operating characteristic curve statistics, with Area Under the Curve values exceeding 0.95. The model architecture was designed to use near-real-time data, and it can be implemented in a cloud computing environment (i.e., Google Cloud Platform) for the routine assessment of landslide hazards in the Lower Mekong region. This work was developed in collaboration with scientists at the Asian Disaster Preparedness Center as part of the NASA SERVIR Program’s Mekong hub. The goal of this work is to develop a suite of tools and services on accessible open-source platforms that support and enable stakeholder communities to better assess landslide hazard and exposure at local to regional scales for decision making and planning.
NASA produces a large volume and variety of data products that are used every day to support research, decision making, and education. The widespread use of NASA’s Earth Science data is enabled by NASA’s Earth Science Data System (ESDS) program, which oversees the archiving and distribution of these data and invests in the development of new data systems and tools. However, NASA’s current approach to Earth Science data distribution — based on distributed institutional archives with individual on-premises high-performance computing capabilities — faces some significant challenges, including massive increases in data volume from upcoming missions, a greater need for transdisciplinary science that synthesizes many different kinds of observations, and a push to make science more open, inclusive, and accessible. To address these challenges, NASA is aggressively migrating its Earth Science data and related tools and services into the commercial cloud. Migration of data into the commercial cloud can significantly improve NASA’s existing data system capabilities by (1) providing more flexible options for storage and compute (including rapid, as-needed access to state-of-the-art capabilities); (2) by centralizing and standardizing data access, which gives all of NASA’s institutional data centers access to all of each other’s datasets; and (3) by facilitating “analysis-in-place”, whereby users can bring their own computational workflows and tools to the data rather than having to maintain their own copies of NASA datasets. However, migration to the commercial cloud also poses some significant challenges, including (1) managing costs under a “pay-as-you-go” model; (2) incompatibility with existing tools and data formats with object-based storage and network access; (3) vendor lock-in; (4) challenges with data access for workflows that mix on-premise and cloud computing; and (5) standardization for highly diverse data as is present in NASA’s data archive. I conclude with two examples of recent NASA activities showcasing capabilities enabled by the commercial cloud: An interactive analysis and development platform for analyzing airborne imaging spectroscopy data, and a new collection of tools and services for data discovery, analysis, publication, and data-driven storytelling (Visualization, Exploration, and Data Analysis, VEDA).
Advancing water security in overallocated river basins globally requires consistent and reproducible information on consumptive use of water that can anchor the development of data-driven solutions to the challenge of balancing water supply and demand. OpenET is a fully automated system for field-scale (30 m), satellite-based mapping of evapotranspiration (ET) at daily, monthly and annual timesteps. OpenET currently provides spatially contiguous data throughout the 23 westernmost states in the continental US, and includes both current information as well as multi-year timeseries of ET. The OpenET consortium has implemented an ensemble of satellite-based ET models (ALEXI/DisALEXI, eeMETRIC, PT-JPL, geeSEBAL, SIMS and SSEBop) on Google Earth Engine, which provides a shared computing platform for collaboration on processing of data from Landsat and other satellites, land cover and meteorological inputs, leading to increased consistency and accuracy across the ensemble of models. Earth Engine also facilitates hosting and distribution of data via open data collections and an application programming interface. We provide updates on the OpenET framework, open data services and data access tools, approach to geographic expansion, recent accuracy assessments, and describe how a user-driven design approach has facilitated successful applications of OpenET data for a wide range of water resource management activities. Applications to date include: use of ET data to improve quantification of ET and consumptive use in Oregon, Utah and the Upper Colorado River Basin; streamlining of water use reporting requirements in the California Delta; support for calculation of water budgets for the implementation of the Sustainable Groundwater Management Act in California; and integration into decision support tools for irrigation management. The use cases demonstrate how satellite-derived ET data that are easily accessed and seen as broadly accepted can accelerate adoption of innovative water management practices at scale, and support advances in the sustainability of water supplies. Uptake and use of data by the OpenET science community has also led to advances in our understanding of the impacts of landcover change, irrigation intensification and wildfire events on hydrology and the water security.
Smart and connected communities use technology to better manage their urban energy systems and improve the quality and performance of government services by leveraging big data for data-driven decisions. NREL helps these communities reach their clean energy goals through cutting-edge expertise in planning, data, analytical tools, and technical support.
Community and city leaders are interested in achieving sustainability goals, providing resilient energy infrastructure, and improving economic competitiveness. Community-level data acquisition and analysis can provide energy and associated benefits that are not possible at the single building level. However, there is a lack of organizational structure, common semantic data models, interoperable systems, and methods to support data-driven decision-making for community-scale energy supply and demand systems. We explored the need and opportunity for a Community Energy Operations and Planning System (Community EOPS), a potential data exchange platform. We conducted “customer discovery” interviews, and reviewed literature, public tools, and technology platforms to identify key energy data “users” and use cases in communities. The key users of the Community EOPS could be developers of mixed-use districts, corporate, defense and university campus energy managers, and city managers of cities that own their energy utility. The value could be for community planning and reporting (for energy data-integrated land use planning and community infrastructure investments in microgrids, storage, district heating and cooling), energy efficiency (leveraging optimizations for community scale energy supply and demand), flexible load management (grid-edge load management to offset, shift, and flatten loads for multiple buildings and EV fleets), cost savings and revenue generation (participating in grid services), and social benefits such as energy resilience, equity, and awareness. We developed a conceptual Community EOPS architecture with recommendations for streamlined and prioritized data acquisition, sharing, and integration driven by prioritized use cases, common metrics, and actionable visualizations that can provide value to a community’s users.
The National Renewable Energy Laboratory's (NREL) Hydrogen Safety Research and Development (HSR&D) program in collaboration with the University of Maryland's Systems Risk and Reliability Analysis Laboratory (SyRRA) are working to improve reliability and reduce risk in hydrogen systems. This approach strives to use quantitative data on component leaks and failures, together with Prognosis and Health Management (PHM), and Quantitative Risk Assessment (QRA) to identify at-risk components, reduce component failures and downtime, and predict when components require maintenance. Hydrogen component failures increase facility maintenance cost, facility downtime, and reduce public acceptance of hydrogen technologies, ultimately increasing facility size and cost because of potentially overly conservative requirements. Leaks are a predominant failure mode for hydrogen components. However, uncertainties in the amount of hydrogen emitted from leaking components and the frequency of those failure events limit the understanding of the risks that they present under real-world operational conditions. NREL has deployed a test fixture, the Leak Rate Quantification Apparatus (LRQA), to quantify the mass flow rate of leaking gases from medium and high-pressure components that have failed while in service. Quantitative hydrogen leak rate data from this system could ultimately be used to better inform risk assessment and Regulation Codes and Standards (RCS). Parallel activity explores the use of PHM and QRA techniques to assess and reduce risk, thereby improving safety and reliability of hydrogen systems. The results of QRAs could further provide a systematic and science-based foundation for the design and implementation of RCS, as in the latest versions of the NFPA 2 code for gaseous hydrogen stations. Alternatively, data-driven techniques of PHM could provide new damage diagnosis and health-state prognosis tools. This research will help end users, station owners and operators, and regulatory bodies move towards risk-informed preventative maintenance versus emergency corrective maintenance, reducing cost and improving reliability. Predictive modelling of failures could improve safety and affect RCS requirements such as setback distances at liquid refuelling sites. The combination of leak rate quantification research, PHM, and QRA can lead to better informed models enabling data-based decision to be made for hydrogen system safety improvements.
This article, the fourth and final installment in the DOE–IDEA series, focuses on organizational execution and how strong management practices, teamwork, and preparedness contribute to successful district energy systems. It highlights case studies from Ashley Energy and Cornell University to illustrate effective operational strategies. Ashley Energy demonstrates the importance of emergency preparedness and rapid response. After a major flood disrupted its plant, the organization restored service in under 72 hours by relying on pre-established plans, vendor relationships, and trained staff. The case emphasizes proactive contingency planning, understanding insurance processes, and empowering skilled personnel to improvise during crises. Cornell University’s example highlights the role of collaboration and transparency in long-term success. Its district energy system benefits from strong data sharing, real-time energy monitoring, and active involvement of faculty and students in system planning and innovation. This culture of teamwork and data-driven decision-making supports sustainability goals and continuous system improvement. Overall, the article shows that effective organizational execution—through preparedness, collaboration, and data transparency—is essential for maintaining reliable, efficient, and sustainable district energy systems.