Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Distribution Service”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Striped Data Server for Scalable Parallel Data Analysis

A columnar data representation is known to be an efficient way for data storage, specifically in cases when the analysis is often done based only on a small fragment of the available data structures. A data representation like Apache Parquet is a step forward from a columnar representation, which splits data horizontally to allow for easy parallelization of data analysis. Based on the general idea of columnar data storage, working on the [LDRD Project], we have developed a striped data representation, which, we believe, is better suited to the needs of High Energy Physics data analysis. A traditional columnar approach allows for efficient data analysis of complex structures. While keeping all the benefits of columnar data representations, the striped mechanism goes further by enabling easy parallelization of computations without requiring special hardware. We will present an implementation and some performance characteristics of such a data representation mechanism using a distributed no-SQL database or a local file system, unified under the same API and data representation model. The representation is efficient and at the same time simple so that it allows for a common data model and APIs for wide range of underlying storage mechanisms such as distributed no-SQL databases and local file systems. Striped storage adopts Numpy arrays as its basic data representation format, which makes it easy and efficient to use in Python applications. The Striped Data Server is a web service, which allows to hide the server implementation details from the end user, easily exposes data to WAN users, and allows to utilize well known and developed data caching solutions to further increase data access efficiency. We are considering the Striped Data Server as the core of an enterprise scale data analysis platform for High Energy Physics and similar areas of data processing. As a result, we have been testing this architecture with a 2TB dataset from a CMS dark matter search and plan to expand it to multiple 100 TB or even PB scale. We will present the striped format, Striped Data Server architecture and performance test results.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Building ControlScore: General Service Administration Office Building Deployment

Improvements to building control systems can lead to energy savings and increased occupant comfort. In an optimized system, process variables such as air temperature will closely follow their desired setpoints and avoid excess energy use. Typically, experts must manually inspect individual control loops to identify poor performance and opportunities for improvement. However, this approach is difficult in modern buildings that have a prohibitively large number of controllers. To address this issue, Pacific Northwest National Laboratory (PNNL) created the ControlScore concept, which takes operating data from the many controllers within a building and generates standardized scores for each loop on a scale of 0 to 10 (a score of 0 indicates poor control, a score of 10 indicates good control). PNNL applied the Building ControlScore application to all available data from a General Services Administration office building within the period of January 1, 2023, to March 9, 2023. The building scored a 4.7 overall, with all 74 of the building’s loops fitting a roughly normal distribution centered around 5. These results indicate that the analyzed systems have below-average performance with room for improvement, especially in the poorly scored systems. Airflow loops tended to have much lower scores than zone temperature loops. The lowest and highest performing systems in the building section were identified, as were all loops with a score less than 1. While the ControlScore identifies loops and systems that aren’t meeting their designated setpoints, it does not indicate the cause of those issues. For example, consider a supply air terminal unit’s airflow loop that received a low score due to it delivering less air than specified by the setpoint. The lower-than-desired airflow could be due to equipment limitations (e.g., the terminal unit or duct serving is too small to accommodate that airflow), malfunctioning equipment (e.g., a stuck damper or bad sensor), or something else entirely. The ControlScore does not diagnose problems it simply identifies the symptoms that can be explored and addressed by building operators.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Development of a Knowledgebase to Integrate, Analyze, Distribute, and Visualize Microbial Community Systems Biology Data

We have developed a flexible knowledgebase system, ggKbase, (http://gg.berkeley.edu), to enable effective data analysis and knowledge generation from samples from which metagenomic and other ‘omics’ data are obtained. Within ggKbase, data can be interpreted, integrated and linked to other databases and services. Sequence information from complex metagenomic samples can be quickly and effectively resolved into genomes and biologically meaningful investigations of an organism’s metabolic potential can then be conducted. Critical features make analyses efficient, allowing analysis of hundreds of genomes at a time. The system is being used to support research in multiple DOE-relevant systems, including the LBNL SFA subsurface science biogeochemical cycling research at Rifle, Colorado. ggKbase is supporting the research of a rapidly growing group of users. It has enabled studies of carbon cycling in acid mine drainage ecosystems, biologically-mediated transformations in deep subsurface biomes sampled from mines and the north slope of Alaska, to study the human microbiome and for laboratory bioreactor-based remediation investigations.

59 BASIC BIOLOGICAL SCIENCES↗

Evaluation of the Simplified Assessment Peak Equivalent Stress Design Limit Probability of Failure

The ASME Boiler and Pressure Vessel Code Section III Division 5 Subsection HH Subpart A (HHA) outlines requirements and qualification methods for graphite core components in high temperature reactor environments. One such structural qualification method is the Simplified Assessment outlined in HHA-3220, which represents a comparison of service level specific equivalent stresses to allowable stress values derived from target probability of failures (POFs). The target POF is based on the structural reliability class (SRC) for the components and service condition of interest, and is used to establish design limits for combined membrane and peak equivalent stresses. Combined membrane stress limits are derived from two parameter Weibull distribution fits of the graphite grade specific tensile strength data and the target POF. The combined membrane stress limits are then scaled based on the ratio of flexural strength to tensile strength (referred hereinafter as Rtf) to establish the corresponding peak equivalent stress limits. It is desired that the target POF is at least maintained for both combined membrane and peak equivalent design limits, especially with the use of Rtf. The assessment presented herein examines tensile and flexural data for medium and fine grade graphites relative to the calculated combined membrane and peak equivalent stress limits per HHA-3220 and illustrates the simplified assessment method results in anticipated, or at least conservative, peak equivalent design stress limits.

36 - MATERIALS SCIENCE↗

Calculating uncertainty on K-effective with MONK10 - 14583

Criticality safety assessments require a demonstration that a particular configuration of fissile material has an adequate sub-critical margin (k-effective sufficiently below unity) to ensure that the risk of criticality under normal operation and accident conditions is acceptable. The required sub-critical margin depends upon the uncertainty in the estimated value of k-effective. The uncertainty in the calculated value of k-effective arises from a number of sources, including: manufacturing tolerances on input data to the code (affecting geometry, compositions and densities), uncertainty in the nuclear data used by the code, stochastic uncertainty resulting from Monte Carlo simulation and modelling approximations/errors, including the inevitable bugs in the software. The ANSWERS Software Service, in collaboration with industrial partners, is developing a number of techniques to better understand and quantify uncertainty on predicted values of k-effective, using MONK. The SPRUCE utility code has been developed to allow uncertainty to be estimated using sampling methods. This can include the sampling of input parameters (including dimensions, compositions and densities) from statistical distributions. It can also include sampling different nuclear data libraries. A set of nuclear data libraries has been generated for this purpose by sampling from statistical distributions that represent the uncertainties in the published nuclear data evaluated files; a set of libraries has been produced for Latin Hypercube Sampling. By varying the input data and nuclear data, separate and combined uncertainties due to manufacturing tolerances and nuclear data can be derived. By performing least squares fitting on the results it is also possible to estimate the contribution of each of the uncertain inputs and a sensitivity method in MONK can break down the nuclear data uncertainty. (authors)

42 ENGINEERING↗

VisIO: enabling interactive visualization of ultra-scale, time-series data via high-bandwidth distributed I/O systems

Petascale simulations compute at resolutions ranging into billions of cells and write terabytes of data for visualization and analysis. Interactive visuaUzation of this time series is a desired step before starting a new run. The I/O subsystem and associated network often are a significant impediment to interactive visualization of time-varying data; as they are not configured or provisioned to provide necessary I/O read rates. In this paper, we propose a new I/O library for visualization applications: VisIO. Visualization applications commonly use N-to-N reads within their parallel enabled readers which provides an incentive for a shared-nothing approach to I/O, similar to other data-intensive approaches such as Hadoop. However, unlike other data-intensive applications, visualization requires: (1) interactive performance for large data volumes, (2) compatibility with MPI and POSIX file system semantics for compatibility with existing infrastructure, and (3) use of existing file formats and their stipulated data partitioning rules. VisIO, provides a mechanism for using a non-POSIX distributed file system to provide linear scaling of 110 bandwidth. In addition, we introduce a novel scheduling algorithm that helps to co-locate visualization processes on nodes with the requested data. Testing using VisIO integrated into Para View was conducted using the Hadoop Distributed File System (HDFS) on TACC's Longhorn cluster. A representative dataset, VPIC, across 128 nodes showed a 64.4% read performance improvement compared to the provided Lustre installation. Also tested, was a dataset representing a global ocean salinity simulation that showed a 51.4% improvement in read performance over Lustre when using our VisIO system. VisIO, provides powerful high-performance I/O services to visualization applications, allowing for interactive performance with ultra-scale, time-series data.

97 MATHEMATICS AND COMPUTING↗

Secure Interoperable Open Smart Grid Demonstration Project

The Consolidated Edison, Inc., of New York (Con Edison) Secure Interoperable Open Smart Grid Demonstration Project (SGDP), sponsored by the United States (US) Department of Energy (DOE), demonstrated that the reliability, efficiency, and flexibility of the grid can be improved through a combination of enhanced monitoring and control capabilities using systems and resources that interoperate within a secure services framework. The project demonstrated the capability to shift, balance, and reduce load where and when needed in response to system contingencies or emergencies by leveraging controllable field assets. The range of field assets includes curtailable customer loads, distributed generation (DG), battery storage, electric vehicle (EV) charging stations, building management systems (BMS), home area networks (HANs), high-voltage monitoring, and advanced metering infrastructure (AMI). The SGDP enables the seamless integration and control of these field assets through a common, cyber-secure, interoperable control platform, which integrates a number of existing legacy control and data systems, as well as new smart grid (SG) systems and applications. By integrating advanced technologies for monitoring and control, the SGDP helps target and reduce peak load growth, improves the reliability and efficiency of Con Edison’s grid, and increases the ability to accommodate the growing use of distributed resources. Con Edison is dedicated to lowering costs, improving reliability and customer service, and reducing its impact on the environment for its customers. These objectives also align with the policy objectives of New York State as a whole. To help meet these objectives, Con Edison’s long-term vision for the distribution grid relies on the successful integration and control of a growing penetration of distributed resources, including demand response (DR) resources, battery storage units, and DG. For example, Con Edison is expecting significant long-term growth of DG. The SGDP enables the efficient, flexible integration of these disparate resources and lays the architectural foundations for future scalability. Con Edison assembled an SGDP team of more than 16 different project partners, including technology vendors, and participating organizations, and the Con Edison team provided overall guidance and project management. Project team members are listed in Table 1-1.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Measurements and analytics of wide-area file transfers over dedicated connections

Distributed scientific and big-data computations are becoming increasingly dependent on access to remote files. Wide-area file transfers are supported by two basic schemes: (i) application-level tools, such as GridFTP, that provide transport services between file systems housed at geographically separated sites, and (ii) file systems mounted over wide-area networks, using mechanisms such as LNet routers that make them transparently available. In both cases, the file transfer performance critically depends on the configuration consisting of host, file, IO, and disk subsystems, which are complex by themselves, as well as on their complex compositions implemented using buffers and IO-network data transitions. We present extensive file transfer rate measurements collected over dedicated 10 Gbps connections with 0-366 ms round-trip times, using GridFTP and XDD file transfer tools, and Lustre file system extended over wide-area networks using LNet routers. Our test configurations are composed of: three types of host systems; XFS, Lustre, and ext3 file systems; and Ethernet and SONET wide-area connections. We present analytics based on the convexity-concavity of throughput profiles which provide insights into throughput and its superior or inferior trend compared to linear interpolations. We propose the utilization-concavity coefficient, a scalar metric that characterizes the overall performance of any file transfer method consisting of specific configuration and scheme. Our results enable performance optimizations by highlighting the significant roles of (i) buffer sizes and parallelism in GridFTP and XDD, and (ii) buffer utilization and credit mechanism in LNet routers.

Rao, Nageswara↗

Scalable in-memory RDFS closure on billions of triples.

We present an RDFS closure algorithm, specifically designed and implemented on the Cray XMT supercomputer, that obtains inference rates of 13 million inferences per second on the largest system configuration we used. The Cray XMT, with its large global memory (4TB for our experiments), permits the construction of a conceptually straightforward algorithm, fundamentally a series of operations on a shared hash table. Each thread is given a partition of triple data to process, a dedicated copy of the ontology to apply to the data, and a reference to the hash table into which it inserts inferred triples. The global nature of the hash table allows the algorithm to avoid a common obstacle for distributed memory machines: the creation of duplicate triples. On LUBM data sets ranging between 1.3 billion and 5.3 billion triples, we obtain nearly linear speedup except for two portions: file I/O, which can be ameliorated with the additional service nodes, and data structure initialization, which requires nearly constant time for runs involving 32 processors or more.

99 GENERAL AND MISCELLANEOUS↗

Software-defined network for end-to-end networked science at the exascale

Domain science applications and workflow processes are currently forced to view the network as an opaque infrastructure into which they inject data and hope that it emerges at the destination with an acceptable Quality of Experience. There is little ability for applications to interact with the network to exchange information, negotiate performance parameters, discover expected performance metrics, or receive status/troubleshooting information in real time. The work presented here is motivated by a vision for a new smart network and smart application ecosystem that will provide a more deterministic and interactive environment for domain science workflows. The Software-Defined Network for End-to-end Networked Science at Exascale (SENSE) system includes a model-based architecture, implementation, and deployment which enables automated end- to-end network service instantiation across administrative domains. An intent based interface allows applications to express their high-level service requirements, an intelligent orchestrator and resource control systems allow for custom tailoring of scalability and real-time responsiveness based on individual application and infrastructure operator requirements. This allows the science applications to manage the network as a first-class schedulable resource as is the current practice for instruments, compute, and storage systems. Deployment and experiments on production networks and testbeds have validated SENSE functions and performance. Emulation based testing verified the scalability needed to support research and education infrastructures. Key contributions of this work include an architecture definition, reference implementation, and deployment. This provides the basis for further innovation of smart network services to accelerate scientific discovery in the era of big data, cloud computing, machine learning and artificial intelligence.

47 OTHER INSTRUMENTATION↗

Cloud Services Enable Efficient AI-Guided Simulation Workflows across Heterogeneous Resources

Applications which fuse machine learning and simulation are rarely best served by a single computing resource. Highly parallel simulation codes are best deployed on super- computers, while AI tasks used to decide which simulations to perform may be best suited to specialized accelerators. Here we present a Function-as-a-Service (FaaS) system for executing complex, distributed computational campaigns that achieves performance parity with conventional workflow systems without the complexities of secure network connections between compute providers. One innovation enabling high performance is a subsystem that directly moves task data between sites, separate from the cloud-hosted FaaS system used to distribute task instructions. We also introduce a flexible scheduling system that allows us access factor of 2 trade offs between the amount of resources required to solve a problem at each compute site. We anticipate that this system will upgrade multi-site applications from demonstration projects to routine practice in computational science.

Ward, Logan↗

A global map of saltmarshes

Saltmarshes are extremely valuable but often overlooked ecosystems, contributing to livelihoods locally and globally through the associated ecosystem services they provide, including fish production, carbon storage and coastal protection. Despite their importance, knowledge of the current spatial distribution (occurrence and extent) of saltmarshes is incomplete. In light of increasing anthropogenic and environmental pressures on coastal ecosystems, global data on the occurrence and extent of saltmarshes are needed to draw attention to these critical ecosystems and to the benefits they generate for people. Such data can support resource management, strengthen decision-making and facilitate tracking of progress towards global conservation targets set by multilateral environmental agreements, such as the Aichi Biodiversity Targets of the United Nations' (UN's) Strategic Plan for Biodiversity 2011-2020, the Sustainable Development Goals of the UN's 2030 Agenda for Sustainable Development and the Ramsar Convention.

54 ENVIRONMENTAL SCIENCES↗

Examination of Industry Payments to Radiation Oncologists in 2014 Using the Centers for Medicare and Medicaid Services Open Payments Database

Purpose: To use the Centers for Medicare and Medicaid Services Open Payments database to characterize payments made to radiation oncologists and compare their payment profile with that of medical and surgical oncologists. Methods and Materials: The June 2015 release of the Open Payments database was accessed, containing all payments made to physicians in 2014. The general payments dataset was used for analysis. Data on payments made to medical, surgical, and radiation oncologists was obtained and compared. Within radiation oncology, data regarding payment category, sponsorship, and geographic distribution were identified. Basic statistics including mean, median, range, and sum were calculated by provider and by transaction. Results: Among the 3 oncologic specialties, radiation oncology had the smallest proportion (58%) of compensated physicians and the lowest mean ($1620) and median ($112) payment per provider. Surgical oncology had the highest proportion (84%) of compensated physicians, whereas medical oncology had the highest mean ($6371) and median ($448) payment per physician. Within radiation oncology, nonconsulting services accounted for the most money to physicians ($1,042,556), whereas the majority of the sponsors were medical device companies (52%). Radiation oncologists in the West accepted the most money ($2,041,603) of any US Census region. Conclusions: Radiation oncologists in 2014 received a large number of payments from industry, although less than their medical or surgical counterparts. As the Open Payments database continues to be improved, it remains to be seen whether this information will be used by patients to inform choice of providers or by lawmakers to enact policy regulating physician–industry relationships.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

OpenET: Filling a Critical Data Gap in Water Management for the Western United States

The lack of consistent, accurate information on evapotranspiration (ET) and consumptive use of water by irrigated agriculture is one of the most important data gaps for water managers in the western United States (U.S.) and other arid agricultural regions globally. The ability to easily access information on ET is central to improving water budgets across the West, advancing the use of data-driven irrigation management strategies, and expanding incentive-driven conservation programs. Recent advances in remote sensing of ET have led to the development of multiple approaches for field-scale ET mapping that have been used for local and regional water resource management applications by U.S. state and federal agencies. The OpenET project is a community-driven effort that is building upon these advances to develop an operational system for generating and distributing ET data at a field scale using an ensemble of six well-established satellite-based approaches for mapping ET. Key objectives of OpenET include: Increasing access to remotely sensed ET data through a web-based data explorer and data services; supporting the use of ET data for a range of water resource management applications; and development of use cases and training resources for agricultural producers and water resource managers. Here we describe the OpenET framework, including the models used in the ensemble, the satellite, meteorological, and ancillary data inputs to the system, and the OpenET data visualization and access tools. We also summarize an extensive intercomparison and accuracy assessment conducted using ground measurements of ET from 139 flux tower sites instrumented with open path eddy covariance systems. Results calculated for 24 cropland sites from Phase I of the intercomparison and accuracy assessment demonstrate strong agreement between the satellite-driven ET models and the flux tower ET data. For the six models that have been evaluated to date (ALEXI/DisALEXI, eeMETRIC, geeSEBAL, PT-JPL, SIMS, and SSEBop) and the ensemble mean, the weighted average mean absolute error (MAE) values across all sites range from 13.6 to 21.6 mm/month at a monthly timestep, and 0.74 to 1.07 mm/day at a daily timestep. At seasonal time scales, for all but one of the models the weighted mean total ET is within ±8% of both the ensemble mean and the weighted mean total ET calculated from the flux tower data. Overall, the ensemble mean performs as well as any individual model across nearly all accuracy statistics for croplands, though some individual models may perform better for specific sites and regions. We conclude with three brief use cases to illustrate current applications and benefits of increased access to ET data, and discuss key lessons learned from the development of OpenET.

54 ENVIRONMENTAL SCIENCES↗

Pipelining RDP Data to the "Taxomatic" (Final Report)

This project builds on the results of previously funded research by integrating data and software that had been previously used in building resources used in the preparation of Bergey's Manual of Systematic Bacteriology, 2 nd Edition (Volumes 1 & 2A-C) and the Ribosomal Database Project-II (RDP-II) so as to both enhance the value of the data and create a pipeline approach to keeping the data current. Earlier, we demonstrated the value of using exploratory data analysis (EDA) to visualize large sets of sequence data (notably SSU rRNA gene sequences used in constructing a comprehensive phylogeny of prokaryotes. While the Self- Organizing Self-Correcting Classification (SOSCC) algorithms we developed were computationally efficient and useful for unraveling problems within the underlying data (e.g., identification of annotation errors, detection of unresolved synonymies, taxonomic and nomenclatural errors), bottlenecks at the preprocessing stage limited deployment of our applications as tools for end-users. To overcome the bottlenecks (which included hand alignment and computation of large matrices of pair-wise evolutionary distances), we proposed building a data pipeline between the "Taxomatic" application and RDP-II. The objectives were to accelerate the production and distribution of the updated versions of the prokaryotic taxonomy in lock-step with publication of new taxa and rearrangement of existing taxa, and to distribute these data more readily with the RDP-II and and other stakeholders in the community. A related goal of the current project is to deploy our visualization techniques as an interactive web application by which end-users can view manipulate, and select datasets of particular interest based upon phylogenetic and genomic information, access sequence data, and ultimately the scientific literature where the original observations were made and those that build on the original observations. The Taxomatic is a web-based tool to visualize distance matrices. The tool accepts raw distance matrices or aligned sequence information as data sources. When sequence information is provided the distance matrix is computed using the uncorrected distance model. Users can upload files to the Taxomatic website or sequences can be submitted by a SOAP service. This SOAP service is used by RDP to streamline Taxomatic use with RDP data. In addition to supplying source information, users can either supply their own taxonomic information by uploading it in XML, retrieve data taxonomic information from the RDP using either RDP or Genbank identifiers as source data, with or without classification by the RDP Classifier web service, or completely omit taxonomic data. In the latter case, the input distance matrix can be viewed in the order in which it was loaded.

59 BASIC BIOLOGICAL SCIENCES↗

EV Charging Infrastructure Energization An Overview of Approaches for Simplifying and Accelerating Timelines to Processing EV Charging Load Service Requests

The United States has seen significant growth in electric vehicle (EV) adoption, leading to increased demand for EV charging infrastructure. Over the past decade, EV charging infrastructure site developers, site hosts, and electric distribution utilities have navigated the process to integrate chargers onto the electric grid. Site developers and site hosts have raised the alarm that the integration process for high-powered EV charging projects does not meet the needs of the EV market for timeliness or cost. High-powered charging stations typically require a load service request or an agreement with the local utility to connect to the grid. The process of energizing a new high-powered charging site can be complex and time-consuming, often taking up to 2 years. This timeline is the result of current utility energization processes having been designed for construction projects that take longer to build (i.e., buildings). The specific challenges stem from various factors, including compartmentalization in application processes, the integration of EV charging process approvals with other distributed energy resources (DERs), and the need to ensure grid reliability. The energization process needs to evolve to meet the growing demand for high-powered EV charging. This white paper compiles information gathered through various conversations with key stakeholders, including utilities, utility regulators, EV charging operators, site developers, and authorities having jurisdiction (AHJ) as well as through an extensive literature review. This document identifies the challenges and provides potential solutions to streamline the process of connecting EV charging infrastructure to the power grid in the United States, serving as a starting point for future conversations around these solutions. The solutions noted in this white paper require collaborative efforts among utilities, regulators, and EV charging infrastructure developers to streamline the grid connection process for EV charging infrastructure. They are broadly organized into four areas: 1. Increase data access and transparency: Develop automated load service request tools, integrate hosting capacity and load service request analyses, incorporate EV adoption forecasts, and provide transparency on the processing queue. 2. Improve energization processes and timing: Create fast-track options based on prescreening criteria, provide flexibility or phased approvals in the load service request/interconnection process, build internal knowledge within utilities about EV charging technologies, and provide standardized workforce training. 3. Promote economic efficiency: Right size distribution components to accurately reflect the load requirements of EV charging infrastructure, make proactive investments in grid infrastructure based on EV adoption forecasts and growth projections, and consider energy equity and environmental justice factors such as equitable access to EV charging when planning infrastructure. 4. Improve grid reliability and resilience: Use load management/power control systems (PCS) at EV charging stations, adopt and implement harmonized standards for communication protocols and information models between the EV charging and grid control infrastructure, and address cybersecurity considerations by implementing robust security measures and standards for EV charging infrastructure—with particular emphasis on clarifying the security requirements for the interface to the grid. The objective of the solutions proposed in this white paper is to accelerate the timeline and decrease costs associated with connecting EV charging infrastructure to the grid. Electric utilities, utility regulators, EV charging infrastructure developers, and site hosts will first need to understand which solutions are available in their service territory, and if warranted, which combination of solutions would support their specific needs. Through the successful implementations of solutions at scale detailed here, industry will demonstrate a new and innovative ecosystem where timely deployment and energization of EV charging infrastructure with greater grid resiliency and reliability is a reality.

24 POWER TRANSMISSION AND DISTRIBUTION↗

MeDICi: An Open Platform for Sensor Integration

MeDICi (Middleware for Data Intensive Computing) is a platform for developing high performance, distributed streaming analytic and scientific applications. Developed at Pacific Northwest National Laboratory (PNNL), MeDICi has been released under an open source license and is based on enterprise-proven middleware technologies including a widely used Enterprise Service Bus (ESB), the standard Business Process Execution Language (BPEL), and open source message brokers. Wherever possible, we have built on existing open source, standards-based systems and integrated them into a coherent whole by creating simplified graphical programming tools such as a Workflow Designer and an easy to use and well-documented integration API. This software development approach allows us to: avoid re-creating complex service integration and orchestration systems, reap the benefits of continual improvements to the technology base, and focus on creating tools and APIs which allow for the creation of re-usable component-based software components applications and workflows. These aspects have facilitated rapid adoption of the platform within PNNL for demonstration and operational applications. In fact, MeDICi has been used for a wide range of integration projects including two sensor integration applications described later on in this paper. The remainder of this article white paper is organized as follows: Section 2 provides a high-level description of the MeDICi architecture. In Section 3, the open aspects of the API and tool development are highlighted. Section 4 explains system readiness by presenting relevant demonstrations and deployments. Finally documentation and licensing details are provided in Section 5

99 GENERAL AND MISCELLANEOUS↗

A Two-Step Time-Series Data Clustering Method for Building-Level Load Profile

Residential and commercial buildings have huge potential to contribute value to improve grid resilience by participating grid services. To reveal the significant value, it is critical to estimate the grid service capability from these buildings. Unlike the large-scale distributed energy resources such as wind and solar farms, those buildings need to participate grid services in aggregation, not by individual. Therefore, it is important to appropriately group buildings for aggregation. The load profiles in the same group will have similar characteristics at the same time step, so grid operators can send the grid service signal to the customer group with a higher chance to respond at that time step. In this paper, we develop a load profile clustering method to classify the building-level load profiles for grid service capability estimation. In our two-step clustering approach, we first calculate the total load consumption for each building, clustering the load profiles based on energy consumption level. Then, we further cluster the load profiles in each energy cluster based on the load shape. The parameter selection for each clustering step is discussed. The proposed method is applied on actual building-level load profiles, and the results have proved the effectiveness of this method.

advanced metering infrastructure (AMI)↗