Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Distribution Service”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

Field To Farm Aggregation For Agricultural Systems

The Fields to Farms methodology illustrates the generation of farm parcels from the Crop Data Layer (CDL), a raster dataset containing 133 categories representing various crop types and land uses. This methodology involves two primary steps: Field Delineation and Farm Aggregation. The code specifically addresses the aggregation of pre-delineated fields within a county to form farms, adhering to predefined criteria for farm size categories. It is assumed that the field delineation process, which involves creating vector polygons from CDL raster, has been completed beforehand, possibly through external tools or methods. Upon initialization, the script processes county-level fields, preparing them for farm aggregation. In the Farm Aggregation phase, the code iteratively combines delineated fields into farms based on specified criteria, continuing until the aggregated farm size meets predefined thresholds derived from data from the 2017 National Agricultural Statistics Service (NASS) census. Throughout this iterative process, the script dynamically adjusts the aggregation to ensure alignment with the desired distribution reported by NASS. The resulting output of the script is a GeoDataFrame containing classified farms, which are subsequently saved as GeoPackage files. These files enable further analysis and visualization, facilitating comprehensive exploration of the farm landscape generated through the methodology.

Paudel, Rajiv [Idaho National Laboratory (INL), Id↗

eMosaic: Electrification Mosaic Platform for Grid Informed Smart Charging Management (Final Scientific/Technical Report)

ABB (Prime Contractor), in collaboration with its partners at the Utah State University (USU), Idaho National Laboratory, Rocky Mountain Power (RMP), and Electric Power Engineers (EPE), have performed research, development, and wide scale demonstration of a scalable and resilient Electrification Mosaic (eMosaic) platform for Smart Charge Management (SCM) for Electric Vehicle Infrastructure. Work was completed under DE EE0009194, titled “eMosaic Electrification Mosaic Platform for Grid Informed Smart Charging Management”, funded by the US Department of Energy. The project members developed algorithms that provide localized and bulk grid services and that reduce and stabilize costs all the way down the supply chain to the PEV owner through SCM. This platform aggregates telemetry from multiple data sources as pieces of the larger picture including personal, private fleet or transportation EVs, fast chargers and other supply equipment, weather service information, and geographically distributed charging sites such as public lots, garage and retail, and private or shared usage depots. ABB and the project team designed, tested, and improved a charging management system at local/edge and cloud levels. The ultimate objective of the project was to convincingly demonstrate that the developed secure eMosaic plat-form can be readily and favorably adopted by diverse utilities and site owners at scale. This was achieved through a demonstration plan with field deployment at several physical sites across 4 states and additional scalable simulation from high fidelity charging models.

24 POWER TRANSMISSION AND DISTRIBUTION↗

R&D100: Lightweight Distributed Metric Service

On today's High Performance Computing platforms, the complexity of applications and configurations makes efficient use of resources difficult. The Lightweight Distributed Metric Service (LDMS) is monitoring software developed by Sandia National Laboratories to provide detailed metrics of system performance. LDMS provides collection, transport, and storage of data from extreme-scale systems at fidelities and timescales to provide understanding of application and system performance with no statistically significant impact on application performance.

97 MATHEMATICS AND COMPUTING↗

A Distributed Middleware Architecture for Attack-Resilient Communications in Smart Grids

Distributed Energy Resources (DERs) are being increasingly accepted as an excellent complement to traditional energy sources in smart grids. As most of these generators are geographically dispersed, dedicated communications investments for every generator are capital cost prohibitive. Real-time distributed communications middleware, which supervises, organizes and schedules tremendous amounts of data traffic in smart grids with high penetrations of DERs, allows for the use of existing network infrastructure. In this paper, we propose a distributed attack-resilient middleware architecture that detects and mitigates the congestion attacks by exploiting the Quality of Experience (QoE) measures to complement the conventional Quality of Service (QoS) information to detect and mitigate the congestion attacks effectively. The simulation results illustrate the efficiency of our proposed communications middleware architecture.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NREL and San Diego Gas & Electric Company

San Diego Gas & Electric Company (SDG&EA(R)) and NREL are collaborating to evaluate advanced metering infrastructure (AMI)-based controls for grid operations. With millions of new smart meters being deployed around the country, this collaboration will take the lead in determining how to generate insights from the AMI data, and to use these insights for real-time operations. Smart meters ensure availability of pervasive and continuous data from the grid-edge, a service that is not possible with traditional grid sensors.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Components in the Pipeline

Scientists commonly describe their data processing systems metaphorically as software pipelines. These pipelines input one or more data sources and apply a sequence of processing steps to transform the data and create useful results. While conceptually simple, pipelines often adopt complex topologies and must meet stringent quality of service requirements that place stress on the software infrastructure used to construct the pipeline. In this paper we describe the MeDICi Integration Framework, which is a component-based framework for constructing complex software pipelines. The framework supports composing pipelines from distributed heterogeneous software components and provides mechanisms for controlling qualities of service to meet demanding performance, reliability and communication requirements.

99 GENERAL AND MISCELLANEOUS↗

Distribution Transformer Health Monitoring using Smart Meter Data

The distribution electric grid has become a highly complex and intelligent network with changing load and customer types. This has generated unprecedented challenges and opportunities for utility companies–opportunities especially in the area of asset health/performance management. Moreover, several utilities are increasingly moving from the traditional reactive and time-based asset monitoring approach to a more proactive condition based method. However, this needs to be done in a low-cost and efficient manner. Here, this paper explores how existing sensor infrastructure such as smart meters can be utilized to provide utility operators with more visibility into the health and operation of their assets. The paper focuses primarily on service transformers.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Geospatial Visualization of Global Satellite Images with Vis-EROS

The Earth Resources Observation and Science (EROS) Center of U.S. Geological Survey is currently managing and maintaining the world largest satellite images distribution system, which provides 24/7 free download service for researchers all over the globe in many areas such as Geology, Hydrology, Climate Modeling, and Earth Sciences. A large amount of geospatial data contained in satellite images maintained by EROS is generated every day. However, this data is not well utilized due to the lack of efficient data visualization tools. This software implements a method for visualizing various characteristics of the global satellite image download requests. More specifically, Keyhole Markup Language (KML) files are generated which can be loaded into an earth browser such as Google Earth. Colored rectangles associated with stored satellite scenes are painted onto the earth browser; and the color and opacity of each rectangle is varied as a function of the popularity of the corresponding satellite image. An analysis of the geospatial information obtained relative to specified time constraints provides an ability to relate image download requests to environmental, political, and social events.

54 ENVIRONMENTAL SCIENCES↗

BESPOKV: Application Tailored Scale-Out Key-Value Stores

Enterprise KV stores are not well suited for HPC applications, and entail customization and cumbersome end-to-end KV design to extract the HPC application needs. To this end, in this paper we present BESPOKV, an adaptive, extensible, and scale-out KV store framework. BESPOKV decouples the KV store design into the control plane for distributed management and the data plane for local data store. BESPOKV takes as input a single-server KV store, called a datalet, and transparently enables a scalable and fault-tolerant distributed KV store service. The resulting distributed stores are also adaptive to consistency or topology requirement changes and can be easily extended for new types of services. Experiments show that BESPOKV-enabled distributed KV stores scale horizontally to a large number of nodes, and performs comparably and sometimes better than the state-of-the-art systems.

Anwar, Ali↗

Site Environmental Report for 2009, Volume I

Each year, the University of California (UC), as the managing and operating contractor of the Ernest Orlando Lawrence Berkeley National Laboratory, prepares an integrated report regarding its environmental programs to satisfy the requirements of United States Department of Energy (DOE) Order 231.1A, Environment, Safety, and Health Reporting.1 The Site Environmental Report for 2009 summarizes Berkeley Lab's environmental management performance, presents environmental monitoring results, and describes significant programs for calendar year (CY) 2009. Throughout this report, 'Berkeley Lab' or 'LBNL' refers both to (1) the multiprogram scientific facility the UC manages and operates on the 202-acre university-owned site located in the hills above the UC Berkeley campus, and the site itself, and (2) the UC as managing and operating contractor for Ernest Orlando Lawrence Berkeley National Laboratory. The report is separated into two volumes. Volume I is organized into an executive summary followed by six chapters that contain an overview of LBNL, a discussion of its environmental management system (EMS), the status of environmental programs, summarized results from surveillance and monitoring activities, and quality assurance (QA) measures. Volume II contains individual data results from surveillance and monitoring activities. The Site Environmental Report is distributed by releasing it on the World Wide Web (Web) from the Berkeley Lab Environmental Services Group (ESG) home page, which is located at www.lbl.gov/ehs/esg/. Many of the documents cited in this report also are accessible from the ESG Web page. Links to documents available on the Web are given with the citations in the References section. CD and printed copies of this Site Environmental Report are available upon request. The report follows Berkeley Lab's policy of using the International System of Units (SI), also known as the metric system of measurements. Whenever possible, results are also reported using the more conventional (non-SI) system of measurements, because the non-SI system is referenced by several current regulatory standards and is more familiar to some readers. Two tables are provided at the end of the Glossary to help readers: Table G-1 defines the prefixes used with SI units of measurement, and Table G-2 provides conversions to non-SI units. Years mentioned in this report refer to calendar years unless specified as fiscal year(s). Berkeley Lab's fiscal year (FY) is October 1 to September 30, and begins in the year previous to its name, i.e., FY 2009 was from October 1, 2008, to September 30, 2009. For ease of reference, a key to acronyms and abbreviations used in this report can be found directly after the text, at the end of Chapter 6. Following that is also a glossary for readers who may be unfamiliar with some of the terms used in this report. This report was prepared under the direction of Ron Pauer of ESG. Please address any questions regarding this report to him by telephone at 510-486-7614, or by e-mail at ropauer@lbl.gov. The primary contributors were David Baskin, Tim Bauters, Ned Borglin, Robert Fox, John Jelinski, Ginny Lackner, Patrick Thorson, Linnea Wahl, and Suying Xu (Volume II). Readers are encouraged to comment on this report by completing the survey form found at the ESG Web page where this report is available.

57↗

A Framework for Federated Two-Factor Authentication Enabling Cost-Effective Secure Access to Distributed Cyberinfrastructure

As cyber attacks become increasingly sophisticated, the security measures used to mitigate the risks must also increase in sophistication. One time password (OTP) systems provide strong authentication because security credentials are not reusable, thus thwarting credential replay attacks. The credential changes regularly, making brute-force attacks significantly more difficult. In high performance computing, end users may require access to resources housed at several different service provider locations. The ability to share a strong token between multiple computing resources reduces cost and complexity. The National Science Foundation (NSF) Extreme Science and Engineering Discovery Environment (XSEDE) provides access to digital resources, including supercomputers, data resources, and software tools. XSEDE will offer centralized strong authentication for services amongst service providers that leverage their own user databases and security profiles. This work implements a scalable framework built on standards to provide federated secure access to distributed cyberinfrastructure.

99 GENERAL AND MISCELLANEOUS↗

U.S. residential consumer product information: Validation of methods for post-stratification weighting of Amazon Mechanical Turk surveys

We present two post-stratification weighting methods to validate survey data collected using Amazon Mechanical Turk (AMT). Two surveys focused on appliance and consumer electronics devices were administered in the spring and summer of 2012 to each of approximately 3,000 U.S. households. Specifically, the surveys asked questions about residential refrigeration products, televisions (TVs) and set-top boxes (STBs). Filtered data were assigned weights using each of two weighting methods, termed “sequential” and “simultaneous,” by examining up to eight demographic variables (income, education, gender, race, Hispanic origin, number of occupants, ages of occupants, and geographic region) in comparison to reference U.S. demographic data from the 2009 Residential Energy Consumption Survey (RECS). Five key questions from the surveys (number of refrigerators, number of freezers, number of TVs, number of STBs and primary service provider) were evaluated with a set of statistical tests to determine whether either method improved the agreement of AMT with reference data, and if so, which method was better. The statistical tests used were: differences in proportions, distributions of proportions (using Pearson’s chi-squared test), and differences in average numbers of devices as functions of all demographic variables. The results indicated that both methods generally improved the agreement between AMT and reference data, sometimes greatly, but that the simultaneous method was usually superior to the sequential method. Some differences in sample populations were found between the AMT and reference data. Differences in the proportion of STBs reflected large changes in the STB market since the time our reference data was acquired in 2009. Differences in the proportions of some primary service providers suggested real sample bias, with the possible explanation that AMT user are more likely to subscribe to providers who also provide home internet service. Differences in other variables, while statistically significant in some cases, were nonetheless considered to be minor. Depending on the intended purpose of the data collected using AMT, these biases may or may not be important; to correct them, additional questions and/or further post-survey adjustments could be employed. In general, based on the analysis methods and the sample datasets used in this study, AMT surveys appeared to provide useful data on appliance and consumer electronics devices.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Additive Manufacturing Flaw Assessment Methodology

An evolution fatigue data and flaw tolerance of components produced using the Powder Bed Fusion (PBF) Additive Manufacturing (AM) process is documented in this report. Initial differences in fatigue data for AM components compared to smooth bar fatigue data indicated a need for a more detailed analysis of AM data available in technical literature. The investigation was initiated to support the development of a fatigue analysis methodology for AM components to support the of codification of AM technology for pressure equipment. The project was initiated to collect and analyze stainless steel 304L and 316L AM fatigue test data and corresponding process and quality information to develop S-N and E-N based fatigue data representation. Additional AM fatigue test data including Inconel Ti-6-4 and aluminum alloys were also considered for comparison purposes Metallic AM parts tend to contain various forms of defects distributed throughout the part. If an AM part is subjected to fatigue loading in service, a fatigue analysis needs to be performed during the design process to ensure an acceptable service life for the part. Post-process machining and polishing do not to improve fatigue resistance in any significant degree. The low cycle fatigue regime is of particular interest to this project in support of flaw acceptance criteria currently under development by ASME’s BPTCS/BNCS. Internal defects become exposed as external surface defects during machining for the machining of the AM part to final dimensions. This implies that as long as inherent AM defects are within a controlled limit in terms of both size and distribution characteristics, the corresponding fatigue test data in terms of either S-N (stress life) or E-N (strain life) can be investigated and characterized to establish fatigue properties of AM parts for design and fatigue evaluation purposes. The resulting S-N or E-N curves and their scatter bands can be used to derive fatigue design allowable stress values by capturing the effects of distributed discontinuities within an acceptable limit.

36 MATERIALS SCIENCE↗

Scale tests of the new DUNE data pipeline

In preparation for the second runs of the ProtoDUNE detectors at CERN (NP02 and NP04)[1], DUNE has established a new data pipeline for bringing the data from the EHN-1 experimental hall at CERN to primary tape storage at Fermilab and CERN, and then spreading it out to a distributed disk data store at many locations around the world. This system includes a new Ingest Daemon and a new Declaration Daemon. The Rucio[2] replica catalog, and FTS3 transport are used to transport all files. All file metadata is declared to the new MetaCat[3] metadata service. All of these new components have been successfully tested at a scale equal to the expected output of the detector data acquisition system (~2-4 GB/s), and the expected network bandwidth out of the experimental hall. We present the procedure that was used to test and the results of the test.

Timm, Steven↗

Optimal Power Flow Pursuit

Past works that focused on addressing power-quality and reliability concerns related to renewable energy resources (RESs) operating with business-as-usual practices have looked at the design of Volt/VAr and Volt/Watt strategies to regulate real or reactive powers based on local voltage measurements, so that terminal voltages are within acceptable levels. These control strategies have the potential of operating at the same time scale of distribution-system dynamics, and can therefore mitigate disturbances precipitated fast time-varying loads and ambient conditions; however, they do not necessarily guarantee system-level optimality, and stability claims are mainly based on empirical evidences. On a different time scale, centralized and distributed optimal power flow (OPF) algorithms have been proposed to compute optimal steady-state inverter setpoints, so that power losses and voltage deviations are minimized and economic benefits to end-users providing ancillary services are maximized. However, traditional OPF schemes may offer decision making capabilities that do not match the dynamics of distribution systems. Particularly, during the time required to collect data from all the nodes of the network (e.g., loads), solve the OPF, and subsequently dispatch setpoints, the underlying load, ambient, and network conditions may have already changed; in this case, the DER output powers would be consistently regulated around outdated setpoints, leading to suboptimal system operation and violation of relevant electrical limits. The present work focuses on the synthesis of distributed RES-inverter controllers that leverage the opportunities for fast feedback offered by power-electronics interfaced RESs. The overarching objective is to bridge the temporal gap between long-term system optimization and real-time control, to enable seamless RES integration in large scale with stability and efficiency guarantees, while congruently pursuing system-level optimization objectives. The design of the control framework is based on suitable linear approximations of the AC power-flow equations as well as Lagrangian regularization methods. The proposed controllers enable an update of the power outputs at a time scale that is compatible with the underlying dynamics of loads and ambient conditions, and continuously drive the system operation towards OPF-based solutions.

24 POWER TRANSMISSION AND DISTRIBUTION↗

A synthesis dataset of permafrost-affected soil thermal conditions for Alaska, USA

Recent observations of near-surface soil temperatures over the circumpolar Arctic show accelerated warming of permafrost-affected soils. The availability of a comprehensive near-surface permafrost and active layer dataset is critical to better understanding climate impacts and to constraining permafrost thermal conditions and its spatial distribution in land system models. We compiled a soil temperature dataset from 72 monitoring stations in Alaska using data collected by the U.S. Geological Survey, the National Park Service, and the University of Alaska Fairbanks permafrost monitoring networks. The array of monitoring stations spans a large range of latitudes from 60.9 to 71.3°N and elevations from near sea level to ~1300 m, comprising tundra and boreal forest regions. This dataset consists of monthly ground temperatures at depths up to 1 m, volumetric soil water content, snow depth, and air temperature during 1997–2016. These data have been quality controlled in collection and processing. Meanwhile, we implemented data harmonization evaluation for the processed dataset. The final product (PF-AK, v0.1) is available at the Arctic Data Center (https://doi.org/10.18739/A2KG55).

58 GEOSCIENCES↗

Distributed Resources for the Earth System Grid Federation (ESGF) Advanced Management (DREAM). Final Report

Distributed Resources for the Earth System Grid Federation (ESGF) Advanced Management (DREAM) is a proposed system that will enable data from an infinite number of diverse sources to be organized and accessed from anywhere using any handheld or other computer device. The approach offers a powerful roadmap for the creation and integration of a unified knowledge base of an entire ecosystem, including its many geophysical, geographical, social, political, agricultural, energy, transportation, and cyber aspects. The resulting aggregation of data has the potential to generate an informational universe of unprecedented size that has never before been possible due to the prohibitive costs, managerial complexity, and technical barriers associated with ever-changing exponential-growth data flows. We envision that DREAM will accelerate discovery by enabling climate researchers, among other types of researchers, to manage, analyze, and visualize data from earth-scale measurements and simulations. DREAM’s success will be built on proven components that leverage existing services and resources. A key building block for DREAM will be the ESGF, chaired by Dean N. Williams. Expanding on the existing ESGF, the project will ensure that the access, storage, movement, and analysis of the large quantities of data that are processed and produced by diverse science projects can be dynamically distributed with proper resource management. Much of the Office of Science data is currently generated by multiple stand-alone facilities. DREAM can collect data accumulated from these facilities and incorporate it into a fully integrated network accessible from anywhere in the world. The result is a completely new paradigm shift for data management, analysis, and visualization enabling researchers to: Manage their calculations, data, tools, and research results; Ensure that all data are sharable, reproducible and (re)usable—accompanied by appropriate metadata describing its provenance, syntax, and semantics at creation; Advance application performance by selectively adapting APIs and services in response to scientific requirements and architectural complexities; and Provide scalable interactive resource management—navigate data and metadata at multiple levels, provide architecture-aware data integration, analysis and visualization tools. We will engage closely with DOE, NASA, and NOAA science groups working at the leading edge of computing. These engagements—in domains such as biology, climate, and hydrology—will allow us to advance disciplinary science goals and inform our development of technologies that can accelerate discovery across DOE more broadly. We will advertise and promote our technologies via dedicated workshops, tutorials, and sessions at conferences, stand-alone events with broad inter-disciplinary invitation, and engagements with leadership facilities.

54 ENVIRONMENTAL SCIENCES↗

Improving Solar and Solar+Storage Screening Techniques to Reduce Utility Interconnection Time and Costs (Final Technical Report)

Residential PV installations have increased rapidly over the last decade, and the increased application volume has caused permitting delays and lower overall adoption rates. In this project, we developed and evaluated whether data-driven secondary modeling and screening techniques can help utilities assess customer applications more accurately than traditional screening shortcuts. Secondary topologies are predicted using decision trees and commonly available information, such as service transformer, customer, and street locations. Conductors were predicted using a logistic regression method based on real world object (RWO) types, service transformer ratings, conductor length, and distance to transformer. After developing the combined primary and secondary distribution network model, hosting capacity results were used to train a random forest model to predict the pass/fail likelihood of a customer application. Powerflow based models with predicted secondaries and data-driven methods both increased the screening success rate, relative to common utility heuristics, by as much as 55 percentage points. Data-driven screening techniques were described by one utility as a "right-sized" approach for residential customers given the low-risk of small errors and the high-cost of accurate modeling.

14 SOLAR ENERGY↗