Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

AGGREGATE: dAta-driven modelinG preservinG contRollable dEr for outaGe mAnagemenT and rEsiliency (Report for Task 13: Functional Specification Report(Deliverable D9))

This report provides functional specification for the AGGREGATE project and associated modules/ tools being developed for outage management and restoration with the Distributed Energy Resources (DERs). This report also provide initial validation of the developed modules and performance metrics. The AGGREGATE team at this stage has been collaborating with General Electric (GE) and Seattle City Light (SCL) to move forward from offline validation to online validation using commercial advanced distribution management system (ADMS). Hence, the functional specifications documents, which outline the requirement of various modules, is provided to address the integration of the modules, including 1) real-time model update and aggregation tools module, 2) observability and controllability metric module, 3) fault location solution service restoration module, 4) transmission-distribution co-simulation module, and 5) advanced distribution management system module. integration of different AGGREGATE modules and functional specification provides the foundation for online multi-scenario testing using SCL system model.

42 ENGINEERING↗
DOE OSTI2019

Data-based Aggregate Model of Refrigerators for Electric Power Grid Services

Electrical power system stability under high penetration of intermittent renewable energy generation, such as wind and solar energy, has become a high concern. It is unpredictable and hard to control due to little inertial behavior characteristic to these generation resources. To help mitigate possibly undesirable frequency excursions, demand-side frequency control presents an emerging solution. By observing the frequency deviation, devices could switch operating modes for a short time to rebalance generation. To analyze device level control algorithms, aggregate models for their behavior are needed. Refrigerators are household devices that could help with load modulation. This paper uses real measured data to analyze the refrigerator behavior, learn about its operating mode and power consumption ranges. A Markov chain process is proposed to model the transition of a refrigerator population from one operating mode to another, such that the overall load is modulated according to the control input. The Markov-chain model can be used to design control so that the overall load is modulated according to the control input.

Markov chain model, demand response, refrigerator,↗
DOE OSTI2020

AGGREGATE: dAta-driven modelinG preservinG contRollable dEr for outaGe mAnagemenT and rEsiliency (Report for Task 14: Model Verification/Validation Test Plan (Deliverable D10))

This report summarizes the technical progress and validation plan for each of AGGREGATE project modules, and integrated modules. The overall validation plan can be divided into two major parts. Firstly, it provides offline validation results of each module using the IEEE-123 node distribution system with certain modifications to support individual module results verification. The offline validation plan tentatively validated the usability and practicability of developed AGGREGATE tools. Secondly, the realworld test system - Seattle City Light (SCL) test feeder for verifying each module and the integrated modules using GE advanced distribution management system (ADMS) software.

42 ENGINEERING↗
DOE OSTI2018

Specific Features of the Electron Structure of ZnTPP Aggregated Forms: Data of Optical Measurements and Quantum-Chemical Calculations

Zinc tetraphenylporphyrin (ZnTPP) films and ZnTPP-based composites are promising materials of organic photonics. Porphyrins are inclined towards self-assembly and the formation of molecular ensembles or differently structured aggregates. The energies of the formation of aggregates and ordered structures and modifications of their electron structures compared to that of a free ZnTPP molecule have not been adequately explored. In the study, the first comprehensive investigation of the structure, absorption and luminescence spectra, and photoluminescence kinetics of structurally perfect ZnTPP thin films produced under quasi-equilibrium conditions in vacuum is conducted. It is shown that changes in the absorption spectra and the red shift of the luminescence spectra of films by 0.15 eV from the spectra observed for ZnTPP solutions in toluene can be interpreted as a result of the formation of an ordered thin-film structure through the dimerization of porphyrin planar molecules under nearly equilibrium conditions. The optimal geometric structure, the energy of the ground state, and the electronic spectrum of excitations of the dimerized (ZnTPP){sub 2} state are calculated in the context of density functional theory and time-dependent density functional theory. The energy gain on the formation of a dimer of symmetry C{sub i} compared to two separate molecules is 0.23 eV per dimer; the HOMO–LUMO gap for the dimer is decreased by 70 meV. The radiative emission time in the ordered solid phase is an order of magnitude shorter than that in the solution in toluene and corresponds to 277 ps, which is typical of J aggregates of porphyrins.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2022

Bayesian Framework for Multi-Timescale State Estimation in Low-Observable Distribution Systems

To support the smart grid paradigm, there has been a significant increase in sensor deployments and metering infrastructure in distribution systems. However, the measurements provided by these sensors and metering devices are typically sampled at different rates and could suffer from losses during the aggregation process. It is crucial to effectively reconcile the time-series measurements for a reliable state estimation. While weighted least squares has been the traditional approach for state estimation, sparsity-based approaches like matrix completion have become popular due to their superior performance in low-observability conditions. This paper proposes a Bayesian framework for both multi-timescale data aggregation and matrix completion based state estimation. Specifically, the multiscale time-series data aggregated from heterogenous sources are reconciled using a multitask Gaussian process that exploits the spatio-temporal correlations. Here, the resulting consistent timeseries alongwith the confidence bound on the imputations are fed into a Bayesian matrix completion method augmented with linearized power-flow constraints to accurately estimate the states in low-observability conditions. Results on three phase unbalanced IEEE 37 and IEEE 123 bus test systems reveal the superior performance of the proposed Bayesian framework. The computational complexity for the proposed Bayesian framework is also quantified.

42 ENGINEERING↗
DOE OSTI2024

Homomorphic Encryption for Electrical Metering Aggregation: Protecting the Privacy of Building Tenants

Electrical meters are devices that measure consumer electricity usage. The data collected by these meters is necessary for utility billing and electrical grid management but can also be used to assess the environmental impact of buildings. Prior research has found that unprotected metering data could potentially be used to infer some information about the behaviors of building tenants by detecting changes in electricity usage. For example, a period of low electricity usage could suggest that the tenants are not in the building. As smart metering becomes more common, there is a growing need for data privacy protections for metering data that do not negatively impact the quality and availability of data used for energy management and billing applications. To identify potential solutions, we developed a Python-based data aggregation platform to analyze the potential efficacy of privacy-enhancing technologies for energy metering applications. This platform aggregates groups of metering sites into virtual buildings, which could potentially detach changes in electrical activity from individual tenants, making it more difficult to track the activity of a specific tenant. To further protect data during analysis, this project utilizes homomorphic encryption as part of its initial approach. Homomorphic encryption offers a means of protecting energy consumption data while permitting mathematical operations to be performed without the need to know the data contents. This allows for data to be processed into usable statistics without revealing energy consumption information. A series of homomorphic encryption libraries were evaluated to determine their applicability and limitations in the context of metering data. The use of these techniques may help to reassure consumers and encourage further adoption of smart grid infrastructure.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

EXSCLAIM!

Due to recent improvements in image resolution and acquisition speed, materials microscopy is experiencing an explosion of published imaging data. The standard publication format, while sufficient for traditional data ingestion scenarios where a select number of images can be critically examined and curated manually, is not conducive tolarge-scale data aggregation or analysis, hindering data sharing and reuse. Most images in publications are presented as components of a larger figure with their explicit context buried in the main body or caption text, so even if aggregated, collections of images with weak or no digitized contextual labels have limited value. To solve the problem of curating labeled microscopy data from literature, we introduce the EXSCLAIM! Python toolkit for the automatic EXtraction, Separation, and Caption-based natural Language Annotation of IMages from scientific literature. The software is implemented through a three part pipeline: the JournalScraper, which searches the web and downloads figures and captions based on a user provided query, the CaptionDistributor, which separates caption text based on the subfigure each portion of the caption refers to, and the FigueSeparator, which separates figures into component subfigures and extracts other visual information. Also included is a Django user interface for exploring the resulting dataset.

CHAN, MARIA↗
DOE OSTI2021

Report for Task 15: Developed Tool Verification/Validation Report (Deliverable D11): AGGREGATE: dAta-driven modelinG preservinG contRollable dEr for outaGe mAnagemenT and rEsiliency

This report summarizes the verification report for the developed modules. The team conducted verification in two phases. In phase 1, the modules were validated using offline validation methods and in phase 2 the modules were subjected to real-time validation. This report outlines the approach and the results for the modules under offline and real-time settings. The rest of the report details the functional workflow of AGGREGATE modules, we then transition to offline validation followed by real-time validation of modules. We conclude the report by discussing some of the key points of this report.

14 SOLAR ENERGY↗
DOE OSTI2025

Results of an interlaboratory study on the working curve in vat photopolymerization II: Towards a standardized method

The working curve measurement in photopolymer additive manufacturing is a ubiquitous measure of the cure depth of a printing resin as a function of radiant exposure of light. The fit parameters from this measurement (the depth of light penetration D p and the critical exposure E c ) are used to evaluate and report a resin’s printability, optimize processing parameters, and inform print and resin quality control. Despite its widespread use, the working curve lacks a standard measurement method. Here, following up on our paper “Results of an Interlaboratory Study on the Working Curve in Vat Photopolymerization” from last year, an interlaboratory study on the working curve was performed using calibrated, reproducible, bandpass filtered light sources. With these light sources, the variability between labs in measured working curves was dramatically reduced from the initial interlaboratory study. Aggregate data from this experiment produced reliable D p and E c measurements at 385 nm of 39.2 ± 3.7 µm and 12.3 ± 3.0 mJ cm −2 , respectively. At 405 nm the values of D p and E c are 69.3 ± 3.8 µm and 17.9 ± 2.3 mJ cm −2 , respectively. The results are agnostic to the thickness measurement tool utilized by participants, ensuring broad applicability across laboratories. We also tested the generalizability of the proposed method of using a filtered light source by filtering a commercial 405 nm light source and obtaining a working curve in agreement with the aggregate data from the interlaboratory study. This interlaboratory study provides a basis for a documentary standard for the working curve, so that the entire photopolymer additive manufacturing industry can share reproducible and interoperable working curve data.

36 MATERIALS SCIENCE↗
DOE OSTI2015

Indexing amyloid peptide diffraction from serial femtosecond crystallography: new algorithms for sparse patterns

Special methods are required to interpret sparse diffraction patterns collected from peptide crystals at X-ray free-electron lasers. Bragg spots can be indexed from composite-image powder rings, with crystal orientations then deduced from a very limited number of spot positions. Still diffraction patterns from peptide nanocrystals with small unit cells are challenging to index using conventional methods owing to the limited number of spots and the lack of crystal orientation information for individual images. New indexing algorithms have been developed as part of the Computational Crystallography Toolbox (cctbx) to overcome these challenges. Accurate unit-cell information derived from an aggregate data set from thousands of diffraction patterns can be used to determine a crystal orientation matrix for individual images with as few as five reflections. These algorithms are potentially applicable not only to amyloid peptides but also to any set of diffraction patterns with sparse properties, such as low-resolution virus structures or high-throughput screening of still images captured by raster-scanning at synchrotron sources. As a proof of concept for this technique, successful integration of X-ray free-electron laser (XFEL) data to 2.5 Å resolution for the amyloid segment GNNQQNY from the Sup35 yeast prion is presented.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2020

AGGREGATE: dAta-driven modelinG preservinG contRollable dEr for outaGe mAnagemenT and rEsiliency (Report for Task 9: Viability of Data-Driven Approach for Outage Management (Deliverable D6))

This technical report is provided to US Department of Energy for progress made on the AGGERGATE project led by the Washington State University. This report is specifically related to the deliverable: ’D6: Technical report discussing the viability of the developed data-driven models and co-simulation architecture for models for operation in a real-world operational setting and the results of Tasks 1 to 8, summarizing the project findings and accomplishments to date’.

42 ENGINEERING↗
DOE OSTI2020

AGGREGATE: dAta-driven modelinG preservinG contRollable dEr for outaGe mAnagemenT and rEsiliency (Report for Task 10: Controllability and Observability Analysis (Deliverable D7))

It is common to have less than full observability for distribution systems, as the number of available measurements are not sufficient. Considering increasing penetration of distributed energy resources (DERs), it is imperative to understand and control distribution systems better. Developed for the transmission system, the original numerical observability analysis methods were based on the canonical DC Power Flow (DC-PF) model. However, the DC-PF model is ill-suited for the distribution system in which line resistance to reactance (R/X) ratios can be near unity. In this report, we apply the non-iterative Observability Test (OT) and Observable Island Identification Procedure (OIIP) in to the gain matrix from the distribution system state estimator. Because sensors are few in distribution systems, pseudo-measurements based on historical load profiles and zero-injection virtual measurements are often employed to ensure observability. However, both the observability and the accuracy of the state estimates are uncertain when historical data is limited/unavailable.

42 ENGINEERING↗
NASA NTRS2019

Forming Aggregations using Virtual Sharding: Lessons Learned from Simple Scalable Storage (S3)

Data aggregation is the ability to combine separate datasets to form a single new logical dataset provides users with a powerful abstraction. The advantage of an aggregate dataset is that the users are freed from having to understand, and incorporate into their workflow, knowledge about the (ad hoc) organization of the constituent datasets. However, aggregating large numbers of files can be computationally complex with data server systems performing many repetitive operations. As part of the authors work on subsetting data stored on Amazon Web Service (AWS) Simple Storage Service (S3), we developed technology to read portions of otherwise monolithic data files. This enables the formation of virtual shards for user in subsetting data stored in HDF5 (hierarchical data format, version 5) files. This same tool can be used to form aggregations that combine data stored in many HDF5 files when those files are stored on S3. The nature of the virtual sharding and the algorithm that exploits it for subsetting is such that it can also be used for aggregation with the need for many of the repetitive operations required by the per file aggregation techniques. We will present timing information that demonstrates the flexibility of this approach. However, the lessons learned is that while this is a useful result in and of itself, these very same techniques can be applied in other contexts where data are stored in services and on media other than S3. For example, this same technique can be applied to data stored on spinning disk. Pushing the envelope for S3 forced a reexamination of our data access techniques which lead to unexpected positive benefits.

Gallagher, James↗
DOE OSTI2021

A case study on parallel HDF5 dataset concatenation for high energy physics data analysis

In High Energy Physics (HEP), experimentalists generate large volumes of data that, when analyzed, helps us better understand the fundamental particles and their interactions. This data is often captured in many files of small size, creating a data management challenge for scientists. In order to better facilitate data management, transfer, and analysis on large scale platforms, it is advantageous to aggregate data further into a smaller number of larger files. However, this translation process can consume significant time and resources, and if performed incorrectly the resulting aggregated files can be inefficient for highly parallel access during analysis on large scale platforms. In this paper, we present our case study on parallel I/O strategies and HDF5 features for reducing data aggregation time, making effective use of compression, and ensuring efficient access to the resulting data during analysis at scale. We focus on NOvA detector data in this case study, a large-scale HEP experiment generating many terabytes of data. Here, the lessons learned from our case study inform the handling of similar datasets, thus expanding community knowledge related to this common data management task.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Data Fusion to Enhance Quality Control and Analysis with Instruments at the Marine and Coastal Research Laboratory

Deploying environmental monitoring instruments in the marine environment can be challenging, facing challenges around device survivability, biofouling and corrosion, and consistent data collection. This project explores the use of data fusion – the process of integrating multiple data sources to produce more consistent, accurate, and useful information – to build a consistent long-term monitoring system at the Marine and Coastal Research Laboratory (MCRL) in Sequim, Washington. Unused instruments that had been acquired from past projects were inventoried and deployments planned on the MCRL pier and floating dock. A total of 8 instruments were deployed including a tide gauge, hydrophone, acoustic Doppler current profiler (ADCP), photosynthetically active radiation (PAR) sensors, meteorological station, and three water quality sensors. Deployments were planned to be well-protected around the pier structure and a maintenance schedule was created for cleaning and recalibration. An automated data pipeline was created to aggregate data on edge computers that push data to Amazon Web Services (AWS) cloud storage every 15 minutes, performing automated quality control and data transformations using the Time Series Data Analytical Toolkit (TSDAT). Continued efforts are underway to maintain this system into the future, take a data-driven approach to maintenance scheduling, improve the reliability of the system, and share the data with a variety of end-users.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Vegetation Warming Experiment: Environmental Conditions, Utqiagvik (Barrow), Alaska, 2018

Environmental conditions measured in five warming chambers and paired ambient control plots located on the Barrow Environmental Observatory (BEO), Utqiagvik, Alaska from 16 June - 24 September, 2018. These data were recorded in support of the Zero Power Warming (ZPW) vegetation warming experiment, a series of single season vegetation warming treatments conducted over four years from 2017-2021 (no experiment in 2020). Air temperature and humidity, infrared surface (canopy) temperature, soil temperature, soil moisture, NDVI (normalized difference vegetation index), PRI (photochemical reflectance index), solar radiation and chamber venting were recorded in each chamber at 1 minute intervals. Ambient air temperature, humidity, solar radiation and uplooking PRI and NDVI were measured at a centrally located meteorology station. Vapor pressure deficit (VPD) was calculated and included in the final processed data products. Data has undergone full QA/QC and is presented as 1 minute data, and hourly and daily aggregate data products. This data package includes unprocessed raw data (*.dat files), processed data (*.csv) and metadata including a full description of sensors, calculations and processing (*.csv, *.pdf). See related NGEE-Arctic "Vegetation Warming Experiment" data packages for leaf-level gas exchange and other leaf trait data; chamber, plot and landscape phenocamera images; thaw depth, and GPS locations of chambers and ambient plots.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗