Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Monthly Quality-filtered Aggregation of NOAA Climate Data Record (CDR) of AVHRR (Version 5) and VIIRS (Version 1) Leaf Area Index (LAI) and Fraction of Absorbed Photosynthetically Active Radiation (FAPAR)

This dataset contains gridded monthly Leaf Area Index (LAI) derived from the daily NOAA Climate Data Record (CDR) of AVHRR (Version 5) and VIIRS (Version 1) Leaf Area Index (LAI) and Fraction of Absorbed Photosynthetically Active Radiation (FAPAR). This data record spans from 1981 to 2024 using data from NOAA polar orbiting satellites: NOAA-7, -9, -11, -14, -16, -17, -18, -19 and S-NPP. The data are projected on a 0.05 degree x 0.05 degree global grid, as in the original CDR. The original CDR is one of the Land Surface CDR products produced by the NASA Goddard Space Flight Center (GSFC) and the University of Maryland (UMD), which is accompanied by algorithm documentation, data flow diagram and source code for the NOAA CDR Program. This dataset is in the netCDF-4 file format following ACDD and CF Conventions. This dataset has applied quality assurance information to only include "OK" data from the original CDR in the monthly aggregation.

Vermote, Eric [NASA Goddard Space Flight Center (G↗

Accelerating shared file checkpoint with local burst buffers

A data management system and method for accelerating shared file checkpointing. Written application data is aggregated in an application data file created in a local burst buffer memory at a compute node, and an associated data mapping built index to maintain information related to the offsets into a shared file at which segments of the application data is to be stored in a parallel file system, and where in the buffer those segments are located. The node asynchronously transfers a data file containing the application data and the associated data mapping index to a file server for shared file storage. The data management system and method further accelerates shared file checkpointing in which a shared file, together with a map file that specifies how the shared file is to be distributed, is asynchronously transferred to local burst buffer memories at the nodes to accelerate reading of the shared file.

Gooding, Thomas↗

Commercial and Multifamily Building Tenant Energy Usage Aggregation and Privacy

In a number of cities and states, building owners are required to disclose and/or benchmark their building energy use. This requires the building owner to possess monthly whole-building energy usage information, which can be challenging for buildings in which individual tenants have their own utility meters and accounts with the utility. Some utilities and utility regulators have turned to aggregation of customer data as a way to give building owners the whole-building energy usage data while protecting customer privacy. However, no utilities or regulators appear to have conducted a concerted statistical, cybersecurity, and privacy analysis to justify the level of aggregation selected. Therefore, the Tennant Data Aggregation Task was established to help utilities address these issues and provide recommendations as well as a theoretical justification of the aggregation threshold. This study is focused on the use case of submitting data for ENERGY STAR Portfolio Manager (ESPM), but it also looks at other potential use cases for monthly energy consumption data.

data aggergation↗

PV System Availability from Commercial and Utility-Scale Systems [Slides]

In the PV Fleet Performance Data Initiative, we partner with photovoltaic (PV) fleet owners to collect time-series PV production data, and publish aggregated, anonymized results. An assessment of system availability is conducted on 1128 systems which passed our data quality checks, and include cumulative energy meter data. Overall inverter availability is low in the first 6 months of system performance before reaching steady-state by the end of the first year. System-level aggregated data shows a median (P50) system availability of 0.99, and a lower P90 value of 0.95. A dependence on system size is also identified, with worse inverter availability results for larger PV systems. Potential causes of this effect are under investigation.

14 SOLAR ENERGY↗

PV Inverter Availability from the U.S. PV Fleet

In the PV Fleet Performance Data Initiative, we partner with photovoltaic (PV) fleet owners to collect time-series PV production data, and publish aggregated, anonymized results. An assessment of system availability is conducted on 1128 systems which passed our data quality checks, and include cumulative energy meter data. Overall inverter availability is low in the first 6 months of system performance before reaching steady-state by the end of the first year. System-level aggregated data shows a median (P50) system availability of 0.99, and a lower P90 value of 0.95. A dependence on system size is also identified, with better inverter availability results for smaller PV systems. Potential causes of this effect may include the selection of inverter itself: smaller inverters 6kW-250kW showed better average availability than inverters 300kW-5MW. The elimination of string combiner boxes and lower energy impact when one particular inverter goes off-line are potential benefits of a string inverter-based PV system architecture. DNV also analyzed availability data from over 1100 operating systems and found similar trends. DNV's P50 industry guidance on expected availability has been updated to reflect the data and the following observations: utility scale systems have lower availability than DG systems, availability is lower in first year compared to subsequent years, and that actual availability is lower than expected.

fleet↗

Carbon Storage Site Mapping Inquiry Tool (MapIT)

To date, 48 projects, consisting of 139 wells, are currently under review with the Environmental Protection Agency’s (EPA) Underground Injection Control (UIC) Program for Class VI – wells used for geologic sequestration of carbon dioxide. The number of applications submitted is expected to increase in coming years with the increase of the 45Q tax credit available to projects that initiate construction prior to 2033. The amount of data collected to submit a Class VI permit is vast, and often disparate, coming from state, federal, and commercial entities, as well as field-specific data collected within an area of interest. When preparing for site selection and permitting, the initial aggregation of relevant public data can be time intensive. The Carbon Storage Site Mapping Inquiry tool (MapIT) was created to support and accelerate the discovery and accessibility of open-source data and information available across the USA. Data was aggregated and organized based on data types described within the EPA UIC Class VI permit documentation. The online tool enables users to explore hundreds of geospatial data layers and connect to additional external resources, leveraging API and REST services where possible to ensure updates to data in real time. MapIT enables users to explore state and federal data related to geologic, geophysical, structural, hydrologic, and contextual information. In addition to displaying spatial data and linking to external resources, MapIT leverages custom widgets to ensure that internal data and external data are discoverable and accessible. The widgets connect users to resources such as the USGS publications and the USGS Earthquake Catalog based on a user-defined location. This talk will describe data aggregation workflows, data types, data preparation, and tool development for MapIT. The Carbon Storage Site Mapping Inquiry Tool and underlying database are valuable, intuitive resources that empower government, academic, commercial and industry stakeholders to explore, analyze, and acquire carbon storage related data.

Morkner, Paige↗

A Generalized approach to the operationalization of Software Quality Models

Comprehensive measures of quality are a research imperative, yet the development of software quality models is a wicked problem. Definitive solutions do not exist and quality is subjective at its most abstract. Definitional measures of quality are contingent on a domain, and even within a domain, the choice of representative characteristics to decompose quality is subjective. Thus, the operationalization of quality models brings even more challenges. A promising approach to quality modeling is the use of hierarchies to represent characteristics, where lower levels of the hierarchy represent concepts closer to real-world observations. Building upon prior hierarchical modeling approaches, we developed the Platform for Investigative software Quality Understanding and Evaluation (PIQUE). PIQUE surmounts several quality modeling challenges because it allows modelers to instantiate abstract hierarchical models in any domain by leveraging organizational tools tailored to their specific contexts. Here, we introduce PIQUE; exemplify its utility with two practical use cases; address challenges associated with parameterizing a PIQUE model; and describe algorithmic techniques that tackle normalization, aggregation, and interpolation of measurements.

Data aggregation↗

Storages Are Not Forever

Not unlike the concern over diminishing fossil fuel, information technology is bringing its own share of future worries. Here, we chose to look closely into one concern in this paper, namely the limited amount of data storage. By a simple extrapolatory analysis, it is shown that we are on the way to exhaust our storage capacity in less than two centuries with current technology and no recycling. This can be taken as a note of caution to expand research initiative in several directions: firstly, bringing forth innovative data analysis techniques to represent, learn, and aggregate useful knowledge while filtering out noise from data; secondly, tap onto the interplay between storage and computing to minimize storage allocation; thirdly, explore ingenious solutions to expand storage capacity. Throughout this paper, we delve deeper into the state-of-the-art research and also put forth novel propositions in all of the abovementioned directions, including space- and time-efficient data representation, intelligent data aggregation, in-memory computing, extra-terrestrial storage, and data curation. The main aim of this paper is to raise awareness on the storage limitation we are about to face if current technology is adopted and the storage utilization growth rate persists. In the manuscript, we propose some storage solutions and a better utilization of storage capacity through a global DIKW hierarchy.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

SU-E-T-186: Cloud-Based Quality Assurance Application for Linear Accelerator Commissioning

Purpose: To identify anomalies and safety issues during data collection and modeling for treatment planning systems Methods: A cloud-based quality assurance system (AQUIRE - Automated QUalIty REassurance) has been developed to allow the uploading and analysis of beam data aquired during the treatment planning system commissioning process. In addition to comparing and aggregating measured data, tools have also been developed to extract dose from the treatment planning system for end-to-end testing. A gamma index is perfomed on the data to give a dose difference and distance-to-agreement for validation that a beam model is generating plans consistent with the beam data collection. Results: Over 20 linear accelerators have been commissioning using this platform, and a variety of errors and potential saftey issues have been caught through the validation process. For example, the gamma index of 2% dose, 2mm DTA is quite sufficient to see curves not corrected for effective point of measurement. Also, data imported into the database is analyzed against an aggregate of similar linear accelerators to show data points that are outliers. The resulting curves in the database exhibit a very small standard deviation and imply that a preconfigured beam model based on aggregated linear accelerators will be sufficient in most cases. Conclusion: With the use of this new platform for beam data commissioning, errors in beam data collection and treatment planning system modeling are greatly reduced. With the reduction in errors during acquisition, the resulting beam models are quite similar, suggesting that a common beam model may be possible in the future. Development is ongoing to create routine quality assurance tools to compare back to the beam data acquired during commissioning. I am a medical physicist for Alzyen Medical Physics, and perform commissioning services.

60 APPLIED LIFE SCIENCES↗

Medium Duty ARRA Data Reporting and Analysis (Presentation)

This project compiles medium-duty (MD) aggregated deployment data and provides the compiled detailed analyses to industry. The U.S. Department of Energy's (DOE's) American Recovery and Reinvestment Act (ARRA) deployment and demonstration projects are helping to commercialize technologies for all-electric vehicles, electrified accessories, and electric charging infrastructure. Over 3.2 million miles of in-service all-electric MD truck data from 560 different vehicles have been collected since 2011, and usage data from over 1,000 truck electrification sites have been collected since 2013. Through the DOE's Vehicle Technologies Office, NREL is working to analyze real-time data from these deployment and demonstration projects to quantify the benefits: results and summary statistics are made available through the NREL website as quarterly and annual reports; 23 aggregated reports have been published on the performance and operation of these vehicles; and detailed data are being extracted to help further understand battery use and performance.

2014 AMR↗

TU-C-BRE-03: Aggregation of Linac Measurement Data

Purpose: Accurate data of linear accelerator radiation characteristics is important for treatment planning system commissioning as well as regular quality assurance of the machine. The RPC has performed site visits of numerous machines . Data gathered from Varian machines from the past 15 years are presented. The data collected can be used as a secondary check or when commissioning a new machine to verify that values are reasonable. Methods: Data from the past 15 years of RPC site visits was compiled and analyzed. Data was composed from measurements from approximately 400 Varian machines. Each dataset consists of several point measurements at various locations in a water phantom to measure percentage depth dose, output factors, including small MLC fields, off-axis factors, and wedge factors if applicable. Common statistical values are presented for each machine type. Where applicable, data was compared to other reference data given by the vendor or a select number of previous researchers. Results: Data is separated by energy and parameter and then analyzed by machine class. Data distributions of the parameter data were normal except occasionally at the tails. Distributions of the data for each class and parameter are tabulated to give not simply a singular reference value, but metrics about the distribution: 5th and 95th percentile values and the standard deviation as well as the median. Conclusion: The RPC has collected numerous data on Varian linacs and presented the finding of the past 15 years. The data can be used as a reference data set for physicists to compare against. A linac that deviates from the values does not necessarily indicate there is a problem as long as the treatment planning system correlates to the machine. Comparison of linac and treatment planning system data to external reference data can prevent serious treatment errors.

07 ISOTOPE AND RADIATION SOURCES↗

Oak Ridge National Laboratory Pilot Demonstration of an Attestation and Anomaly Detection Framework using Distributed Ledger Technology for Power Grid Infrastructure

This report summarizes the design and pilot demonstration of a framework called Grid Guard that was created to provide increased data and device trustworthiness to electric grid devices by leveraging distributed ledger technology (DLT), specifically blockchain. Grid Guard contains a combination of core cryptographic methods such as the secure hash algorithm (SHA), and asymmetric cryptography, private permissioned blockchain, baselining configuration data, consensus algorithm (Raft) and the Hyperledger Fabric (HLF) framework. The system implements a low energy, fast, and robust enhancement to system trustworthiness within and across electric grid systems such as substations, control centers and metering infrastructures. Blockchain is a distributed database structured that provides a practically unalterable (immutable) timeline of stored transactions. By relying on hashing and the Raft consensus algorithm, if an entity tries to illegitimately alter a record at one instance of the database the other ledger nodes are not altered. They work to cross-reference each other and easily locate any incorrectly added data and remove it. The bulk raw data is stored in an off-chain storage (outside of the blockchain ledger) and a hash of this baseline data is stored in the Blockchain ledger via hashing windows of time-series and configuration data, after aggregation and filtering. The bulk off-chain data repository is then considered to be trust-anchored using the hashes stored in the blockchain. To secure the electric grid testbed devices and data, device configuration baselines were compared to those baselines that had been previously stored in the ledger. Statistical baselines for device configurations, network communication patterns, and high-speed sensor data are calculated and then stored off-chain and hashes stored in the ledger. Measurements such as three-phase voltage and current, frequency, breaker status, protection scheme settings, network configuration settings (and other device configuration artifacts) and network traffic features (packet interarrival times) are compared every minute or other selected time windows. During phase 1 of the Grid Guard DLT project different DLT technologies were studies, and an assessment was performed on DLT technology vulnerabilities, uses, and key characteristics. DLT consensus protocols were studies (e.g., RAFT, named after Reliable, Replicated, Redundant, And Fault-Tolerant). Also, cryptography, public, private and permissioned or permissionless systems were assessed. Grid Guard implements a permissioned private DLT. Consensus algorithm selection and choice of DLT implementation depended heavily on the use-case. For this use-case, parameters were selected to measure performance and existing tools for assessment. Benchmarking was performed theoretically and practically. During phase 2 hashed transactions/blocks were inserted into the ledger every second. During phase 2 of the Grid Guard DLT project, a prototype framework was developed and demonstrated for attestation of critical substation devices and data using precision timing systems that use PTP and IRIG-B protocols) on a testbed of operational devices that emulated a distribution substation, control center, and power metering infrastructure using real Operational Technology (OT). The testbed includes OT devices such as protective relays, human machine interfaces (HMI), and power meters. To determine when to collect and compare system and network baselines, an initial examination of an anomaly detection capability to identify malicious manipulation of data streams was conducted. The resulting anomaly detection was demonstrated in a set of experiments and leveraged to trigger device artifact attestation checks. Attestation checks occur against device configuration baselines when compared with the immutable blockchain-stored baselines, which provided a cryptographically supported means by which to store baselines. The electrical substation-grid testbed was created to test the Grid Guard framework. The testbed emulates the operations of a portion of a power grid and SCADA systems as closely as possible. The testbed integrates real protocols, mainly IEC 61850 standard protocols, such as the Sampled Value (SV) and the GOOSE protocols. The testbed also supports DNP3 and other layer 2 and layer 3 protocols such as Telnet, SSH, SFTP/FTP and other proprietary protocols needed to connect to industrial control system equipment. The testbed emulates real power conditions using the OpalRT hardware-in-the-loop (HIL) device which can create fault situations that cannot be easily tested on real systems. The electrical substation-grid testbed was created using real measurement, communication, and protection devices that electrical utilities commonly use.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Availability and Performance Loss Factors for U.S. PV Fleet Systems

In the PV Fleet Performance Data Initiative, we partner with photovoltaic (PV) fleet owners to collect time-series PV production data and publish aggregated, anonymized results. This report is an update of our previous publications, specifically a FY 2021 performance index publication and a FY 2022 fleet degradation analysis. In this analysis, we have increased our data participants and system totals by around 10% to 8.5 GW and 24,000 separate inverter data channels. Four major analysis topics are considered in this report: Performance Index (PI) trends, PV system availability, soiling losses, and PV system degradation. Performance Index and inverter availability are assessed on a larger set of data from our FY 2021 report: 1,128 systems compared with 200 systems from before. The increased number of systems is due to an improved data quality methodology, as well as introducing new systems to the analysis. Overall results are similar to previously published values - overall inverter availability is low in the first six months of system performance before reaching steady-state by the end of the first year. Excluding this six-month startup period, system-level aggregated data shows a median (P50) system availability of 0.99 and a lower 10th percentile (P90) value of 0.95 (Figure ES-1). A dependence on system size is also demonstrated, with worse inverter availability results for larger PV systems. Causes of this effect are under investigation, but may be impacted by inverter size, which also show lower availability for larger inverter sizes. This report also investigates PI, correcting for degradation, soiling, snow, and availability. Following these corrections, the median system PI over its entire lifetime is 0.95. PI values reported here are approximately 3% lower than what we presented in our previous FY 2021 report. Soiling loss is assessed in a comprehensive way for the first time in this report. Results are presented using the COmbined Degradation and Soiling (CODS) method, as implemented in RdTools (v3.0.0a4). Soiling values are presented for 255 systems, which indicated irradiance-weighted soiling loss greater than 1%. The values have been published in an updated NREL soiling map at nrel.gov/pv/soiling.html. Finally, we investigated system degradation using three different data analysis techniques: conventional RdTools (year-on-year (YOY)), CODS, and Performance Loss Rate (PLR) analysis. Overall degradation results are consistent with our previous publications. Rerunning conventional RdTools on our updated fleet shows that some data partners have systematically fallen below the median system degradation rate (change over time) of -0.75 %/year. A comparison with PLR analysis, which looks at change in annual PI over time, shows that median system degradation is consistent with -0.5% to -0.75% per year change. However, at the P90 value, system degradation is substantially faster. These two results are consistent and indicate that resulting degradation statistics depend to a great degree on the population of PV systems making up the analysis cohort and whether soiling impacts the systems. The use of CODS for degradation analysis provides a different method for degradation assessment, which explicitly excludes the impact of recoverable soiling on degradation analysis. Excluding soiling effects yields an annual system degradation around -0.5% per year on average. This indicates that a portion of system performance loss may be attributed to periodic soiling that is not fully recovered. This report provides PV system owners/operators with background and methods to analyze PV system performance, give guidance for expected cohort performance, and performance loss values for use in pro-forma financial models, which guide new-build system design and bankability reports.

14 SOLAR ENERGY↗

Uranium particle age dating, aggregation, and model age best estimators

We present important aspects of uranium particle age dating by Large-Geometry Secondary Ion Mass Spectrometry (LG-SIMS) that can introduce bias and increase model age uncertainties, especially for small, young, and/or low-enriched particles. This metrology is important for applications related to International Nuclear Safeguards. We explore influential factors related to model age estimation, including the effects of evolving surface chemistry on inter-element measurements of particles (e.g., Th and U), detector background, and aggregation methods using simulated and actual particle samples. We introduce a new model age estimator, called “mid68”, that supplements 95% confidence intervals, providing a “best estimate” and uncertainty about the most likely age. The mid68 estimator can be calculated using the Feldman and Cousins method or Bayesian methods and provides a value with a symmetric uncertainty that can be used for calculations and approximate aggregation of processed model age values when the raw data and correction factors are not available. For particles yielding low 230 Th counts amidst nonzero detector background, their underlying model age probability distributions are asymmetric, so the mid68 estimator provides additional robust information regarding the underlying model age likelihood. This study provides a comprehensive and timely examination of critical aspects of uranium particle age dating as more laboratories establish particle chronometry capabilities.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

BASIN-3D Data Integration for Selected ARM Data Field Campaign Report

The purpose of this data services request was to demonstrate integration of the Atmospheric Radiation Measurement (ARM) User Facility’s “met” datastreams with time series data from other earth science data sources using the BASIN-3D data synthesis software tool. BASIN-3D is an open-source Python library that enables researchers to integrate data across configured public and private data sources. It provides a common query language for researchers to request measurement locations and time series data based on specified locations, variables, time period, statistics, aggregation, and data quality. BASIN-3D acquires the data that match the query from each configured data source and translates the results into harmonized vocabularies, thus reducing researchers' data-wrangling effort. In addition, because the queries are executed on demand, researchers can easily regenerate their synthesized data sets as new data and/or data updates become available, eliminating one-off data products. BASIN-3D can output data using a variety of different data structures for end-user applications including Python pandas data frames and hdf5 output formats.

54 ENVIRONMENTAL SCIENCES↗

Kinetic regime of aggregation of UV-irradiated glyceraldehyde-3-phosphate dehydrogenase from rabbit skeletal muscle

Highlights: • Kinetic regime of aggregation of UV-irradiated glyceraldehyde-3-phosphate dehydrogenase at 37°C has been established. • The rate of stage of aggregate growth is limited by monomolecular structural reorganization of the irradiated protein. • 2-Hydroxypropyl-b-cyclodextrin decelerates stage of structural reorganization of irradiated enzyme and nucleation stage. The study of the kinetics of aggregation of UV-irradiated proteins has attracted considerable interest, since test systems based on aggregation of proteins denatured by UV radiation can be used for screening of the natural and artificial agents possessing chaperone-like activity (anti-aggregation activity). To provide the proper interpretation of the effects caused by the agents under study, the kinetic mechanism of the aggregation process should be established. In the present work the kinetic data on aggregation of UV-irradiated glyceraldehyde-3-phosphate dehydrogenase (GAPDH) from rabbit skeletal muscle at 37 °C (O.I. Maloletkina et al. Biophys. Chem. 163–164 (2012) 11–20) have been analyzed. It has been shown that the stage of aggregate growth follows the first-order kinetics and the experimentally measured rate constant of the first order corresponds to heat-induced structural reorganization of UV-irradiated GAPDH containing concealed damage.

60 APPLIED LIFE SCIENCES↗

Offshore Geologic Carbon Storage Data Collection and International Project Inventory

We present an interactive data collection to aggregate, understand, and disseminate the data that are publicly available to support offshore GCS which can be leveraged by stakeholders to understand where GCS may be viable offshore, create GCS project analogs, and address challenges to GCS in offshore environments. The Offshore Geologic Carbon Storage Data Collection is an Experience Builder web application of multiple web mapping applications, aggregated into a single tool for each data type for access, visualization, and exploration. We also present a spatial inventory of global offshore GCS efforts to visualize the scale and locations of actualized and potential offshore GCS. It includes project location, project type and stage, CO2 storage resource potential, injection rate, reservoir and seal geology, and key literature references. Quantitative and qualitative comparisons of the distribution and magnitude of projects by their attributes lends spatial insight into the status of global GCS operations and storage resource potential, thereby enabling comparative assessments and cross-cutting knowledge transfer for projects in development. These datasets illuminate trends in ongoing offshore projects and can be leveraged by stakeholders to estimate storage resources, identify subsurface analogs, review regulations, and address challenges to offshore GCS. Additionally, opportunities for concurrent decarbonization strategies can be identified.

Mulhern, Julia↗