SEARCH · Engineering Papers

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

All sources NASA NTRS DOE OSTI

DOE OSTI2024

A Generalized approach to the operationalization of Software Quality Models

Comprehensive measures of quality are a research imperative, yet the development of software quality models is a wicked problem. Definitive solutions do not exist and quality is subjective at its most abstract. Definitional measures of quality are contingent on a domain, and even within a domain, the choice of representative characteristics to decompose quality is subjective. Thus, the operationalization of quality models brings even more challenges. A promising approach to quality modeling is the use of hierarchies to represent characteristics, where lower levels of the hierarchy represent concepts closer to real-world observations. Building upon prior hierarchical modeling approaches, we developed the Platform for Investigative software Quality Understanding and Evaluation (PIQUE). PIQUE surmounts several quality modeling challenges because it allows modelers to instantiate abstract hierarchical models in any domain by leveraging organizational tools tailored to their specific contexts. Here, we introduce PIQUE; exemplify its utility with two practical use cases; address challenges associated with parameterizing a PIQUE model; and describe algorithmic techniques that tackle normalization, aggregation, and interpolation of measurements.

Data aggregation↗

DOE OSTI2017

Storages Are Not Forever

Not unlike the concern over diminishing fossil fuel, information technology is bringing its own share of future worries. Here, we chose to look closely into one concern in this paper, namely the limited amount of data storage. By a simple extrapolatory analysis, it is shown that we are on the way to exhaust our storage capacity in less than two centuries with current technology and no recycling. This can be taken as a note of caution to expand research initiative in several directions: firstly, bringing forth innovative data analysis techniques to represent, learn, and aggregate useful knowledge while filtering out noise from data; secondly, tap onto the interplay between storage and computing to minimize storage allocation; thirdly, explore ingenious solutions to expand storage capacity. Throughout this paper, we delve deeper into the state-of-the-art research and also put forth novel propositions in all of the abovementioned directions, including space- and time-efficient data representation, intelligent data aggregation, in-memory computing, extra-terrestrial storage, and data curation. The main aim of this paper is to raise awareness on the storage limitation we are about to face if current technology is adopted and the storage utilization growth rate persists. In the manuscript, we propose some storage solutions and a better utilization of storage capacity through a global DIKW hierarchy.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2015

SU-E-T-186: Cloud-Based Quality Assurance Application for Linear Accelerator Commissioning

Purpose: To identify anomalies and safety issues during data collection and modeling for treatment planning systems Methods: A cloud-based quality assurance system (AQUIRE - Automated QUalIty REassurance) has been developed to allow the uploading and analysis of beam data aquired during the treatment planning system commissioning process. In addition to comparing and aggregating measured data, tools have also been developed to extract dose from the treatment planning system for end-to-end testing. A gamma index is perfomed on the data to give a dose difference and distance-to-agreement for validation that a beam model is generating plans consistent with the beam data collection. Results: Over 20 linear accelerators have been commissioning using this platform, and a variety of errors and potential saftey issues have been caught through the validation process. For example, the gamma index of 2% dose, 2mm DTA is quite sufficient to see curves not corrected for effective point of measurement. Also, data imported into the database is analyzed against an aggregate of similar linear accelerators to show data points that are outliers. The resulting curves in the database exhibit a very small standard deviation and imply that a preconfigured beam model based on aggregated linear accelerators will be sufficient in most cases. Conclusion: With the use of this new platform for beam data commissioning, errors in beam data collection and treatment planning system modeling are greatly reduced. With the reduction in errors during acquisition, the resulting beam models are quite similar, suggesting that a common beam model may be possible in the future. Development is ongoing to create routine quality assurance tools to compare back to the beam data acquired during commissioning. I am a medical physicist for Alzyen Medical Physics, and perform commissioning services.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2014

Medium Duty ARRA Data Reporting and Analysis (Presentation)

This project compiles medium-duty (MD) aggregated deployment data and provides the compiled detailed analyses to industry. The U.S. Department of Energy's (DOE's) American Recovery and Reinvestment Act (ARRA) deployment and demonstration projects are helping to commercialize technologies for all-electric vehicles, electrified accessories, and electric charging infrastructure. Over 3.2 million miles of in-service all-electric MD truck data from 560 different vehicles have been collected since 2011, and usage data from over 1,000 truck electrification sites have been collected since 2013. Through the DOE's Vehicle Technologies Office, NREL is working to analyze real-time data from these deployment and demonstration projects to quantify the benefits: results and summary statistics are made available through the NREL website as quarterly and annual reports; 23 aggregated reports have been published on the performance and operation of these vehicles; and detailed data are being extracted to help further understand battery use and performance.

2014 AMR↗

DOE OSTI2014

TU-C-BRE-03: Aggregation of Linac Measurement Data

Purpose: Accurate data of linear accelerator radiation characteristics is important for treatment planning system commissioning as well as regular quality assurance of the machine. The RPC has performed site visits of numerous machines . Data gathered from Varian machines from the past 15 years are presented. The data collected can be used as a secondary check or when commissioning a new machine to verify that values are reasonable. Methods: Data from the past 15 years of RPC site visits was compiled and analyzed. Data was composed from measurements from approximately 400 Varian machines. Each dataset consists of several point measurements at various locations in a water phantom to measure percentage depth dose, output factors, including small MLC fields, off-axis factors, and wedge factors if applicable. Common statistical values are presented for each machine type. Where applicable, data was compared to other reference data given by the vendor or a select number of previous researchers. Results: Data is separated by energy and parameter and then analyzed by machine class. Data distributions of the parameter data were normal except occasionally at the tails. Distributions of the data for each class and parameter are tabulated to give not simply a singular reference value, but metrics about the distribution: 5th and 95th percentile values and the standard deviation as well as the median. Conclusion: The RPC has collected numerous data on Varian linacs and presented the finding of the past 15 years. The data can be used as a reference data set for physicists to compare against. A linac that deviates from the values does not necessarily indicate there is a problem as long as the treatment planning system correlates to the machine. Comparison of linac and treatment planning system data to external reference data can prevent serious treatment errors.

07 ISOTOPE AND RADIATION SOURCES↗

DOE OSTI2022

Oak Ridge National Laboratory Pilot Demonstration of an Attestation and Anomaly Detection Framework using Distributed Ledger Technology for Power Grid Infrastructure

This report summarizes the design and pilot demonstration of a framework called Grid Guard that was created to provide increased data and device trustworthiness to electric grid devices by leveraging distributed ledger technology (DLT), specifically blockchain. Grid Guard contains a combination of core cryptographic methods such as the secure hash algorithm (SHA), and asymmetric cryptography, private permissioned blockchain, baselining configuration data, consensus algorithm (Raft) and the Hyperledger Fabric (HLF) framework. The system implements a low energy, fast, and robust enhancement to system trustworthiness within and across electric grid systems such as substations, control centers and metering infrastructures. Blockchain is a distributed database structured that provides a practically unalterable (immutable) timeline of stored transactions. By relying on hashing and the Raft consensus algorithm, if an entity tries to illegitimately alter a record at one instance of the database the other ledger nodes are not altered. They work to cross-reference each other and easily locate any incorrectly added data and remove it. The bulk raw data is stored in an off-chain storage (outside of the blockchain ledger) and a hash of this baseline data is stored in the Blockchain ledger via hashing windows of time-series and configuration data, after aggregation and filtering. The bulk off-chain data repository is then considered to be trust-anchored using the hashes stored in the blockchain. To secure the electric grid testbed devices and data, device configuration baselines were compared to those baselines that had been previously stored in the ledger. Statistical baselines for device configurations, network communication patterns, and high-speed sensor data are calculated and then stored off-chain and hashes stored in the ledger. Measurements such as three-phase voltage and current, frequency, breaker status, protection scheme settings, network configuration settings (and other device configuration artifacts) and network traffic features (packet interarrival times) are compared every minute or other selected time windows. During phase 1 of the Grid Guard DLT project different DLT technologies were studies, and an assessment was performed on DLT technology vulnerabilities, uses, and key characteristics. DLT consensus protocols were studies (e.g., RAFT, named after Reliable, Replicated, Redundant, And Fault-Tolerant). Also, cryptography, public, private and permissioned or permissionless systems were assessed. Grid Guard implements a permissioned private DLT. Consensus algorithm selection and choice of DLT implementation depended heavily on the use-case. For this use-case, parameters were selected to measure performance and existing tools for assessment. Benchmarking was performed theoretically and practically. During phase 2 hashed transactions/blocks were inserted into the ledger every second. During phase 2 of the Grid Guard DLT project, a prototype framework was developed and demonstrated for attestation of critical substation devices and data using precision timing systems that use PTP and IRIG-B protocols) on a testbed of operational devices that emulated a distribution substation, control center, and power metering infrastructure using real Operational Technology (OT). The testbed includes OT devices such as protective relays, human machine interfaces (HMI), and power meters. To determine when to collect and compare system and network baselines, an initial examination of an anomaly detection capability to identify malicious manipulation of data streams was conducted. The resulting anomaly detection was demonstrated in a set of experiments and leveraged to trigger device artifact attestation checks. Attestation checks occur against device configuration baselines when compared with the immutable blockchain-stored baselines, which provided a cryptographically supported means by which to store baselines. The electrical substation-grid testbed was created to test the Grid Guard framework. The testbed emulates the operations of a portion of a power grid and SCADA systems as closely as possible. The testbed integrates real protocols, mainly IEC 61850 standard protocols, such as the Sampled Value (SV) and the GOOSE protocols. The testbed also supports DNP3 and other layer 2 and layer 3 protocols such as Telnet, SSH, SFTP/FTP and other proprietary protocols needed to connect to industrial control system equipment. The testbed emulates real power conditions using the OpalRT hardware-in-the-loop (HIL) device which can create fault situations that cannot be easily tested on real systems. The electrical substation-grid testbed was created using real measurement, communication, and protection devices that electrical utilities commonly use.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS1994

Regional rainfall climatologies derived from Special Sensor Microwave Imager (SSM/I) data

Climatologies of convective precipitation were derived from passive microwave observations from the Special Sensor Microwave Imager using a scattering-based algorithm of Adler et al. Data were aggregated over periods of 3-5 months using data from 4 to 5 years. Data were also stratified by satellite overpass times (primarily 06 00 and 18 00 local time). Four regions (Mexico, Amazonia, western Africa, and the western equatorial Pacific Ocean (TOGA COARE area) were chosen for their meteorological interest and relative paucity of conventional observations. The strong diurnal variation over Mexico and the southern United States was the most striking aspect of the climatologies. Pronounced morning maxima occured offshore, often in concativities in the coastline, the result of the increased convergence caused by the coastline shape. The major feature of the evening rain field was a linear-shaped maximum along the western slope of the Sierra Madre Occidental. Topography exerted a strong control on the rainfall in other areas, particularly near the Nicaragua/Honduras border and in Guatemala, where maxima in excess of 700 mm/month were located adjacent to local maxima in terrain. The correlation between the estimates and monthly gage data over the southern United States was low (0.45), due mainly to poor temporal sampling in any month and an inadequate sampling of the diurnal cycle. Over the Amazon Basin the differences in morning versus evening rainfall were complex, with an alternating series of morning/evening maxima aligned southwest to northeast from the Andes to the northeast Brazilian coast. A real extent of rainfall in Amazonia was slightly higher in the evening, but a maximum in morning precipitation was found on the Amazon River just east of Manaus. Precipitation over the water in the intertropical convergence zone (ITCZ) north of Brazil was more pronounced in the morning, and a pronounced land-/sea-breeze circulation was found along the northeast coast of Brazil. Inter-comparison of four years revealed 1992 to be the driest over Amazonia, with about a 23% decrease in mean rate compared to the 4-year mean estimated rain rate.

Negri, Andrew J.↗

DOE OSTI2024

Availability and Performance Loss Factors for U.S. PV Fleet Systems

In the PV Fleet Performance Data Initiative, we partner with photovoltaic (PV) fleet owners to collect time-series PV production data and publish aggregated, anonymized results. This report is an update of our previous publications, specifically a FY 2021 performance index publication and a FY 2022 fleet degradation analysis. In this analysis, we have increased our data participants and system totals by around 10% to 8.5 GW and 24,000 separate inverter data channels. Four major analysis topics are considered in this report: Performance Index (PI) trends, PV system availability, soiling losses, and PV system degradation. Performance Index and inverter availability are assessed on a larger set of data from our FY 2021 report: 1,128 systems compared with 200 systems from before. The increased number of systems is due to an improved data quality methodology, as well as introducing new systems to the analysis. Overall results are similar to previously published values - overall inverter availability is low in the first six months of system performance before reaching steady-state by the end of the first year. Excluding this six-month startup period, system-level aggregated data shows a median (P50) system availability of 0.99 and a lower 10th percentile (P90) value of 0.95 (Figure ES-1). A dependence on system size is also demonstrated, with worse inverter availability results for larger PV systems. Causes of this effect are under investigation, but may be impacted by inverter size, which also show lower availability for larger inverter sizes. This report also investigates PI, correcting for degradation, soiling, snow, and availability. Following these corrections, the median system PI over its entire lifetime is 0.95. PI values reported here are approximately 3% lower than what we presented in our previous FY 2021 report. Soiling loss is assessed in a comprehensive way for the first time in this report. Results are presented using the COmbined Degradation and Soiling (CODS) method, as implemented in RdTools (v3.0.0a4). Soiling values are presented for 255 systems, which indicated irradiance-weighted soiling loss greater than 1%. The values have been published in an updated NREL soiling map at nrel.gov/pv/soiling.html. Finally, we investigated system degradation using three different data analysis techniques: conventional RdTools (year-on-year (YOY)), CODS, and Performance Loss Rate (PLR) analysis. Overall degradation results are consistent with our previous publications. Rerunning conventional RdTools on our updated fleet shows that some data partners have systematically fallen below the median system degradation rate (change over time) of -0.75 %/year. A comparison with PLR analysis, which looks at change in annual PI over time, shows that median system degradation is consistent with -0.5% to -0.75% per year change. However, at the P90 value, system degradation is substantially faster. These two results are consistent and indicate that resulting degradation statistics depend to a great degree on the population of PV systems making up the analysis cohort and whether soiling impacts the systems. The use of CODS for degradation analysis provides a different method for degradation assessment, which explicitly excludes the impact of recoverable soiling on degradation analysis. Excluding soiling effects yields an annual system degradation around -0.5% per year on average. This indicates that a portion of system performance loss may be attributed to periodic soiling that is not fully recovered. This report provides PV system owners/operators with background and methods to analyze PV system performance, give guidance for expected cohort performance, and performance loss values for use in pro-forma financial models, which guide new-build system design and bankability reports.

14 SOLAR ENERGY↗

DOE OSTI2025

Uranium particle age dating, aggregation, and model age best estimators

We present important aspects of uranium particle age dating by Large-Geometry Secondary Ion Mass Spectrometry (LG-SIMS) that can introduce bias and increase model age uncertainties, especially for small, young, and/or low-enriched particles. This metrology is important for applications related to International Nuclear Safeguards. We explore influential factors related to model age estimation, including the effects of evolving surface chemistry on inter-element measurements of particles (e.g., Th and U), detector background, and aggregation methods using simulated and actual particle samples. We introduce a new model age estimator, called “mid68”, that supplements 95% confidence intervals, providing a “best estimate” and uncertainty about the most likely age. The mid68 estimator can be calculated using the Feldman and Cousins method or Bayesian methods and provides a value with a symmetric uncertainty that can be used for calculations and approximate aggregation of processed model age values when the raw data and correction factors are not available. For particles yielding low 230 Th counts amidst nonzero detector background, their underlying model age probability distributions are asymmetric, so the mid68 estimator provides additional robust information regarding the underlying model age likelihood. This study provides a comprehensive and timely examination of critical aspects of uranium particle age dating as more laboratories establish particle chronometry capabilities.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2026

BASIN-3D Data Integration for Selected ARM Data Field Campaign Report

The purpose of this data services request was to demonstrate integration of the Atmospheric Radiation Measurement (ARM) User Facility’s “met” datastreams with time series data from other earth science data sources using the BASIN-3D data synthesis software tool. BASIN-3D is an open-source Python library that enables researchers to integrate data across configured public and private data sources. It provides a common query language for researchers to request measurement locations and time series data based on specified locations, variables, time period, statistics, aggregation, and data quality. BASIN-3D acquires the data that match the query from each configured data source and translates the results into harmonized vocabularies, thus reducing researchers' data-wrangling effort. In addition, because the queries are executed on demand, researchers can easily regenerate their synthesized data sets as new data and/or data updates become available, eliminating one-off data products. BASIN-3D can output data using a variety of different data structures for end-user applications including Python pandas data frames and hdf5 output formats.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2014

Simplified Access to Massive Climate Modeling Datasets to Support End Users

Explore the source record for details and available documents.

NetCDF, SQL, efficient data subsetting, Climate Mo↗

DOE OSTI2018

Kinetic regime of aggregation of UV-irradiated glyceraldehyde-3-phosphate dehydrogenase from rabbit skeletal muscle

Highlights: • Kinetic regime of aggregation of UV-irradiated glyceraldehyde-3-phosphate dehydrogenase at 37°C has been established. • The rate of stage of aggregate growth is limited by monomolecular structural reorganization of the irradiated protein. • 2-Hydroxypropyl-b-cyclodextrin decelerates stage of structural reorganization of irradiated enzyme and nucleation stage. The study of the kinetics of aggregation of UV-irradiated proteins has attracted considerable interest, since test systems based on aggregation of proteins denatured by UV radiation can be used for screening of the natural and artificial agents possessing chaperone-like activity (anti-aggregation activity). To provide the proper interpretation of the effects caused by the agents under study, the kinetic mechanism of the aggregation process should be established. In the present work the kinetic data on aggregation of UV-irradiated glyceraldehyde-3-phosphate dehydrogenase (GAPDH) from rabbit skeletal muscle at 37 °C (O.I. Maloletkina et al. Biophys. Chem. 163–164 (2012) 11–20) have been analyzed. It has been shown that the stage of aggregate growth follows the first-order kinetics and the experimentally measured rate constant of the first order corresponds to heat-induced structural reorganization of UV-irradiated GAPDH containing concealed damage.

60 APPLIED LIFE SCIENCES↗

NASA NTRS2010

Community-Based Services that Facilitate Interoperability and Intercomparison of Precipitation Datasets from Multiple Sources

Over the past 12 years, large volumes of precipitation data have been generated from space-based observatories (e.g., TRMM), merging of data products (e.g., gridded 3B42), models (e.g., GMAO), climatologies (e.g., Chang SSM/I derived rain indices), field campaigns, and ground-based measuring stations. The science research, applications, and education communities have greatly benefited from the unrestricted availability of these data from the Goddard Earth Sciences Data and Information Services Center (GES DISC) and, in particular, the services tailored toward precipitation data access and usability. In addition, tools and services that are responsive to the expressed evolving needs of the precipitation data user communities have been developed at the Precipitation Data and Information Services Center (PDISC) (http://disc.gsfc.nasa.gov/precipitation or google NASA PDISC), located at the GES DISC, to provide users with quick data exploration and access capabilities. In recent years, data management and access services have become increasingly sophisticated, such that they now afford researchers, particularly those interested in multi-data set science analysis and/or data validation, the ability to homogenize data sets, in order to apply multi-variant, comparison, and evaluation functions. Included in these services is the ability to capture data quality and data provenance. These interoperability services can be directly applied to future data sets, such as those from the Global Precipitation Measurement (GPM) mission. This presentation describes the data sets and services at the PDISC that are currently used by precipitation science and applications researchers, and which will be enhanced in preparation for GPM and associated multi-sensor data research. Specifically, the GES-DISC Interactive Online Visualization ANd aNalysis Infrastructure (Giovanni) will be illustrated. Giovanni enables scientific exploration of Earth science data without researchers having to perform the complicated data access and match-up processes. In addition, PDISC tool and service capabilities being adapted for GPM data will be described, including the Google-like Mirador data search and access engine; semantic technology to help manage large amounts of multi-sensor data and their relationships; data access through various Web services (e.g., OPeNDAP, GDS, WMS, WCS); conversion to various formats (e.g., netCDF, HDF, KML (for Google Earth)); visualization and analysis of Level 2 data profiles and maps; parameter and spatial subsetting; time and temporal aggregation; regridding; data version control and provenance; continuous archive verification; and expertise in data-related standards and interoperability. The goal of providing these services is to further the progress towards a common framework by which data analysis/validation can be more easily accomplished.

Liu, Zhong↗

DOE OSTI2024

Offshore Geologic Carbon Storage Data Collection and International Project Inventory

We present an interactive data collection to aggregate, understand, and disseminate the data that are publicly available to support offshore GCS which can be leveraged by stakeholders to understand where GCS may be viable offshore, create GCS project analogs, and address challenges to GCS in offshore environments. The Offshore Geologic Carbon Storage Data Collection is an Experience Builder web application of multiple web mapping applications, aggregated into a single tool for each data type for access, visualization, and exploration. We also present a spatial inventory of global offshore GCS efforts to visualize the scale and locations of actualized and potential offshore GCS. It includes project location, project type and stage, CO2 storage resource potential, injection rate, reservoir and seal geology, and key literature references. Quantitative and qualitative comparisons of the distribution and magnitude of projects by their attributes lends spatial insight into the status of global GCS operations and storage resource potential, thereby enabling comparative assessments and cross-cutting knowledge transfer for projects in development. These datasets illuminate trends in ongoing offshore projects and can be leveraged by stakeholders to estimate storage resources, identify subsurface analogs, review regulations, and address challenges to offshore GCS. Additionally, opportunities for concurrent decarbonization strategies can be identified.

Mulhern, Julia↗

DOE OSTI2024

International Offshore Geologic Carbon Storage Project Inventory and Data Collection

Mulhern, Julia↗

DOE OSTI2024

International Offshore Geologic Carbon Storage Inventory and Data Collection

Mulhern, Julia↗

DOE OSTI2017

Comparison of Clustering Techniques for Residential Energy Behavior using Smart Meter Data

Current practice in whole time series clustering of residential meter data focuses on aggregated or subsampled load data at the customer level, which ignores day-to-day differences within customers. This information is critical to determine each customer’s suitability to various demand side management strategies that support intelligent power grids and smart energy management. Clustering daily load shapes provides fine-grained information on customer attributes and sources of variation for subsequent models and customer segmentation. In this paper, we apply 11 clustering methods to daily residential meter data. We evaluate their parameter settings and suitability based on 6 generic performance metrics and post-checking of resulting clusters. Finally, we recommend suitable techniques and parameters based on the goal of discovering diverse daily load patterns among residential customers. To the authors’ knowledge, this paper is the first robust comparative review of clustering techniques applied to daily residential load shape time series in the power systems’ literature.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS2014

Data Albums: An Event Driven Search, Aggregation and Curation Tool for Earth Science

Approaches used in Earth science research such as case study analysis and climatology studies involve discovering and gathering diverse data sets and information to support the research goals. To gather relevant data and information for case studies and climatology analysis is both tedious and time consuming. Current Earth science data systems are designed with the assumption that researchers access data primarily by instrument or geophysical parameter. In cases where researchers are interested in studying a significant event, they have to manually assemble a variety of datasets relevant to it by searching the different distributed data systems. This paper presents a specialized search, aggregation and curation tool for Earth science to address these challenges. The search rool automatically creates curated 'Data Albums', aggregated collections of information related to a specific event, containing links to relevant data files [granules] from different instruments, tools and services for visualization and analysis, and information about the event contained in news reports, images or videos to supplement research analysis. Curation in the tool is driven via an ontology based relevancy ranking algorithm to filter out non relevant information and data.

Ramachandran, Rahul↗