Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

Developing and tuning a community scale energy model for a disadvantaged community

This work describes the development of a community-scale energy model for a mixed-use low-income community located in Huntington Beach, CA. An accurate community-scale energy model is useful for evaluating the use of limited capital resources used to invest in clean energy technologies. This work lays out the process of developing such a model while relying primarily on publicly available data and highlighting critical partnerships necessary for model development success. The primary contribution of this work is the demonstration of the process used to develop an accurate energy model for a disadvantaged community when minimal building and energy use data is available. The heart of the model is the physics-based community scale energy modeling platform URBANopt. Using a bottom-up load modeling approach, energy simulated energy use falls within 3% or less of aggregate annual utility data, and within 10% or less aggregate monthly utility data. The demonstrated model development and tuning process can be used by others to characterize other atypical communities, which may differ significantly from prototypical models.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Commercial Building Tenant Energy Usage Aggregation and Privacy

A growing number of building owners are benchmarking their building energy use. This requires the building owner to acquire monthly whole-building energy usage information, which can be challenging for buildings in which individual tenants have their own utility meters and accounts with the utility. Some utilities and utility regulators have turned to aggregation of customer energy use data (CEUD) as a way to give building owners whole-building energy usage data while protecting customer privacy. Meter profile aggregation adds a layer of protection that decreases the risk of revealing CEUD as the number of meters aggregated increases. The report statistically characterizes the similarity between individual energy usage patterns and whole-building totals at various levels of meter aggregation.

Data aggregation,. Aggregation threhold selection↗

Current Trends in the Detection of Sociocultural Signatures: Data-Driven Models

The harvesting of behavioral data and their analysis through evidence-based reasoning enable the detection of sociocultural signatures in their context to support situation awareness and decision making. Harvested data are used as training materials from which to infer computational models of sociocultural behaviors or calibrate parameters for such models. Harvested data also serve as evidence input that the models use to provide insights about observed and future behaviors for targets of interest. The harvested data is often the result of assembling diverse data types and aggregating them into a form suitable for analysis. Data need to be analyzed to bring out the categories of content that are relevant to the domain being addressed in order to train or run a model. If, for example, we are modeling the intent of a group to engage in violent behavior using messages that the group has broadcasted, then these messages need to be processed to extract and measure indicators of violent intent. The extracted indicators and the associated measurements (e.g. rates or counts of occurrence) can then be used to train/calibrate and run computational models that assess the propensity for violence expressed in the source message. Ubiquitous access to the Internet, mobile telephony and technologies such as digital photography and digital video have enabled social media application platforms such as Facebook, YouTube, and Twitter that are altering the nature of human social interaction. The fast increasing pace of online social interaction introduces new challenges and opportunities for gathering sociocultural data. Challenges include the development of harvesting and processing techniques tailored to new data environments and formats (e.g. Twitter, Facebook), the integration of social media content with traditional media content, and the protection of personal privacy. As these and other challenges are addressed, a new wealth of behavioral data and data analysis methods becomes available that are shaping social computing as a strongly data-driven experimental discipline with an increasingly stronger impact on the decision-making process of groups and individuals alike. In this chapter, we review current advances and trends in the detection of sociocultural signatures. Specific embodiments of the issues discussed are provided with respect to the assessment of violent intent and sociopolitical contention. We begin by reviewing current approaches to the detection of sociocultural signatures in these domains. Next, we turn to the review of novel data harvesting methods for social media content. Finally, we discuss the application of sociocultural models to social media content, and conclude by commenting on current challenges and future developments.

Sanfilippo, Antonio P.↗

DOME: Directional medical embedding vectors from Electronic Health Records

Motivation: The increasing availability of Electronic Health Record (EHR) systems has created enormous potential for translational research. Recent developments in representation learning techniques have led to effective large-scale representations of EHR concepts along with knowledge graphs that empower downstream EHR studies. However, most existing methods require training with patient-level data, limiting their abilities to expand the training with multi-institutional EHR data. On the other hand, scalable approaches that only require summary-level data do not incorporate temporal dependencies between concepts. Methods: We introduce a DirectiOnal Medical Embedding (DOME) algorithm to encode temporally directional relationships between medical concepts, using summary-level EHR data. Specifically, DOME first aggregates patient-level EHR data into an asymmetric co-occurrence matrix. Then it computes two Positive Pointwise Mutual Information (PPMI) matrices to correspondingly encode the pairwise prior and posterior dependencies between medical concepts. Following that, a joint matrix factorization is performed on the two PPMI matrices, which results in three vectors for each concept: a semantic embedding and two directional context embeddings. They collectively provide a comprehensive depiction of the temporal relationship between EHR concepts. Results: We highlight the advantages and translational potential of DOME through three sets of validation studies. First, DOME consistently improves existing direction-agnostic embedding vectors for disease risk prediction in several diseases, for example achieving a relative gain of 5.5% in the area under the receiver operating characteristic (AUROC) for lung cancer. Second, DOME excels in directional drug-disease relationship inference by successfully differentiating between drug side effects and indications, correspondingly achieving relative AUROC gain over the state-of-the-art methods by 10.8% and 6.6%. Finally, DOME effectively constructs directional knowledge graphs, which distinguish disease risk factors from comorbidities, thereby revealing disease progression trajectories. The source codes are provided at https://github.com/celehs/Directional-EHRembedding.

60 APPLIED LIFE SCIENCES↗

Offshore Geologic Carbon Storage Data Collection and Data Gaps Analysis

This is a TRS documenting the Offshore Geologic Carbon Storage Data Collection. It describes the Data Collection web application and its creation as well as an accompanying Data Gaps Assessment. We present an interactive data collection and data gaps analysis to aggregate, understand, and disseminate the data that are publicly available to support offshore GCS in the United States. This data collection and data gaps analysis can be leveraged by stakeholders to understand where GCS may be viable offshore, create GCS project analogs, and address challenges to GCS in offshore environments.

58 GEOSCIENCES↗

Data-Driven Pricing Strategy for Demand-Side Resource Aggregators

We consider a utility who seeks to coordinate the energy consumption of multiple demand-side flexible resource aggregators. For the purpose of privacy protection, the utility has no access to the detailed information of loads of resource aggregators. Instead, we assume that the utility can directly observe each aggregator's aggregate energy consumption outcomes. Furthermore, the utility can leverage resource aggregator energy consumption via time-varying electricity price profiles. Based on inverse optimization technique, we propose an estimation method for the utility to infer the energy requirement information of aggregators. Subsequently, we design a data-driven pricing scheme to help the utility achieve system-level control objectives (e.g., minimizing peak demand) by combining hybrid particle swarm optimizer with mutation algorithm and an iterative algorithm. Case studies have demonstrated the effectiveness of the proposed approach against two benchmark pricing strategies-a flat-rate scheme and a time-of-use scheme.

data-driven optimization↗

Reducing Uncertainty of Fielded Photovoltaic Performance (Final Technical Report)

Improved analysis and reporting of photovoltaic (PV) field performance increases the certainty of owners and financiers that systems will perform as expected. Advanced module technologies (e.g., PERC, HJT, and bifacial) introduce new degradation mechanisms and performance characteristics. The FY19-21 Reducing Uncertainty project leveraged data from the ever-increasing PV fleet to develop models and understanding of the field performance of existing and new technologies. Specifically, we accomplished: report on field performance and degradation rates for high-efficiency silicon (HJT, PERC, IBC) and more conventional technologies; developed automated analysis techniques to quantify system performance (performance ratio, energy yield) and production shortfalls (soiling, degradation, availability); refined the RdTools software toolkit to bring standard, validated analysis techniques to bear on third-party data; analyzed and reported on large datasets including Treasury data and Lawrence Berkeley National Laboratory's Utility-Scale dataset to expand the high-quality degradation-rate histogram published previously; worked with industry partners and the DuraMAT data hub to enable private parties to share and aggregate PV production data anonymously, leveraging cloud-based data analysis infrastructure and publishing on US fleet-scale performance comprising over 7GW of operating systems. (https://www.nrel.gov/pv/fleet-performance-data-initiative.html). Through our industry collaborations we have engaged in NDA-covered data transfer with twelve PV fleet owners as of January 2022, with more agreements in negotiation. Our scalable cloud-based time series database contains over 30 billion rows (20TB) of PV time series data, representing over 1700 commercial and utility-scale systems, and over 7.2 GW of DC capacity (Fig 1). Initial field performance results have been distributed in several public reports. Because our fleet composition and data quality methods are continually improving, annual updates to these results are published to our PV Fleet webpage [ https://www.nrel.gov/pv/fleet-performance-data-initiative.html ] and DuraMAT data hub [DOI: 10.21948/1842958]. Another existing dissemination channel used for observed soiling losses is a map we maintain for soiling losses. Additional products developed include a report detailing fleet-wide performance index, availability, startup loss and snow loss factors, a detailed report on the 1603 grant dataset comprising over 100,000 PV systems with failure and performance details and a utility-scale report coauthored with LBNL on 31 GW of system performance.

14 SOLAR ENERGY↗

Initial Mobility Analysis for ORNL VA-EDH Synthetic Populations

Travel burdens are a major barrier to healthcare access among US Veteran patient populations, particularly those residing in rural areas. Spatial accessibility to points of care for US Veteran populations is commonly assessed in two ways. The first approach uses open data from the US Census to represent collective travel burdens, for example the distance between population-weighted census tract centroids and VHA points of care. The second approach uses restricted-access VHA patient data to measure travel costs (e.g., distance, time) for accessing points of care with respect to geolocated patient addresses and real or approximated transportation networks. While the advantage of the open data approach lies in its reproducibility, it has notable limitations in its tendency to infer individual travel behavior from aggregate population characteristics, a problem known as ecological fallacy. Conversely, while the patient data approach is able to account for individual travel behavior, its ability to account for localized access disparities (e.g., a neighborhood with exceptionally high transportation costs) and patient demographics is limited as protecting individual patient data requires their storage in closed systems with limited capacity for adequately modeling real-world travel patterns or for supplementing patient attributes. Additionally, the patient data approach cannot account for veterans who are not enrolled in the VHA system but who may be eligible for care. These challenges limit the ability to perform “what if” analyses on the effects of place-specific interventions on veteran populations with high access barriers to healthcare. To address these challenges, we explore the application of realistic synthetic populations to examine travel burdens and spatial accessibility issues among veteran patient populations. Synthetic populations provide a virtual, individually-resolved and cross-sectional representation of the veteran patient population that enables investigation of spatial access to points of care in ways in which aggregate data and patient data do not. First, synthetic populations allow one to directly assess how individuals access points of care, from synthesized residential locations to outpatient facilities on real-world transportation networks. Modeling access to points of care at the individual scale addresses the ecological fallacy problem associated with using aggregated census data to represent veteran populations and patterns of movement. Second, synthetic populations provide a means of completely representing an area’s veteran population using only publicly available, anonymized census microdata from the American Community Survey (ACS) to ensure the privacy of real-world individuals. Generating synthetic populations from the ACS also expands descriptive characteristics beyond what patient data typically offers to include socio-demographic, economic, housing, and mobility attributes. More detailed profiles of both VHA patient populations and veterans not enrolled in the VA system will provide a comprehensive picture of groups that may benefit from interventions or outreach. As an initial exercise for using synthetic populations to measure veteran travel burdens to VA care, we apply Oak Ridge National Laboratory’s (ORNL) UrbanPop capability to generate a series of synthetic VHA patient populations for 9 Veterans Integrated Services Networks (VISN) market areas in 9 Census Divisions across the continental United States, which are listed in Table 1. We use UrbanPop to produce synthetic populations for the VISN markets selected for each US Census Division, then assign VA outpatient clinic destinations to synthetic VHA patients based on travel about each VISN market’s road network. To demonstrate using the synthetic populations to evaluate healthcare travel burdens, we compare the time-based impedance between simulated home locations and VA outpatient clinics in each VISN market. We then perform validation exercises on the synthetic populations with respect to neighborhood (block group) demographic composition as well as patient mobility, comparing aggregate origin-destination statistics for the synthetic population to outpatient visits available in restricted patient data from the VA’s Corporate Data Warehouse (CDW) database.

97 MATHEMATICS AND COMPUTING↗

Curating Carbon Storage Data for Reuse: Enabling Research and Modeling from Earth’s Surface to Subsurface

The volume of public geologic carbon storage (GCS) data resources has continued to increase in recent years as the result of an increase in funding from government, industry, and academia towards national, basin, regional and field scale studies to ensure carbon capture and storage becomes a commercially viable operation. Despite the increasing volume of data, GCS data applied towards analyses such as geologic, cost, and risk modeling continues to be multi-sourced and often disparate in nature, published across government agencies, websites, data repositories and buried in derivative reports and documents. Much of the time preparing for an analysis and derivative product development is spent collecting, aggregating, transforming and preparing input data. There have been significant efforts within the DOE National Energy Technology Laboratory’s Carbon Storage Program to optimize multi-source, multi-scale subsurface geologic data curation and aggregation to support data discovery, interoperability, and reuse. Methods include the use of artificial intelligence, machine learning, and data science techniques. This talk will discuss the workflows, best practices, and processes developed to support the aggregation and curation of data through the whole system – surface to subsurface data - that support multi-scale, multi-purpose analysis for carbon storage research.

Morkner, Paige↗

Soil pore network response to freeze-thaw cycles in permafrost aggregates

This dataset contains data used for the paper "Pore network response to freeze-thaw cycles in permafrost aggregates". The Related References field will be updated with a full citation when available.Climate change in Arctic landscapes may increase freeze-thaw frequency within the active layer as well as newly thawed permafrost. A highly disruptive process, freeze-thaw can deform soil pores and alter the architecture of the soil pore network with varied impacts to water transport and retention, redox conditions, and microbial activity. Our objective was to investigate how freeze-thaw cycles impacted the pore network of newly thawed permafrost aggregates to improve understanding of what type of transformations can be expected from warming Arctic landscapes. We measured the impact of freeze-thaw on pore morphology, pore throat diameter distribution, and pore connectivity with X-ray computed tomography (XCT) using six permafrost aggregates with sizes of 2.5 cm3 from a mineral soil horizon (Bw; 28-50 cm depths) in Toolik, Alaska. Freeze-thaw cycles were performed using a laboratory incubation consisting of five freeze-thaw cycles (-10˚C to 20˚C) over five weeks. Our findings indicated decreasing spatial connectivity of the pore network across all aggregates with higher frequencies of singly connected pores following freeze-thaw. Water-filled pores that were connected to the pore network decreased in volume while the overall connected pore volumetric fraction was not affected. Shifts in the pore throat diameter distribution were mostly observed in pore throats ranges of 100 microns or less with no corresponding changes to the pore shape factor of pore throats. Responses of the pore network to freeze-thaw varied with aggregate, suggesting that initial pore morphology may play a role in driving freeze-thaw response. Our research suggests that freeze-thaw alters the microenvironment of permafrost aggregates during the incipient stage of deformation following permafrost thaw, impacting soil properties and function in Arctic landscapes undergoing transition. This dataset contains a compressed (.zip) archive of the data and R scripts used for this manuscript. The dataset includes files in .csv format, which can be accessed and processed using MS Excel or R. This archive can also be accessed on GitHub at https://github.com/Erin-Rooney/XCT-freezethaw (DOI: 10.5281/zenodo.5816355).

54 ENVIRONMENTAL SCIENCES↗

Incoherent range-based holography of far-distant objects using LIDAR

Los Alamos National Laboratory has developed an incoherent, long-range, sub-centimeter resolution (LIDAR) with which we achieve centimeter-scale reflection holography at extremely long ranges. The system consists of a pulsed laser and photon-counting receiver. This combination yields round-trip time of flight data to illuminate parts of the object of interest. The aggregation of these data for many LIDAR pulses yields a plot with a range on the X axis and reflectance on the Y axis, which we refer to as a range profile. Observing that the range profile is a projection of the reflection map of the object onto the view vector, we collect profiles from a variety of viewing angles and invert these data to form an image. We adapt imaging algorithms from the field of computer aided tomography to suit our application and present results from imaging demonstrations at a 10 km range.

Hoffmann, Mitchell P. (ORCID:0000000170551324)↗

Changes in Exercise Data Management

The suite of exercise hardware aboard the International Space Station (ISS) generates an immense amount of data. The data collected, treadmill, cycle ergometer, and resistance strength training hardware, are basic exercise parameters (time, heart rate, speed, load, etc.). The raw data are processed in the laboratory and more detailed parameters are calculated from each exercise data file. Updates recently have been made to how these valuable data are stored, adding an additional level of security, increasing accessibility, and resulting in overall increased efficiency of medical report delivery. Questions regarding exercise performance or how exercise may influence other variables of crew health frequently arise within the crew health care community. Inquiries regarding the health of the exercise hardware often need quick analysis and response to ensure the exercise system is operable on a continuous basis. Consolidating all of the exercise system data in a single repository enables a quick response to both the medical and engineering communities. A SQL server database is currently in use, and provides a secure location for all of the exercise data starting at ISS Expedition 1 to current date. The database has been structured to update derived metrics automatically, making analysis and reporting available within minutes of dropping the in-flight data into the database. Commercial tools were evaluated to help aggregate and visualize data from the SQL database. The Tableau software provides manageable interface, which has improved the laboratory’s output time of crew reports by 67%. Expansion of the SQL database, to be inclusive of additional medical requirement metrics, addition of ‘app-like’ tools for mobile visualization, and collaborative use (e.g., operational support teams, research groups, and International Partners) of the data system, is currently being explored.

Buxton, R. E.↗

Finding Atmospheric Composition (AC) Metadata

The Atmospheric Composition Portal (ACP) is an aggregator and curator of information related to remotely sensed atmospheric composition data and analysis. It uses existing tools and technologies and, where needed, enhances those capabilities to provide interoperable access, tools, and contextual guidance for scientists and value-adding organizations using remotely sensed atmospheric composition data. The initial focus is on Essential Climate Variables identified by the Global Climate Observing System CH4, CO, CO2, NO2, O3, SO2 and aerosols. This poster addresses our efforts in building the ACP Data Table, an interface to help discover and understand remotely sensed data that are related to atmospheric composition science and applications. We harvested GCMD, CWIC, GEOSS metadata catalogs using machine to machine technologies - OpenSearch, Web Services. We also manually investigated the plethora of CEOS data providers portals and other catalogs where that data might be aggregated. This poster is our experience of the excellence, variety, and challenges we encountered.Conclusions:1.The significant benefits that the major catalogs provide are their machine to machine tools like OpenSearch and Web Services rather than any GUI usability improvements due to the large amount of data in their catalog.2.There is a trend at the large catalogs towards simulating small data provider portals through advanced services. 3.Populating metadata catalogs using ISO19115 is too complex for users to do in a consistent way, difficult to parse visually or with XML libraries, and too complex for Java XML binders like CASTOR.4.The ability to search for Ids first and then for data (GCMD and ECHO) is better for machine to machine operations rather than the timeouts experienced when returning the entire metadata entry at once. 5.Metadata harvest and export activities between the major catalogs has led to a significant amount of duplication. (This is currently being addressed) 6.Most (if not all) Earth science atmospheric composition data providers store a reference to their data at GCMD.

metadata search↗

Spatial and Temporal Uncertainty of Crop Yield Aggregations

The aggregation of simulated gridded crop yields to national or regional scale requires information on temporal and spatial patterns of crop-specific harvested areas. This analysis estimates the uncertainty of simulated gridded yield time series related to the aggregation with four different harvested area data sets. We compare aggregated yield time series from the Global Gridded Crop Model Inter-comparison project for four crop types from 14 models at global, national, and regional scale to determine aggregation-driven differences in mean yields and temporal patterns as measures of uncertainty. The quantity and spatial patterns of harvested areas differ for individual crops among the four datasets applied for the aggregation. Also simulated spatial yield patterns differ among the 14 models. These differences in harvested areas and simulated yield patterns lead to differences in aggregated productivity estimates, both in mean yield and in the temporal dynamics. Among the four investigated crops, wheat yield (17% relative difference) is most affected by the uncertainty introduced by the aggregation at the global scale. The correlation of temporal patterns of global aggregated yield time series can be as low as for soybean (r = 0.28).For the majority of countries, mean relative differences of nationally aggregated yields account for10% or less. The spatial and temporal difference can be substantial higher for individual countries. Of the top-10 crop producers, aggregated national multi-annual mean relative difference of yields can be up to 67% (maize, South Africa), 43% (wheat, Pakistan), 51% (rice, Japan), and 427% (soybean, Bolivia).Correlations of differently aggregated yield time series can be as low as r = 0.56 (maize, India), r = 0.05∗Corresponding (wheat, Russia), r = 0.13 (rice, Vietnam), and r = −0.01 (soybean, Uruguay). The aggregation to sub-national scale in comparison to country scale shows that spatial uncertainties can cancel out in countries with large harvested areas per crop type. We conclude that the aggregation uncertainty can be substantial for crop productivity and production estimations in the context of food security, impact assessment, and model evaluation exercises.

Aggregation uncertainty↗

Analysis for Building Envelopes and Mechanical Systems Using 2012 CBECS Data

This report describes the aggregation and mapping of certain building characteristics data available in the most recent Commercial Building Energy Consumption Survey (CBECS) (DOE EIA 2012) to describe most typical building construction practices. This report provides summary data for potential use in the support of modifications to the Pacific Northwest National Laboratory’s commercial building prototypes used for building energy code analysis. Specifically, this report outlines findings and most typical design choices for certain building envelope and heating, ventilating, and air-conditioning (HVAC) system choices.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Artificial Intelligence for Data Center Operations (AIOps): Cooperative Research and Development (Final Report)

High performance computing data centers will increasingly need to rely on automation to keep pace with exascale growth in compute capability and to manage and optimize the data center environment and facility resources. Artificial intelligence and machine learning approaches provide the means to improve HPC data center operational efficiency, by learning historical trends and training models to operate on real-time data collected from both IT and facilities sources. NREL has developed methods of real-time collection, aggregation and streaming of these data in the ESIF HPC Data Center and has collected a significant dataset of relevant metrics across computer systems, racks, environmental, building and utility sources for research into various predictive analytics problems. HPE's Advanced Technology Group (ATG) is doing comprehensive research into exascale monitoring and management for High Performance Computing (HPC) systems (hereinafter HPE's Data Monitoring/ Management Technology). NREL and HPE will collaborate to add Artificial Intelligence (AI) to NREL's real-time data collection/ aggregation/ streaming system and HPE's Data Monitoring/ Management System, with the goal of improving the operational efficiency of NREL's Energy Systems Integration Facility (ESIF) HPC Data Center through data analytics on both historical and real-time data from IT systems and facilities operations. This collaboration will consist of efforts in Data Management, Data Analytics, and AI/ML Optimization for both manual and autonomous intervention in data center operations. This will be a multi-year, multi-staged effort with a goal towards building capabilities for an Advanced Smart Facility, and demonstration of these techniques in the NREL ESIF HPC Data Center.

97 MATHEMATICS AND COMPUTING↗

Changes in Exercise Data Management

The suite of exercise hardware aboard the International Space Station (ISS) generates an immense amount of data. The data collected from the treadmill, cycle ergometer, and resistance strength training hardware are basic exercise parameters (time, heart rate, speed, load, etc.). The raw data are post processed in the laboratory and more detailed parameters are calculated from each exercise data file. Updates have recently been made to how this valuable data are stored, adding an additional level of data security, increasing data accessibility, and resulting in overall increased efficiency of medical report delivery. Questions regarding exercise performance or how exercise may influence other variables of crew health frequently arise within the crew health care community. Inquiries over the health of the exercise hardware often need quick analysis and response to ensure the exercise system is operable on a continuous basis. Consolidating all of the exercise system data in a single repository enables a quick response to both the medical and engineering communities. A SQL server database is currently in use, and provides a secure location for all of the exercise data starting at ISS Expedition 1 - current day. The database has been structured to update derived metrics automatically, making analysis and reporting available within minutes of dropping the inflight data it into the database. Commercial tools were evaluated to help aggregate and visualize data from the SQL database. The Tableau software provides manageable interface, which has improved the laboratory's output time of crew reports by 67%. Expansion of the SQL database to be inclusive of additional medical requirement metrics, addition of 'app-like' tools for mobile visualization, and collaborative use (e.g. operational support teams, research groups, and International Partners) of the data system is currently being explored.

Buxton, R. E.↗