SEARCH · Engineering Papers

Results for “Data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Vegetation Warming Experiment: Environmental conditions, Utqiagvik (Barrow), Alaska, 2021

Environmental conditions measured in five warming chambers and paired ambient control plots located on the Barrow Environmental Observatory (BEO), Utqiagvik, Alaska from 19 June - 17 September, 2021. These data were recorded in support of the Zero Power Warming (ZPW) vegetation warming experiment, a series of single season vegetation warming treatments conducted over four years from 2017-2021 (no experiment in 2020). Air temperature and humidity, infrared surface (canopy) temperature, soil temperature, soil moisture, NDVI (normalized difference vegetation index), PRI (photochemical reflectance index), solar radiation and chamber venting were recorded in each chamber at 1 minute intervals. Ambient air temperature, humidity, solar radiation and uplooking PRI and NDVI were measured at a centrally located meteorology station. Vapor pressure deficit (VPD) was calculated and included in the final processed data products. Data has undergone full QA/QC and is presented as 1 minute data, and hourly and daily aggregate data products. This data package includes unprocessed raw data (*.dat files), processed data (*.csv) and metadata including a full description of sensors, calculations and processing (*.csv, *.pdf). See related NGEE-Arctic "Vegetation Warming Experiment" data packages for leaf-level gas exchange and other leaf trait data; chamber, plot and landscape phenocamera images; thaw depth, and GPS locations of chambers and ambient plots. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2020

AUTONOMIE VID

Autonomie Vehicle Information Database (VID) offers a comprehensive list of vehicle specifications since 1990. The database details more than 65,000 vehicles with hundreds of attributes. The database is the result of the development of a general automated data collection framework as well as the development of building blocks for processing, cleaning, integrating and analyzing complex data. The data has undergone several layers of outlier detections processes, machine learning based imputations methods have been used to deal with missing data problems, and new fields have been created according to the rules of feature engineering. Thanks to this streamlined data pipelines, the resulting processed aggregated data should deliver a unique level of information to the user in which the content can be efficiently maintained and updated.

Moswd, Ayman↗

DOE OSTI2023

Projections of Hourly Meteorology by Balancing Authority Based on the IM3/HyperFACETS Thermodynamic Global Warming (TGW) Simulations

This dataset contains 40 years (1980-2019) of historical hourly meteorology and 80 years (2020-2099) of projected hourly meteorology for 54 Balancing Authorities (BAs) in the conterminous United States. Details about the scenarios and variables included in this dataset are in the readme.pdf file. This dataset is derived from the IM3/HyperFACETS Thermodynamic Global Warming (TGW) simulations (https://doi.org/10.57931/1885756). More details on the TGW approach can be found at: https://tgw-data.msdlive.org/. If you use this dataset please also cite the raw TGW dataset (Jones, A. D., Rastogi, D., Vahmani, P., Stansfield, A., Reed, K., Thurber, T., Ullrich, P., & Rice, J. S. (2022). IM3/HyperFACETS Thermodynamic Global Warming (TGW) Simulation Datasets (v1.0.0) [Data set]. MSD-LIVE Data Repository. https://doi.org/10.57931/1885756). To go from the TGW data to these BA-level aggregated data we first averaged the raw gridded data by county in the United States. That intermediate data step is also stored in MSD-LIVE (https://doi.org/10.57931/1960548). We then population-weight the county-level hourly data in order to create population-weighted meteorology time series for each BA. Historical populations are from the United States Census Bureau and the evolving future populations are based on Shared Socioeconomic Pathways (SSPs) 3 and 5. The four climate scenarios crossed with the two SSPs yield eight different future projections for each BA: rcp45cooler_ssp3, rcp45cooler_ssp5, rcp45hotter_ssp3, rcp45hotter_ssp5, rcp85cooler_ssp3, rcp85cooler_ssp5, rcp85hotterssp3, rcp85hotterssp5. For the historical period and each of the eight future scenarios the dataset has hourly estimates of the population-weighted average of five meteorological variables: Temperature, specific humidity, shortwave radiation, longwave radiation, and wind speed. All times are in Coordinated Universal Time (UTC). The code to go from the raw TGW data to county-level and then BA-level projections is available at: https://github.com/IMMM-SFA/im3components/tree/main/im3components/wrf_to_tell.

Balancing Authority↗

DOE OSTI2023

The New Small Wheel electronics

The increase in luminosity, and consequent higher backgrounds, of the LHC upgrades require improved rejection of fake tracks in the forward region of the ATLAS Muon Spectrometer. The New Small Wheel upgrade of the Muon Spectrometer aims to reduce the large background of fake triggers from track segments that don't originate from the interaction point. The New Small Wheel employs two detector technologies, the resistive strip Micromegas detectors and the “small” Thin Gap Chambers, with a total of 2.45 million electrodes to be sensed. The two technologies require the design of a complex electronics system given that it consists of two different detector technologies and is required to provide both precision readout and a fast trigger. It will operate in a high background radiation region up to about 20 kHz/cm 2 at the expected HL-LHC luminosity of $\mathcal{L}$= 7.5 × 10 34 cm -2 s -1 . The architecture of the system is strongly defined by the GBTx data aggregation ASIC, the newly-introduced FELIX data router and the software based data handler of the ATLAS detector. The electronics complex of this new detector was designed and developed in the last ten years and consists of multiple radiation tolerant Application Specific Integrated Circuits, multiple front-end boards, dense boards with FPGA's and purpose-built Trigger Processor boards within the ATCA standard. The New Small Wheel has been installed in 2021 and is undergoing integration within ATLAS for LHC Run 3. It should operate through the end of Run 4 (December 2032). In this manuscript, the overall design of the New Small Wheel electronics is presented.

47 OTHER INSTRUMENTATION↗

DOE OSTI2022

Aggregation bias and its drivers in large‐scale flood loss estimation: A Massachusetts case study

Abstract Large‐scale estimations of flood losses are often based on spatially aggregated inputs. This makes risk assessments vulnerable to aggregation bias, a well‐studied, sometimes substantial outcome in analyses that model fine‐grained spatial phenomena at coarse spatial units. To evaluate this potential in the context of large‐scale flood risk assessments, we use data from a high‐resolution flood hazard model and structure inventory for over 1.3 million properties in Massachusetts and examine how prominent data aggregation approaches affect the magnitude and spatial distribution of flood loss estimates. All considered aggregation approaches rely on aggregate structure inventories but differ in whether flood hazard is also aggregated. We find that aggregating only structure inventories slightly underestimates overall losses (−10% bias), and when flood hazard data is spatially aggregated to even relatively small spatial units (census block), statewide aggregation bias can reach +366%. All aggregation‐based procedures fail to capture the spatial covariation of inputs distributions in the upper tails that disproportionately generate total expected losses. Our findings are robust to several key assumptions, add important context to published risk assessments and highlight opportunities to improve flood loss estimation uncertainty quantification.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Radio Afterglow Detection and AI-driven Response (RADAR): A Federated Framework for Gravitational-wave Event Follow-up

The landmark detection of both gravitational waves (GWs) and electromagnetic (EM) radiation from the binary neutron star merger GW170817 has spurred efforts to streamline the follow-up of GW alerts in current and future observing runs of ground-based GW detectors. Within this context, the radio band of the EM spectrum presents unique challenges. Sensitive radio facilities capable of detecting the faint radio afterglow seen in GW170817, and with sufficient angular resolution, have small fields of view compared to typical GW localization areas. Additionally, theoretical models predict that the radio emission from binary neutron star mergers can evolve over weeks to years, necessitating long-term monitoring to probe the physics of the various postmerger ejecta components. These constraints, combined with limited radio observing resources, make the development of more coordinated follow-up strategies essential—especially as the next generation of GW detectors promises a dramatic increase in detection rates. Here, we present RADAR, a framework designed to address these challenges by promoting community-driven information sharing, federated data analysis, and system resilience, while integrating AI methods for both GW signal identification and radio data aggregation. We show that it is possible to preserve data rights while sharing models that can help design and/or update follow-up strategies. We demonstrate our approach through a case study of GW170817, and discuss future directions for refinement and broader application.

Gravitational waves↗

DOE OSTI2021

Simple and Scalable Streaming: The GRETA Data Pipeline

The Gamma Ray Energy Tracking Array (GRETA) is a state of the art gamma-ray spectrometer being built at Lawrence Berkeley National Laboratory to be first sited at the Facility for Rare Isotope Beams (FRIB) at Michigan State University. A key design requirement for the spectrometer is to perform gamma-ray tracking in near real time. To meet this requirement we have used an inline, streaming approach to signal processing in the GRETA data acquisition system, using a GPU-equipped computing cluster. The data stream will reach 480 thousand events per second at an aggregate data rate of 4 gigabytes per second at full design capacity. We have been able to simplify the architecture of the streaming system greatly by interfacing the FPGA-based detector electronics with the computing cluster using standard network technology. A set of highperformance software components to implement queuing, flow control, event processing and event building have been developed, all in a streaming environment which matches detector performance. Prototypes of all high-performance components have been completed and meet design specifications.

Cromaz, Mario↗

DOE OSTI2026

Predicting Particle-Size Distributions in Fuel-Cell Inks

Prediction of the catalyst-activated carbon particle sizes of fuel-cell inks remains a critical challenge in enhancing the performance and durability of fuel cells. The performance and structural integrity of the catalyst layers in the cell depend not only on the properties of the ionomer, but also on the carbon supports that host the catalyst. To investigate how these carbon aggregate structures form, we present a model that calculates the cooperative size distributions of ionomer and carbon aggregates in various water/alcohol mixtures and compares those results to available experimental data. Aggregation of both the suspended ionomers and the carbon particles is interwoven as the carbon aggregation depends heavily on the ionic strength of its environment, namely protons dissociating from the ionomer’s sulfonic-acid-group side chains. We demonstrate that the surrounding mixed solvent as well as Nafion TM binder concentration strongly influence the degree of aggregation for carbon particles. The validated model provides a framework for exploring aggregation under a wide range of conditions for various electrochemical-technology inks.

Srivastav, Harsh [University of California, Berkel↗

DOE OSTI2023

Secure Data Logging and Processing with Blockchain and Machine Learning (Final Report)

Secure Data Logging and Processing with Blockchain and Machine Learning (ML) research is focused on the development of a platform to securely log and process sensor data in fossil power plants. The platform integrates two emerging technologies, blockchain and ML, and incorporates several innovative mechanisms to ensure the integrity, reliability, and resiliency of power systems. The goal is to protect the power plant from various cyberattacks such as false data injection and denial of service attacks using these technologies. The research goal was enabled by the following Research Project Objectives: 1) Secure authentication and identity verification of sensor nodes, actuators, and other equipment within a network. 2) Development of mechanisms that ensure only data sent by legitimate sensors are accepted and stored in the data repository. 3) Development of data aggregation methodologies using ML / Deep Learning (DL) algorithms to minimize noise / faulty data. 4) Implementation of the blockchain technologies to provide data security using secured IOTA framework & nodes.

20 FOSSIL-FUELED POWER PLANTS↗

DOE OSTI2026

Investigating the Determinants of Household Capabilities Burden During Power Outages: The Case of Winter Storm Uri

Existing research primarily uses census data to identify the vulnerability of communities to hazards. These vulnerability indices provide aggregated data and are not hazard-specific nor well-validated with post-event data. In contrast, our study uses household survey data (n=1065) to understand which Texan households suffered the greatest loss of their capabilities due to power outages and other utility service disruptions during Winter Storm Uri. Inspired by the Capabilities Approach, our measures of burden include the number of household capability types disrupted during the outages (e.g., cooking, heating, refrigeration), the severity of impact for each disrupted capability, and the additional time and financial costs of coping with these disruptions. We perform a clustering analysis, and find two distinct groups in our data, consisting of ‘lesser burden' and ‘heavier burden' households. Results indicate that the households experiencing the heaviest capabilities burden were most likely to experience longer power outages and the loss of water services. They were also more likely to have a Hispanic-Latino household member, lack access to a generator, live in a rented home, have larger households with more young children, fewer adults over 65, lower household incomes, been impacted by the COVID-19 pandemic, and more family characteristics that made life harder. We also fit a logistic regression model to assess the role of outage, household, and community characteristics in predicting differences in capabilities burden. Our results offer insights into enumerating the consequences of utility service disruptions on households, which can inform more targeted and equitable resilience strategies.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

Modifying the Asynchronous Jacobi Method for Data Corruption Resilience

Moving scientific computation from high-performance computing (HPC) and cloud computing (CC) environments to devices on the edge, i.e., physically near instruments of interest, has received tremendous interest in recent years. Such edge computing environments can operate on data in situ, offering enticing benefits over data aggregation to HPC and CC facilities that include avoiding costs of transmission, increased data privacy, and real-time data analysis. Because of the inherent unreliability of edge computing environments, new fault-tolerant approaches must be developed before the benefits of edge computing can be realized. Motivated by algorithm-based fault tolerance, a variant of the asynchronous Jacobi (ASJ) method is developed that achieves resilience to data corruption by rejecting solution approximations from neighbor devices according to a bound derived from convergence theory. Numerical results on a two-dimensional Poisson problem show that the new rejection criterion, along with a novel approximation to the shortest path length on which the criterion depends, restores convergence for the ASJ variant in the presence of certain types data corruption. Numerical results are obtained for when the singular values in the analytic bound are approximated. Additional linear systems are also explored, one with a more dense sparsity pattern and one that includes advection. All results indicate that successful resilience to data corruption depends on whether the bound tightens fast enough to reject corrupted data before the iteration evolution deviates significantly from that predicted by the convergence theory defining the bound. This observation generalizes to future work on algorithm-based fault tolerance for other asynchronous algorithms, including upcoming approaches that leverage Krylov subspaces.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Impact Mapping for Geospatial Reasoning and Decision Making

Objective The reported study evaluated a novel approach to aiding geospatial reasoning and decision making. Background Impact mapping aims to alleviate the cognitive demands of geospatial tasks in part by externalizing data in the form of an integrated decision surface. This is achieved by aggregating data across multiple sources of information and visualizing their combined utility rather than objective measurements or individual utility. Previous research has shown that geospatial decisions improve when aided in this manner, but it remains unknown if dynamic decision making, often plagued by fatigue and anchoring bias, would benefit similarly. Method The experiment implemented a systematic manipulation of the presence of a composite impact map and the number of attributes present in a two-stage disaster relief, resource allocation task to investigate when and how impact mapping is beneficial or deleterious to decision makers. Results The presence of the composite impact map increased the utility of selected sites, increased re-planning decisions, reduced information display views, and reduced workload. Generally, the effect of the composite impact map was greater when participants were asked to evaluate more attributes. Conclusion Composite impact maps appear to improve repeated geospatial reasoning and minimize anchoring bias because they alleviate the cognitive demands otherwise necessary to interpret and maintain information from multiple attributes. Application Data visualization techniques, such as impact mapping, can improve repeated geospatial decision making in environments that include high cognitive demand.

Illingworth, David A.↗

DOE OSTI2021

ShopperWorkerRatio

These data characterize spatial and temporal variation in the ratio between shoppers and workers in public places - points of interest - in the United States between 2019 and 2020. The underlying data on foot traffic to public places is collected by SafeGraph, a commercial data aggregator. We estimate the ratio of shoppers to workers based on recorded visit duration.These data may be useful for understanding how use of public spaces have changed during the Coronavirus pandemic, and more generally for understanding activity patterns in public.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2022

The ABCflux database: Arctic–boreal CO 2 flux observations and ancillary information aggregated to monthly time steps across terrestrial ecosystems

Past efforts to synthesize and quantify the magnitude and change in carbon dioxide (CO 2 ) fluxes in terrestrial ecosystems across the rapidly warming Arctic–boreal zone (ABZ) have provided valuable information but were limited in their geographical and temporal coverage. Furthermore, these efforts have been based on data aggregated over varying time periods, often with only minimal site ancillary data, thus limiting their potential to be used in large-scale carbon budget assessments. To bridge these gaps, we developed a standardized monthly database of Arctic–boreal CO 2 fluxes (ABCflux) that aggregates in situ measurements of terrestrial net ecosystem CO 2 exchange and its derived partitioned component fluxes: gross primary productivity and ecosystem respiration. The data span from 1989 to 2020 with over 70 supporting variables that describe key site conditions (e.g., vegetation and disturbance type), micrometeorological and environmental measurements (e.g., air and soil temperatures), and flux measurement techniques. Here, we describe these variables, the spatial and temporal distribution of observations, the main strengths and limitations of the database, and the potential research opportunities it enables. In total, ABCflux includes 244 sites and 6309 monthly observations; 136 sites and 2217 monthly observations represent tundra, and 108 sites and 4092 observations represent the boreal biome. The database includes fluxes estimated with chamber (19 % of the monthly observations), snow diffusion (3 %) and eddy covariance (78 %) techniques. The largest number of observations were collected during the climatological summer (June–August; 32 %), and fewer observations were available for autumn (September–October; 25 %), winter (December–February; 18 %), and spring (March–May; 25 %). ABCflux can be used in a wide array of empirical, remote sensing and modeling studies to improve understanding of the regional and temporal variability in CO 2 fluxes and to better estimate the terrestrial ABZ CO 2 budget.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2021

A Multi-criteria CCUS Screening Evaluation of the Gulf of Mexico, USA - Supplementary Data

The Wendt et al. study aims to incorporate multiple and disparate carbon capture, utilization, and storage (CCUS) decision-making criteria into a systematic, quantitative analytical approach to help identify areas with potentially high suitability to serve as offshore CO2 storage or EOR regions. Spatially-distributed data from publicly-available sources within the Gulf of Mexico (GOM) study area (limited to federal waters; state waters were not evaluated) was compiled using the U.S. Department of Energy's (DOE) National Energy Technology Laboratory (NETL) Cumulative Spatial Impact Layers™ (CSIL) tool to easily aggregate data based on evenly-distributed grids across the study region set at a resolution of approximately 25 square miles (65 square kilometers). The data included in this Microsoft Excel™ workbook provide the aggregated scores for each grid point across the study domain as well the weighting for each criterion under the four scenarios evaluated.

CCUS↗

DOE OSTI2021

Accelerating Multigrid-based Hierarchical Scientific Data Refactoring on GPUs

Rapid growth in scientific data and a widening gap between computational speed and I/O bandwidth make it increasingly infeasible to store and share all data produced by scientific simulations. Instead, we need methods for reducing data volumes: ideally, methods that can scale data volumes adaptively so as to enable negotiation of performance and fidelity tradeoffs in different situations. Multigrid-based hierarchical data representations hold promise as a solution to this problem, allowing for flexible conversion between different fidelities so that, for example, data can be created at high fidelity and then transferred or stored at lower fidelity via logically simple and mathematically sound operations. However, the effective use of such representations has been hindered until now by the relatively high costs of creating, accessing, reducing, and otherwise operating on such representations. We describe here highly optimized data refactoring kernels for GPU accelerators that enable efficient creation and manipulation of data in multigrid-based hierarchical forms. We demonstrate that our optimized design can achieve up to 250 TB/s aggregated data refactoring throughput—83% of theoretical peak—on 1024 nodes of the Summit supercomputer. We showcase our optimized design by applying it to a large-scale scientific visualization workflow and the MGARD lossy compression software.

Data refactoring↗

DOE OSTI2021

Accelerating Multigrid-based Hierarchical Scientific Data Refactoring on GPUs

Chen, Jieyang↗

DOE OSTI2024

Federated Machine Learning-Based Anomaly Detection System for Synchrophasor Network Using Heterogeneous Data Sets: Preprint

Synchrophasor technology is widely deployed in the energy management system to monitor the grid health at micro level and perform necessary corrective actions in real time; however, integrated phasor devices and data aggregators are exposed to several cybersecurity threats. This paper proposes a federated ML(FML)-based ADS to detect several data integrity attacks in the synchrophasor network. The proposed approach integrates the horizontal FML technique and consists of substation-based local models and a control center-based global model. The proposed methodology includes training local models using heterogeneous data sets that include network and grid information and updating the global model through multiple iterations by sharing model gradients. Finally, the trained global model is applied to identify cyberattacks, normal operation, and physical events. To validate the proof of concept, we used synthetic data sets generated by Mississippi State University and Oak Ridge National Laboratory for training and testing the classification models using the National Renewable Energy Laboratory's high performance computing resources. Our experimental results, computed through several performance measures, reveal that the proposed approach shows consistent performance during the binary, three-class, and multiclass classifications while ensuring privacy of synchrophasor data.

anomaly detection system↗