Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

eMosaic: Electrification Mosaic Platform for Grid Informed Smart Charging Management (Final Scientific/Technical Report)

ABB (Prime Contractor), in collaboration with its partners at the Utah State University (USU), Idaho National Laboratory, Rocky Mountain Power (RMP), and Electric Power Engineers (EPE), have performed research, development, and wide scale demonstration of a scalable and resilient Electrification Mosaic (eMosaic) platform for Smart Charge Management (SCM) for Electric Vehicle Infrastructure. Work was completed under DE EE0009194, titled “eMosaic Electrification Mosaic Platform for Grid Informed Smart Charging Management”, funded by the US Department of Energy. The project members developed algorithms that provide localized and bulk grid services and that reduce and stabilize costs all the way down the supply chain to the PEV owner through SCM. This platform aggregates telemetry from multiple data sources as pieces of the larger picture including personal, private fleet or transportation EVs, fast chargers and other supply equipment, weather service information, and geographically distributed charging sites such as public lots, garage and retail, and private or shared usage depots. ABB and the project team designed, tested, and improved a charging management system at local/edge and cloud levels. The ultimate objective of the project was to convincingly demonstrate that the developed secure eMosaic plat-form can be readily and favorably adopted by diverse utilities and site owners at scale. This was achieved through a demonstration plan with field deployment at several physical sites across 4 states and additional scalable simulation from high fidelity charging models.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Data-Driven and Multiscale Modeling of DNA-Templated Dye Aggregates

Dye aggregates are of interest for excitonic applications, including biomedical imaging, organic photovoltaics, and quantum information systems. Dyes with large transition dipole moments (μ) are necessary to optimize coupling within dye aggregates. Extinction coefficients (ε) can be used to determine the μ of dyes, and so dyes with a large ε (>150,000 M –1 cm –1 ) should be engineered or identified. However, dye properties leading to a large ε are not fully understood, and low-throughput methods of dye screening, such as experimental measurements or density functional theory (DFT) calculations, can be time-consuming. In order to screen large datasets of molecules for desirable properties (i.e., large ε and μ), a computational workflow was established using machine learning (ML), DFT, time-dependent (TD-) DFT, and molecular dynamics (MD). ML models were developed through training and validation on a dataset of 8802 dyes using structural features. A Classifier was developed with an accuracy of 97% and a Regressor was constructed with an R2 of above 0.9, comparing between experiment and ML prediction. Using the Regressor, the ε values of over 18,000 dyes were predicted. The top 100 dyes were further screened using DFT and TD-DFT to identify 15 dyes with a μ relative to a reference dye, pentamethine indocyanine dye Cy5. Two benchmark MD simulations were performed on Cy5 and Cy5.5 dimers, and it was found that MD could accurately capture experimental results. The results of this study exhibit that our computational workflow for identifying dyes with a large μ for excitonic applications is effective and can be used as a tool to develop new dyes for excitonic applications.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2026

Solar and Battery Storage Permitting and Siting Requirements - Solar Prize Round 7 (CRADA Final Report)

The purpose of this research project was to aggregate zoning and permitting data for utility-scale solar PV and battery energy storage systems. The ultimate goal of this collaboration is to lower solar and battery energy storage system soft costs by streamlining regulatory due diligence and reducing the burden of conducting feasibility assessments for solar and storage systems. The below sections describe the specific research completed by NLR (the contractor) in furtherance of the agreement with Vanox (the participant).

14 SOLAR ENERGY↗
DOE OSTI2021

Empirical prediction of saline water atomization pressure loss and spray phase change using local flow pressure analysis

A pressure analysis technique was developed to obtain spray spatial evaporation profile in a thermal desalination process. The technique can replace temperature-based evaporation measurement methods that are challenged by liquid-phase interference. It also provides an alternative to optical methods with high cost and complexity. Fundamental analysis was developed to determine local humidity ratios from changes in local static and dynamic pressures. The analysis was applied to data obtained from an external mixing air-assist atomizer. An empirical model was developed for the pressure loss in atomization process and was used in the analysis of humidity ratio. The results were used to develop empirical correlations for local humidity ratio in spray mediums using inlet conditions. Utilizing inlet conditions to predict evaporation profiles is an important contribution as it eliminates the need to take local spray measurements. Obtaining local data is often difficult and expensive; Our method and correlation circumvent the need for that. Finally, the correlations presented in this paper apply to saline water sprays with total dissolved salt of 0-10 wt%. The salinity-specific correlations are accurate to within ±10% of experimental data. The individual models were aggregated into a unified correlation for salinities within 0-10 wt%. The unified correlation is accurate to within ±18% of experimental data.

42 ENGINEERING↗
DOE OSTI2023

A riming‐dependent parameterization of scattering by snowflakes using the self‐similar Rayleigh–Gans approximation

Abstract Riming is a key process of precipitation formation in ice‐containing clouds, but quantifying riming from observations is challenging, limiting our ability to evaluate the riming process in numerical weather models. One challenge for radar observations is that riming changes both the physical properties (mass, area cross‐section) and scattering properties of ice particles. These changes need to be implemented consistently as a function of riming in radar forward operators, which are required for retrievals and model evaluation in observation space. In this study, mass–size, cross‐section area–size, and backscattering cross‐section relations are developed as a function of the normalized rime mass for aggregates composed of various monomer types (columns, dendrites, needles, plates, and rosettes). The proposed framework allows us to simulate scattering properties of aggregated ice particles consistently as a function of riming in retrievals and radar forward operators. The parameterizations are developed from a large data set of simulated rimed aggregates of different sizes and monomer crystal types. The backscattering cross‐section parameterization (the “riming‐dependent parameterization”) is evaluated for radar frequencies of 35.6 and 94.0 GHz and is based on the Self‐Similar Rayleigh–Gans approximation (SSRGA), which is increasingly used to calculate microwave scattering of ice crystals and snowflakes. Compared with parameterizations from the literature that do not consider riming, the riming‐dependent parameterization leads to significantly smaller biases in terms of backscattering cross‐section. When using the particle masses and scattering properties of the individual particles simulated by the aggregation and riming model as a reference, the bias of our parameterization is below 1 dB when integrating over an exponential particle size distribution with sizes from 0.1–10 mm.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Quota management in dCache or making a perfectly normal file system normal

dCache (https://dcache.org) is a highly scalable storage system providing location-independent access to data. The data are stored across multiple data servers as complete files presented to the end-user via a single-rooted namespace. From its inception, dCache has been designed as a caching disk buffer to a tertiary tape storage system with the assumption that the latter has virtually unlimited capacity. dCache can also be configured as a disk-only storage system with no tape backend. Owing to the idea that a tape resource is infinite, or purely physically limited by budget considerations, the system has never provided for any restrictions on how much data can be stored on tape. Likewise, in the disk-only configuration, the capacity of the system is only limited by the aggregate disk capacity of the data servers. In a multi-user environment, however, this has become problematic. This presentation will describe the design and implementation of a user- and group-based quota system, that allows to manage tape and disk space allocations, as part of dCache namespace.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Proximity Portability and in Transit , M-to-N Data Partitioning and Movement in SENSEI [Book Chapter]

In high-performance parallel in situ processing, the term in transit processing refers to those configurations where data must move from a producer to a consumer that runs on separate resources. In the context of parallel and distributed computing on an HPC platform one of the central challenges is to determine a mapping of data from producer ranks to consumer ranks. This problem is complicated by the heterogeneity that arises in producer-consumer pairs, such as when producer and consumer codes have different levels of concurrency, different scaling characteristics, or different data models. The resulting mapping and movement of data from M producer to N consumer ranks can have a significant impact on aggregate application performance, particularly when the data consumer requires only a subset of the overall data for its task. This chapter focuses on the design considerations that underlie SENSEI’s implementation to this challenging problem. These design considerations extend the core SENSEI architecture and include ideas like the need to accommodate flexibility in the choice of different partitioning methods, the ability for a data consumer to request and receive only the subset of data needed for its particular operation, and the ability to leverage any of several different data transport tools. The idea of proximity portability, being able to use different data transport methods as part of an in transit workflow, is illustrated through the use of three different transport layers where switching from one transport tool to another is accomplished with only a configuration file change. Here, the chapter also includes a performance analysis summary showing the performance gains that are possible in terms of multiple metrics, such as memory footprint, time to solution, and amount of data moved, when using optimized partitioners in an in transit setting, gains that are made possible by the implementation shaped by specific design considerations.

Bethel, E. Wes↗
DOE OSTI2023

Decoding Ethiopian Abodes: Towards Classifying Buildings by Occupancy Type Using Footprint Morphology

Building occupancy classification plays a crucial role in urban planning, disaster management, and population modeling. Traditional methods often require extensive field surveys or detailed datasets, which can be time-consuming, expensive, and may yield incomplete or erroneous data. In this paper, we present a novel approach for classifying buildings as residential or non-residential using only building footprint data. By extracting geometric shape derivatives that characterize building morphology, we developed a high-accuracy classification model employing a combination of unsupervised and supervised learning methods. We utilized open-source data from Open Street Map, aggregating it to create binary labels for buildings based on their respective human use type. Our approach demonstrates the potential for scalability without the need for additional data sources other than building footprints and labels, offering a more efficient solution for building occupancy classification.

Adams, Daniel↗
DOE OSTI2020

The Manufacturing Data and Machine Learning Platform: Enabling Real-time Monitoring and Control of Scientific Experiments via IoT

IoT devices and sensor networks present new opportunities for measuring, monitoring, and guiding scientific experiments. Sensors, cameras, and instruments can be combined to provide previously unachievable insights into the state of ongoing experiments. However, IoT devices can vary greatly in the type, volume, and velocity of data they generate, making it challenging to fully realize this potential. Indeed, synergizing diverse IoT data streams in near-real time can require the use of machine learning (ML). In addition, new tools and technologies are required to facilitate the collection, aggregation, and manipulation of sensor data in order to simplify the application of ML models and in turn, fully realize the utility of IoT devices in laboratories. Here we will demonstrate how the use of the Argonne-developed Manufacturing Data and Machine Learning (MDML) platform can analyze and use IoT devices in a manufacturing experiment. MDML is designed to standardize the research and operational environment for advanced data analytics and AI-enabled automated process optimization by providing the infrastructure to integrate AI in cyber-physical systems for in situ analysis. We will show that MDML is capable of processing diverse IoT data streams, using multiple computing resources, and integrating ML models to guide an experiment.

Machine Learning↗
DOE OSTI2023

De-Risking Large-Scale PV Systems Through Data Analytics

Large-scale photovoltaic system performance analysis is being conducted within the US Department of Energy's-sponsored PV Fleet Performance Data Initiative. This collaboration with commercial PV system owners collects and evaluates PV field performance data, and provides reports on aggregated results. Drawing on over 2200 sites across the US and over 24,000 separate PV inverters we have collected in excess of 8.3 gigawatts (GW) of performance data, representing 6-7% of the entire US installed PV capacity. A mixture of utility-scale and large commercial systems are represented, averaging 4.1 megawatts (MW) in size and 5 years in age. Initial results show average system degradation rates at -0.75% / year, which is slightly higher than historically reported module-level values of -0.5%/year. We also found that the availability of systems averaged 97.7%, which is lower than the typical 99% uptime assumed by many project economic forecasts. Given these results, we compared monthly performance with expected production values, based on satellite weather data and a simple PVWatts performance model. We found that systems were performing within 10% of monthly expectation over 90% of the time, with a fleet average vs expected monthly value of 0.994. We also evaluated the impact of extreme weather events on system performance, and found a range of short-term and longer-term performance effects ranging from grid outage, system downtime, module damage and accelerated long-term degradation rate.

analysis↗
DOE OSTI2025

Short-Term Electric Load Forecasting for a Residential Household in Alaska

Accurate short-term load forecasting at a fine scale is essential for demand response programs, peak shaving, and load-shedding strategies [1]. While traditionally, only aggregate short-term consumption data was available, advanced metering infrastructure (AMI) now provides data at the individual consumer level [1]. There is increasing interest in utilizing this data for short-term load forecasting (from an hour to a few days) to optimize grid operations. Electricity consumption in individual households is highly influenced by residents’ personal behaviors [2]. As a result, unlike aggregate loads, electrical power usage in single households often shows significant volatility, making meter-level load forecasting for individual users particularly challenging [3], [4]. Deep learning methods, with their strong ability to model nonlinear data, have become popular for improving the accuracy of household electricity consumption forecasting [4]. Notably, the Long ShortTerm Memory (LSTM) has attracted significant attention [5], [6].

42 ENGINEERING↗
DOE OSTI2022

Storm and Annual Time Scale Hydrological Data for the Russian River Watershed 1996-2022

These files contains observed and simulated hydrological data (discharge, precipitation) that have been aggregated to the storm-event scale and to the annual time scale for the Russian River Watershed (RRW), California. Observed data are obtained from 12 different USGS hydrological stations located throughout the watershed, and discharge is simulated using the GR5H hourly hydrological model. Precipitation data are obtained from the NASA NLDAS NOAH Community Land Surface Model. All simulated and observed discharge are aggregated to the storm and annual time scales. We investigated changes in watershed hydrological conditions in the Russian River Watershed, a Mediterranean, drought prone, wildfire-adapted ecosystem, following eleven wildfires that occurred from 2017-2020. We ask two research questions: 1) How do interacting wildfire events, drought, and atmospheric rivers impact hydrological conditions of the watershed and in particular streamflow?, and 2) What percentage of new and/or cumulative wildfire disturbance is required to initiate hydrological change? We hypothesize that sub-watersheds of the RRW Mediterranean ecoregion have not burned beyond an intrinsic, and still unknown, threshold required to initiate change. Using a series of paired burned/unburned catchments nested within the larger watershed, we examined temporal and spatial patterns of pre-and-post wildfire water hydrology using a rainfall runoff hydrological model compared with data.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

A Guideline for the Assessment of Uniaxial Creep and Creep-Fatigue Data and Models

Of primary concern to FE practitioners is a determination of which constitutive models are the “best”, capable of reproducing the mechanisms expected in an intended design accurately; as well as what experimental datasets are proper or “best” to use for fitting the constitutive parameters needed for the model(s) of interest. In addition, the inherent uncertainty in experiments and the repeatability and stability of extrapolations (for long-lived FE components) is a major concern. The goal of this project is to develop a guideline for the assessment of uniaxial creep and creep-fatigue data and models. The research objectives are RO1) develop an aggregated database of creep and creep-fatigue data for P91 steel and 316 stainless steel and perform a systematic assessment of the uncertainty and integrity of the datasets. RO2) employ the database to benchmark existing creep and creep-fatigue models to evaluate the model’s ability to predict standard and nonstandard boundary conditions.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2023

Assessing individual movement, habitat use, and behavior of non-breeding marine birds in relation to prey availability in the US Atlantic

Resource availability is a key factor driving marine bird movements and distributions, but direct information on prey availability is difficult to obtain at relevant scales. We present novel methods for describing multi-scale trophic associations, combining movement analyses of marine birds with estimates of forage fish surface aggregations from digital aerial survey data and species occupancy from bottom trawl survey data. We analyzed satellite telemetry data from northern gannets Morus bassanus , red-throated loons Gavia stellata , and long-tailed ducks Clangula hyemalis in the US Atlantic during the non-breeding period. Using discrete-time hidden Markov models to distinguish area-restricted (i.e. putative foraging) from transit movements, we examined how environmental factors influence movement, and how forage fish species distributions and surface aggregations influence habitat use by gannets and loons that have greater dietary reliance. Our results suggest that chlorophyll a concentration significantly affected movement behavior across species, highlighting the importance of higher-productivity areas around estuaries during colder months when regional productivity is low. Though variable across species and seasons, spatial cross-correlation analysis revealed that herring species (Family Clupeidae), including Atlantic menhaden Brevoortia tyrannus , may be important resources; it also showed positive spatial correlations with forage fish aggregations. This suggests that prey patch dynamics and factors driving aggregation formation may be as important as species composition. However, spatial patterns were generally low (<0.3), suggesting a mismatch in spatiotemporal resolution, exemplifying the challenges in quantifying trophic relationships in marine systems. Disentangling predator-prey relationships is critical to understanding the mechanisms driving marine bird behavior in rapidly changing marine systems.

Environmental Sciences & Ecology↗
DOE OSTI2024

A consistent dataset for the net income distribution for 190 countries and aggregated to 32 geographical regions from 1958 to 2015

Abstract. Data on income distributions within and across countries are becoming increasingly important for informing analysis of income inequality and understanding the distributional consequences of climate change. While datasets on income distribution collected from household surveys are available for multiple countries, these datasets often do not represent the same concept of inequality (or income concept) and therefore make comparisons across countries, over time and across datasets difficult. Here, we present a consistent dataset of income distributions across 190 countries from 1958 to 2015 measured in terms of net income. We complement the observed values in this dataset with values imputed from a summary measure of the income distribution, specifically the Gini coefficient. For the imputation, we use a recently developed nonparametric principal-component-based approach that shows an excellent fit to data on income distributions compared to other approaches. We also present another version of this dataset aggregated from the country level to 32 geographical regions. Our dataset is developed for the purpose of calibrating models such as integrated human–Earth system models with detailed data on income distributions. This dataset will enable more robust analysis of income distribution at multiple scales. The latest version of our data are available on Zenodo: https://doi.org/10.5281/zenodo.7093997 (Narayan et al., 2022b).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Development of a Benchmark Eddy Flux Evapotranspiration Dataset for Evaluation of Satellite-Driven Evapotranspiration Models Over the CONUS

A large sample of ground-based evapotranspiration (ET) measurements made in the United States, primarily from eddy covariance systems, were post-processed to produce a benchmark ET dataset. The dataset was produced primarily to support the intercomparison and evaluation of the OpenET satellite-based remote sensing ET (RSET) models and could also be used to evaluate ET data from other models and approaches. OpenET is a web-based service that makes field-delineated and pixel-level ET estimates from well-established RSET models readily available to water managers, agricultural producers, and the public. The benchmark dataset is composed of flux and meteorological data from a variety of providers covering native vegetation and agricultural settings. Flux footprint predictions were developed for each station and included static flux footprints developed based on average wind direction and speed, as well as dynamic hourly footprints that were generated with a physically based model of upwind source area. The two footprint prediction methods were rigorously compared to evaluate their relative spatial coverage. Data from all sources were post-processed in a consistent and reproducible manner including data handling, gap-filling, temporal aggregation, and energy balance closure correction. The resulting dataset included 243,048 daily and 5,284 monthly ET values from 194 stations, with all data falling between 1995 and 2021. We assessed average daily energy imbalance using 172 flux sites with a total of 193,021 days of data, finding that overall turbulent fluxes were understated by about 12% on average relative to available energy. Multiple linear regression analyses indicated that daily average latent energy flux may be typically understated slightly more than sensible heat flux. This dataset was developed to provide a consistent reference to support evaluation of RSET data being developed for a wide range of applications related to water accounting and water resources management at field to watershed scales.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Simulation of PV Variability as a Function of PV Generation and Plant Size

The deployment of photovoltaic (PV) systems continues to show significant expansion; however, this growth has brought added attention to issues around the variability of the solar resource. Both spatial and temporal variability exist. Temporal scales can range from the sub-second to multiyear, whereas spatial scales can range from a few meters to tens of kilometers. There are multiple methods described in the literature to quantify PV variability at various spatial and temporal scales. This study focuses on short-term temporal variability and uses similar approaches with the addition of PV plant size a parameter to quantify variability. The method employed here incorporates the normalization of clear-and cloudy-sky conditions and PV plant size to quantify nominal variability metrics. The distribution and fluctuations of these metrics provide relevant information that is useful for system operations. The National Solar Radiation Database (NSRDB) is used to simulate PV variability as a function of PV generation and plant size. Hypothetical but realistic system information at 33 locations is used to model PV generation by feeding NSRDB solar irradiance data to the National Renewable Energy Laboratory’s System Advisor Model (SAM). Over the selected region, it is found that the aggregated ramp rates for the 1-minute data are associated with standard deviations ranging from 0.002–0.055 on a daily basis; however, hourly intervals induce higher aggregated ramp rates than the other timescales. Even though minute-to-minute variations are significant for the 1-minute time-scale, the standard deviation aggregated into a daily metric is smaller because of the cancellation of values.

irradiance↗
DOE OSTI2021

Simulation of PV Variability as a Function of PV Generation and Plant Size: Preprint

The deployment of photovoltaic (PV) systems continues to show significant expansion; however, this growth has brought added attention to issues around the variability of the solar resource. Both spatial and temporal variability exist. Temporal scales can range from the sub-second to multiyear, whereas spatial scales can range from a few meters to tens of kilometers. There are multiple methods described in the literature to quantify PV variability at various spatial and temporal scales. This study focuses on short-term temporal variability and uses similar approaches with the addition of PV plant size a parameter to quantify variability. The method employed here incorporates the normalization of clear- and cloudy-sky conditions and PV plant size to quantify nominal variability metrics. The distribution and fluctuations of these metrics provide relevant information that is useful for system operations. The National Solar Radiation Database (NSRDB) is used to simulate PV variability as a function of PV generation and plant size. Hypothetical but realistic system information at 33 locations is used to model PV generation by feeding NSRDB solar irradiance data to the National Renewable Energy Laboratory’s System Advisor Model (SAM). Over the selected region, it is found that the aggregated ramp rates for the 1-minute data are associated with standard deviations ranging from 0.002–0.055 on a daily basis; however, hourly intervals induce higher aggregated ramp rates than the other timescales. Even though minute-to-minute variations are significant for the 1-minute timescale, the standard deviation aggregated into a daily metric is smaller because of the cancellation of values.

41 EE - Solar Energy Technologies Office (EE-4S)↗