SEARCH · Engineering Papers

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

All sources NASA NTRS DOE OSTI

NASA NTRS2016

Information Management Platform for Data Analytics and Aggregation (IMPALA) System Design Document

The System Design document tracks the design activities that are performed to guide the integration, installation, verification, and acceptance testing of the IMPALA Platform. The inputs to the design document are derived from the activities recorded in Tasks 1 through 6 of the Statement of Work (SOW), with the proposed technical solution being the completion of Phase 1-A. With the documentation of the architecture of the IMPALA Platform and the installation steps taken, the SDD will be a living document, capturing the details about capability enhancements and system improvements to the IMPALA Platform to provide users in development of accurate and precise analytical models. The IMPALA Platform infrastructure team, data architecture team, system integration team, security management team, project manager, NASA data scientists and users are the intended audience of this document. The IMPALA Platform is an assembly of commercial-off-the-shelf (COTS) products installed on an Apache-Hadoop platform. User interface details for the COTS products will be sourced from the COTS tools vendor documentation. The SDD is a focused explanation of the inputs, design steps, and projected outcomes of every design activity for the IMPALA Platform through installation and validation.

Carnell, Andrew↗

DOE OSTI2019

Proactive Data Containers for Scientific Storage (Final Report)

Emerging HPC systems are expected to be deployed with an unprecedented level of complexity, due to a deep system memory/storage hierarchy and heterogeneity of the storage hardware. This hierarchy is expected to range from CPU cache through several levels of volatile memory to nonvolatile memory, traditional hard disks, and tape. Simple and efficient methods of data management and movement through this hierarchy is critical for scientific applications using exascale systems. Existing storage system and I/O (SSIO) technologies face severe challenges in dealing with these requirements. POSIX and MPI I/O standards that are the basis for existing I/O libraries and parallel file systems present fundamental challenges in the areas of scalable metadata operations, semantics-based data movement performance tuning, asynchronous operation, and support for scalable consistency of distributed operations. Moving toward new paradigms for SSIO in the extreme-scale era, we have proposed to investigate novel object-based data abstractions and storage mechanisms that take advantage of the deep storage hierarchy and enable proactive automated performance tuning. In order to achieve these overarching goals, we initiated an effort to develop a fundamental new data abstraction, called Proactive Data Containers (PDC). A PDC is a container within a locus of storage (memory, NVRAM, disk, etc.) that stores science data in an object-centric manner. Managing data as objects enables powerful optimization opportunities for data movement and transformations. The R&D focus of this project are: 1) formulation of object-oriented PDCs and their mapping in different levels of the exascale storage hierarchy; 2) efficient strategies for moving data in deep storage hierarchies using PDCs; 3) techniques for transforming and reorganizing data based on application requirements; and 4) novel analysis paradigms for enabling data transformations and user-defined analysis on data in PDCs. Toward achieving these overarching goals, we designed an object-centric application programing interface (API) for HPC, scalable metadata management for object-centric storage systems, and data movement optimizations such as Data Elevator for moving data between two levels of storage devices and TAPIOCA for efficient aggregation of data on compute nodes. We then implemented several components of the PDC system. They include metadata management, data placement services, remote procedure calls, data aggregation, etc. We have put them together into the overall PDC framework.

97 MATHEMATICS AND COMPUTING↗

NASA NTRSDate not supplied

Observed Cloud Type‐Sorted Cloud Property and Radiative Flux Changes With the Degree of Convective Aggregation From CERES Data

Explore the source record for details and available documents.

Kuan‐Man Xu↗

DOE OSTI2025

BAMCensus (The Behavior and Advanced Mobility Census Dataset Aggregator) [SWR-25-120]

This software is a high-performance tool developed in Rust for downloading and processing large-scale geospatial datasets, specifically focusing on US Census data. It is designed to address scaling limitations found in existing tools, such as R's [tidycensus](https://walker-data.com/tidycensus/), by providing performant streaming dataset JOIN operations between various US Census datasets (like ACS and LEHD) and their corresponding geometries stored on the TIGER/Lines web server. The tool automates the process of joining these data sources, returning aggregated data to the user based on a specified census GEOID type. The tool automates the process of joining these data sources, returning aggregated data to the user based on a specified census GEOID type. Its primary motivation stems from the need for a high-performance solution to combine spatial datasets with graph traversals within the context of mobility analysis tooling being developed at NREL's Behavior and Advanced Mobility (BAM) group.

Fitzgerald, Robert [National Renewable Energy Labo↗

NASA NTRSDate not supplied

Identifying Trends in Landing Profiles After Long Duration Spaceflight

BACKGROUND As NASA’s mission programs begin to shift from Low Earth Orbit (LEO) to planetary mission profiles, the importance of understanding the phenomena a human experience when returning from spaceflight is of increased importance. The purpose of this study was to assess trends across individuals in the symptoms experienced during the landing process of long duration ISS missions. METHODS Data for this study is an aggregation of mission level data from forms used by crew surgeons at R+0, electronic medical records, notes from weekly Space Medicine Operation Team meetings, records in the Shuttle Mission Data Archive, and data collected by research teams in the Human Research Program (HRP). Aggregated data was used in descriptive analysis of the trends in medical events and countermeasure use. The data was also used to identify high value targets for emphasis in on going surveillance efforts. RESULTS Completeness of reporting varied greatly over time and across missions dependent on information sources available for data extraction. Landing forms provided the most complete data but still missed or left unconfirmed, several, common symptoms, and countermeasures. Data aggregation did show to cover gaps within individual data sources. Commonly reported symptoms included nystagmus, vertigo, and nausea. Common protocols for fluid loading and meclizine are reflected within the data set. DISCUSSION The data set generated in this project provides a foundation that show how symptoms of astronauts as they return to Earth varies across the long duration ISS missions. While there is an abundance of information provided within the data set, there are still inconsistencies in reporting especially in the absence of the landing day form. Further data collection practices should be standardized to ease the burden on crew surgeons during R+0 exams and better collect high value variables.

Sam Jacobs↗

DOE OSTI2014

Atmospheric Moisture Budget and Spatial Resolution Dependence of Precipitation Extremes in Aquaplanet Simulations

This study investigates the resolution dependency of precipitation extremes in an aqua-planet framework. Strong resolution dependency of precipitation extremes is seen over both tropics and extra-tropics, and the magnitude of this dependency also varies with dynamical cores. Moisture budget analyses based on aqua-planet simulations with the Community Atmosphere Model (CAM) using the Model for Prediction Across Scales (MPAS) and High Order Method Modeling Environment (HOMME) dynamical cores but the same physics parameterizations suggest that during precipitation extremes moisture supply for surface precipitation is mainly derived from advective moisture convergence. The resolution dependency of precipitation extremes mainly originates from advective moisture transport in the vertical direction. At most vertical levels over the tropics and in the lower atmosphere over the subtropics, the vertical eddy transport of mean moisture field dominates the contribution to precipitation extremes and its resolution dependency. Over the subtropics, the source of moisture, its associated energy, and the resolution dependency during extremes are dominated by eddy transport of eddies moisture at the mid- and upper-troposphere. With both MPAS and HOMME dynamical cores, the resolution dependency of the vertical advective moisture convergence is mainly explained by dynamical changes (related to vertical velocity or omega), although the vertical gradients of moisture act like averaging kernels to determine the sensitivity of the overall resolution dependency to the changes in omega at different vertical levels. The natural reduction of variability with coarser resolution, represented by areal data averaging (aggregation) effect, largely explains the resolution dependency in omega. The thermodynamic changes, which likely result from non-linear feedback in response to the large dynamical changes, are small compared to the overall changes in dynamics (omega). However, after excluding the data aggregation effect in omega, thermodynamic changes become relatively significant in offsetting the effect of dynamics leading to reduce differences between the simulated and aggregated results. Compared to MPAS, the simulated stronger vertical motion with HOMME also results in larger resolution dependency. Compared to the simulation at fine resolution, the vertical motion during extremes is insufficiently resolved/parameterized at the coarser resolution even after accounting for the natural reduction in variability with coarser resolution, and this is more distinct in the simulation with HOMME. To reduce uncertainties in simulated precipitation extremes, future development in cloud parameterizations must address their sensitivity to spatial resolution as well as dynamical cores.

aqua-planet↗

DOE OSTI2025

Aggregated carbon dioxide flux and hydrometeorology data from an Amazonian palm swamp peatland in Peru: 2018, 2019, and 2022

This dataset contains eddy covariance carbon dioxide flux and hydrometeorological measurements made in an Amazonian palm swamp peatland near Iquitos, Peru. These data have been aggregated from half-hourly observations that are available from AmeriFlux (https://ameriflux.lbl.gov/; site PE-QFR). These data files are CSV (comma separated values) format and can be imported using Matlab, R, or Excel. Three full years of data are reported (2018, 2019 and 2022) during which time there were large differences in annual net ecosystem carbon dioxide exchange. The gap was caused by instrument malfunction and extended delays in repairs because of the Covid-19 pandemic. This research was conducted to better understand the carbon cycle of tropical peatlands, and was supported by the U.S. Department of Energy, Office of Science, Office of Biological and Environmental Research, Terrestrial Ecosystem Science Program, under Award Number DE-SC0020167.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Real-Time Lifetime Prediction of Semiconductor Devices Using Hardware-in-the-Loop

This paper presents a unique approach to enable real-time lifespan prediction of semiconductor power modules using a Hardware-in-the-Loop (HIL) system. By integrating the module's overall loss characteristics-specifically switching and conduction losses-with a thermoelectric model of the thermal management system, this research demonstrates that the model can dynamically estimates the junction temperature profile of the semiconductor devices in response to a changing torque demand profile for the motor drive system. This capability enables continuous monitoring of the module's operational time and cumulative stress induced on the devices to compute accumulated remaining lifetime or time-to-failure (TTF). This study provides an architectural framework for the HIL system with high-fidelity component models of multiple physical domains, allowing simulation of dynamic behaviors of a closely-coupled motor drive system. The advanced real-time computation and measurement functionalities of the HIL system allow for both dynamic lifetime calculations based on simulated data and aggregate lifetime predictions utilizing historical data. Moreover, this paper details an algorithm that not only computes cumulative damage but also synthesizes these data into a comprehensive aggregated lifetime metric. This methodology can enhance the maintenance scheduling strategies and operational reliability of semiconductor devices in critical applications, ultimately extending their service life while optimizing performance.

hardware-in-the-loop (HIL)↗

DOE OSTI2021

INGENIOUS Thermal Conductivity Measurement Source Categorization

Thermal conductivity (TC) data taken for different wells at a specified drill depth. This is an abridged version of the complete SMU heat flow database, downloaded from the SMU node of the NGDS at the beginning of INGENIOUS (approximately April 2021), and filtered to the INGENIOUS study area. This National Geothermal Data System (NGDS) project aggregates geothermal data collected and curated by the SMU Geothermal Laboratory and its partner organizations. All columns in this database are the same as the SMU database, except for 2 additions associated with this project. Repeated columns are for data correlation purposes. Column descriptions and data types are the same as previous iterations of the SMU database. The new values that are the addition are two new columns developed as part of the INGENIOUS project: INGENIOUS TC Value | INGENIOUS notes INGENIOUS notes are individual notes that were written for specific data points during the analysis process. There are not always notes associated with each input value. INGENIOUS TC Value includes 4 values: 1. Assumed Measured These are values that are assumed to be measured thermal conductivity values, either within a specific well or within the same study region. Many of these have either a published reference, a reported standard deviation, or a unique thermal conductivity value. 2. Data release - assumed measured These are values in the SMU database that are from proprietary data that were added to the SMU database and are labeled as data release for their reference. These values were searched for in person at the SMU Geothermal Laboratory as well as virtual examination of data available on the NGDS. For many of these, there are reported thermal conductivity values associated with the heat flow data in the database, but no specific table or reference to measurements in the original data release files. 3. Known measured These are values that have a reported measurement, either as an original file in the SMU data files on the NGDS or a reported table in a publication. In the rare circumstances, Maria Richards or David Blackwell confirmed measurement. Confirmation of measurement would be written in the INGENIOUS notes column. 4. Unmeasured Unmeasured values are those that are known to be unmeasured, either estimated from another report or no information given. In the SMU database, there are wells that have a heat flow but no thermal conductivity. These are categorized as unmeasured. There are also heat flow values that are stated to have estimated or generalized average thermal conductivity values for the region and rock type. Because these are known to be unmeasured, they are categorized as such. 5. Blank Blank values are either A quality or X quality. These quality values are stated in the INGENIOUS notes. These values were not going to change associated with the heat flow analysis, so these were not examined.

15 GEOTHERMAL ENERGY↗

DOE OSTI2025

Monthly Quality-filtered Aggregation of NOAA Climate Data Record (CDR) of AVHRR Leaf Area Index (LAI) and Fraction of Absorbed Photosynthetically Active Radiation (FAPAR), Version 5

This dataset contains gridded monthly Leaf Area Index (LAI) derived from the daily NOAA Climate Data Record (CDR) of AVHRR Leaf Area Index (LAI) and Fraction of Absorbed Photosynthetically Active Radiation (FAPAR), Version 5. This data record spans from 1981 to 2018 using data from eight NOAA polar orbiting satellites: NOAA-7, -9, -11, -14, -16, -17, -18 and -19. The data are projected on a 0.05 degree x 0.05 degree global grid, as in the original CDR. The original CDR is one of the Land Surface CDR Version 5 products produced by the NASA Goddard Space Flight Center (GSFC) and the University of Maryland (UMD), which is accompanied by algorithm documentation, data flow diagram and source code for the NOAA CDR Program. This dataset is in the netCDF-4 file format following ACDD and CF Conventions. This dataset has applied quality assurance information to only include "OK" data from the original CDR in the monthly aggregation.

Vermote, Eric [NASA Goddard Space Flight Center (G↗

DOE OSTI2026

Monthly Quality-filtered Aggregation of NOAA Climate Data Record (CDR) of AVHRR (Version 5) and VIIRS (Version 1) Leaf Area Index (LAI) and Fraction of Absorbed Photosynthetically Active Radiation (FAPAR)

This dataset contains gridded monthly Leaf Area Index (LAI) derived from the daily NOAA Climate Data Record (CDR) of AVHRR (Version 5) and VIIRS (Version 1) Leaf Area Index (LAI) and Fraction of Absorbed Photosynthetically Active Radiation (FAPAR). This data record spans from 1981 to 2024 using data from NOAA polar orbiting satellites: NOAA-7, -9, -11, -14, -16, -17, -18, -19 and S-NPP. The data are projected on a 0.05 degree x 0.05 degree global grid, as in the original CDR. The original CDR is one of the Land Surface CDR products produced by the NASA Goddard Space Flight Center (GSFC) and the University of Maryland (UMD), which is accompanied by algorithm documentation, data flow diagram and source code for the NOAA CDR Program. This dataset is in the netCDF-4 file format following ACDD and CF Conventions. This dataset has applied quality assurance information to only include "OK" data from the original CDR in the monthly aggregation.

Vermote, Eric [NASA Goddard Space Flight Center (G↗

DOE OSTI2022

Accelerating shared file checkpoint with local burst buffers

A data management system and method for accelerating shared file checkpointing. Written application data is aggregated in an application data file created in a local burst buffer memory at a compute node, and an associated data mapping built index to maintain information related to the offsets into a shared file at which segments of the application data is to be stored in a parallel file system, and where in the buffer those segments are located. The node asynchronously transfers a data file containing the application data and the associated data mapping index to a file server for shared file storage. The data management system and method further accelerates shared file checkpointing in which a shared file, together with a map file that specifies how the shared file is to be distributed, is asynchronously transferred to local burst buffer memories at the nodes to accelerate reading of the shared file.

Gooding, Thomas↗

NASA NTRS1979

Trace elements in the Allende meteorite. IV - Amoeboid olivine aggregates

INAA data for Ca, Sc, Hf, La, Ce, Sm, Eu, Tb, Yb, Lu, Os, Ir, Ru, Na, Cl, Br, Fe, Mn, Cr, Co, Au, As, and Sb are presented for ten amoeboid aggregates from the Allende meteorite. Only one lacks olivine. Seven of the remainder, as a group, have cosmic proportions of refractory lithophile and siderophile elements and appear to have formed when coarse-grained Allende inclusion material underwent partial reaction with a low-temperature nebular gas and mixture with FeO-rich olivine. The other two have highly fractionated abundances of refractory elements relative to one another compared to Cl chondrites, including Group II REE patterns, and probably formed by the mixing of fine-grained Allende inclusion material with FeO-rich olivine. Non-refractory siderophile components are also different in composition in each type of amoeboid olivine aggregate.

Grossman, L.↗

DOE OSTI2014

Commercial and Multifamily Building Tenant Energy Usage Aggregation and Privacy

In a number of cities and states, building owners are required to disclose and/or benchmark their building energy use. This requires the building owner to possess monthly whole-building energy usage information, which can be challenging for buildings in which individual tenants have their own utility meters and accounts with the utility. Some utilities and utility regulators have turned to aggregation of customer data as a way to give building owners the whole-building energy usage data while protecting customer privacy. However, no utilities or regulators appear to have conducted a concerted statistical, cybersecurity, and privacy analysis to justify the level of aggregation selected. Therefore, the Tennant Data Aggregation Task was established to help utilities address these issues and provide recommendations as well as a theoretical justification of the aggregation threshold. This study is focused on the use case of submitting data for ENERGY STAR Portfolio Manager (ESPM), but it also looks at other potential use cases for monthly energy consumption data.

data aggergation↗

NASA NTRS1980

Global crop forecasting

The needs for and remote sensing means of global crop forecasting are discussed, and key results of the Large Area Crop Inventory Experiment (LACIE) are presented. Current crop production estimates provided by foreign countries are shown often to be inadequate, and the basic elements of crop production forecasts are reviewed. The LACIE project is introduced as a proof-of-concept experiment designed to assimilate remote sensing technology, monitor global wheat production, evaluate key technical problems, modify the technique accordingly and demonstrate the feasibility of a global agricultural monitoring system. The global meteorological data, sampling and aggregation techniques, Landsat data analysis procedures and yield forecast procedures used in the experiment are outlined. Accuracy assessment procedures employed to evaluate LACIE technology performance are presented, and improvements in system efficiency and capacity during the three years of operation are pointed out. Results of LACIE estimates of Soviet, U.S. and Canadian wheat production are presented which demonstrate the feasibility and accuracy of the remote-sensing approach for global food and fiber monitoring.

Macdonald, R. B.↗

DOE OSTI2024

PV System Availability from Commercial and Utility-Scale Systems [Slides]

14 SOLAR ENERGY↗

DOE OSTI2024

PV Inverter Availability from the U.S. PV Fleet

In the PV Fleet Performance Data Initiative, we partner with photovoltaic (PV) fleet owners to collect time-series PV production data, and publish aggregated, anonymized results. An assessment of system availability is conducted on 1128 systems which passed our data quality checks, and include cumulative energy meter data. Overall inverter availability is low in the first 6 months of system performance before reaching steady-state by the end of the first year. System-level aggregated data shows a median (P50) system availability of 0.99, and a lower P90 value of 0.95. A dependence on system size is also identified, with better inverter availability results for smaller PV systems. Potential causes of this effect may include the selection of inverter itself: smaller inverters 6kW-250kW showed better average availability than inverters 300kW-5MW. The elimination of string combiner boxes and lower energy impact when one particular inverter goes off-line are potential benefits of a string inverter-based PV system architecture. DNV also analyzed availability data from over 1100 operating systems and found similar trends. DNV's P50 industry guidance on expected availability has been updated to reflect the data and the following observations: utility scale systems have lower availability than DG systems, availability is lower in first year compared to subsequent years, and that actual availability is lower than expected.

fleet↗

DOE OSTI2024

Carbon Storage Site Mapping Inquiry Tool (MapIT)

To date, 48 projects, consisting of 139 wells, are currently under review with the Environmental Protection Agency’s (EPA) Underground Injection Control (UIC) Program for Class VI – wells used for geologic sequestration of carbon dioxide. The number of applications submitted is expected to increase in coming years with the increase of the 45Q tax credit available to projects that initiate construction prior to 2033. The amount of data collected to submit a Class VI permit is vast, and often disparate, coming from state, federal, and commercial entities, as well as field-specific data collected within an area of interest. When preparing for site selection and permitting, the initial aggregation of relevant public data can be time intensive. The Carbon Storage Site Mapping Inquiry tool (MapIT) was created to support and accelerate the discovery and accessibility of open-source data and information available across the USA. Data was aggregated and organized based on data types described within the EPA UIC Class VI permit documentation. The online tool enables users to explore hundreds of geospatial data layers and connect to additional external resources, leveraging API and REST services where possible to ensure updates to data in real time. MapIT enables users to explore state and federal data related to geologic, geophysical, structural, hydrologic, and contextual information. In addition to displaying spatial data and linking to external resources, MapIT leverages custom widgets to ensure that internal data and external data are discoverable and accessible. The widgets connect users to resources such as the USGS publications and the USGS Earthquake Catalog based on a user-defined location. This talk will describe data aggregation workflows, data types, data preparation, and tool development for MapIT. The Carbon Storage Site Mapping Inquiry Tool and underlying database are valuable, intuitive resources that empower government, academic, commercial and industry stakeholders to explore, analyze, and acquire carbon storage related data.

Morkner, Paige↗