SEARCH · Engineering Papers

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Assessing the large-scale drivers of precipitation in the northeastern United States via linear orthogonal decomposition

This study examines the linear orthogonal modes associated with monthly precipitation in the northeastern United States, from CESM1 LENS (35 ensemble members, 1979–2005) and two reanalysis datasets (ERA5, 1950–2018 and NOAA-CIRES-DOE 20CRv3, 1950–2015). Calendar months are aggregated together, and any linear trends in data are removed. Using region-averaged precipitation anomaly time series and monthly anomalies for several global 2D atmospheric fields, a linear orthogonal decomposition method is implemented to iteratively extract time series (based on field and geographic location) of absolute maximum correlation. Linear modes associated with this method are then projected onto the full set of 2D fields to provide physical insight into the mechanisms involved in generating precipitation. In this region, the first mode is associated with vapor transport from the Atlantic seaboard, the second mode is characterized by westward vapor transport associated with extratropical cyclones, and the third mode captures vapor transport from the Gulf of Mexico during the fall and winter. However, the third mode is less robust in the spring and summer. Results are generally consistent across the datasets, and applying multiple linear regression with the linear modes to predict the precipitation anomalies produces R-squared values of around 0.54–0.65 for CESM1 LENS, and around 0.58–0.88 for reanalysis, with the lowest values generally in the spring and late summer. The influence of low-frequency climate variability on the modes is considered for CESM1 LENS, and the modes in late winter can be predicted with some success via a combination of several, prominent large-scale teleconnection patterns.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Parametric dynamic mode decomposition for reduced order modeling

Dynamic Mode Decomposition (DMD) is a model-order reduction approach, whereby spatial modes of fixed temporal frequencies are extracted from numerical or experimental data sets. The DMD low-rank or reduced operator is typically obtained by singular value decomposition of the temporal data sets. For parameter-dependent models, as found in many multi-query applications such as uncertainty quantification or design optimization, the only parametric DMD technique developed was a stacked approach, with data sets at multiple parameter values were aggregated together, increasing the computational work needed to devise low-rank dynamical reduced-order models. Here in this paper, we present two novel approach to carry out parametric DMD: one based on the interpolation of the reduced-order DMD eigen-pair and the other based on the interpolation of the reduced DMD (Koopman) operator. Numerical results are presented for diffusion-dominated nonlinear dynamical problems, including a multiphysics radiative transfer example. All three parametric DMD approaches are compared.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

Toward Molecule-Specific Geochemistry of Heavy Ends: Application to the Upstream Oil Industry

Detailed molecular analysis of the composition of petroleum is typically performed on only the light (volatile) fraction, using analyses involving gas chromatography (GC). Heavier (nonvolatile) components such as kerogen, bitumen, and asphaltenes, which are relevant to conventional reservoirs and particularly to unconventional gas shale and tight oil formations, are not amenable to analysis by GC and instead are typically studied by more phenomenological techniques. This report describes a suite of analytical techniques that can provide sufficient average molecular information on the heavy ends (defined as the nonvolatile fraction) to be useful for a variety of industrial applications, expanding the discipline of molecule-specific geochemistry into the non-GC amenable fractions of petroleum materials. Heavy ends are composed of the elements C, H, N, S, and O, and the bonding environments of each can be measured by various solid state spectroscopies: nuclear magnetic resonance (NMR) and infrared (IR) spectroscopies for C, H, and O; X-ray absorption near-edge structure (XANES) spectroscopy for S and N. Additionally, advances in mass spectrometry, including techniques such as laser desorption laser ionization mass spectrometry (L2MS), can be used to measure the molecular weight and dominant molecular architecture of asphaltenes. Moreover, higher-order structures can be measured: samples of kerogen with representative pore geometry can be prepared, and those pore systems are analyzed with techniques such as X-ray diffraction and neutron scattering. Colloidal structures of asphaltenes can be measured with a variety of techniques, which reveal a hierarchical aggregation structure. These analyses provide sufficient structural data to enable structure–property and structure–function relationships to be determined, which can be applied to the characterization of conventional and unconventional resources. In gas shale and tight oil formations, which contain abundant inorganic minerals as well as kerogen, diffuse reflectance infrared Fourier transform spectroscopy (DRIFTS) is used for simultaneous analysis of mineralogy, kerogen content, and maturity, typically performed at the wellsite. In conventional reservoirs, spatial variations in the asphaltene concentration measured by downhole fluid analysis logging tools are being interpreted with a recently developed equation of state to address upstream issues such as reservoir connectivity, tar mat formation, and fault block migration. Furthermore, as more heavy end analyses are developed, further scientific knowledge and industrial applications of heavy end geochemistry are anticipated.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2025

Linking structural and rheological memory in disordered soft materials

Linking the macroscopic flow properties and nanoscopic structure is a fundamental challenge to understanding, predicting, and designing disordered soft materials. Under small stresses, these materials are soft solids, while larger loads can lead to yielding and the acquisition of plastic strain, which adds complexity to the task. In this work, we connect the transient structure and rheological memory of a colloidal gel under cyclic shearing across a range of amplitudes via a generalized memory function using rheo-X-ray photon correlation spectroscopy (rheo-XPCS). Our rheo-XPCS data show that the nanometer scale aggregate-level structure recorrelates whenever the change in recoverable strain over some interval is zero. The macroscopic recoverable strain is therefore a measure of the nano-scale structural memory. We further show that yielding in disordered colloidal materials is strongly heterogeneous and that memories of prior deformation can exist even after the material has been subjected to flow.

Kamani, Krutarth M. [Univ. of Illinois at Urbana-C↗

DOE OSTI2017

Pattern Discovery in Internet Background Radiation

Internet Background Radiation (IBR) is observed in empty network address spaces. No traffic should arrive there, but it does in overwhelming quantities, gathering evidences of attacks, malwares and misconfigurations. The study of IBR helps to detect spreading network problems, common vulnerabilities and attack trends. However, network traffic data evolves quickly and is of high volume and diversity, i.e., an outstanding big data challenge. When used to assist network security, it also requires the online classification of dynamic streaming data. In this paper, we introduce an AGgregation & Mode (AGM) vector to represent network traffic. The AGM format characterizes IP hosts by extracting aggregated and mode values of IP header fields, and without inspecting payloads. We performed clustering and statistical analysis to explore six months of IBR from 2012 with the AGM mapping. The discovered patterns allow building a classification of IBR, which identifies phenomena that have been actively polluting the Internet for years. The AGM representation is light and tailored for monitoring and pattern discovery. We show that AGM vectors are suitable to analyze large volumes of network traffic: they capture permanent operations, such as long term scanning, as well as bursty events from targeted attacks and short term incidents.

Iglesias, Felix↗

DOE OSTI2014

MO-G-18C-05: Real-Time Prediction in Free-Breathing Perfusion MRI

Purpose: The aim is to minimize frame-wise difference errors caused by respiratory motion and eliminate the need for breath-holds in magnetic resonance imaging (MRI) sequences with long acquisitions and repeat times (TRs). The technique is being applied to perfusion MRI using arterial spin labeling (ASL). Methods: Respiratory motion prediction (RMP) using navigator echoes was implemented in ASL. A least-square method was used to extract the respiratory motion information from the 1D navigator. A generalized artificial neutral network (ANN) with three layers was developed to simultaneously predict 10 time points forward in time and correct for respiratory motion during MRI acquisition. During the training phase, the parameters of the ANN were optimized to minimize the aggregated prediction error based on acquired navigator data. During realtime prediction, the trained ANN was applied to the most recent estimated displacement trajectory to determine in real-time the amount of spatial Results: The respiratory motion information extracted from the least-square method can accurately represent the navigator profiles, with a normalized chi-square value of 0.037±0.015 across the training phase. During the 60-second training phase, the ANN successfully learned the respiratory motion pattern from the navigator training data. During real-time prediction, the ANN received displacement estimates and predicted the motion in the continuum of a 1.0 s prediction window. The ANN prediction was able to provide corrections for different respiratory states (i.e., inhalation/exhalation) during real-time scanning with a mean absolute error of < 1.8 mm. Conclusion: A new technique enabling free-breathing acquisition during MRI is being developed. A generalized ANN development has demonstrated its efficacy in predicting a continuum of motion profile for volumetric imaging based on navigator inputs. Future work will enhance the robustness of ANN and verify its effectiveness with human subjects. Research supported by National Institutes of Health National Cancer Institute Grant R01 CA159471-01.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2019

LBNL-Load

This software package is a set of Python-based tools for forecasting future time series of electrical power demand for a granular set of customer types in California, based on historical data on individual customers' demand, combined with aggregate forecasts of demand growth.

Gerke, Brian↗

DOE OSTI2026

ComStock Measure Documentation: High-Efficiency Rooftop Unit

Building on the 3-year End-Use Load Profiles project to calibrate and validate the U.S. Department of Energy's ResStock and ComStock models, this work produces national data sets that enable cities, states, utilities, and other stakeholders to answer a broad range of questions regarding their commercial building stock. ComStock is a highly granular, bottom-up model that uses various data sources, statistical sampling methods, and advanced building energy simulations to estimate the annual subhourly energy consumption of the commercial building stock across the United States. The "baseline" model intends to represent the U.S. commercial building stock as it existed in 2018. The methodology of the baseline model is discussed in the ComStock Reference Documentation. The goal of this work is to develop energy efficiency and demand flexibility measures that cover market-ready technologies and study their mass adoption impact on the baseline building stock. "Measures" refers to various "what-if" scenarios that can be applied to buildings. The results for the baseline and measure scenario simulations are published in public data sets that provide insights into building stock characteristics, operational behaviors, utility bill impacts, and annual and sub-hourly energy usage by fuel type and end use. This report describes the modeling methodology for a single ComStock measure scenario - high-efficiency rooftop unit (RTU) - and briefly introduces key results. The full public data set can be accessed on the Comstock data lake or via the Data Viewer at comstock.nlr.gov. The public data set enables users to create custom aggregations of results for their use case (e.g., filter to a specific county or building type).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2026

ComStock Measure Documentation: Variable-Speed Pumps

Building on the 3-year End-Use Load Profiles project to calibrate and validate the U.S. Department of Energy's ResStock and ComStock models, this work produces national data sets that enable cities, states, utilities, and other stakeholders to answer a broad range of questions regarding their commercial building stock. ComStock is a highly granular, bottom-up model that uses various data sources, statistical sampling methods, and advanced building energy simulations to estimate the annual subhourly energy consumption of the commercial building stock across the United States. The "baseline" model intends to represent the U.S. commercial building stock as it existed in 2018. The methodology of the baseline model is discussed in the ComStock Reference Documentation. The goal of this work is to develop energy efficiency and demand flexibility measures that cover market-ready technologies and study their mass adoption impact on the baseline building stock. "Measures" refers to various "what-if" scenarios that can be applied to buildings. The results for the baseline and measure scenario simulations are published in public data sets that provide insights into building stock characteristics, operational behaviors, utility bill impacts, and annual and sub-hourly energy usage by fuel type and end use. This report describes the modeling methodology for a single ComStock measure scenario - variable speed pumps - and briefly introduces key results. The full public data set can be accessed on the ComStock data lake or via the Data Viewer at comstock.nlr.gov. The public data set enables users to create custom aggregations of results for their use case (e.g., filter to a specific county or building type).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NASA NTRS1974

Earth Observatory Satellite system definition study. Report no. 3: Design/cost tradeoff studies. Appendix A: EOS program WBS dictionary. Appendix B: EOS mission functional analysis

The work breakdown structure (WBS) dictionary for the Earth Observatory Satellite (EOS) is defined. The various elements of the EOS program are examined to include the aggregate of hardware, computer software, services, and data required to develop, produce, test, support, and operate the space vehicle and the companion ground data management system. A functional analysis of the EOS mission is developed. The operations for three typical EOS missions, Delta, Titan, and Shuttle launched are considered. The functions were determined for the top program elements, and the mission operations, function 2.0, was expanded to level one functions. Selection of ten level one functions for further analysis to level two and three functions were based on concern for the EOS operations and associated interfaces.

Source record↗

NASA NTRS2011

What We've Learned from Approximately 12 Years of MISR Aerosol Observations

The NASA Earth Observing System's Multi-angle Imaging SpectroRadiometer (MISR) has been imaging the planet about once per week for nearly 12 years. These data contain a wealth of information about the current state, as well as the variability of Earth's surface and atmosphere. For airborne particles specifically, horizontal distributions of aerosol optical depth, aerosol type, and near-source aerosol plume height are now routinely derived. The results are being applied to a wide range of problems, from constraining aerosol direct radiative forcing of climate and assessing its seasonal and secular trends, to material transports of smoke and dust, to mapping volcanic ash plumes and near-surface aerosol pollution. The value of these data is multiplied many times by combining it with data from other sources, and using the aggregate to constrain models. This talk will summarize the latest developments, and will also discuss current work aimed at making further advancements in these areas.

Kahn, Ralph↗

DOE OSTI2018

Novel capabilities for examining and summarizing hierarchical and geographic resiliency profiles

The geospatial research and development team in the National and Homeland Security Division at Idaho National Laboratory was tasked with providing tools to derive insight from the substantial amount of data currently available - and continuously being produced - associated with the critical infrastructure of the US. This effort is in support of the Department of Homeland Security, whose mission includes the protection of this infrastructure and the enhancement of its resilience to hazards, both natural and human. We present geovisual-analytics-based approaches for analysis of vulnerabilities and resilience of critical infrastructure, designed so that decision makers, analysts, and infrastructure owners and managers can manage risk, prepare for hazards, and direct resources before and after an incident that might result in an interruption in service. Our designs are based on iterative discussions with DHS leadership and analysts, who in turn will use these tools to explore and communicate data in partnership with utility providers, law enforcement, and emergency response and recovery organizations, among others. In most cases these partners desire summaries of large amounts of data, but increasingly, our users seek the additional capability of focusing on, for example, a specific infrastructure sector, a particular geographic region, or time period, or of examining data in a variety of generalization or aggregation levels. These needs align well with tenets of in-formation-visualization design; in this paper, selected applications among those that we have designed are described and positioned within geovisualization, geovisual analytical, and information visualization frameworks.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2012

Linking NASA Environmental Data with a National Public Health Cohort Study and a CDC On-Line System to Enhance Public Health Decision Making

The overall goal of this study is to address issues of environmental health and enhance public health decision making by utilizing NASA remotely-sensed data and products. This study is a collaboration between NASA Marshall Space Flight Center, Universities Space Research Association (USRA), the University of Alabama at Birmingham (UAB) School of Public Health and the Centers for Disease Control and Prevention (CDC) National Center for Public Health Informatics. The objectives of this study are to develop high-quality spatial data sets of environmental variables, link these with public health data from a national cohort study, and deliver the linked data sets and associated analyses to local, state and federal end-user groups. Three daily environmental data sets were developed for the conterminous U.S. on different spatial resolutions for the period 2003-2008: (1) spatial surfaces of estimated fine particulate matter (PM2.5) exposures on a 10-km grid utilizing the US Environmental Protection Agency (EPA) ground observations and NASA s MODerate-resolution Imaging Spectroradiometer (MODIS) data; (2) a 1-km grid of Land Surface Temperature (LST) using MODIS data; and (3) a 12-km grid of daily Solar Insolation (SI) and maximum and minimum air temperature using the North American Land Data Assimilation System (NLDAS) forcing data. These environmental datasets were linked with public health data from the UAB REasons for Geographic and Racial Differences in Stroke (REGARDS) national cohort study to determine whether exposures to these environmental risk factors are related to cognitive decline and other health outcomes. These environmental national datasets will also be made available to public health professionals, researchers and the general public via the CDC Wide-ranging Online Data for Epidemiologic Research (WONDER) system, where they can be aggregated to the county, state or regional level as per users need and downloaded in tabular, graphical, and map formats. The linkage of these data provides a useful addition to CDC WONDER, allowing public health researchers and policy makers to better include environmental exposure data in the context of other health data available in this online system. It also substantially expands public access to NASA data, making their use by a wide range of decision makers feasible.

Al-Hamdan, Mohammad↗

DOE OSTI2026

IntraShuffler: A Privacy Preserving Framework for Heterogeneous DP Federated Learning

Heterogeneous Differential Privacy (HDP) in Federated Learning (FL) allows clients to select individual privacy budgets () according to institutional policies and data sensitivity. In practice, many HDP-FL systems employ -aware server aggregation to improve model utility by re-weighting client updates according to their declared privacy budgets. However, gradient updates in FL retain structural patterns induced by non-independent and identically-distributed (non-IID) data, and these additional signals exposed by -aware aggregation create new opportunities for inference by an honest-but-curious server. In this work, we first show that a server equipped with gradient denoising and surrogate modeling can mount a Privacy Inference Attack that infers distributional attributes of clients and links updates from the same client across training rounds, measured via surrogate inference accuracy and linkage success, under realistic knowledge constraints. The Shuffle-Model has been widely studied as a defense against such inference risks by anonymizing update sources, but it is fundamentally incompatible with HDP-FL -aware aggregation. To address this challenge, we propose IntraShuffler, a middleware defense framework designed for HDP-FL systems. IntraShuffler introduces a privacy-aware shuffling mechanism that groups clients into privacy-compatible buckets and performs parameter-level shuffling within each bucket to disrupt persistent gradient structure while preserving -aware aggregation. Experiments across four different datasets show that IntraShuffler reduces gradient recoverability by over 60% and decreases surrogate inference accuracy from 0.78 to 0.33 while maintaining comparable model utility across multiple FL aggregation rules.

Riya, Farhin Farhad [ORNL]↗

DOE OSTI2025

Front-of-Meter Model Results

These files contains aggregations of key variables from the NREL Distributed Wind Futures Study using full parcel level data. These variables describe total technical and economic potential for distributed wind turbine deployment. Aggregations are available at the (1) county, (2) zipcode (zip code tabulation area or zcta), and (3) US Census block group level. Each scenario is coded with the scenario name (e.g., baseline) and year (e.g., 2022). Those files postfixed with 'econpot' contain results for only those parcels that are economically viable while the files postfixed with 'techpot' include results for all parcels that are technically feasible. Hence these correspond to technoeconomic and technical potential respectively. The data are available as CSV or Geopackage. Columns in the files are as follows: * geoid: geographic identifier (FIPS code or similar) * min_techpot_sum_kw: technical potential for all parcels in kW using turbines downsized to demand when appropriate * max_techpot_sum_kw: technical potential for all parcels in kW without downsizing turbines * aep_sum_kwh: annual energy production estimate in kWh * cf_mean_ratio: mean capacity factor * lcoe_mean_cents_per_kwh: mean levelized cost of energy for parcels in geography in cents per kWh * lcoe_std_cents_per_kwh: standard deviation of the above * parcel_area_sum_acres: total area of viable parcels in acres * n_turbines: number of cited turbines (one per viable parcel currently) Note: These are preliminary results from the full-parcel 2024 update of the Distributed Wind Energy Futures study. Please take care when making use of the data, and feel free to contact the team with any questions. Full documentation in support of these data is in progress and will follow.

17 WIND ENERGY↗

DOE OSTI2025

Behind-the-Meter Model Results

17 WIND ENERGY↗

DOE OSTI2024

Individual Data Sparsity in Smart Thermostat Big Data: Impacts on Modeling Thermostat Use Behavior Dynamics

This study explores the impacts of the sparsity of individual thermostat interaction data on modeling thermostat use behavior dynamics using a dataset of over 100,000 smart thermostats. In developing a data-driven model of Thermal Frustration Theory (TFT), we investigate the challenges and trade-offs in clustering occupant data to enhance predictive accuracy. Our findings reveal that a single, aggregated model fails to capture the diversity of occupant behaviors, resulting in extremely poor prediction performance. Conversely, excessive clustering exacerbates data sparsity, undermining model reliability. By identifying an optimal clustering strategy, we achieve a balance that significantly improves the prediction of manual setpoint changes during demand response (DR) events, enhancing energy management and occupant comfort

Fannon, David↗

NASA NTRS1988

Geobotanical determination of aggregate source material using Airborne Thematic Mapper imagery

The possible use of vegetation to discriminate parent materials for suitability as aggregate source material is examined. Airborne Thematic Mapper data of two test sites representing potential alluvial and residual source areas in central California were analyzed. It is found that the most useful images were composites that included principal components bands and a Perpendicular Vegetation Index band. The image processing demarcated species compositional differences which characterized a shale site and revealed differences in an alluvial site caused by moisture stress due to aggregate size and sorting.

Minor, Timothy↗