SEARCH · Engineering Papers

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42

All sources NASA NTRS DOE OSTI

NASA NTRS2012

Constructing a Database from Multiple 2D Images for Camera Pose Estimation and Robot Localization

The LMDB (Landmark Database) Builder software identifies persistent image features (landmarks) in a scene viewed multiple times and precisely estimates the landmarks 3D world positions. The software receives as input multiple 2D images of approximately the same scene, along with an initial guess of the camera poses for each image, and a table of features matched pair-wise in each frame. LMDB Builder aggregates landmarks across an arbitrarily large collection of frames with matched features. Range data from stereo vision processing can also be passed to improve the initial guess of the 3D point estimates. The LMDB Builder aggregates feature lists across all frames, manages the process to promote selected features to landmarks, and iteratively calculates the 3D landmark positions using the current camera pose estimations (via an optimal ray projection method), and then improves the camera pose estimates using the 3D landmark positions. Finally, it extracts image patches for each landmark from auto-selected key frames and constructs the landmark database. The landmark database can then be used to estimate future camera poses (and therefore localize a robotic vehicle that may be carrying the cameras) by matching current imagery to landmark database image patches and using the known 3D landmark positions to estimate the current pose.

Wolf, Michael↗

NASA NTRSDate not supplied

High Performance Access to Archival Data Stored in HDF4 and HDF5 on Cloud Object Stores Without Reformatting the Files

Cloud computing offers numerous advantages for users of extensive Earth science data collections. These benefits encompass direct online access to data files and granules from any location, scalable access supporting parallel computing workflows, and flexible computing tools enabling innovative experimentation with processing techniques. However, older archival file formats designed for distinct computing systems hinder efficient access to decade-long time-series data when compared to data stored in modern cloud-optimized formats like Web Object Stores (WOS), exemplified by Amazon Web Services’ Simple Storage Service (S3). We describe DMR++ (Dataset Metadata Response plus plus), a technology facilitating efficient access to HDF5 (Hierarchical Data Format, version 5) and HDF4 files stored on WOS systems without requiring data reformatting. DMR++ achieves performance comparable to technologies like Zarr while preserving the original file structure, a substantial benefit considering the vast quantity of archival files held by organizations such as NASA. Moreover, DMR++ typically outperforms cloud-optimized versions of HDF5. Essentially an XML (Extensible Markup Language) document usually stored alongside the described data, DMR++ can also be generated on-the-fly but is generally created during data staging to the WOS. Archival files that use HDF4/5 often store large arrays of numerical data. The data in these files is often compressed, typically reducing their size by a factor of four or more. To achieve efficient access to portions of those arrays, they are 'chunked' into smaller sub-arrays, each individually compressed. The chunk size is a compromise, where spinning disks can efficiently access data in smaller chunks while S3 favors larger chunks. A simple optimization of aggregating smaller chunks that are stored adjacently, transferring them in a single access and then individually decompressing them will improve performance. NASA data pose an additional challenge: special Application Programmer Interface (API) libraries are often needed to compute some variables. These libraries are incompatible with WOS environments. Our solution involves storing computed values in the DMR++ document or a companion file, making them accessible like other variables and eliminating the need for specialized APIs. We outline specific optimizations for both satellite grid and swath data stored in HDF4-EOS2 (Earth Observing System).

James Gallagher↗

DOE OSTI2014

Radiation effects in concrete for nuclear power plants, Part II: Perspective from micromechanical modeling

The need to understand and characterize the effects of neutron irradiation on concrete has become urgent because of the possible extension of service life of many nuclear power generating stations. Current knowledge is primarily based on a collection of data obtained in test reactors. These results are inherently difficult to interpret because materials and testing conditions are inconsistent. A micromechanical approach based on the Hashin composite sphere model is presented to derive a first-order separation of the effects of radiation on cement paste and aggregate, and, also, on their interaction. Although the scarcity of available data limits the validation of the model, it appears that, without negating a possible gamma-ray induced effect, the neutron-induced damage and swelling of aggregate plays a predominant role on the overall concrete expansion and the damage of the cement paste. Finally, the radiation-induced volumetric expansion (RIVE) effects can also be aided by temperature elevation and shrinkage in the cement paste.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

NASA NTRS1990

The role of data management in discipline-independent data visualization

The common data format (CDF) is described in terms of its support applications for the database management of visualization systems. The CDF is a self-describing data abstraction technique for the storage and manipulation of multidimensional data that are based on block structures. The discipline-independent approach is designed to manage, manipulate, archive, display, and analyze data, and can be applied to heterogeneous equipment communicating different data structures over networks. An improved CDF version incorporates a hyperplane access allowing random aggregate access to subdimensional blocks within a multidimensional variable. The visualization pipeline is also discussed, which controls the flow of data and permits the visualization of different classes of data representation techniques. The system is found to accommodate a large variety of scientific data structures and large disk-based data sets.

Treinish, Lloyd A.↗

NASA NTRS1998

Scale in Remote Sensing and GIS: An Advancement in Methods Towards a Science of Scale

The term "scale", both in space and time, is central to remote sensing and Geographic Information Systems (GIS). The emergence and widespread use of GIS technologies, including remote sensing, has generated significant interest in addressing scale as a generic topic, and in the development and implementation of techniques for dealing explicitly with the vicissitudes of scale as a multidisciplinary issue. As science becomes more complex and utilizes databases that are capable of performing complex space-time data analyses, it becomes paramount that we develop the tools and techniques needed to operate at multiple scales, to work with data whose scales are not necessarily ideal, and to produce results that can be aggregated or disaggregated ways that suit the decision-making process. Contemporary science is constantly coping with compromises, and the data available for a particular study rarely fit perfectly with the scales at which the processes being investigated operate, or the scales that policy-makers require to make sound, rational decisions. This presentation discusses some of the problems associated with scale as related to remote sensing and GIS, and describes some of the questions that need to be addressed in approaching the development of a multidisciplinary "science of scale". Techniques for dealing with multiple scaled data that have been developed or explored recently are described as a means for recognizing scale as a generic issue, along with associated theory and tools that can be of simultaneous value to a large number of disciplines. These can be used to seek answers to a host of interrelated questions in the interest of providing a formal structure for the management and manipulation of scale and its universality as a key concept from a multidisciplinary perspective.

Quattrochi, D. A.↗

DOE OSTI2019

NGEE Arctic Meteorological Data from Instrumented PCDC Stations, Council Site, Seward Peninsula, Alaska, 2018 - 2023

This dataset provides meteorological data collected at two instrumented Power, Communication, and Data Collection (PCDC) Stations, located at Council Road Mile Marker 71 (CN_MM71) and Council Road Mile Marker 68 (Rocky Top). Measurements began in June of 2018 at CN_MM71 and in September 2018 at Rocky Top and ended in 2023. Raw data included. Meteorological data include air temperature, relative humidity, wind speed and direction, and snow depth. Measurements have been aggregated to both 10-minute and 30-minute reporting intervals provided in four comma-separated (*.csv) data files.• The 30-minute data files include only the primary meteorological variables, leaf wetness, and snow depth.. The 30-minute reporting interval matches other coincident site data collections.• The 10-minute files provide the primary meteorological variables and numerous system operational parameters such as battery charge and solar and wind turbine status. This 10-minute data product is valuable for analyzing solar and wind charging system performance in arctic landscapes.Daily photographic images (09:00, 12:00, and 15:00 AKST/AKDT) of the PCDC Stations and the surrounding landscape that are captured by an adjacent web camera, are also provided.The daily 12:00 images from CN_MM71 camera have been compiled into time-lapse movies:• October 2018 – June 12, 2019 = Council_winter_2018_2019.mov• January 2019 – December 2019 = Council_2019.mp4The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Day-Ahead Forecasting with Federated LSTM to Plan Energy Sharing in a Community Microgrid

Energy balancing in microgrids is a key enabler of resilience. Community microgrids located close to each other have the added benefit of networking and sharing surplus energy, if available. Such complex decision-making runs on optimization that requires reliable short-term (up to very-short-term) forecasts of energy generation and consumption for scheduling or trading. Each microgrid may also opt to not expose their sensitive data such as consumption patterns of individual businesses or residences. This paper investigates a federated approach to dayahead forecasting that trains naive long short-term memory (LSTM) at each business in a microgrid and aggregates weights at the microgrid controller using proximal regularization. This approach ensures that the controller has access only to energy surplus/deficit and not the actual generation or consumption values, avoiding unwanted exposure of sensitive data. A community microgrid in Adjuntas, Puerto Rico with 3 businesses is selected as a case study with a laboratory-scale computing setup. A central LSTM forecaster, where sensitive data from businesses are aggregated at the controller, is implemented as a baseline for qualifying the results. This work serves as a proof-of-concept for scaling the approach to networked and nested microgrids with more complex control options.

Sundararajan, Aditya [ORNL] (ORCID:000000033577854↗

NASA NTRS2012

Using NASA Remotely Sensed Data to Help Characterize Environmental Risk Factors for National Public Health Applications

The overall goal of this study is to address issues of environmental health and enhance public health decision making by using NASA remotely sensed data and products. This study is a collaboration between NASA Marshall Space Flight Center, Universities Space Research Association (USRA), the University of Alabama at Birmingham (UAB) School of Public Health and the Centers for Disease Control and Prevention (CDC) Office of Surveillance, Epidemiology and Laboratory Services. The objectives of this study are to develop high-quality spatial data sets of environmental variables, link these with public health data from a national cohort study, and deliver the environmental data sets and associated public health analyses to local, state and federal end ]user groups. Three daily environmental data sets were developed for the conterminous U.S. on different spatial resolutions for the period 2003-2008: (1) spatial surfaces of estimated fine particulate matter (PM2.5) on a 10-km grid using US Environmental Protection Agency (EPA) ground observations and NASA's MODerate-resolution Imaging Spectroradiometer (MODIS) data; (2) a 1-km grid of MODIS Land Surface Temperature (LST); and (3) a 12-km grid of daily incoming solar radiation and maximum and minimum air temperature using the North American Land Data Assimilation System (NLDAS) data. These environmental datasets were linked with public health data from the UAB REasons for Geographic and Racial Differences in Stroke (REGARDS) national cohort study to determine whether exposures to these environmental risk factors are related to cognitive decline, stroke and other health outcomes. These environmental national datasets will also be made available to public health professionals, researchers and the general public via the CDC Wide-ranging Online Data for Epidemiologic Research (WONDER) system, where they can be aggregated to the county-level, state-level, or regional-level as per users f need and downloaded in tabular, graphical, and map formats. This provides a significant addition to the CDC WONDER online system, allowing public health researchers and policy makers to better include environmental exposure data in the context of other health data available in CDC WONDER. It also substantially expands public access to NASA data, making their use by a wide range of decisionmakers feasible.

Al-Hamdan, Mohammad↗

DOE OSTI2021

Tri-Level Scheduling Model Considering Residential Demand Flexibility of Aggregated HVACs and EVs Under Distribution LMP

Residential loads, especially heating, ventilation and air conditioners (HVACs) and electric vehicles (EVs), have great potentials to provide demand flexibility which is an attribute of grid-interactive efficient buildings (GEB). Under this new paradigm, first, EV and HVAC aggregator models are developed in this paper to represent the fleet of GEBs, in which the aggregated parameters are obtained based on a new approach of data generation and least squares parameter estimation (DG-LSPE), which can deal with heterogeneous HVACs. Then, a tri-level bidding and dispatching framework is established based on competitive distribution operation with distribution locational marginal price (DLMP). Furthermore, the first two levels form a bilevel model to optimize the aggregators’ payment and to represent the interdependency between load aggregators and the distribution system operator (DSO) using DLMP, and the third level is to dispatch the optimal load aggregation to all residents by the proposed priority list-based demand dispatching algorithm. Finally, case studies on a modified IEEE 33-Bus system illustrate three main technical reasons of payment reduction due to demand flexibility: load shift, DLMP step changes, and power losses. They can be used as general guidelines for better decision-making for future planning and operation of demand response programs.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2023

Using observational mean-flow data to drive large-eddy simulations of a diurnal cycle at the SWiFT site

Reproducing realistic date- and site-specific unsteady wind conditions in large-eddy simulations is becoming increasingly useful in wind energy. How to run a large-eddy simulation to match observed conditions, however, remains an open research question. One approach that has received considerable attention is mesoscale-to-microscale coupling, in which information about the mesoscale weather, most commonly acquired from a mesoscale numerical weather model, is passed on to a microscale model. In this paper, we demonstrate how the recently developed profile-assimilation technique, a form of mesoscale-to-microscale coupling, can be used to drive large-eddy simulations solely based on observed mean-flow profiles at a single location, bypassing the need for auxiliary mesoscale simulations. The new approach is evaluated for a diurnal cycle at the Scaled Wind Farm Technology site. Observed mean-flow profiles from the ground up to a height of 2 km are reconstructed by aggregating measurements from multiple instruments, and gaps in the data are infilled with natural neighbor interpolation. We perform nine simulations using various forcing approaches to deal with data limitations. The results show that it is indeed possible to drive microscale large-eddy simulation with observations using the profile-assimilation technique, notwithstanding large gaps in virtual potential temperature measurements. However, profile assimilation with vertical smoothing of the error between the desired and actual profiles is required. Without that smoothing, the microscale simulations develop unrealistically high turbulence levels under many situations. Finally, we show that simulated mesoscale data can account for missing observations, although care is needed as both data sources are not necessarily compatible.

17 WIND ENERGY↗

NASA NTRS1998

Scale in Remote Sensing and GIS: An Advancement in Methods Towards a Science of Scale

The term "scale", both in space and time, is central to remote sensing and geographic information systems (GIS). The emergence and widespread use of GIS technologies, including remote sensing, has generated significant interest in addressing scale as a generic topic, and in the development and implementation of techniques for dealing explicitly with the vicissitudes of scale as a multidisciplinary issue. As science becomes more complex and utilizes databases that are capable of performing complex space-time data analyses, it becomes paramount that we develop the tools and techniques needed to operate at multiple scales, to work with data whose scales are not necessarily ideal, and to produce results that can be aggregated or disaggregated in ways that suit the decision-making process. Contemporary science is constantly coping with compromises, and the data available for a particular study rarely fit perfectly with the scales at which the processes being investigated operate, or the scales that policy-makers require to make sound, rational decisions. This presentation discusses some of the problems associated with scale as related to remote sensing and GIS, and describes some of the questions that need to be addressed in approaching the development of a multidisciplinary "science of scale". Techniques for dealing with multiple scaled data that have been developed or explored recently are described as a means for recognizing scale as a generic issue, along with associated theory and tools that can be of simultaneous value to a large number of disciplines. These can be used to seek answers to a host of interrelated questions in the interest of providing a formal structure for the management and manipulation of scale and its universality as a key concept from a multidisciplinary perspective.

Quattrochi, Dale A.↗

DOE OSTI2016

Multilayer spectral graph clustering via convex layer aggregation

Multilayer graphs are commonly used for representing different relations between entities and handling heterogeneous data processing tasks. New challenges arise in multilayer graph clustering for assigning clusters to a common multilayer node set and for combining information from each layer. This paper presents a theoretical framework for multilayer spectral graph clustering of the nodes via convex layer aggregation. Under a novel multilayer signal plus noise model, we provide a phase transition analysis that establishes the existence of a critical value on the noise level that permits reliable cluster separation. The analysis also specifies analytical upper and lower bounds on the critical value, where the bounds become exact when the clusters have identical sizes. Numerical experiments on synthetic multilayer graphs are conducted to validate the phase transition analysis and study the effect of layer weights and noise levels on clustering reliability.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Understanding the trade-offs of national municipal solid waste estimation methods for circular economy policy

Policies embracing circular economy concepts have taken hold in national legislation around the world. As the number of governments and organizations adopting circular economy policies increases, so does the need for accurate and timely measurement of material resource flows. Since many countries do not have access to centrally reported municipal solid waste (MSW) data, estimation and modeling are critical in evaluating circular economy policy effectiveness. The purpose of this paper is to examine three modeling approaches estimating national MSW data in the United States, including industry-based material flow analysis, waste-extended input-output modeling, and aggregated regional waste reporting. We establish five criteria to guide the analysis through the context of policy monitoring (data quality, flow totality, update frequency, sensitivity to disruption, and product granularity) and use these criteria to analyze and score each model. We then use a literature search to identify five, internationally-implemented options for circular economy policy and determine the data and modeling components that are most helpful in evaluating policy effectiveness. Finally, we provide a crosswalk of the model scores and policy needs to inform the suitability of model selection by policy type. We found that data quality and update frequency are identified as critical components for evaluating circular economy policies within the models evaluated, and can both be fulfilled by aggregated regional waste reporting. Flow totality, sensitivity to disruption, and product granularity requirements vary by both model and policy types. While none of the evaluated models satisfy the combination of requirements for any of the five policies, industry-based material flow analysis offers flow totality for extended producer responsibility, landfill bans, and recycling rate target policies that typically require it. Here, the waste-extended input-output model can provide disruption sensitivity and product granularity as needed for policies like minimum recycled content and market restrictions. Policy developers in areas where strong centralized data collection is not an option should design policy action(s) with modeling tradeoffs in mind, including the potential hybridization of modeling approaches that may provide the most accurate national MSW estimates.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2019

Stability of Transactive Energy Market-Based Power Distribution System Under Data Integrity Attack

Future smart power distribution system with multiple active consumers and aggregators exchanging real-time electricity pricing and electricity consumption information over a communication network is prone to cyberattack. The goal of this paper is to understand the impact of attacks on pricing/load signals on the physical grid within a transactive energy market framework. First, this paper models the interaction between real-time electricity price and total energy demand in the form of a discrete time nonlinear autonomous dynamical system. Second, equilibrium electricity price and energy demand associated with this coupled dynamical system is derived and conditions for bounded input bounded output (BIBO) stability are identified. Third, a BIBO stable algorithm to design real-time electricity pricing scheme from a techno-economic perspective is developed. Finally, the impact of various levels of false data injection (FDI) attack on price of electricity, demand, and distribution system voltage is investigated. The proposed model is analyzed using simulations on the IEEE 69-bus test system and the impact of FDI attack on both electricity price/demand and distribution grid voltage is quantified. In conclusion, this paper shows that impact of FDI attack on electricity prices is more severe than an attack on electricity demand.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

NASA NTRS2017

Task 28: Web Accessible APIs in the Cloud Trade Study

This study explored three candidate architectures for serving NASA Earth Science Hierarchical Data Format Version 5 (HDF5) data via Hyrax running on Amazon Web Services (AWS). We studied the cost and performance for each architecture using several representative Use-Cases. The objectives of the project are: Conduct a trade study to identify one or more high performance integrated solutions for storing and retrieving NASA HDF5 and Network Common Data Format Version 4 (netCDF4) data in a cloud (web object store) environment. The target environment is Amazon Web Services (AWS) Simple Storage Service (S3).Conduct needed level of software development to properly evaluate solutions in the trade study and to obtain required benchmarking metrics for input into government decision of potential follow-on prototyping. Develop a cloud cost model for the preferred data storage solution (or solutions) that accounts for different granulation and aggregation schemes as well as cost and performance trades.

cost model↗

NASA NTRS2011

Object-based Mapping of the Circumpolar Taiga-Tundra Ecotone with MODIS Tree Cover

The circumpolar taiga-tundra ecotone was delineated using an image segmentation based mapping approach with multi-annual MODIS Vegetation Continuous Fields (VCF) tree cover data. Circumpolar tree canopy cover (TCC) throughout the ecotone was derived by averaging MODIS VCF data from 2000 - 2005 and adjusting the averaged values using linear equations relating MODIS TCC to Quickbird-derived tree cover estimates. The adjustment helped mitigate VCF's overestimation of tree cover in lightly forested regions. An image segmentation grouped pixels representing similar tree cover into polygonal features (objects) that form the map of the transition zone. Eachfeature represents an area much larger than the 500m MODIS pixel to characterize thepatterns of sparse forest patches on a regional scale. Comparisons of the adjusted average tree cover data were made with (1) two existing tree line definitions aggregated for each 1deg longitudinal interval in North America and Eurasia and (2) Landsat-derived Canadianproportion of forest cover for Canada. The adjusted TCC from MODIS VCF shows, on average, greater than 12% TCC for all but one regional zone at the intersection with independently delineated tree lines. Adjusted values track closely with Canadian proportion of forest cover data in areas of low tree cover. Those polygons near the boreal/tundra interface with either (1) mean adjusted TCC values between 5-20% , or (2) mean adjusted TCC values <5% but with a standard deviation > 5% were used to identify the ecotone.

Ransom, Kenneth J.↗

DOE OSTI2021

Added value for integrated marine energy data systems

Launched in 2019, the Portal and Repository for Information on Marine Renewable Energy (PRIMRE) provides centralized access, standardization, community building, and integration of United States (U.S.) databases, tools and codes, and other resources that cover a range of marine energy information. The PRIMRE universe contains a series of Knowledge Hubs that represent data and information on testing of marine energy devices (MHKDR); environmental effects (Tethys); engineering and technical papers (Tethys Engineering); descriptions of marine energy companies and technologies (Marine Energy Projects Database); codes and models (Marine Energy Software); and guidance on testing and measurements (Telesto). Content is added to PRIMRE by applying a set of Guidelines and Best Practices (PRIMRE Guidelines). An aggregate search across the PRIMRE site enables users to find data and information from all the PRIMRE Knowledge Hubs simultaneously, using a single entry-point (PRIMRE Search). In addition to providing access to a range of data and information on marine energy development, testing, and effects, PRIMRE allows for the development of value added products and processes that will help move the marine energy industry forward. The PRIMRE team has recently launched two key initiatives in the U.S.—Signature Projects and Lessons Learned. Outputs and outcomes from these two initiatives will be highlighted in this paper. The Signature Projects initiative is intended to bring focus to a selection of ongoing and completed marine energy projects funded by the U.S. Department of Energy’s Water Power Technologies Office (WPTO), and to inform the marine energy community of what investigations have been undertaken, what tools are available, and where gaps in information persist. Each Signature Project tags papers, reports, and data from large marine energy research projects, providing easy access and attention to all the output and associated products from each project. The Lessons Learned initiative is intended to ensure that hard-won achievements are recognized and available for those who come later, that missteps and unfortunate outcomes can be prevented in future, and that efficiencies and effective shortcuts can be publicized and used as the marine energy industry moves forward. This initiative builds off the Knowledge Hubs and reaches out to members of the marine energy community, particularly technology developers and researchers, to integrate experience in the development, deployment, assessment, success, and challenges while creating this new industry and field of study.

Copping, Andrea E.↗

DOE OSTI2021

Added value for integrated marine energy data systems

Copping, Andrea E.↗