Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

hfem3d_parfrac

A toolkit of FORTRAN90 and PYTHON routines to automate calculation of electrostatic response of fracture networks in geomaterials, specifically designed for distributed memory compute clusters. Software integrates pre-processing of Cubit-generated meshes with calculations from HFEM3D and post-processing for aggregate data analysis.SAND2019-11083 M Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Beskardes, Gungor↗

Quantum teleportation coexisting with classical communications in optical fiber

The ability for quantum and conventional networks to operate in the same optical fibers would aid the deployment of quantum network technology on a large scale. Quantum teleportation is a fundamental operation in quantum networking, but has yet to be demonstrated in fibers populated with high-power conventional optical signals. Here we report, to the best of our knowledge, the first demonstration of quantum teleportation over fibers carrying conventional telecommunications traffic. Quantum state transfer is achieved over a 30.2-km fiber carrying 400-Gbps C-band classical traffic with a Bell state measurement performed at the fiber’s midpoint. To protect quantum fidelity from spontaneous Raman scattering noise, we use optimal O-band quantum channels, narrow spectro-temporal filtering, and multi-photon coincidence detection. Fidelity is shown to be well maintained with an elevated C-band launch power of 18.7 dBm for the single-channel 400-Gbps signal, which we project could support multiple classical channels totaling many terabits/s aggregate data rates. These results show the feasibility of advanced quantum and classical network applications operating within a unified fiber infrastructure.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Manuscript Workflows from and Processed Organic Matter Composition of Experimentally Burned Open Air and Muffle Furnace Vegetation Chars across Differing Burn Severity and Feedstock Types from Pacific Northwest, USA (v3)

This dataset includes processed organic matter chemistry data from an experimental study designed to compare how the chemical composition of organic matter changes across different burn conditions and vegetation materials representative of major land cover types of the Pacific Northwest, USA. Chars were created in a closed muffle furnace or on an open burn table from four different feedstock species representing vegetation commonly impacted by fire regimes across the Pacific Northwest, USA. Source data and associated metadata (including methods and geospatial information) can be found at https://data.ess-dive.lbl.gov/datasets/doi:10.15485/1894135 (Grieger et al. 2022). This dataset provides processing scripts and processed data for both solid and dissolved phase organic matter characterization data from experimentally generated chars. These processed data can be used to compare how different burn conditions may influence resultant organic matter chemistry and help further our understanding of potential biogeochemical impacts on river corridors post-fire. The processed data were subsequently analyzed; and the results and ecological implications of the findings were published in peer-reviewed manuscripts. The scripts and workflows used to develop the manuscripts are also included in this data package.This data package was originally published June 2024. It was updated September 2024 (new and modified files) and in January 2025 (modified files). See the change history section in the readme for more details.This dataset is comprised of one data package readme, one data dictionary (dd), one file level metadata (flmd), and folders containing (A) processed data; (B) general processing scripts; and (C) additional folders with specific manuscript analysis scripts and processed data. Step-by-step instructions to assist the user in recreating the workflow used to generate the results in the manuscripts is also provided. The processed data folder includes (1) a folder of processed Parallel Factor Analysis (PARAFAC) and spectra indices outputs from excitation emissions matrix (EEM) fluorescence and absorbance data; (2) a folder of processed solid state carbon-13 (13-C NMR) integrals; (3) folder of high resolution characterization of organic matter via 21 Tesla Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) generated through the Environmental Molecular Sciences Laboratory (EMSL; https://www.pnnl.gov/environmental-molecular-sciences-laboratory) processed data outputs from Formultitude (https://github.com/PNNL-Comp-Mass-Spec/Formultitude), blank corrections and data aggregation, and calculated molecular indices. All files are .pdf, .csv, .html, .Rmd, .R, or .RData.

54 ENVIRONMENTAL SCIENCES↗

PEC Performance Measures from Literature Reports

This dataset is a capture of PEC experimental performance metrics data aggregated from publications and included here in the measurement and format provided by the authors. The measurements captured are based on rate of hydrogen production by PEC devices under illumination. Original creator of this dataset: Wen-Hui (Sophia) Cheng (National Cheng Kung University) Contributors to this dataset: Joel Ager (LBNL), H. Nishiyama et al. (Meiji University), I. Holmes-Gentle (EPFL), Yanfa Yan (University of Toledo), Aditya Mohite (Rice University), Zetian Mi (University of Michigan).

08 HYDROGEN↗

gaia: An R package to estimate crop yield responses to temperature and precipitation

gaia is an open-source R package designed to estimate crop yield shocks in response to annual weather variations and CO 2 concentrations at the country scale for 17 major crops. This innovative tool streamlines the workflow from raw climate data processing to projections of annual shocks to crop yields at the country level, using the response surfaces from an empirical econometric model developed and documented in Waldhoff et al. (2020), which leverages historical weather, CO 2 , and crop yield data for robust empirical fitting for 17 crops. gaia uses these response surfaces with monthly temperature and precipitation projections (e.g., from the Coupled Model Intercomparison Project Phase 6 (CMIP6) (O’Neill et al., 2016) climate data bias-adjusted and statistically downscaled by the ISIMIP3BASD approach (Lange, 2019) in the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP) (Warszawski et al., 2014)) to project yield shocks that can be applied to agricultural productivity changes at the country level for use in multisectoral economic models. The historical and future projections use gridded, country-and-crop specific monthly growing season precipitation and temperature data, aggregated to the national level, and weighted by cropland area derived from the global Monthly Irrigated and Rainfed Crop Areas around the year 2000 (MIRCA2000) dataset (Portmann et al., 2010). These annual, country, and crop-specific yield shocks can be aggregated to different definitions of regions, crop commodities, and time periods, as needed by specific multisectoral economic models. gaia serves as a lightweight, powerful tool that can aid exploration of crop yield responses under a broad range of future climate projections, enhancing human-Earth system analysis capabilities.

60 APPLIED LIFE SCIENCES↗

What kind of charging infrastructure do Nissan Leaf drivers in The EV Project use?

This document will describe the charging behavior of Nissan Leaf battery electric vehicles that were enrolled in the EV Project. It will include aggregated data from several thousand vehicles regarding time-of-day, power level, and location of charging and driving events. This document is a white paper that will be published on the INL AVTA website.

33 ADVANCED PROPULSION SYSTEMS↗

What Kind of Charging Infrastructure Do Chevrolet Volt Drivers in The EV Project Use and When Do They Use It?

This document will present information describing the charging behavior of Chevrolet Volts that were enrolled in the EV Project. It will included aggregated data from more than 1,800 vehicles regarding locations, power levels, and time-of-day of charging events performed by those vehicles. This document will be published to the INL AVTA website.

33 ADVANCED PROPULSION SYSTEMS↗

Capturing the Sun: A Roadmap for Navigating Data-Access Challenges and Auto-Populating Solar Home Sales Listings

Capturing the Sun: A Roadmap for Navigating Data-Access Challenges and Auto-Populating Solar Home Sales Listings supports a vision of solar photovoltaic (PV) advocates and real estate advocates evolving together to make information about solar homes more accessible to home buyers and sellers and to simplify the process when these homes are resold. The Roadmap is based on a concept in the real estate industry known as automatic population of fields. Auto-population (also called auto-pop in the industry) is the technology that allows data aggregated by an outside industry to be matched automatically with home sale listings in a multiple listing service (MLS).

14 SOLAR ENERGY↗

Scalability of Several Asynchronous Many-Task Models for In Situ Statistical Analysis.

This report is a sequel to [PB16], in which we provided a first progress report on research and development towards a scalable, asynchronous many-task, in situ statistical analysis engine using the Legion runtime system. This earlier work included a prototype implementation of a proposed solution, using a proxy mini-application as a surrogate for a full-scale scientific simulation code. The first scalability studies were conducted with the above on modestly-sized experimental clusters. In contrast, in the current work we have integrated our in situ analysis engines with a full-size scientific application (S3D, using the Legion-SPMD model), and have conducted nu- merical tests on the largest computational platform currently available for DOE science ap- plications. We also provide details regarding the design and development of a light-weight asynchronous collectives library. We describe how this library is utilized within our SPMD- Legion S3D workflow, and compare the data aggregation technique deployed herein to the approach taken within our previous work.

97 MATHEMATICS AND COMPUTING↗

The Value of Transparency in Distributed Solar PV Markets

Distributed solar photovoltaic (PV) markets are relatively non-transparent: PV price and product information is not readily available, searching for this information is costly (in terms of time and effort), and customers are mostly unfamiliar with the new technology. Quote aggregation, where third-party companies collect PV quotes on behalf of customers, may be one way to increase PV market transparency. In this paper, quote aggregation data are analyzed to study the value of transparency for distributed solar PV markets. The results suggest that easier access to more quotes results in lower prices. We find that installers tend to offer lower prices in more competitive market environments. We supplement the empirical analysis with key findings from interviews of residential PV installers.

14 SOLAR ENERGY↗

Utilizing Commercial Real Estate Owner and Investor Data to Analyze the Financial Performance of Energy Efficient, High-Performance Office Buildings

Evidence has shown that owning and operating energy-efficient, high-performance, “green” properties results in multiple benefits including lower utility bills, higher rents, improved occupancy, and greater net operating income. However, it is difficult to isolate and control moderating factors to identify the specific drivers behind improved financial performance and value to investors that results from sustainability in real estate. DOE is interested in facilitating deeper investigation of the correlation between energy efficiency and financial performance, reducing data acquisition and matching challenges, and developing a stronger understanding of how sustainable design and energy efficiency impact value. DOE commissioned this pilot study to test the logistical and empirical procedures required to establish a Commercial Real Estate Data Aggregation & Trends Analysis lab, determine the potential benefits available through the lab, and contribute to the existing body of evidence in this field.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

A Parallel Graph Environment for Real-World Data Analytics Workflows

Economic competitiveness and national security depend increasingly on the insightful analysis of large data sets. The diversity of real-world data sources and analytic workflows impose challenging hardware and software requirements for parallel graph platforms. The irregular nature of graph methods is not supported well by the deep memory hierarchies of conventional distributed systems, requiring new processor and runtime system designs to tolerate memory and synchronization latencies. Moreover, the efficiency of relational table operations and matrix computations are not attainable when data is stored in common graph data structures. In this paper, we present HAGGLE, a high-performance, scalable data analytics platform. The platform’s hybrid data model supports a variety of distributed, thread-safe data structures, parallel programming constructs, and persistent and streaming data. An abstract runtime layer enables us to map the stack to conventional, distributed computer systems with accelerators. The runtime uses multithreading, active messages, and data aggregation to hide memory and synchronization latencies on large-scale systems.

Graph Analytics↗

Model run output for WM-ABM 2024 HESS Publication

Model output for computational experiments introducing WM-ABM. See associated HESS publication here: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-1604/ all_nldas_cdl_data_v3.txt - Cropland Data Layer data aggregated to 1/8 degree resolution for input into the PMP calibration process. Variables include NLDAS ID, crop type, year, the count of pixels in the grid cell of the crop type, CDL crop ID, and GCAM crop ID. 20230115 ABM runs - WM-ABM model outputs for various scenarios described in HESS publication. If multiple versions exist (e.g., v2, v3, etc.) latest version indicates dataset generated for manuscript. The "baseline" run is the primary run, where as the "memXX" runs indicate sensitivity runs testing different parameters of agent memory. Output files starting with "abm_results_" include groundwater irrigated acres and surface water irrigated acres for each farm/agent combination. Other CSV outputs provide summary statistics of various agent and water shortage outcomes, used for generation of figures in the HESS manuscript.

ABM↗

Integrated GW Farm ABM

This Data Repository includes data used for the integrated groundwater- farm ABM model, raw model output from scenario ensemble, and processed outputs that isolate the groundwater storage depletion outcomes for the 35,000 farm cells. Model Inputs: Farm ABM Inputs: This folder contains the input data used by the integrated groundwater - farm ABM modelling script (Python file) used for the high performance computing (HPC) experiments. The sub-folder "data inputs" contains all of the farm attribute data, while the three files in the folder have the hydrogeological data lookup table (NLDAS Cost Curve Attributes.csv), a lookup table (Theis well function table.csv) for the groundwater cost curve function, and the farm indexes and corresponding NLDAS ids for all of the cells run in this experiment (nldas farms subset final.csv). NLDAS Cost curve hydrogeological data: Hydrogeological data aggregated to 1/8 degree resolution and aligned with the NLDAS grid. Parameters include: water depth below ground surface [meters], subsurface porosity [unitless], aquifer depth from ground surface to aquifer bottom [meters], annual average recharge (USGS: mm, Doll: meters), and three different hydraulic conductivity (K) values (meters/day). The three K values represent the mean value from Gleeson et al. (2018), one standard deviation above the mean from Gleeson et al. (2018), and the de Graaf et al. 2020 modifications to certain lithologies. Additional information about these datasets and their processing are documented in the supplement to Yoon et al. 2025 (in review). Output: Raw outputs: This folder contains a .zip file that has model outputs for the entire scenario ensemble. There is one csv for each farm id, using the format "farm farmid cases.csv". The relationship between the farm id and NLDAS id is defined by the "nldas farms subset final.csv" located in the Farm ABM Inputs folder. Each csv has 625 rows, corresponding to 625 combinations of different scenario parameter values. Each row (scenario) represents the outcome of a 100 year simulation. Columns define scenario settings and summary statistics for each scenario. The first four columns define the scenario settings: "hydro ratio," "econ ratio," "K scenario," and "gamma scenario." The hydro and econ ratios are values passed to the modeling script that influence multipliers for other model parameters, as documented in the supplement to Yoon et al. 2025 (in review). The gamma multiplier is a coefficient multiplier applied to the baseline gamma values (values below 1 represent lower unobserved costs compared to baseline, values above 1 represent higher costs). The K scenario names represent K values of: "low": 0.5 m/d, "int 1": 2.5 m/d, "int 2": 10 m/d, "high": 50 m/d, and "gleeson": mean Gleeson K value. "Perc vol depleted" is the fraction of groundwater depleted at the end of the 100 simulation. Processed Output: Derived depletion outcomes from raw outputs: All of the individual csv files from the Raw outputs were aggregated into a single file that has the scenario settings and fraction depletion "Perc vol depleted" for every farm cell, for every scenario. The other two files define relationships between the farm id, NLDAS id, and local and major aquifer units, used for aquifer-level depletion analysis.

Agent based modeling↗

SHAPA: An interactive software tool for protocol analysis applied to aircrew communications and workload

As modern transport environments become increasingly complex, issues such as crew communication, interaction with automation, and workload management have become crucial. Much research is being focused on holistic aspects of social and cognitive behavior, such as the strategies used to handle workload, the flow of information, the scheduling of tasks, the verbal and non-verbal interactions between crew members. Traditional laboratory performance measures no longer sufficiently meet the needs of researchers addressing these issues. However observational techniques are better equipped to capture the type of data needed and to build models of the requisite level of sophistication. Presented here is SHAPA, an interactive software tool for performing both verbal and non-verbal protocol analysis. It has been developed with the idea of affording the researchers the closest possible degree of engagement with protocol data. The researcher can configure SHAPA to encode protocols using any theoretical framework or encoding vocabulary that is desired. SHAPA allows protocol analysis to be performed at any level of analysis, and it supplies a wide variety of tools for data aggregation, manipulation. The output generated by SHAPA can be used alone or in combination with other performance variables to get a rich picture of the influences on sequences of verbal or nonverbal behavior.

James, Jeffrey M.↗

Algorithms for Automatic Alignment of Arrays

Aggregate data objects (such as arrays) are distributed across the processor memories when compiling a data-parallel language for a distributed-memory machine. The mapping determines the amount of communication needed to bring operands of parallel operations into alignment with each other. A common approach is to break the mapping into two stages: an alignment that maps all the objects to an abstract template, followed by a distribution that maps the template to the processors. This paper describes algorithms for solving the various facets of the alignment problem: axis and stride alignment, static and mobile offset alignment, and replication labeling. We show that optimal axis and stride alignment is NP-complete for general program graphs, and give a heuristic method that can explore the space of possible solutions in a number of ways. We show that some of these strategies can give better solutions than a simple greedy approach proposed earlier. We also show how local graph contractions can reduce the size of the problem significantly without changing the best solution. This allows more complex and effective heuristics to be used. We show how to model the static offset alignment problem using linear programming, and we show that loop-dependent mobile offset alignment is sometimes necessary for optimum performance. We describe an algorithm with for determining mobile alignments for objects within do loops. We also identify situations in which replicated alignment is either required by the program itself or can be used to improve performance. We describe an algorithm based on network flow that replicates objects so as to minimize the total amount of broadcast communication in replication.

Chatterjee, Siddhartha↗

Biomimetic Models for An Ecological Approach to Massively-Deployed Sensor Networks

Promises of ubiquitous control of the physical environment by massively-deployed wireless sensor networks open avenues for new applications that will redefine the way we live and work. Due to small size and low cost of sensor devices, visionaries promise systems enabled by deployment of massive numbers of sensors ubiquitous throughout our environment working in concert. Recent research has concentrated on developing techniques for performing relatively simple tasks with minimal energy expense, assuming some form of centralized control. Unfortunately, centralized control is not conducive to parallel activities and does not scale to massive size networks. Execution of simple tasks in sparse networks will not lead to the sophisticated applications predicted. We propose a new way of looking at massively-deployed sensor networks, motivated by lessons learned from the way biological ecosystems are organized. We demonstrate that in such a model, fully distributed data aggregation can be performed in a scalable fashion in massively deployed sensor networks, where motes operate on local information, making local decisions that are aggregated across the network to achieve globally-meaningful effects. We show that such architectures may be used to facilitate communication and synchronization in a fault-tolerant manner, while balancing workload and required energy expenditure throughout the network.

Jones, Kennie H.↗