Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,153 records · Page 64
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

A global anthropogenic emission inventory of atmospheric pollutants from sector- and fuel-specific sources (1970–2017): an application of the Community Emissions Data System (CEDS)

Global anthropogenic emission inventories remain vital for understanding the sources of atmospheric pollution and the associated impacts on the environment, human health, and society. Rapid changes in today's society require that these inventories provide contemporary estimates of multiple atmospheric pollutants with both source sector and fuel type information to understand and effectively mitigate future impacts. To fill this need, we have updated the open-source Community Emissions Data System (CEDS) (Hoesly et al., 2019) to develop a new global emission inventory, CEDSGBD-MAPS. This inventory includes emissions of seven key atmospheric pollutants (NOx; CO; SO2; NH3; non-methane volatile organic compounds, NMVOCs; black carbon, BC; organic carbon, OC) over the time period from 1970–2017 and reports annual country-total emissions as a function of 11 anthropogenic sectors (agriculture; energy generation; industrial processes; on-road and non-road transportation; separate residential, commercial, and other sectors (RCO); waste; solvent use; and international shipping) and four fuel categories (total coal, solid biofuel, the sum of liquid-fuel and natural-gas combustion, and remaining process-level emissions). The CEDSGBD-MAPS inventory additionally includes monthly global gridded (0.5°?×?0.5°) emission fluxes for each compound, sector, and fuel type to facilitate their use in earth system models. CEDSGBD-MAPS utilizes updated activity data, updates to the core CEDS default scaling procedure, and modifications to the final procedures for emissions gridding and aggregation. Relative to the previous CEDS inventory (Hoesly et al., 2018), these updates extend the emission estimates from 2014 to 2017 and improve the overall agreement between CEDS and two widely used global bottom-up emission inventories. The CEDSGBD-MAPS inventory provides the most contemporary global emission estimates to date for these key atmospheric pollutants and is the first to provide global estimates for these species as a function of multiple fuel types and source sectors. Dominant sources of global NOx and SO2 emissions in 2017 include the combustion of oil, gas, and coal in the energy and industry sectors as well as on-road transportation and international shipping for NOx. Dominant sources of global CO emissions in 2017 include on-road transportation and residential biofuel combustion. Dominant global sources of carbonaceous aerosol in 2017 include residential biofuel combustion, on-road transportation (BC only), and emissions from the waste sector. Global emissions of NOx, SO2, CO, BC, and OC all peak in 2012 or earlier, with more recent emission reductions driven by large changes in emissions from China, North America, and Europe. In contrast, global emissions of NH3 and NMVOCs continuously increase between 1970 and 2017, with agriculture as a major source of global NH3 emissions and solvent use, energy, residential, and the on-road transport sectors as major sources of global NMVOCs. Due to similar development methods and underlying datasets, the CEDSGBD-MAPS emissions are expected to have consistent sources of uncertainty as other bottom-up inventories. The CEDSGBD-MAPS source code is publicly available online through GitHub: https://github.com/emcduffie/CEDS/tree/CEDS_GBD-MAPS (last access: 1 December 2020). The CEDSGBD-MAPS emission inventory dataset (both annual country-total and monthly global gridded files) is publicly available under https://doi.org/10.5281/zenodo.3754964 (McDuffie et al., 2020c).

Mcduffie, Erin↗
DOE OSTI2025

High Resolution Siting Suitability of Various Power Plant Technologies

Energy sector planning models determine the aggregate need for new generation, but these models are typically at the state or regional scale and are not equipped to address the wide range of location- and technology-specific issues that are increasingly a factor in power plant siting. These animations demonstrate the aggregate siting suitability of various power plant technology configurations, considering technology-specific factors that can prohibit development. The data presented is from the GRIDCERF (Geospatial Raster Input Data for Capacity Expansion Regional Feasibility) data package. GRIDCERF is a harmonized, open-source geospatial product that can be used to evaluate siting suitability for renewable and non-renewable power plants in the conterminous United States. The animations presented here demonstrate a curated selection of the full suite of technology configurations available. GRIDCERF provides the necessary inputs for models that simulate power plant siting for regional capacity expansion planning such as the Capacity Expansion Regional Feasibility (CERF) model.

Mongird, Kendall [Pacific Northwest National Labor↗
NASA NTRS2002

Global Aerosol Remote Sensing from MODIS

The physical characteristics, composition, abundance, spatial distribution and dynamics of global aerosols are still very poorly known, and new data from satellite sensors have long been awaited to improve current understanding and to give a boost to the effort in future climate predictions. The derivation of aerosol parameters from the MODerate resolution Imaging Spectro-radiometer (MODIS) sensors aboard the Earth Observing System (EOS) Terra and Aqua polar-orbiting satellites ushers in a new era in aerosol remote sensing from space. Terra and Aqua were launched on December 18, 1999 and May 4, 2002 respectively, with daytime equator crossing times of approximately 10:30 am and 1:30 pm respectively. Several aerosol parameters are retrieved at 10-km spatial resolution (level 2) from MODIS daytime data. The MODIS aerosol algorithm employs different approaches to retrieve parameters over land and ocean surfaces, because of the inherent differences in the solar spectral radiance interaction with these surfaces. The parameters retrieved include: aerosol optical thickness (AOT) at 0.47, 0.55 and 0.66 micron wavelengths over land, and at 0.47, 0.55, 0.66, 0.87, 1.2, 1.6, and 2.1 micron over ocean; Angstrom exponent over land and ocean; and effective radii, and the proportion of AOT contributed by the small mode aerosols over ocean. To ensure the quality of these parameters, a substantial part of the Terra-MODIS aerosol products were validated globally and regionally, based on cross correlation with corresponding parameters derived from ground-based measurements from AERONET (AErosol RObotic NETwork) sun photometers. Similar validation efforts are planned for the Aqua-MODIS aerosol products. The MODIS level 2 aerosol products are operationally aggregated to generate global daily, eight-day (weekly), and monthly products at one-degree spatial resolution (level 3). MODIS aerosol data are used for the detailed study of local, regional, and global aerosol concentration, distribution, and temporal dynamics, as well as for radiative forcing calculations. We show several examples of these results and comparisons with model output.

Ichoku, Charles↗
DOE OSTI2018

Privacy-Preserving Aggregation of Controllable Loads to Compensate Fluctuations in Solar Power

Cybersecurity and privacy are of the utmost importance for safe, reliable operation of the electric grid. It is well known that the increased connectivity/interoperability between all stakeholders (e.g., utilities, suppliers, and consumers) will enable personal information collection. Significant advanced metering infrastructure (AMI) deployment and demand response (DR) programs across the country, while enable enhanced automation, also generate energy data on individual consumers that can potentially be used for exploiting privacy. Inspired by existing works which consider DR, battery-based perturbation, and differential privacy noise adding, we novelly consider the aggregator (cluster) level privacy issue in the DR framework of solar photovoltaic (PV) generation following. Different from most of the existing works which mainly rely on the charging/discharging scheduling of rechargeable batteries, we utilize controllable building loads to serve as virtual storage devices to absorb a large portion of the PV genertaion while delicately keeping desired noisy terms to satisfy the differential privacy for the raw load profiles at the aggregator level. This not only ensures differential privacy, but also improves the DR efficiency in load following since part of the noisy signal in solar PV generation has been filtered out. In particular, a mixed integer quadratic optimization problem is formulated to optimally dispatch a population of on/off controllable loads to achieve this privacy-preserving DR service.

Dong, Jin↗
DOE OSTI2017

Temperature-dependent self-assembly and rheological behavior of a thermoreversible pmma-P n BA-PMMA triblock copolymer gel

Self-assembly and mechanical properties of triblock copolymers in a mid-block selective solvent are of interest in many applications. Herein, we report physical assembly of an ABA triblock copolymer, [PMMA–PnBA–PMMA] in two different mid-block selective solvents, n-butanol and 2-ethyl-1-hexanol. Gel formation resulting from end-block associations and the corresponding changes in mechanical properties have been investigated over a temperature range of -80 °C to 60 °C, from near the solvent melting points to above the gelation temperature. Shear-rheometry, thermal analysis, and small-angle neutron scattering data reveal formation and transition of structure in these systems from a liquid state to a gel state to a percolated cluster network with decrease in temperature. The aggregated PMMA end-blocks display a glass transition temperature. Our results provide new understanding into the structural changes of a self-assembled triblock copolymer gel over a large length scale and wide temperature range.

36 MATERIALS SCIENCE↗
NASA NTRS1995

Smoke detection in low-G fires

Fires in spacecraft are considered a credible risk. To respond to this risk, NASA flew fire detectors on Skylab and the Space Shuttle (STS) and included them in the design for International Space Station Alpha (ISSA). In previous missions (Mercury, Gemini and Apollo), the crew quarters were so cramped that it was not considered credible that the astronauts could fail to observe a fire. The Skylab nodule included approximately 20 UV fire detectors. The space shuttle has 9 ionization detectors in the mid deck and flight deck and Spacelab has six additional ionization detectors. The planned detectors for ISSA are laser-diode, forward-scattering, smoke or particulate detectors. Current plans for the ISSA call for two detectors in the open area of the module and detectors in racks that have both cooling air flow and electrical power. Due to the complete absence of data concerning the nature of particulate and radiant emission from low-g fires, all three of these detector systems were designed based upon 1-g test data. As planned mission durations and complexity increase and the volume of spacecraft increases, the need for and importance of effective, crew independent, fire detection grows significantly. This requires more knowledge concerning low-gravity fires and how they might be detected. To date, no combustion-generated particulate samples have been collected for well-developed microgravity flames. All of the extant data come from drop tower tests and therefore only correspond to the early stages of a fire. The fuel sources were restricted to laminar gas-jet diffusion flames and rapidly overheated wire insulation. These gas-jet drop tower tests indicate, through thermophoretic sampling, that soot primaries and aggregates (groups of primary particles) in micro-g may be significantly larger than those in normal-g (ng). This raises new scientific questions about soot processes as well as practical issues for particulate detection/alarm threshold levels used in on-orbit smoke detectors. Furthermore, it is widely speculated but unverified that the aggregates will grow to very large scales in a microgravity fire of longer duration than available on the ground. Preliminary tests in the 2.2 second drop tower suggest that particulate generated by overheated wire insulation will also be larger in microgravity than in normal gravity. TEM grids downstream of the fire region in the WIF experiment as well as visual observation of long string-like aggregates, further confirm this suggestion. The combined impact of these limited results and theoretical predictions is that direct knowledge of low-g combustion particulate as opposed to extrapolation from 1-g data is needed for a more confident design of smoke detectors for spacecraft.

Urban, David L.↗
DOE OSTI2019

Hydrothermally robust Ti/SiO 2 epoxidation catalysts via surface modification with oligomeric PMHS

We investigate grafting of poly(methylhydrosiloxane) (PMHS) onto Ti/SiO 2 catalysts for improving their robustness to water, using olefin epoxidation catalysis with organic hydroperoxide as oxidant as a relevant probe reaction. Our approach involves triggering catalyst deactivation with added water in the reactants as a means to simulate catalyst deactivation, thereby causing grafted Ti-site aggregation. When using harsh aqueous Soxhlet extraction conditions as a trigger for this aggregation, significant differences are observed via diffuse-reflectance UV-Vis spectrscopy on samples before and after treatment, for Ti/SiO 2 catalysts containing grafted PMHS. Not only was there a lack of appearance of red-shifted bands in the UV-Vis spectrum following Soxhlet extraction, the batch catalytic activity for catalysts before and after PMHS grafting remained virtually unchanged. This was in stark contrast to an unmodified Ti/SiO 2 catalyst, which showed a more than 60% decrease in catalytic activity following Soxhlet extraction as a result of aggregation, as ascertained by red shifts in its UV-Vis spectrum after Soxhlet extraction. Similar red shifts were also observed for a hydrophobically modified TMS Ti/SiO 2 catalyst. However, when comparing these catalysts under more gentle olefin epoxidation catalysis conditions with added water in a flow reactor, it was more difficult to differentiate the TMS Ti/SiO2 and grafted PMHS catalysts. Both catalysts exhibited similar activity and selectivity after a prolonged time on stream, though the rate of initial deactivation in the latter was less pronounced. Altogether, our data demonstrate that grafted PMHS catalysts are positively differentiated from conventional catalysts based on either TMS surface modification or no surface modification, under conditions involving aggregation induced via extreme hydrothermal environmentst. Thus, we expect that PMHS grafting of Lewis acid catalysts can offer an alternative approach for endowing hydrolytic stability to such catalysts, as demonstrated here for Ti/SiO 2 epoxidation catalysts.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2018

Photoactive Blend Morphology Engineering through Systematically Tuning Aggregation in All-Polymer Solar Cells

Polymer aggregation plays a critical role in the miscibility of materials and the performance of all-polymer solar cells (APSCs). However, many aspects of how polymer texturing and aggregation affect photoactive blend film microstructure and photovoltaic performance are poorly understood. Here the effects of aggregation in donor–acceptor blends are studied, in which the number-average molecular weights (Mns) of both an amorphous donor polymer, poly[4,8-bis(5-(2-ethylhexyl)thiophen-2-yl)benzo[1,2-b;4,5-b]dithiophene-2,6-diyl-alt-(4-(2-ethylhexyl)-3-fluorothieno[3,4-b]thiophene-)-2-carboxylate-2-6-diyl)] (PBDTT-FTTE) and a semicrystalline acceptor polymer, poly{[N,N'-bis(2-octyldodecyl)naphthalene-1,4,5,8-bis(dicarboximide)-2,6-diyl]-alt-5,5'-(2,2'-bithiophene)} (P(NDI2OD-T2)) are systematically varied. The photovoltaic performance is correlated with active layer microstructural and optoelectronic data acquired by in-depth transmission electron microscopy, grazing incidence wide-angle X-ray scattering, thermal analysis, and optical spectroscopic measurements. Coarse-grained modeling provides insight into the effects of polymer aggregation on the blend morphology. Notably, the computed average distance between the donor and the acceptor polymers correlates well with solar cell photovoltaic metrics such as short-circuit current density (Jsc) and represents a useful index for understanding/predicting active layer blend material intermixing trends. Importantly, these results demonstrate that for polymers with different texturing tendencies (amorphous/semicrystalline), the key for optimal APSC performance, photovoltaic blend morphology can be controlled via both donor and acceptor polymer aggregation.

bio-inspired↗
DOE OSTI2018

Impact of acidic pH on plant cell wall polysaccharide structure and dynamics: insights into the mechanism of acid growth in plants from solid-state NMR

Acidification of plant primary cell walls causes cell wall expansion and plant growth. To understand how acidic pH affects the molecular structure and dynamics of wall polysaccharides, we have now characterized and compared Arabidopsis thaliana primary cell walls in neutral (pH 6.8) and acidic (pH 4.0) conditions using solid-state NMR spectroscopy. Quantitative 13C solid-state NMR spectra indicate that the pH 4.0 cell wall has neutral galacturonic acid residues in homogalacturonan (HG) and rhamnogalacturonan (RG). 13C INEPT spectra, which selectively detect highly dynamic polymers, indicate that some of the HG and RG chains in the interfibrillar region have become more dynamic in the acidic wall compared to the neutral cell wall, whereas other chains have become more rigid. Consistent with this increased dynamic heterogeneity, C–H dipolar couplings and 2D 13C–13C correlation spectra indicate that some of the HG backbones are partially aggregated in the acidic cell wall. Moreover, 2D correlation spectra measured with long mixing times indicate that the acidic cell wall has weaker cellulose–pectin interactions, and water-polysaccharide 1H spin diffusion data show that cellulose microfibrils are better hydrated at low pH. Taken together, these results indicate a cascade of chemical and conformational changes of wall polysaccharides due to cell wall acidification. These changes start with neutralization of the pectic polysaccharides, which disrupts calcium crosslinking of HG, causes partial aggregation of the interfibrillar HG, weakens cellulose–pectin interactions, and increases the hydration of both cellulose microfibrils and matrix polysaccharides. These molecular-level structural and dynamical changes are expected to facilitate polysaccharide slippage, which underlies cell wall loosening and expansion, and may occur both independent of and as a consequence of protein-mediated wall loosening.

biofuels (including algae and biomass), bio-inspir↗
DOE OSTI2025

Data Assimilation for Robust UQ Within Agent-Based Simulation on HPC Systems

Agent-based simulation provides a powerful tool for in silico system modeling. However, these simulations do not provide built-in methods for uncertainty quantification (UQ). Within these types of models a typical approach to UQ is to run multiple realizations of the model then compute aggregate statistics. This approach is limited due to the compute time required for a solution. When faced with an emerging biothreat, public health decisions need to be made quickly and solutions for integrating near real-time data with analytic tools are needed. We propose an integrated Bayesian UQ framework for agent-based models based on sequential Monte Carlo sampling. Given streaming or static data about the evolution of an emerging pathogen this Bayesian framework provides a distribution over the parameters governing the spread of a disease through a population. These estimates of the spread of a disease may be provided to public health agencies seeking to abate the spread. By coupling agent-based simulations with Bayesian modeling in a data assimilation, our proposed framework provides a powerful tool for modeling dynamical systems in silico. We propose a method which reduces model error and provides a range of realistic possible outcomes. Moreover, our method addresses two primary limitations of ABMs: the lack of UQ and an inability to assimilate data. Our proposed framework combines the flexibility of an agent-based model with UQ provided by the Bayesian paradigm in a workflow which scales well to HPC systems. We provide algorithmic details and results on a simulated outbreak with both static and streaming data.

Spannaus, Adam [ORNL] (ORCID:0000000225213657)↗
DOE OSTI2022

Simultaneous Insight into Dissolution and Aggregation of Metal Sulfide Nanoparticles through Single-Particle Inductively Coupled Plasma Mass Spectrometry

Nanoparticles (NPs) and their colloidal aggregates are ubiquitous and play important roles in the transport and release of metals. Knowledge of their dissolution rates and aggregation behavior in solution are crucial for better prediction of their fate in biogeochemical cycling, ecotoxicity, and environmental remediation. There are however significant technical challenges to accurately obtain such information as a result of the heterogeneity and highly dynamic transformation exhibited by NPs, particularly at relatively low particle concentrations in aqueous systems. Here, we quantitatively examine the simultaneous dissolution and aggregation behavior of metal sulfide NPs using single-particle inductively coupled plasma mass spectrometry (spICP–MS). We focus on nickel sulfide (NiS), with additional data presented for copper sulfide (CuS) and cobalt sulfide (CoS). The kinetics of metal release (dissolution and disaggregation) of NiS was fastest under strongly oxidizing conditions (from 0.04 to >3 min –1 with H 2 O 2 ) and were slower under near-neutral (HEPES buffer/H 2 O) and acidic (1 mM HNO 3 ) conditions (≤0.006 min –1 ). Metal release kinetics in HNO 3 was not faster than in H 2 O or HEPES, suggesting that the solution pH has an influence over both the dissolution kinetics of individual particles and the NP aggregation states, which in combination affect metal release rates over time. Between different metal sulfides, the measured metal release rates were largely consistent with predictions based on the crystallinity, solubility products, and specific surface areas of the NPs, following an order of CoS > NiS > CuS. Here, the spICP–MS approach described here can be easily applied to the characterization of metal release and aggregation of other NPs at low concentrations (~10 5 particles/mL) typically found in natural environments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2012

Image Mapping and Visual Attention on the Sensory Ego-Sphere

The Sensory Ego-Sphere (SES) is a short-term memory for a robot in the form of an egocentric, tessellated, spherical, sensory-motor map of the robot s locale. Visual attention enables fast alignment of overlapping images without warping or position optimization, since an attentional point (AP) on the composite typically corresponds to one on each of the collocated regions in the images. Such alignment speeds analysis of the multiple images of the area. Compositing and attention were performed two ways and compared: (1) APs were computed directly on the composite and not on the full-resolution images until the time of retrieval; and (2) the attentional operator was applied to all incoming imagery. It was found that although the second method was slower, it produced consistent and, thereby, more useful APs. The SES is an integral part of a control system that will enable a robot to learn new behaviors based on its previous experiences, and that will enable it to recombine its known behaviors in such a way as to solve related, but novel, task problems with apparent creativity. The approach is to combine sensory-motor data association and dimensionality reduction to learn navigation and manipulation tasks as sequences of basic behaviors that can be implemented with a small set of closed-loop controllers. Over time, the aggregate of behaviors and their transition probabilities form a stochastic network. Then given a task, the robot finds a path in the network that leads from its current state to the goal. The SES provides a short-term memory for the cognitive functions of the robot, association of sensory and motor data via spatio-temporal coincidence, direction of the attention of the robot, navigation through spatial localization with respect to known or discovered landmarks, and structured data sharing between the robot and human team members, the individuals in multi-robot teams, or with a C3 center.

Fleming, Katherine Achim↗
DOE OSTI2014

Communication: Orientational self-ordering of spin-labeled cholesterol analogs in lipid bilayers in diluted conditions

Lipid-cholesterol interactions are responsible for different properties of biological membranes including those determining formation in the membrane of spatial inhomogeneities (lipid rafts). To get new information on these interactions, electron spin echo (ESE) spectroscopy, which is a pulsed version of electron paramagnetic resonance (EPR), was applied to study 3β-doxyl-5α-cholestane (DCh), a spin-labeled analog of cholesterol, in phospholipid bilayer consisted of equimolecular mixture of 1,2-dipalmitoyl-sn-glycero-3-phosphocholine and 1,2-dioleoyl-sn-glycero-3-phosphocholine. DCh concentration in the bilayer was between 0.1 mol.% and 4 mol.%. For comparison, a reference system containing a spin-labeled 5-doxyl-stearic acid (5-DSA) instead of DCh was studied as well. The effects of “instantaneous diffusion” in ESE decay and in echo-detected (ED) EPR spectra were explored for both systems. The reference system showed good agreement with the theoretical prediction for the model of spin labels of randomly distributed orientations, but the DCh system demonstrated remarkably smaller effects. The results were explained by assuming that neighboring DCh molecules are oriented in a correlative way. However, this correlation does not imply the formation of clusters of cholesterol molecules, because conventional continuous wave EPR spectra did not show the typical broadening due to aggregation of spin labels and the observed ESE decay was not faster than in the reference system. So the obtained data evidence that cholesterol molecules at low concentrations in biological membranes can interact via large distances of several nanometers which results in their orientational self-ordering.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2019

CostMAP: an open-source software package for developing cost surfaces using a multi-scale search kernel

Cost surfaces are a crucial aspect of route optimization and least cost path (LCP) calculations and are used in awide range of disciplines including computer science, landscape ecology, and energy-infrastructure modeling. Linear features introduce akey weakness to traditional routing calculations along cost surfaces because they cannot identify whether moving from acell to its adjacent neighbors constitutes crossing alinear barrier (increased cost) or following acorridor (reduced cost). Following and avoiding linear features can drastically change predicted routes. We introduce an approach to address this adjacency issue using asearch kernel that determines these critical barriers and corridors. We have built this method into anew Java-based open-source software package– CostMAP (cost surface multi-layer aggregation program)– which calculates cost surfaces and cost networks using the search kernel. CostMAP allows users to input multiple GIS data layers and to set weights and rules for developing aweighted-cost network. We compare CostMAP performance with traditional cost surface approaches and show significant performance gains– both following corridors and avoiding barriers– by modeling the movement of alarge terrestrial animal– the Baird’s Tapir (Tapirus bairdii)– in amovement ecology framework and by modeling pipeline routing for carbon capture and storage (CCS).

58 GEOSCIENCES↗
DOE OSTI2015

In-Memory Graph Databases for Web-Scale Data

RDF databases have emerged as one of the most relevant way for organizing, integrating, and managing expo- nentially growing, often heterogeneous, and not rigidly structured data for a variety of scientific and commercial fields. In this paper we discuss the solutions integrated in GEMS (Graph database Engine for Multithreaded Systems), a software framework for implementing RDF databases on commodity, distributed-memory high-performance clusters. Unlike the majority of current RDF databases, GEMS has been designed from the ground up to primarily employ graph-based methods. This is reflected in all the layers of its stack. The GEMS framework is composed of: a SPARQL-to-C++ compiler, a library of data structures and related methods to access and modify them, and a custom runtime providing lightweight software multithreading, network messages aggregation and a partitioned global address space. We provide an overview of the framework, detailing its component and how they have been closely designed and customized to address issues of graph methods applied to large-scale datasets on clusters. We discuss in details the principles that enable automatic translation of the queries (expressed in SPARQL, the query language of choice for RDF databases) to graph methods, and identify differences with respect to other RDF databases.

GEMS↗
DOE OSTI2021

sycl-reduction benchmark v1.0.0

Some different SYCL implementations of the reduction operation. The parallel reduction kernel aggregates vector elements into a single result. It consists of four ways to implement the reduction operation using two different data management strategies, which we could not find elsewhere as a single package.

Dufek, Amanda↗
DOE OSTI2023

Azure Data Tools (FKA: OEDI (Open Energy Data Initiative) Data Access Tools) [SWR-23-92]

The Open Energy Data Initiative (OEDI) provides a number of tools to enable the use of the open data published through this initiative. The source is largely written in Python, including Jupyter notebooks. The Open Energy Data Initiative (OEDI) is a partnership between the National Renewable Energy Laboratory (NREL), the U.S. Department of Energy (DOE), Amazon, Microsoft, and Google to provide universal access to big data in the cloud. At the heart of OEDI is a centralized repository of high-value energy research datasets aggregated from DOE Program Offices, National Laboratories and other collaborators.

Jonathan, Weers↗