SEARCH · Engineering Papers

Results for “data platform”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

All sources NASA NTRS DOE OSTI

DOE OSTI2025

HyRAM+ (Hydrogen Plus Other Alternative Fuels Risk Assessment Models) v.6.1

SAND2025-11565O HyRAM+ (Hydrogen Plus Other Alternative Fuels Risk Assessment Models) is a tool for conducting quantitative risk assessment (QRA) in transportation systems. HyRAM+ contains validated, simplified release behavior models, engineering models, and generic data relevant to hydrogen installations. HyRAM+’s platform integrates models and data to conduct QRA on user-defined hydrogen or other alternative fuel systems. The software will enable the international safety research community to add validated models to the HyRAM+ platform for use in QRAs. HyRAM (hydrogen-only) versions 1.0 to 3.1 were developed by Sandia for the Department of Energy (DOE) Hydrogen and Fuel Cell Technologies Office. The following agencies contributed to the development of HyRAM+ version 4.0 regarding the addition of methane (natural gas) and propane models: the DOE Vehicle Technologies Office and the Department of Transportation Pipeline and Hazardous Material Safety Administration. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Groth, Katrina [Sandia National Lab. (SNL-NM), Alb↗

DOE OSTI2024

JARVIS-Leaderboard: a large scale benchmark of materials design methods

Abstract Lack of rigorous reproducibility and validation are significant hurdles for scientific development across many fields. Materials science, in particular, encompasses a variety of experimental and theoretical approaches that require careful benchmarking. Leaderboard efforts have been developed previously to mitigate these issues. However, a comprehensive comparison and benchmarking on an integrated platform with multiple data modalities with perfect and defect materials data is still lacking. This work introduces JARVIS-Leaderboard, an open-source and community-driven platform that facilitates benchmarking and enhances reproducibility. The platform allows users to set up benchmarks with custom tasks and enables contributions in the form of dataset, code, and meta-data submissions. We cover the following materials design categories: Artificial Intelligence (AI), Electronic Structure (ES), Force-fields (FF), Quantum Computation (QC), and Experiments (EXP). For AI, we cover several types of input data, including atomic structures, atomistic images, spectra, and text. For ES, we consider multiple ES approaches, software packages, pseudopotentials, materials, and properties, comparing results to experiment. For FF, we compare multiple approaches for material property predictions. For QC, we benchmark Hamiltonian simulations using various quantum algorithms and circuits. Finally, for experiments, we use the inter-laboratory approach to establish benchmarks. There are 1281 contributions to 274 benchmarks using 152 methods with more than 8 million data points, and the leaderboard is continuously expanding. The JARVIS-Leaderboard is available at the website: https://pages.nist.gov/jarvis_leaderboard/

36 MATERIALS SCIENCE↗

DOE OSTI2015

Efficient scatter distribution estimation and correction in CBCT using concurrent Monte Carlo fitting

Purpose: X-ray scatter is a significant impediment to image quality improvements in cone-beam CT (CBCT). The authors present and demonstrate a novel scatter correction algorithm using a scatter estimation method that simultaneously combines multiple Monte Carlo (MC) CBCT simulations through the use of a concurrently evaluated fitting function, referred to as concurrent MC fitting (CMCF). Methods: The CMCF method uses concurrently run MC CBCT scatter projection simulations that are a subset of the projection angles used in the projection set, P, to be corrected. The scattered photons reaching the detector in each MC simulation are simultaneously aggregated by an algorithm which computes the scatter detector response, S{sub MC}. S{sub MC} is fit to a function, S{sub F}, and if the fit of S{sub F} is within a specified goodness of fit (GOF), the simulations are terminated. The fit, S{sub F}, is then used to interpolate the scatter distribution over all pixel locations for every projection angle in the set P. The CMCF algorithm was tested using a frequency limited sum of sines and cosines as the fitting function on both simulated and measured data. The simulated data consisted of an anthropomorphic head and a pelvis phantom created from CT data, simulated with and without the use of a compensator. The measured data were a pelvis scan of a phantom and patient taken on an Elekta Synergy platform. The simulated data were used to evaluate various GOF metrics as well as determine a suitable fitness value. The simulated data were also used to quantitatively evaluate the image quality improvements provided by the CMCF method. A qualitative analysis was performed on the measured data by comparing the CMCF scatter corrected reconstruction to the original uncorrected and corrected by a constant scatter correction reconstruction, as well as a reconstruction created using a set of projections taken with a small cone angle. Results: Pearson’s correlation, r, proved to be a suitable GOF metric with strong correlation with the actual error of the scatter fit, S{sub F}. Fitting the scatter distribution to a limited sum of sine and cosine functions using a low-pass filtered fast Fourier transform provided a computationally efficient and accurate fit. The CMCF algorithm reduces the number of photon histories required by over four orders of magnitude. The simulated experiments showed that using a compensator reduced the computational time by a factor between 1.5 and 1.75. The scatter estimates for the simulated and measured data were computed between 35–93 s and 114–122 s, respectively, using 16 Intel Xeon cores (3.0 GHz). The CMCF scatter correction improved the contrast-to-noise ratio by 10%–50% and reduced the reconstruction error to under 3% for the simulated phantoms. Conclusions: The novel CMCF algorithm significantly reduces the computation time required to estimate the scatter distribution by reducing the statistical noise in the MC scatter estimate and limiting the number of projection angles that must be simulated. Using the scatter estimate provided by the CMCF algorithm to correct both simulated and real projection data showed improved reconstruction image quality.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2016

VOLTTRON 2016

VOLTTRON is a flexible, reliable, and scalable platform for distributed control and sensing. VOLTTRON serves in four primary roles: •A reference platform for researchers to quickly develop control applications for transactive energy. •A reference platform with flexible data store support for energy analytics applications either in academia or in commercial enterprise. •A platform from which commercial enterprise can develop products without license issues and easily integrate into their product line. •An accelerator to drive industry adoption of transactive energy and advanced building energy analytics. Pacific Northwest National Laboratory, with funding from the U.S. Department of Energy’s Building Technologies Office, developed and maintains VOLTTRON as an open-source community project. VOLTTRON source code includes agent execution software; agents that perform critical services that enable and enhance VOLTTRON functionality; and numerous agents that utilize the platform to perform a specific function (fault detection, demand response, etc.). The platform supports energy, operational, and financial transactions between networked entities (equipment, organizations, buildings, grid, etc.) and enhance the control infrastructure of existing buildings through the use of open-source device communication, control protocols, and integrated analytics.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Harmonizing solar induced fluorescence across spatial scales, instruments, and extraction methods using proximal and airborne remote sensing: A multi-scale study in a soybean field

Solar-induced chlorophyll fluorescence (SIF) has been widely used to track vegetation photosynthesis at different scales ranging from in-situ measurements to satellite products. Airborne platforms sample SIF data at a spatial scale intermediate between in-situ and satellite, matching that of ground measurement (e.g. flux tower footprints and other field sampling), enabling us to explore causes of SIF variation and validate satellite-based SIF products. However, harmonizing SIF across sensors and platforms (correcting for systematic errors to yield a consistent, comparable SIF product) is challenging because SIF can be retrieved in different absorption windows, with different instruments and methods complicating the comparison between different observational levels (i.e., ground, airborne, satellites) and between sites equipped with different instruments with varying optical properties (spectral resolution and sampling intervals, spatial resolution). Additionally, the spatial and temporal variability of atmospheric properties can influence the retrieval of the weak SIF signal. Because of these complications, direct comparisons of airborne and ground SIF across scales are rarely attempted. Here, in this study, we combined airborne SIF data with simultaneous ‘ground truth’ data collected by stationary and mobile platforms in a soybean field in Nebraska, USA. In this effort, we tested several SIF extraction methods, including Fraunhofer Line Discrimination (FLD), improved Fraunhofer Line Discrimination (iFLD), Spectral Fitting Method (SFM), SpecFit, and a Singular Vector Decomposition (SVD) method. The SpecFit method was sensitive to the 715–740 nm water bands and removing the water bands in the fitting process yielded better agreement between the airborne and ground SIF spectra. Accurate estimation of the ground level downwelling irradiance obtained by ground measurements over a calibration target improved agreement between airborne and ground SIF retrievals at the O 2 A band, and allowed us to derive a SIF dataset with improved agreement across platforms and sampling scales. This experimental approach provided a method for generating comparable SIF signals across instruments, methods and platforms, which is critical to understanding the SIF-GPP relationship at different scales and to cross-validate the diversity of platforms used for satellite products calibration and validation.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2021

Benchmarking blockchain-based gene-drug interaction data sharing methods: A case study from the iDASH 2019 secure genome analysis competition blockchain track

Blockchain distributed ledger technology is just starting to be adopted in genomics and healthcare applications. Despite its increased prevalence in biomedical research applications, skepticism regarding the practicality of blockchain technology for real-world problems is still strong and there are few implementations beyond proof-of-concept. We focus on benchmarking blockchain strategies applied to distributed methods for sharing records of gene-drug interactions. We expect this type of sharing will expedite personalized medicine. We generated gene-drug interaction test datasets using the Clinical Pharmacogenetics Implementation Consortium (CPIC) resource. We developed three blockchain-based methods to share patient records on gene-drug interactions: Query Index, Index Everything, and Dual-Scenario Indexing. We achieved a runtime of about 60 s for importing 4,000 gene-drug interaction records from four sites, and about 0.5 s for a data retrieval query. Our results demonstrated that it is feasible to leverage blockchain as a new platform to share data among institutions.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2021

National Renewable Energy Laboratory, South Table Mountain Campus, Golden, Colorado

The National Renewable Energy Laboratory (NREL) South Table Mountain campus, located in Golden, Colorado, has multiyear datasets from buildings with approximately 1,100,000 sq ft of floor area. These 16 buildings include a large office, nine laboratories, and other education or public assembly facilities such as an education center, a warehouse, quick service restaurants, two site entrance security buildings, and a parking garage—all constructed and improved from 1982 to 2014. These datasets are also available from the NREL data storage and management platform SkySparks. The datasets contain electricity meter data (poser, voltage, and current) with 1-minute resolution and building automation system (BAS) data with resolutions of 5 or 15 minutes. Some subsystems (e.g., lighting) or equipment electricity data have the same intervals as above. Some mechanical equipment state or condition data, such as flow and temperature, are captured with 5- to 15-minute intervals. The types of measurement points (i.e., electrical meter power and zone temperature) and associated data quality across the campus are typical, which helps reduce the unreliability of data quality.

STM, Systems and equipment operational, Energy use↗

DOE OSTI2020

Assessing Current and Future Infrastructure Hazards

Project Objective Execute intelligent analytics via an advanced analytical framework, to assess the current state of offshore infrastructure, evaluate infrastructure life, and identify technologies to reduce infrastructure hazards, costs, and extend infrastructure life. Approach & Results thus Far• Build comprehensive dataset• Perform data-driven analytics to evaluate infrastructure integrity 1. Remaining lifespan 2. Likelihood of future risk• Apply data-driven advanced spatial, statistical, and Machine Learning (ML) models to quantify existing infrastructure integrity• Release data and models through a smart, online platform hosted by Energy Data eXchange (EDX)

Romeo, Lucy F.↗

DOE OSTI2022

MIRACL Co-Simulation Platform Lab assets and tools integration

Pacific Northwest National Laboratory's (PNNL) co-simulation platform (CSP) for the Microgrids, Infrastructure Resilience, and Advanced Controls Launchpad (MIRACL) project, also known as MIRACL-CSP, is a functional layer designed and developed to oversee the operational exchanges of data at the application level to and from different resources residing on the MIRACL Data Hub shared platform. MIRACL-CSP allows virtual interactions between various data hub resources during co-simulation runtime.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2021

University of Colorado and Black Swift Technologies RPAS-based measurements of the lower atmosphere during LAPSE-RATE

Abstract. Between 14 and 20 July 2018, small remotely piloted aircraft systems (RPASs) were deployed to the San Luis Valley of Colorado (USA) together with a variety of surface-based remote and in situ sensors as well as radiosonde systems as part of the Lower Atmospheric Profiling Studies at Elevation – a Remotely-piloted Aircraft Team Experiment (LAPSE-RATE). The observations from LAPSE-RATE were aimed at improving our understanding of boundary layer structure, cloud and aerosol properties, and surface–atmosphere exchange and provide detailed information to support model evaluation and improvement work. The current paper describes the observations obtained using four different types of RPASs deployed by the University of Colorado Boulder and Black Swift Technologies. These included the DataHawk2, the Talon and the TTwistor (University of Colorado), and the S1 (Black Swift Technologies). Together, these aircraft collected over 30 h of data throughout the northern half of the San Luis Valley, sampling altitudes between the surface and 914 m a.g.l. Data from these platforms are publicly available through the Zenodo archive and are co-located with other LAPSE-RATE data as part of the Zenodo LAPSE-RATE community (https://zenodo.org/communities/lapse-rate/, last access: 27 May 2021). The primary DOIs for these datasets are https://doi.org/10.5281/zenodo.3891620 (DataHawk2, de Boer et al., 2020a, e), https://doi.org/10.5281/zenodo.4096451 (Talon, de Boer et al., 2020d), https://doi.org/10.5281/zenodo.4110626 (TTwistor, de Boer et al., 2020b), and https://doi.org/10.5281/zenodo.3861831 (S1, Elston and Stachura, 2020).

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2020

Innovation in Radiological Security, Part 2 of 2 – Insights into Developing a Cloud-hosted Security Technology

A Sentry Remote Monitoring System (Sentry-RMS) is a stand-alone security system that provides detection, assessment, and communication of priority alarms as an additional means of thwarting internal and external threats to sites that maintain radiological material. The SEntry-RMS CommUnications and REsponse (Sentry-SECURE) platform is an optional feature of the Sentry-RMS that relays priority alarm information to the identified response stakeholders. Sentry-SECURE is hosted in a cloud environment that abstracts the data owner’s and data consumer’s platforms to allow for greater information sharing. This promotes situational awareness amongst authorized users and enables future innovation among modern response platforms. When securely architecting a cloud solution such as this, the use of design paradigms can be an effective tool to increase the accuracy and reliability of cyber- and information-security-related decisions made throughout the development process. This approach also supports the categorization of design considerations into three levels: industry concepts, project approaches, and data protections for digital processes. Industry concepts consist of the notional underpinnings that guide or motivate a security process, system, or design but often lack any tangible attributes. Project approaches represent decisions made during the design and development process to prioritize a solution, method, or practice above another that may provide a comparable functional output but lacks a desired security benefit. Data protections for digital processes represent the selection, integration, and implementation of specific controls for a given asset. This paper will explore specific examples of how Sentry-SECURE has been designed to account for considerations at each of these three levels, while balancing the operational intent of the platform with the security enhancements necessary to maintain data integrity, availability, and confidentiality.

assessment, RMS, security, physical, cloud, Cyber ↗

DOE OSTI2022

The Astronomy Commons Platform: A Deployable Cloud-based Analysis Platform for Astronomy

Abstract We present a scalable, cloud-based science platform solution designed to enable next-to-the-data analyses of terabyte-scale astronomical tabular data sets. The presented platform is built on Amazon Web Services (over Kubernetes and S3 abstraction layers), utilizes Apache Spark and the Astronomy eXtensions for Spark for parallel data analysis and manipulation, and provides the familiar JupyterHub web-accessible front end for user access. We outline the architecture of the analysis platform, provide implementation details and rationale for (and against) technology choices, verify scalability through strong and weak scaling tests, and demonstrate usability through an example science analysis of data from the Zwicky Transient Facility’s 1Bn+ light-curve catalog. Furthermore, we show how this system enables an end user to iteratively build analyses (in Python) that transparently scale processing with no need for end-user interaction. The system is designed to be deployable by astronomers with moderate cloud engineering knowledge, or (ideally) IT groups. Over the past 3 yr, it has been utilized to build science platforms for the DiRAC Institute, the ZTF partnership, the LSST Solar System Science Collaboration, and the LSST Interdisciplinary Network for Collaboration and Computing, as well as for numerous short-term events (with over 100 simultaneous users). In a live demo instance, the deployment scripts, source code, and cost calculators are accessible. 4 4 http://hub.astronomycommons.org/

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

UNH TDP - Concurrent Measurements of Inflow, Power Performance, and Loads for a Grid-Synchronized Vertical Axis Cross-Flow Turbine Operating in a Tidal Estuary

This data was collected between October 12 and December 15 of 2021 at the University of New Hampshire (UNH) and Atlantic Marine Energy Center (AMEC) turbine deployment platform (TDP). This data set includes over 29 days of grid connected turbine operation during this 65 day time frame. The priority for this measurement campaign was to collect data while the turbine was electrically connected to the grid by means of a rectifier and inverter. The Fall_2021_UNH_Measurement_Timeline.png highlights when each instrument was functioning and the Fall_2021_UNH_Test_Log.jpg indicates the four main regions for analysis available from this measurement campaign. The TDP is a floating structure moored on the Portsmouth facing side of Memorial Bridge pier #2, which spans the Piscataqua River between Portsmouth, NH and Kittery, ME. The Piscataqua River connects the Great Bay Estuary to the Gulf of Maine and the river currents are dominated by tidal forcing with water velocities exceeding 2.5 m/s during spring ebb tides at this site which were previously characterized by Kaelin Chancey (Assessment Of The Localized Flow And Tidal Energy Conversion System At An Estuarine Bridge - UNH MS Thesis 2019). The turbine under test was a modified New Energy Corporation (Calgary, CA) model EVG-025 4-blade H-Darrius type vertical axis cross flow turbine that rotates in the clockwise direction with a rotor diameter of 3.2m and blade length of 1.7m. The hydro-foil profile was a NACA 0021 with a 10 inch chord length and a blade preset pitch angle of +4deg with a positive angle corresponding with the toe in direction. The standard EVG-025 has a rotor diameter of 3.4m and its rated power output is 25kW at 3 m/s. The rotor diameter was reduced to accommodate the size of the existing TDP moon-pool. This project was pursued to quantify device performance for cross flow turbines operating in a marine environment. Accurate physical models, to characterize cross flow turbine performance, require real operational data sets due to the complexity of blade fluid interactions. This data can help support model development which will help predict turbine performance when analyzing perspective project locations in the future. Instrumentation was deployed to measure; water speed/direction, electrical power output, turbine shaft speed, turbine thrust force, and platform motion. Concurrent measurements of these parameters allow for correlations (cause and affect) to be inferred, allowing for characterization of device performance over a range of operating conditions. Water currents were measured using Acoustic Doppler Current Profilers (ADCP's) and Acoustic Doppler Velocimeters (ADV's) directly upstream and downstream of the turbine for inflow, wake and turbulence measurements. Electrical power output was measured using the Voltsys rectifier and the Shark power meter. Shaft speed was calculated based on the Voltsys measurements of the permanent magnet three phase generator AC generation frequency, coupled directly to the cross flow turbine under test (i.e., no gear box). Platform motions were captured using a Yost IMU (inertial measurement unit). Turbine thrust loading was measured using a reaction arm about the turbine deployment platform spanning beam, where two bi-directional load cells were connected to the system via a pinned connection. This submission includes zipped folders for each instrument containing quality controlled (QC'd) data in daily .csv files for the relevant duration specific to each instrument, along with separate .csv file that contains the units for each variable. Some instrument daily files are quite large and can pose a challenge for a visual spreadsheet editor to open. A processing software like MATLAB or Python is recommended. Note the degree of QC varied between each instrument due to time constraints. Particular time and attention was given to perform quality control tests on the acoustic based instruments that are particularly suscep...

16 TIDAL AND WAVE POWER↗

DOE OSTI2015

Microbial Community Analysis with Ribosomal Gene Fragments from Shotgun Metagenomes

Shotgun metagenomic sequencing does not depend on gene-targeted primers or PCR amplification; thus, it is not affected by primer bias or chimeras. However, searching rRNA genes from large shotgun Illumina data sets is computationally expensive, and no approach exists for unsupervised community analysis of small-subunit (SSU) rRNA gene fragments retrieved from shotgun data. We present a pipeline, SSUsearch, to achieve the faster identification of short-subunit rRNA gene fragments and enabled unsupervised community analysis with shotgun data. It also includes classification and copy number correction, and the output can be used by traditional amplicon analysis platforms. Shotgun metagenome data using this pipeline yielded higher diversity estimates than amplicon data but retained the grouping of samples in ordination analyses. We applied this pipeline to soil samples with paired shotgun and amplicon data and confirmed bias against Verrucomicrobia in a commonly used V6-V8 primer set, as well as discovering likely bias against Actinobacteria and for Verrucomicrobia in a commonly used V4 primer set. In conclusion, this pipeline can utilize all variable regions in SSU rRNA and also can be applied to large-subunit (LSU) rRNA genes for confirmation of community structure.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2018

Monitoring Surface Phenomena Created by an Underground Chemical Explosion Using Fully Polarimetric VideoSAR

Sandia National Laboratories flew its Facility for Advanced RF and Algorithm Development X-Band (9.6-GHz center frequency), fully polarimetric synthetic aperture radar (PolSAR) in VideoSAR mode to collect complex-valued SAR imagery before, during, and after the sixth Source Physics Experiment's (SPE-6) underground explosion. The VideoSAR products generated from the data sets include "movies'' of single- and quad-polarization coherence maps, magnitude imagery, and polarimetric decompositions. Residual defocus, due to platform motion during data acquisition, was corrected with a digital elevation model-based autofocus algorithm. We generated and exploited the VideoSAR image products to characterize the surface movement effects caused by the underground explosion. Unlike seismic sensors, which measure local area seismic waves using sparse spacing and subterranean positioning, these VideoSAR products captured high-spatial resolution, 2-D, time-varying surface movement. The results from the fifth SPE (SPE-5) used single-polarimetric VideoSAR data. In this paper, we present single-polarimetric and fully polarimetric VideoSAR results while monitoring the SPE-6 underground chemical explosion. We show that fully polarimetric VideoSAR imaging provides a unique, coherent, time-varying measure of the surface expression of the SPE-6 underground chemical explosion. As a result, we include new surface characterization results from the measured PolSAR SPE-6 data via H/A/α polarimetric decomposition.

47 OTHER INSTRUMENTATION↗

DOE OSTI2021

A Hardware Platform for Studying Naval Power Electronic Power Distribution Systems

Abstract – Future intelligent ship system designs will likely include electric propulsion, numerous highpower sensors, and directed energy weapons. Supply and control of these large nonlinear loads will require a networked, multi-converter power electronic power distribution system. This work presents a hardware platform to emulate a microgrid power system with multiple power converters and a power data communication network. The platform is reconfigurable and can include both AC and DC power distribution zones, representative of shipboard power systems. It also allows for the study of both power control actuation and power data communication delays. Since the platform is based on electric power hardware, spatial and temporal uncertainties are inherently embedded in the system. Specifically, this work examines the control actuation of multiple pulsed power loads in a single microgrid. Several pulse load levels and operating scenarios have been implemented and measured. A framework for control parameter quantification is presented, and various metrics are explored to capture pulse signal characteristics. The sensitivity of pulse load metric parameters is analyzed. Dynamic shipboard, mission-specific load profiles coupled with pulse loads can also be emulated in the hardware platform.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2018

SIRIUS: Enabling Progressive Data Exploration for Extreme-Scale Scientific Data

Scientific simulations on high performance computing (HPC) platforms generate large quantities of data. To bridge the widening gap between compute and I/O, and enable data to be more efficiently stored and analyzed, simulation outputs need to be refactored, reduced, and appropriately mapped to storage tiers. However, a systematic solution to support these steps has been lacking in the current HPC software ecosystem. To that end, this paper develops SIRIUS, a progressive JPEG-like data management scheme for storing and analyzing big scientific data. It co-designs data decimation, compression, and data storage, taking the hardware characteristics of each storage tier into considerations. With reasonably low overhead, our approach refactors simulation data, using either topological or uniform decimation, into a much smaller, reduced-accuracy base dataset, and a series of deltas that is used to augment the accuracy if needed. The base dataset and deltas are compressed and written to multiple storage tiers. Data saved on different tiers can then be selectively retrieved to restore the level of accuracy that satisfies data analytics. Thus, SIRIUS provides a paradigm shift towards elastic data analytics and enables end users to make trade-offs between analysis speed and accuracy on-the-fly. This paper further develops algorithms to preserve statistics for data decimation, a common requirement for reducing data. Here, we assess the impact of SIRIUS on unstructured triangular meshes, a pervasive data model used in scientific simulations. In particular, we evaluate two realistic use cases: the blob detection in fusion and high-pressure area extraction in computational fluid dynamics.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2014

Scaling Graph Community Detection on the Tilera Many-core Architecture

In an era when power constraints and data movement are proving to be significant barriers for the application of high-end computing, the Tilera many-core architecture offers a low-power platform exhibiting many important characteristics of future systems, including a large number of simple cores, a sophisticated network-on-chip, and fine-grained control over memory and caching policies. While this emerging architecture has been previously studied for structured compute-intensive kernels, benchmarking the platform for data-bound, irregular applications present significant challenges that have remained unexplored. Community detection is an advanced prototypical graph-theoretic operation with applications in numerous scientific domains including life sciences, cyber security, and power systems. In this work, we explore multiple design strategies toward developing a scalable tool for community detection on the Tilera platform. Using several memory layout and work scheduling techniques we demonstrate speedups of up to 46x on 36 cores of the Tilera TileGX36 platform over the best serial implementation, and also show results that have comparable quality and performance to mainstream x86 platforms. To the best of our knowledge this is the first work addressing graph algorithms on the Tilera platform. This study demonstrates that through careful design space exploration, low-power many-core platforms like Tilera can be effectively exploited for graph algorithms that that embody all the essential characteristics of an irregular application.

community detection↗