Scale-out Edge Storage Systems with Embedded Storage Nodes to Get Better Availability and Cost-Efficiency At the Same Time.
Abstract not provided.
Liu, Jianshen↗
SEARCH · Engineering Papers
Results for “scale-out”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
T-FSM: A Scalable Distributed Task-Based System for Frequent Subgraph Pattern Mining from a Big Graph
Finding frequent subgraph patterns in a big graph is an important problem with many applications such as classifying chemical compounds and building indexes to speed up graph queries. Since this problem is NP-hard, some recent parallel and distributed systems have been developed to accelerate the mining. However, they often have a huge memory cost, very long running time, suboptimal load balancing, poor scale-out capability, and possibly inaccurate results. In this article, we propose an efficient system called T-FSM for parallel mining of frequent subgraph patterns in a big graph. T-FSM supports a new anti-monotonic frequentness measure called Fraction-Score, which is more accurate than the widely used MNI measure. The execution engine of T-FSM supports both intra-machine parallelism and inter-machine parallelism. For intra-machine parallelism, T-FSM adopts a novel task-based execution model to ensure high multithreading concurrency, bounded memory consumption, and effective load balancing. For inter-machine parallelism, T-FSM ensures good scale-out performance with a lightweight pattern rebalancing approach that reduces workload skewness of pattern evaluations among machines. To avoid recomputing the contexts for migrated patterns, we design a novel context cache table to support concurrent and asynchronous requesting and caching of remote context data, which can timely evict and garbage collect used pattern contexts that are no longer needed to keep memory consumption bounded. Extensive experiments show that T-FSM is orders of magnitude faster than existing state-of-the-art parallel systems (more than 10×, 51×, 131×, 55× speedup over ScaleMine, DistGraph, Pangolin and Peregrine, respectively) and distributed systems (more than 42× and 88× over ScaleMine and DistGraph, respectively) for frequent subgraph pattern mining, and it scales out satisfactorily to 512 CPU cores on the Polaris supercomputer at Argonne National Laboratory.
97 MATHEMATICS AND COMPUTING↗
Hydrogen Production System Scaling Using a High-Fidelity Simulation-Optimization Framework
Proton exchange membrane (PEM) electrolyzers are widely used for hydrogen production, yet few validated, high-fidelity tools can reliably guide scale-up. Using measured performance from a 50-hour hardware-in-the-loop pilot test, a physics-based, plant-level model of a 1.25 MW PEM electrolyzer and its balance-of-plant (BoP) subsystems is developed and validated. The model couples electrochemistry and thermal/flow submodels and is calibrated against pilot test data via a genetic algorithm (GA) workflow. Validation yields a mean absolute percentage error (APE) of 0.43% for cell voltage and stack power. Two scale-out strategies are then benchmarked under a common 7-day wind-and-photovoltaic (PV) profile: (i) linear duplication of 1.25 MW blocks and (ii) shared-BoP architectures. Sharing BoP between stacks reduces BoP energy by 27% at 10 MW and 34% at 100 MW (vs. linear duplication) and improves system specific energy consumption (SEC) to 52.9 and 52.6 kWh/kg, respectively (from 54.0 kWh/kg with linear duplication). Partial-load studies (25-100% set-point) show that cumulative hydrogen production remains nearly constant down to 50% load because all cases use the same weekly renewable-energy input. Below 50%, the power cap limits how much energy can be used within 168 h, which reduces hydrogen output. The model further indicates that the practical operating optimum lies between 50% and 85% load, where efficiency gains begin to appear without significant loss in hydrogen output. Moreover, the efficiency gains at lower loads are offset by reduced production. The validated framework supports scenario-based engineering trade-off studies for large configurations (10-100 MW) and for operating policies under variable renewables.
08 HYDROGEN↗
Breaking the mold: Overcoming the time constraints of molecular dynamics on general-purpose hardware
The evolution of molecular dynamics (MD) simulations has been intimately linked to that of computing hardware. For decades following the creation of MD, simulations have improved with computing power along the three principal dimensions of accuracy, atom count (spatial scale), and duration (temporal scale). Since the mid-2000s, computer platforms have, however, failed to provide strong scaling for MD, as scale-out central processing unit (CPU) and graphics processing unit (GPU) platforms that provide substantial increases to spatial scale do not lead to proportional increases in temporal scale. Important scientific problems therefore remained inaccessible to direct simulation, prompting the development of increasingly sophisticated algorithms that present significant complexity, accuracy, and efficiency challenges. While bespoke MD-only hardware solutions have provided a path to longer timescales for specific physical systems, their impact on the broader community has been mitigated by their limited adaptability to new methods and potentials. In this work, we show that a novel computing architecture, the Cerebras wafer scale engine, completely alters the scaling path by delivering unprecedentedly high simulation rates up to 1.144 M steps/s for 200 000 atoms whose interactions are described by an embedded atom method potential. This enables direct simulations of the evolution of materials using general-purpose programmable hardware over millisecond timescales, dramatically increasing the space of direct MD simulations that can be carried out. In this paper, we provide an overview of advances in MD over the last 60 years and present our recent result in the context of historical MD performance trends.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Exotanium DOE SBIR Phase I Results Summary
Exotanium demonstrated this technology with the Idaho National Laboratory’s MASTODON application, a Multiphysics environment designed to run typical high-performance computing (HPC) simulations for structural dynamics, seismic analysis, and risk assessment. The MASTODON application was packaged into a container using Docker, Deployed on Amazon ECS, and managed through a custom Scale-Out Compute on AWS (SOCA) implementation.
97 MATHEMATICS AND COMPUTING↗
Exploring DAOS as a Burst Buffer for a 100 Gbps DAQ Real-Time Streaming System
We present an experimental evaluation of a burst buffer for a real-time DAQ streaming system designed to transmit instrument data to remote data centers. The system is based on EJ-FAT, a load balancing system capable of Nx 100Gbps streams, distributing data from event sources to processing nodes. We explore applying the DAOS system as a burst buffer to serve a number of purposes: improve resiliency, elasticity and add new functions into the processing pipeline. In the evaluation a sender transmits events over a 100Gbps network to a receiver integrated with DAOS to store the reassembled events using DAOS APIs. We evaluate the system for possible bottlenecks and provide end-to-end evaluation with a burst buffer using DAOS storage abstractions. We show that a receiver node can support 38.1 Gbps. This proves the viability of our approach and allows us to extend this work to investigate scale-out properties and new streaming optimizations.
Mei, Xinxin↗
Boost the biopharmaceutical industry’s research efficiency with hyperscale data distribution; DOE SBIR Phase I Final Technical Report
Explore the source record for details and available documents.
97 MATHEMATICS AND COMPUTING↗
When to use rsync
We have endeavored to show, using a series of data transfer results obtained from two testbeds, when to use the popular data copying tool rsync and related tools. Tests have been conducted in local area network (LAN) and wide area network (WAN) environments. We conclude that for files in a certain size range and network latency ≦ 10 ms round trip time (RTT), rsync is still useful for data moving tasks in the category 4 of the U.S. DOE Technical Report “Data Movement Categories”. For more demanding data movement requirements, tools of different classes are suggested. Sample histograms from two DOE user facilities are provided to further support our conclusions.
97 MATHEMATICS AND COMPUTING↗