DYnamic and Asynchronous Data Streamliner

Abstract

DYAD aims to help sharing data files between producer and consumer job elements, especially between co-scheduled jobs or within an ensemble. DYAD provides the service by two components: a FLUX module and a I/O wraper set. DYAD transparently synchronizes file I/O between producer and consumer, and transfers data from the producer location to the consumer location managed by the service. Users only need to use the file path that is under the directory managed by the service.