Engineering topics

Kettimuthu, Raj

Publications and source records attributed to Kettimuthu, Raj.

3 records

DOE OSTI2024

Globus service enhancements for exascale applications and facilities

Many extreme-scale applications require the movement of large quantities of data to, from, and among leadership computing facilities, as well as other scientific facilities and the home institutions of facility users. These applications, particularly when leadership computing facilities are involved, can touch upon edge cases (e.g., terabyte files) that had not been a focus of previous Globus optimization work, which had emphasized rather the movement of many smaller (megabyte to gigabyte) files. We report here on how automated client-driven chunking can be used to accelerate both the movement of large files and the integrity checking operations that have proven to be essential for large data transfers. In conclusion, we present detailed performance studies that provide insights into the benefits of these modifications in a range of file transfer scenarios.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Virtual Framework for Science Federations with Instruments Access and Control

Experimental science workflows require federations of geographically dispersed science instruments and computing systems connected over a wide-area network. We develop a Virtual Infrastructure Twin (VIT) framework, which is a digital twin of the physical infrastructure that utilizes network virtualization and containerization technologies to support the development and testing of science workflow codes. Using VIT, we illustrate the access to instruments via EPICS system and the orchestration of containerized computations across federation computing systems. We also present a machine learning method to convert VIT throughput measurements to closely match the corresponding physical testbed measurements.

Rao, Nageswara↗

DOE OSTI2021

Leveraging In-Network Computing and Programmable Switches for Streaming Analysis of Scientific Data

With the emergence of programmable network devices that match the performance of fixed function devices, several recent projects have explored in-network computing, where the processing that is traditionally done outside the network is offloaded to the network devices. In-network computing has typically been applied to network functions (e.g., load balancing, NAT, and DNS), caching, data reduction/aggregation, and coordination/consensus functions. In some cases it has been used to accelerate stream-processing tasks that involve small payloads and simple operations. In this work we focus on leveraging in-network computing for stream processing of scientific datasets with large payloads that require complex operations such as floating-point computations and logarithmic functions. We demonstrate in-network computing for a real-world scientific application performing streaming normalization of a 2-D image from a light source experiment. We discuss the challenges we encountered and potential approaches to address them.

Sankaran, Ganesh↗