Engineering Papers⌕ Search

Engineering topics

Randles, Timothy C.

Publications and source records attributed to Randles, Timothy C..

29 records · Page 2

BEE - FY20 P6-3: Release BEEWorkflowManager, BEETaskManager, and client application 2.3.6.01 – LANL ATDM ST / STNS01-4 P6 Milestone Completion Documentation

Release BEEWorkflowManager, BEETaskManager, and client software. The BEEWorkflowManager daemon runs on the HPC cluster login node. It accepts workflows submitted by the BEE client. These workflows are specified using the Common Workflow Language (CWL) standard. The BEEWorkflowManager loads workflows into the Neo4j graph database to create the workflow directed acyclic graph (DAG), and submits the workflow tasks to the BEETaskManager for execution. The BEEWorkflowManager records the state of the workflow and its tasks, and communicates this state to the BEE client. The BEEWorkflowManager will start, pause, and cancel a running workflow and its tasks at the command of the BEE client. The BEETaskManager daemon runs on the HPC cluster login node. It accepts tasks from the BEEWorkflowManager, turns those tasks into HPC resource manager jobs (e.g. a slurm job script), and submits the job to the cluster resource manager. The BEETaskManager then tracks the status of the job (pending, running, complete) and updates the BEEWorkflowManager. The BEETaskManager will also cancel a queued or running job when commanded to do so by the BEEWorkflowManager. The first release of the BEETaskManager will support the Slurm resource manager and the Charliecloud linux container runtime.

97 MATHEMATICS AND COMPUTING↗

Linux containers for fun and profit in HPC

This article outlines options for user-defined software stacks from an HPC perspective. Here, we argue that a lightweight approach based on Linux containers is most suitable for HPC centers because it provides the best balance between maximizing service of user needs and minimizing risks. We also discuss how containers work and several implementations, including Charliecloud, our own open-source solution developed at Los Alamos.

97 MATHEMATICS AND COMPUTING↗

Next Generation Infrastructure Plan FY18-FY22

Define a 2-5 year plan that documents facilities, operations, networking, storage and user support efforts to concentrate and coordinate resources within the ASC FOUS Program.

97 MATHEMATICS AND COMPUTING↗

Charliecloud: Unprivileged containers for user-defined software stacks in HPC

Supercomputing centers are seeing increasing demand for user-defined software stacks (UDSS), instead of or in addition to the stack provided by the center. These UDSS support user needs such as complex dependencies or build requirements, externally required configurations, portability, and consistency. The challenge for centers is to provide these services in a usable manner while minimizing the risks: security, support burden, missing functionality, and performance. We present Charliecloud, which uses the Linux user and mount namespaces to run industry-standard Docker containers with no privileged operations or daemons on center resources. Our simple approach avoids most security risks while maintaining access to the performance and functionality already on offer, doing so in less than 500 lines of code. Charliecloud promises to bring an industry-standard UDSS user workflow to existing, minimally altered HPC resources.

97 MATHEMATICS AND COMPUTING↗

Flexible HPC Environments [Poster]

New computational paradigms require new tools and environments. A new class of data-analysis tools are available for scientists. These tools do not integrate well with traditional HPC resource management systems. Differing and often conflicting requirements exist. Some fields require validated environments including high energy physics and cosmology. By adopting cloud technologies in an HPC environment we are merging the compute-intensive and data-intensive worlds in one converged system. HPC Systems will support a Larger Set of Technologies. Scientists will be able to leverage the latest tools on HPC resources at previously unavailable scales. User-defined environments will enable rapid experimentation and adoption of emerging tools. HPC systems will support a consolidated workflow from simulation to analysis and discovery. HPC systems will have increased utility to a broader set of users.

97 MATHEMATICS AND COMPUTING↗