Engineering Papers⌕ Search

Engineering topics

Coimbra, Bruno

Publications and source records attributed to Coimbra, Bruno.

Transitioning GlideinWMS, a multi domain distributed workload manager, from GSI proxies to tokens and other granular credentials

GlideinWMS is a distributed workload manager that has been used in production for many years to provision resources for experiments like CERN’s CMS, many Neutrino experiments, and the OSG. Its security model was based mainly on GSI (Grid Security Infrastructure), using X.509 certificate proxies and VOMS (Virtual Organization Membership Service) extensions. Even when other credentials, like SSH keys, were used to authenticate with resources, proxies were also added all the time, to establish the identity of the requestor and the associated memberships or privileges. This single credential was used for everything and was, often implicitly, forwarded wherever needed. The addition of identity and access tokens and the phase-out of GSI forced us to reconsider the security model of GlideinWMS, to handle multiple credentials which can differ in type, technology, and functionality. Both identity tokens and access tokens are supported. GSI proxies even if no more mandatory, are still used, together with various JWT (JSON Web Token) based tokens and other certificates. The functionality of the credentials, defined by issuer, audience, and scope, also differ: a credential can allow access to a computing resource, or can protect the GlideinWMS framework from tampering, or can grant read or write access to storage, can provide an identity for accounting or auditing, or can provide a combination of any the formers. Furthermore, the tools in use do not include automatic forwarding and renewal of the new credentials so credential lifetime and renewal requirements became part of the discussion as well. In this paper, we will present how GlideinWMS was able to change its design and code to respond to all these changes.

97 MATHEMATICS AND COMPUTING↗

Generalizing HEPCloud Split Starter for Multiple Sites and Experiments

This talk discusses changes we made to the HEPCloud Split Starter to be able to reuse the software in multiple HPC sites across different experiments. I will explain the changes we made to the configuration system, how we generalized the project’s scripts, and the steps we took to reduce the application’s footprint on HPC shared storage, among other details.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

FERRY Frontier Experiments RegistRY

The Frontier Experiments RegistRY (FERRY) service provides a centralized repository for the access control and job management attributes such as batch and storage access policies, quotas, batch priorities and NIS attributes for cluster configuration...

Altunay, Mine↗

Continuous Integration Service at Fermilab

This paper describes the current architecture of Continuous Integration (CI) service developed at Fermilab, encountered successes and difficulties, as well as future development plans. The current experiment code has hundreds of contributors that provide new features, bug fixes, and other improvements. Version control systems help developers to collaborate in contributing software for their experiments, while the CI system helps developers keep their code healthy. The Fermilab CI service allows experiments and projects to test and validate their offline production and analysis code on the supported platforms. It is built on top of Jenkins, designed to be set up from a configuration file that provides implementation for each phase of the CI workflow, and able to validate experiment code through grid jobs. This CI service provides a dashboard for easy access to logs and statistical graphs. Since the CI service has been adopted by Fermilab experiments/projects, it proved to be very useful to intercept issues in their code early on and get them fixed before running it in production. Currently the CI service is in use by the ArgoNeuT, DUNE, g-2, LArIAT, MINERvA, mu2e, NOvA, SBND and uBooNE experiments and by the following projects: ART and LArSoft software suites, GENIE, and Glidein-WMS. The CI service is under active development and planning to support code profiling.

Di benedetto, Vito↗

Advances and Enhancements in the Fabric for Frontier Experiments Project at Fermilab

The FabrIc for Frontier Experiments (FIFE) project within the Fermilab Scientific Computing Division is charged with integrating offline computing components into a common computing stack for the non-LHC Fermilab experiments, supporting experiment offline computing, and consulting on new, novel workflows. We will discuss the general FIFE onboarding strategy, the upgrades and enhancements in the FIFE toolset, and plans for the coming year. These enhancements include: expansion of opportunistic computing resources (including GPU and high-performance computing resources) available to experiments; assistance with commissioning computing resources at European sites for individual experiments; StashCache repositories for experiments; enhanced job monitoring tools; and a custom workflow management service. Additionally we have completed the first phase of a Federated Identity Management system to make it easier for FIFE users to access Fermilab computing resources.

Herner, Kenneth↗