Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “CERN”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 865 records · Page 48
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

STAR Data Production Workflow on HPC: Lessons Learned & Best Practices

The Solenoidal Tracker at RHIC (STAR) is a multi-national supported experiment located at Brookhaven National Lab. The raw physics data captured from the detector is on the order of tens of PBytes per data acquisition campaign, which makes STAR fit well within the definition of a big data science experiment. The production of the data has typically run on standard nodes or on standard Grid computing environments. All embedding simulations (complex workflow mixing real and simulated events) have been run on standard Linux resources at the National Energy Research Scientific Computing Center (NERSC) aka PDSF. However, HPC resources such as Cori have become available for STAR's data production as well as embedding, and STAR has been the very first experiment to show feasibility of running a sustainable data production campaign on this computing resource. The use of Docker containers with Shifter is required to run on HPC @ NERSC - this approach encapsulates the environment in which a standard STAR workflow runs. From the deployment of a tailored Scientific Linux environment (requiring many of its own libraries and special configurations required to run) to the deployment of third-party software and the STAR specific software stack, it has become impractical to rely on a set of containers containing each specific software release. To this extent, solutions based on the CERN VM File System (CVMFS) for the deployment of software and services have been employed in HENP, but one needs to make careful scalability considerations when using a resource like Cori, such as not allowing all software to be deployed in containers or bare node. Additionally, CVMFS clients are not compatible on Cori nodes and one needs to rely on an indirect NFS/DVS mount scheme. In our contribution, we will discuss our strategies from the past and our current solution based on CVMFS. Furthermore, running on HPC is not a simple task as each aspect of the workflow must be enabled to scale, run efficiently, and the workflow needs to fit within the boundaries of the provided queue system (SLURM in this case). Lastly, we will also discuss what we have learned so far about what is the best method for grouping jobs to maximize a single 48 core HPC node within a specific time frame and maximize our workflow efficiency.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

Xenon doping of Liquid Argon in ProtoDUNE Single Phase: first results

Doping Liquid Argon (LAr) with xenon is a known technique to shift the light emitted by argon (128 nm) to a longer wavelength to ease its detection. The largest Xenon doping test ever performed in a LArTPC was carried out in ProtoDUNE Single Phase (ProtoDUNE-SP) at the CERN Neutrino Platform. The response of such a large TPC (7701 of Liquid Argon and 4401 of fiducial mass) has been studied using the ProtoDUNE Photon Detection System (PDS) and a dedicated setup installed before the run. With the first, it was possible to study the light detected in the system as a function of the xenon concentration and to characterise the light collection efficiency with respect to the track position. With the second system it was possible to disentangle the LAr (128 nm) light from the xenon (178 nm) light using two dedicated X-ARAPUCA modules. The run was fully satisfactory, it was possible to measure directly the increase of the xenon light component during doping; furthermore most of the LAr light quenched by impurities was fully recovered even at small Xenon concentration (< 20 ppm in mass). A study of the collected scintillation light as a function of the track position showed an improvement of the detector response uniformity.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

Purity monitoring for ProtoDUNE-SP

The Deep Underground Neutrino Experiment is a next-generation long-baseline neutrino oscillation experiment based on liquid argon time projection chamber technology. DUNE-s single-phase prototype ProtoDUNE-SP at CERN finished its two-year Phase-1 running in July 2021, successfully collected test-beam and cosmic ray data. A key aspect of LArTPC calibration is the lifetime of drift electrons. A purity monitor is a miniature TPC measuring the lifetime of electrons generated from the photocathode via the photoelectric effect. It enables continuous monitoring of the detector status, especially when filling the cryostat and when liquid argon recirculation systems operate. The purity monitoring system in ProtoDUNE-SP Phase-1 monitored liquid argon purity throughout its entire lifetime. It is essential to the experiment’s successful commissioning, operation, and data taking. This poster discusses the design, implementation, and results of purity monitors and plans.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Scintillation light detection in the long-drift ProtoDUNE-DP liquid argon TPC

ProtoDUNE-DP is a 6 × 6 × 6m$^{3}$ liquid argon time-projection-chamber (LArTPC) operated at the Neutrino Platform at CERN in 2019-2020 as a prototype of the DUNE Far Detector. DUNE is a dual-site experiment for long-baseline neutrino oscillation studies, neutrino astrophysics and nucleon decay searches. The light signal in these detectors is crucial to provide precise timing capabilities. In ProtoDUNE-DP, scintillation light produced by cosmic muons in the LArTPC is collected by the photomultiplier tubes (PMTs) placed up to 7 m away from the point of interaction. The scintillation light production and propagation processes are analyzed and compared to simulations, improving the understanding of some liquid argon properties.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Impact of xenon doping in the scintillation light in a large liquid-argon TPC

The use of xenon-doped liquid argon (Xe-doped LAr) is a promising alternative for large-scale liquid argon Time Projection Chambers (LAr-TPC), since it mitigates the light suppression due to impurities and it also improves the photon-detection efficiency and uniformity with the distance. This study analyses the impact of using Xe-doped LAr in ProtoDUNE Dual-Phase, a 750 ton Dual-Phase LAr-TPC placed at CERN. ProtoDUNE Dual-Phase completed a Xe-doping data-taking campaign in summer 2020 by re-filling the detector with 230 tons of Xe-doped LAr contaminated with nitrogen, and performing dedicated nitrogen injections. The effects of the presence of Xe at 5.8 ppm in the scintillation light production and propagation are analysed in this paper, showing an increase of the collected photons, but a suppression of the light signal amplitude. A 60% increase of the light attenuation length is measured. The impact on the scintillation time profile is also studied. A model to fit the time profile is proposed and the time constants of the physics processes are obtained.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Extending the distributed computing infrastructure of the CMS experiment with HPC resources

Particle accelerators are an important tool to study the fundamental properties of elementary particles. Currently the highest energy accelerator is the LHC at CERN, in Geneva, Switzerland. Each of its four major detectors, such as the CMS detector, produces dozens of Petabytes of data per year to be analyzed by a large international collaboration. The processing is carried out on the Worldwide LHC Computing Grid, that spans over more than 170 compute centers around the world and is used by a number of particle physics experiments. Recently the LHC experiments were encouraged to make increasing use of HPC resources. While Grid resources are homogeneous with respect to the used Grid middleware, HPC installations can be very different in their setup. In order to integrate HPC resources into the highly automatized processing setups of the CMS experiment a number of challenges need to be addressed. For processing, access to primary data and metadata as well as access to the software is required. At Grid sites all this is achieved via a number of services that are provided by each center. However at HPC sites many of these capabilities cannot be easily provided and have to be enabled in the user space or enabled by other means. At HPC centers there are often restrictions regarding network access to remote services, which is again a severe limitation. The paper discusses a number of solutions and recent experiences by the CMS experiment to include HPC resources in processing campaigns.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Slicing with deep learning models at ProtoDUNE-SP

DUNE is a cutting-edge experiment aiming to study neutrinos in detail, with a special focus on the flavor oscillation mechanism. The prototype of the DUNE Far Detector Single Phase TPC (ProtoDUNE-SP) was built and operated at CERN with a full set of reconstruction tools. To implement these reconstruction tools, Pandora, a multi-algorithm framework, has been developed. A large number of these algorithms, some of them being exploiting traditional clustering, detector physics and deep learning approaches, have been applied to images to gradually build up a picture out of singular events. One of such algorithms is the Pandora slicing algorithm which aims to partition the detector hits of an event in sets called slices. Each slice represents a single interaction in the detector and should identify all the hits related to the interacting particle and its subsequent decay products. We expect the order of tens of slices per event in ProtoDUNE-SP. In this paper we present a deep learning approach to the problem, designing a model able to outperform the state-of-the-art slicing algorithm which is currently implemented within Pandora. We assess the performance of our tool in terms of efficiency and accuracy, while exploiting hardware accelerating setups. The ultimate goal is to incorporate this deep learning approach in the Pandora reconstruction tool.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Sparse Convolutional Neural Networks for particle classification in ProtoDUNE-SP events

Deep Learning (DL) methods and Computer Vision are becoming important tools for event reconstruction in particle physics detectors. In this work, we report on the use of submanifold sparse convolutional neural networks (SparseNets) for the classification of track and shower hits from a DUNE prototype liquid-argon detector at CERN (ProtoDUNE-SP). By taking advantage of the three-dimensional nature of the problem we use a set of nine input features to classify sparse and locally dense hits associated to track or shower particles. The SparseNet has been trained on a test sample and shows promising results: efficiencies and purities greater than 90%. This has also been achieved with a considerable speedup and substantially less resource utilization with respect to other DL networks such as graph neural networks. This method offers great scalability advantages for future large neutrino detectors such as the planned DUNE experiment.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2014

10 Gbps TCP/IP streams from the FPGA for High Energy Physics

The DAQ system of the CMS experiment at CERN collects data from more than 600 custom detector Front-End Drivers (FEDs). During 2013 and 2014 the CMS DAQ system will undergo a major upgrade to address the obsolescence of current hardware and the requirements posed by the upgrade of the LHC accelerator and various detector components. For a loss-less data collection from the FEDs a new FPGA based card implementing the TCP/IP protocol suite over 10Gbps Ethernet has been developed. To limit the TCP hardware implementation complexity the DAQ group developed a simplified and unidirectional but RFC 793 compliant version of the TCP protocol. This allows to use a PC with the standard Linux TCP/IP stack as a receiver. We present the challenges and protocol modifications made to TCP in order to simplify its FPGA implementation. We also describe the interaction between the simplified TCP and Linux TCP/IP stack including the performance measurements.

Bauer, Gerry↗
DOE OSTI2014

Data Bookkeeping Service 3 - Providing Event Metadata in CMS

The Data Bookkeeping Service 3 provides a catalog of event metadata for Monte Carlo and recorded data of the Compact Muon Solenoid (CMS) experiment at the Large Hadron Collider (LHC) at CERN, Geneva. It comprises all necessary information for tracking datasets, their processing history and associations between runs, files and datasets, on a large scale of about 200, 000 datasets and more than 40 million files, which adds up in around 700 GB of metadata. The DBS is an essential part of the CMS Data Management and Workload Management (DMWM) systems [1], all kind of data-processing like Monte Carlo production, processing of recorded event data as well as physics analysis done by the users are heavily relying on the information stored in DBS.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2014

WLCG and IPv6 - The HEPiX IPv6 working group

The HEPiX (http://www.hepix.org) IPv6 Working Group has been investigating the many issues which feed into the decision on the timetable for the use of IPv6 (http://www.ietf.org/rfc/rfc2460.txt) networking protocols in High Energy Physics (HEP) Computing, in particular in the Worldwide Large Hadron Collider (LHC) Computing Grid (WLCG). RIPE NCC, the European Regional Internet Registry (RIR), ran out ofIPv4 addresses in September 2012. The North and South America RIRs are expected to run out soon. In recent months it has become more clear that some WLCG sites, including CERN, are running short of IPv4 address space, now without the possibility of applying for more. This has increased the urgency for the switch-on of dual-stack IPv4/IPv6 on all outward facing WLCG services to allow for the eventual support of IPv6-only clients. The activities of the group include the analysis and testing of the readiness for IPv6 and the performance of many required components, including the applications, middleware, management and monitoring tools essential for HEP computing. Many WLCG Tier 1/2 sites are participants in the group's distributed IPv6 testbed and the major LHC experiment collaborations are engaged in the testing. We are constructing a group web/wiki which will contain useful information on the IPv6 readiness of the various software components and a knowledge base (http://hepix-ipv6.web.cern.ch/knowledge-base). Furthermore, this paper describes the work done by the working group and its future plans.

43 PARTICLE ACCELERATORS↗
DOE OSTI2015

How do quarks and gluons lose energy in the QGP?

RHIC introduced the method of hard scattering of partons as an in-situ probe of the the medium produced in A+A collisions. A suppression, R AA ≈ 0.2 relative to binary-scaling, was discovered for π⁰ production in the range 5 < ρ T < 20 GeV/c in central Au+Au collisions at √s NN = 200 GeV, and surprisingly also for single-electrons from the decay of heavy quarks. Both these results have been confirmed in Pb+Pb collisions at the LHC at √s NN = 2.76 TeV. Interestingly, in this ρ T range the LHC results for pions nearly overlap the RHIC results. Thus, due to the flatter spectrum, the energy loss in the medium at LHC in this ρ T range must be ~ 40% larger than at RHIC. Unique at the LHC are the beautiful measurements of the fractional transverse momentum imbalance 1 – (ρ-carot T 2 /ρ-carot T 1 ) of di-jets in Pb+Pb collisions. At the Utrecht meeting in 2011, I corrected for the fractional imbalance of di-jets with the same cuts in p-p collisions and showed that the relative fractional jet imbalance in Pb+Pb/p-p is ≈ 15% for jets with 120 < ρ-carot T 1 < 360 GeV/c. CMS later confirmed this much smaller imbalance compared to the same quantity derived from two-particle correlations of di-jet fragments at RHIC corresponding to ρ-carot T jet ≈ 10 – 20 GeV/c, which appear to show a much larger fractional jet imbalance ≈ 45% in this lower ρ-carot T range. The variation of apparent energy loss in the medium as a function of both ρ T and √s NN is striking and presents a challenge to both theory and experiment for improved understanding. There are many other such unresolved issues, for instance, the absence of evidence for a q-carot effect, due to momentum transferred to the medium by outgoing partons, which would widen the away-side di-jet and di-hadron correlations in a similar fashion as the k T -effect. Another issue well known from experiments at the CERN ISR, SpS and SpS collider is that parton-parton hard-collisions make negligible contribution to multiplicity or transverse energy production in p-p collisions–soft particles, with ρ T < 2 GeV/c, predominate. Thus an apparent hard scattering component for A+A multiplicity distributions based on a popular formula, dN AA ch /dη = [(1 - x) (N part )dN pp ch /dη2 + x (N coll dN pp ch /dη], seems to be an unphysical way to understand the deviation from N part scaling. Based on recent p-p and d+A measurements, a more physical way is presented along with several other stimulating results and ideas from recent d+Au (p+Pb) measurements.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2015

Next Generation Workload Management System For Big Data on Heterogeneous Distributed Computing

The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the fundamental nature of matter and the basic forces that shape our universe, and were recently credited for the discovery of a Higgs boson. ATLAS and ALICE are the largest collaborations ever assembled in the sciences and are at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, both experiments rely on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System (WMS) for managing the workflow for all data processing on hundreds of data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data centers are physically scattered all over the world. The scale is demonstrated by the following numbers: PanDA manages O(10 2 ) sites, O(10 5 ) cores, O(10 8 ) jobs per year, O(10 3 ) users, and ATLAS data volume is O(10 17 ) bytes. In 2013 we started an ambitious program to expand PanDA to all available computing resources, including opportunistic use of commercial and academic clouds and Leadership Computing Facilities (LCF). The project titled 'Next Generation Workload Management and Analysis System for Big Data' (BigPanDA) is funded by DOE ASCR and HEP. Extending PanDA to clouds and LCF presents new challenges in managing heterogeneity and supporting workflow. The BigPanDA project is underway to setup and tailor PanDA at the Oak Ridge Leadership Computing Facility (OLCF) and at the National Research Center "Kurchatov Institute" together with ALICE distributed computing and ORNL computing professionals. Our approach to integration of HPC platforms at the OLCF and elsewhere is to reuse, as much as possible, existing components of the PanDA system. Finally, we will present our current accomplishments with running the PanDA WMS at OLCF and other supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facilities infrastructure for High Energy and Nuclear Physics as well as other data-intensive science applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

The diverse use of clouds by CMS

The resources CMS is using are increasingly being offered as clouds. In Run 2 of the LHC the majority of CMS CERN resources, both in Meyrin and at the Wigner Computing Centre, will be presented as cloud resources on which CMS will have to build its own infrastructure. This infrastructure will need to run all of the CMS workflows including: Tier 0, production and user analysis. In addition, the CMS High Level Trigger will provide a compute resource comparable in scale to the total offered by the CMS Tier 1 sites, when it is not running as part of the trigger system. During these periods a cloud infrastructure will be overlaid on this resource, making it accessible for general CMS use. Finally, CMS is starting to utilise cloud resources being offered by individual institutes and is gaining experience to facilitate the use of opportunistically available cloud resources. Lastly, we present a snap shot of this infrastructure and its operation at the time of the CHEP2015 conference.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Open access to high-level data and analysis tools in the CMS experiment at the LHC

The CMS experiment, in recognition of its commitment to data preservation and open access as well as to education and outreach, has made its first public release of high-level data under the CC0 waiver: up to half of the proton-proton collision data (by volume) at 7 TeV from 2010 in CMS Analysis Object Data format. CMS has prepared, in collaboration with CERN and the other LHC experiments, an open-data web portal based on Invenio. The portal provides access to CMS public data as well as to analysis tools and documentation for the public. The tools include an event display and histogram application that run in the browser. In addition a virtual machine containing a CMS software environment along with XRootD access to the data is available. Within the virtual machine the public can analyse CMS data, example code is provided. As a result, we describe the accompanying tools and documentation and discuss the first experiences of data use.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2015

Monitoring data transfer latency in CMS computing operations

During the first LHC run, the CMS experiment collected tens of Petabytes of collision and simulated data, which need to be distributed among dozens of computing centres with low latency in order to make efficient use of the resources. While the desired level of throughput has been successfully achieved, it is still common to observe transfer workflows that cannot reach full completion in a timely manner due to a small fraction of stuck files which require operator intervention.For this reason, in 2012 the CMS transfer management system, PhEDEx, was instrumented with a monitoring system to measure file transfer latencies, and to predict the completion time for the transfer of a data set. The operators can detect abnormal patterns in transfer latencies while the transfer is still in progress, and monitor the long-term performance of the transfer infrastructure to plan the data placement strategy.Based on the data collected for one year with the latency monitoring system, we present a study on the different factors that contribute to transfer completion time. As case studies, we analyze several typical CMS transfer workflows, such as distribution of collision event data from CERN or upload of simulated event data from the Tier-2 centres to the archival Tier-1 centres. For each workflow, we present the typical patterns of transfer latencies that have been identified with the latency monitor.We identify the areas in PhEDEx where a development effort can reduce the latency, and we show how we are able to detect stuck transfers which need operator intervention. Lastly, we propose a set of metrics to alert about stuck subscriptions and prompt for manual intervention, with the aim of improving transfer completion times.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Optimizing CMS build infrastructure via Apache Mesos

The Offline Software of the CMS Experiment at the Large Hadron Collider (LHC) at CERN consists of 6M lines of in-house code, developed over a decade by nearly 1000 physicists, as well as a comparable amount of general use open-source code. A critical ingredient to the success of the construction and early operation of the WLCG was the convergence, around the year 2000, on the use of a homogeneous environment of commodity x86-64 processors and Linux.Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications, or frameworks. It can run Hadoop, Jenkins, Spark, Aurora, and other applications on a dynamically shared pool of nodes. Lastly, we present how we migrated our continuous integration system to schedule jobs on a relatively small Apache Mesos enabled cluster and how this resulted in better resource usage, higher peak performance and lower latency thanks to the dynamic scheduling capabilities of Mesos.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2015

Online Data Handling and Storage at the CMS Experiment

During the LHC Long Shutdown 1, the CMS Data Acquisition (DAQ) system underwent a partial redesign to replace obsolete network equipment, use more homogeneous switching technologies, and support new detector back-end electronics. The software and hardware infrastructure to provide input, execute the High Level Trigger (HLT) algorithms and deal with output data transport and storage has also been redesigned to be completely file- based. All the metadata needed for bookkeeping are stored in files as well, in the form of small documents using the JSON encoding. The Storage and Transfer System (STS) is responsible for aggregating these files produced by the HLT, storing them temporarily and transferring them to the T0 facility at CERN for subsequent offline processing. The STS merger service aggregates the output files from the HLT from ~62 sources produced with an aggregate rate of ~2GB/s. An estimated bandwidth of 7GB/s in concurrent read/write mode is needed. Furthermore, the STS has to be able to store several days of continuous running, so an estimated of 250TB of total usable disk space is required. In this article we present the various technological and implementation choices of the three components of the STS: the distributed file system, the merger service and the transfer system.

Andre, J. M.↗