Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distributed processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

Setting the System Boundaries of “Energy for Water” for Integrated Modeling

Many studies over the last two decades have addressed the “water-energy nexus,” generally defined as the inter-dependency between water and energy in their supply, processing, distribution, and use. The research community currently disaggregates the water-energy nexus into two components: “water for energy” and “energy for water.” While there seems to be clear consensus on the definition of “water for energy”—that is, water required for the extraction, processing, and transformation of energy as well as the irrigation of bioenergy—there has been less agreement on the definition and system boundaries of “energy for water.” Here we represent six integrated assessment modeling teams presently incorporating the hydrologic system and water demands into existing global models of energy, agriculture, land use, and climate. In this article, we propose system boundaries of “energy for water” that are appropriate for integrated energy and water modeling, and introduce a third category of processes that are relevant for the water-energy nexus, but that are not logically classified as either “water for energy” or “energy for water.”

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Mascon distribution techniques for asteroids and comets

The mass-concentration model is an approach that has been used to model the gravitational fields of irregularly shaped bodies such as asteroids and comets. By this approach, the body is treated as a collection of point masses. The method is conceptually simple, easy to program, valid down to the surface, and capable of modeling arbitrary density heterogeneities. How the mass concentrations are distributed as well as how mass is assigned to these concentrations is, however, nontrivial. These aspects significantly affect the accuracy and efficiency of the gravitational model. In this paper, we frame the distribution process in terms of numerical integration applied to finite volume meshes. We describe a new method using unstructured, curvilinear, finite volume meshes to significantly improve the accuracy of the mass-concentration model. We then compare the accuracy and efficiency of several variations of our distribution technique to those from literature using Asteroid Eros and Bennu as example bodies. Our results show that the mascon model can be as accurate as the analytic polyhedral model at the surface using an equivalent number of computational elements—i.e., mascon to surface facets. We report the improvement in the model’s performance can be mainly attributed to the volume mesh topology while mesh curving can provide modest case-dependent improvements.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Data-driven predictive probability density function control of fiber length stochastic distribution shaping in refining process

Pulp is the most important raw material in paper in paper industries, whose Fiber Length Stochastic Distribution (FLSD) shaping directly determines the energy consumption and paper quality of the subsequent papermaking processes. However, the mean and variance are insufficient to describe the output FLSD shaping, which display non-Gaussian distribution properties. Therefore, the traditional control, method based on the mean and variance of fiber length is difficult to control the FLSD shaping effectively. In this paper, a novel data-driven predictive probability density function (PDF) control method is proposed for output FLSD shaping in refining process. Primarily, in order to improve the approximation accuracy of the PDF employing RBF neural network, the parameters (center value and width) of each RBF basis functions are tuned via utilizing iterative learning control (ILC) law, and the corresponding estimations of weights law can be obtained. Secondly, considering that the conventional linear model of weights vector has drawbacks of low accuracy and weak generalization ability, random vector functional link networks (RVFLNs) based data driven nonlinear modelling method is employed to characterize the prediction model between the input variables and weights vector. Finally, in order to reduce the randomness of the output FLSD, a minimum entropy control method under mean constraint is employed to design the predictive PDF controller for the output FLSD shaping with the help of the established nonlinear stochastic distribution model. Both simulations and experiments are given to illustrate the effectiveness and practicability of the proposed method.

Probability debsity function, Fiber length distrib↗
DOE OSTI2012

A Path to Operating System and Runtime Support for Extreme Scale Tools

In this project, we cast distributed resource access as operations on files in a global name space and developed a common, scalable solution for group operations on distributed processes and files. The resulting solution enables tool and middleware developers to quickly create new scalable software or easily improve the scalability of existing software. The cornerstone of the project was the design of a new programming idiom called group file operations that eliminates iterative behavior when a single process must apply the same set of file operations to a group of related files. To demonstrate our novel and scalable ideas for group file operations and global name space composition, we developed a group file system called TBON-FS that leverages a tree-based overlay network (TBON), specifically MRNet, for logarithmic communication and distributed data aggregation. We also developed proc++, a new synthetic file system co-designed for use in scalable group file operations. Over the course of the project, we evaluated the utility and performance of group file operations, global name space composition, TBON-FS, and proc++ in three case studies. The first study focused on the ease in using group file operations and TBON-FS to quickly develop several new scalable tools for distributed system administration and monitoring. The second study evaluated the integration of group file operation and TBON-FS within the Ganglia Distributed Monitoring System to improve its scalability for clusters. The final study involved the integration of group file operations, TBON-FS, and proc++ within TotalView, the widely-used parallel debugger. For this project, the work of the Oak Ridge National Laboratory (ORNL) team occurred primarily in two directions: bringing the MRNet tree-based overlay network (TBON) implementation to the Cray XT platform, and investigating techniques for predicting the performance of MRNet topologies on such systems. Rogue Wave Software (RWS), formerly TotalView Technologies Inc., worked with the University ofWisconsin (UW) team on the design and prototyping of a scalable version of the TotalView debugger. RWS assisted UW with their "proc++" design effort. RWS assisted UW with strategy for integrating proc++ into TotalView.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2012

Quantitative analysis of inclusion distributions in hot pressed silicon carbide

ABSTRACT Depth of penetration measurements in hot pressed SiC have exhibited significant variability that may be influenced by microstructural defects. To obtain a better understanding regarding the role of microstructural defects under highly dynamic conditions; fragments of hot pressed SiC plates subjected to impact tests were examined. Two types of inclusion defects were identified, carbonaceous and an aluminum-iron-oxide phase. A disproportionate number of large inclusions were found on the rubble, indicating that the inclusion defects were a part of the fragmentation process. Distribution functions were plotted to compare the inclusion populations. Fragments from the superior performing sample had an inclusion population consisting of more numerous but smaller inclusions. One possible explanation for this result is that the superior sample withstood a greater stress before failure, causing a greater number of smaller inclusions to participate in fragmentation than in the weaker sample.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Modeling Framework for the Assessment of a Sustainable Hydrogen Production and Supply Chain Network in California

The cost-effective and sustainable deployment of hydrogen supply and demand networks, especially in large economic regions like California, can be challenging considering the spatial-temporal availability and variability of the different actors across the network such as production processes, distribution modes, and end-users. In this presentation, we will provide an overview and demonstration of a modeling framework used to assess the environmental, economic, and human health impacts of plausible hydrogen production and supply chain networks in California. Scenarios focus on green hydrogen production pathways using water electrolysis and biomass gasification. End-use applications included in the model are transit, medium and heavy-duty trucking, port authorities, and power and aviation companies that currently consume natural gas, diesel, and aviation fuel for their day-to-day operation. Representative locations for hydrogen production and end-use are based on recent projections of the hydrogen economy in California. All mass and energy flows, as well as estimated emissions, are based on H2A process model designs and projections of technology performance, literature review, and LBNL process, economic and life cycle modeling, and not on company data for the sake of this presentation. Human health impacts are included following methodologies developed for the University of California Irvine HyDeal project. Life cycle phases associated with hydrogen production include feedstock preparation (water and biomass), energy production and consumption (renewable, grid, and combination of renewable and grid electricity), maintenance (chemical utilization in electrolysis and natural gas combustion in gasification), carbon sequestration, hydrogen storage (compression and liquefaction), and distribution (truck and pipeline). We apply the framework utilizing California specific emission factors, financial data, and human health damages and explore the impact of network characteristics on results. Example variations include: the inclusion of policy incentives or not, different representations of the electricity grid and source, electrolysis versus gasification versus combinations of both for production, liquefaction versus compression based on producer capacity cutoffs, transportation truck versus pipeline based on existing infrastructure, and ultimate end use. Comparison of these different scenarios can help inform future projects by demonstrating the trade-offs among environmental, economic, and human health impacts. This model, automated in R, is a starting platform upon which new analysis, modeling capabilities, locations, and emission factors can be rapidly tested and integrated.

Zaki, Mohammed Tamim↗
DOE OSTI2019

FY 18 Status Report on the Computing Systems for the Yucca Mountain Project TSPA-LA Models and Inventory of Software used for Process Models

Sandia National Laboratories (SNL) continued evaluation of total system performance assessment (TSPA) computing systems for the previously considered Yucca Mountain Project (YMP). This was done to maintain the operational readiness of the computing infrastructure (computer hardware and software) and knowledge capability for total system performance assessment (TSPA) type analysis, as directed by the National Nuclear Security Administration (NNSA), DOE 2010. This work is a continuation of the ongoing readiness evaluation reported in Lee and Hadgu (2014), Hadgu et al. (2015) and Hadgu and Appel (2016), and Hadgu et al. (2017). The TSPA computing hardware (2014 server cluster -CL2014) and storage system described in Hadgu et al. (2015) were used for the current analysis. One floating license of Gold Sim with Versions 9.60.300, 10.5, 11.1 and 12.0 was installed on the cluster head node, and its distributed processing capability was mapped on the cluster processors. Other supporting software were tested and installed to support the TSPA-type analysis on the server cluster. The FY18 task included developing an inventory of software used for the Yucca Mountain Project process models and preliminary assessment of status of the software; enhancing security of the cluster and setting a backup system. The 2014 server cluster and supporting software systems are fully operational to support TSPA-LA type analysis.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2011

Singular value decomposition utilizing parallel algorithms on graphical processors

One of the current challenges in underwater acoustic array signal processing is the detection of quiet targets in the presence of noise. In order to enable robust detection, one of the key processing steps requires data and replica whitening. This, in turn, involves the eigen-decomposition of the sample spectral matrix, Cx = 1/K xKX(k)XH(k) where X(k) denotes a single frequency snapshot with an element for each element of the array. By employing the singular value decomposition (SVD) method, the eigenvectors and eigenvalues can be determined directly from the data without computing the sample covariance matrix, reducing the computational requirements for a given level of accuracy (van Trees, Optimum Array Processing). (Recall that the SVD of a complex matrix A involves determining V, , and U such that A = U VH where U and V are orthonormal and is a positive, real, diagonal matrix containing the singular values of A. U and V are the eigenvectors of AAH and AHA, respectively, while the singular values are the square roots of the eigenvalues of AAH.) Because it is desirable to be able to compute these quantities in real time, an efficient technique for computing the SVD is vital. In addition, emerging multicore processors like graphical processing units (GPUs) are bringing parallel processing capabilities to an ever increasing number of users. Since the computational tasks involved in array signal processing are well suited for parallelization, it is expected that these computations will be implemented using GPUs as soon as users have the necessary computational tools available to them. Thus, it is important to have an SVD algorithm that is suitable for these processors. This work explores the effectiveness of two different parallel SVD implementations on an NVIDIA Tesla C2050 GPU (14 multiprocessors, 32 cores per multiprocessor, 1.15 GHz clock - peed). The first algorithm is based on a two-step algorithm which bidiagonalizes the matrix using Householder transformations, and then diagonalizes the intermediate bidiagonal matrix through implicit QR shifts. This is similar to that implemented for real matrices by Lahabar and Narayanan ("Singular Value Decomposition on GPU using CUDA", IEEE International Parallel Distributed Processing Symposium 2009). The implementation is done in a hybrid manner, with the bidiagonalization stage done using the GPU while the diagonalization stage is done using the CPU, with the GPU used to update the U and V matrices. The second algorithm is based on a one-sided Jacobi scheme utilizing a sequence of pair-wise column orthogonalizations such that A is replaced by AV until the resulting matrix is sufficiently orthogonal (that is, equal to U ). V is obtained from the sequence of orthogonalizations, while can be found from the square root of the diagonal elements of AH A and, once is known, U can be found from column scaling the resulting matrix. These implementations utilize CUDA Fortran and NVIDIA's CUB LAS library. The primary goal of this study is to quantify the comparative performance of these two techniques against themselves and other standard implementations (for example, MATLAB). Considering that there is significant overhead associated with transferring data to the GPU and with synchronization between the GPU and the host CPU, it is also important to understand when it is worthwhile to use the GPU in terms of the matrix size and number of concurrent SVDs to be calculated.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2022

Analysis of contact conditions and microstructure evolution in shear assisted processing and extrusion using smoothed particle hydrodynamics method

Shear assisted processing and extrusion (ShAPE) is a solid-phase processing technique that adds an additional shear force as compared with a conventional extrusion approach. Recently, ShAPE has demonstrated the capability of extruding high-performance aluminum alloy 7075 (AA7075) tubes at speeds up to 12.2 m/min without surface tearing. However, the relationship among the ShAPE processing parameters, thermomechanical conditions, contact conditions, heat generation, and microstructure evolution remains primarily empirical because an insightful understanding of the associated physics is still lacking. To help elucidate these relationships, this work proposes a thermomechanical meshfree model for the first time for ShAPE processing of AA7075 using the smoothed particle hydrodynamics (SPH) method. The meshfree model is first validated thoroughly by experimental data in terms of material flow, die face temperature, and extrusion force with various processing parameters. The validated model is then used to analyze the steady-state contact conditions and heat generation rates during ShAPE processing. Distributions of the average grain size of AA7075 being extruded are calculated using the SPH model output. The meshfree model results reveal that extrusions conducted at lower temperatures and higher strain rates yield more refined grains and possibly higher material strength, which is also consistent with the experimental observations.

36 MATERIALS SCIENCE↗
DOE OSTI2015

EVOLUTION, NUCLEOSYNTHESIS, AND YIELDS OF AGB STARS AT DIFFERENT METALLICITIES. III. INTERMEDIATE-MASS MODELS, REVISED LOW-MASS MODELS, AND THE pH-FRUITY INTERFACE

We present a new set of models for intermediate-mass asymptotic giant branch (AGB) stars (4.0, 5.0, and 6.0 M{sub ⊙}) at different metallicities (−2.15 ≤ [Fe/H] ≤ +0.15). This set integrates the existing models for low-mass AGB stars (1.3 ≤ M/M{sub ⊙} ≤ 3.0) already included in the FRUITY database. We describe the physical and chemical evolution of the computed models from the main sequence up to the end of the AGB phase. Due to less efficient third dredge up episodes, models with large core masses show modest surface enhancements. This effect is due to the fact that the interpulse phases are short and, therefore, thermal pulses (TPs) are weak. Moreover, the high temperature at the base of the convective envelope prevents it from deeply penetrating the underlying radiative layers. Depending on the initial stellar mass, the heavy element nucleosynthesis is dominated by different neutron sources. In particular, the s-process distributions of the more massive models are dominated by the {sup 22}Ne(α,n){sup 25}Mg reaction, which is efficiently activated during TPs. At low metallicities, our models undergo hot bottom burning and hot third dredge up. We compare our theoretical final core masses to available white dwarf observations. Moreover, we quantify the influence intermediate-mass models have on the carbon star luminosity function. Finally, we present the upgrade of the FRUITY web interface, which now also includes the physical quantities of the TP-AGB phase for all of the models included in the database (ph-FRUITY)

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2017

A Systematic Classification for HVAC Systems and Components

Depending on the application, the complexity of an HVAC system can range from a small fan coil unit to a large centralized air conditioning system with primary and secondary distribution loops, and central plant components. Currently, the taxonomy of HVAC systems and the components has various aspects, which can get quite complex because of the various components and system configurations. For example, based on cooling and heating medium delivered to terminal units, systems can be classified as either air systems, water systems or air-water systems. In addition, some of the system names might be commonly used in a confusing manner, such as “unitary system” vs. “packaged system.” Without a systematic classification, these components and system terminology can be confusing to understand or differentiate from each other, and it creates ambiguity in communication, interpretation, and documentation. It is valuable to organize and classify HVAC systems and components so that they can be easily understood and used in a consistent manner. This paper aims to develop a systematic classification of HVAC systems and components. First, we summarize the HVAC component information and definitions based on published literature, such as ASHRAE handbooks, regulations, and rating standards. Then, we identify common HVAC system types and map them to the collected components in a meaningful way. Classification charts are generated and described based on the component information. Six main categories are identified for the HVAC components and equipment, i.e., heating and cooling production, heat extraction and rejection, air handling process, distribution system, terminal use, and stand-alone system. Components for each main category are further analyzed and classified in detail. More than fifty system names are identified and grouped based on their characteristics. The result from this paper will be helpful for education, communication, and systems and component documentation.

Li, Han↗
DOE OSTI2022

Possibility to Identify the Contributions from Collapsars, Supernovae, and Neutron Star Mergers from the Evolution of the r-process Mass Abundance Distribution

We study the evolution of rapid neutron-capture process (r-process) isotopes in the galaxy. We analyze relative contributions from core-collapse supernovae (CCSNe), neutron star mergers, and collapsars under a range of astrophysical conditions and nuclear input data. Here we show that, although the r-process in each of these sites can lead to a similar (universal) elemental distribution, the detailed isotopic abundances can differ from one site to another. These differences may allow for the identification of which sources contributed to the early evolution of r-process material in the galaxy. Our simulations suggest that the early evolution was dominated by CCSNe and collapsar r-process nucleosynthesis. This conclusion may be testable if the next generation of observatories can deduce isotopic r-process abundances.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2019

A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data

Summary There are many data sources that produce large volumes of data. The Big Data nature requires new distributed processing approaches to extract the valuable information. Real‐time sentiment analysis is one of the most demanding research areas that requires powerful Big Data analytics tools such as Spark. Prior literature survey work has shown that, though there are many conventional sentiment analysis researches, there are only few works realizing sentiment analysis in real time. One major point that affects the quality of real‐time sentiment analysis is the confidence of the generated data. In more clear terms, it is a valuable research question to determine whether the owner that generates sentiment is genuine or not. Since data generated by fake personalities may decrease accuracy of the outcome, a smart/intelligent service that can identify the source of data is one of the key points in the analysis. In this context, we include a fake account detection service to the proposed framework. Both sentiment analysis and fake account detection systems are trained and tested using Naïve Bayes model from Apache Spark's machine learning library. The developed system consists of four integrated software components, ie, (i) machine learning and streaming service for sentiment prediction, (ii) a Twitter streaming service to retrieve tweets, (iii) a Twitter fake account detection service to assess the owner of the retrieved tweet, and (iv) a real‐time reporting and dashboard component to visualize the results of sentiment analysis. The sentiment classification performances of the system for offline and real‐time modes are 86.77% and 80.93%, respectively.

Kılınç, Deniz↗
DOE OSTI2011

Parallel application-level behavioral attributes for performance and energy management of high-performance computing systems

Run time variability of parallel applications continues to present significant challenges to their performance and energy efficiency in high-performance computing (HPC) systems. When run times are extended and unpredictable, application developers perceive this as a degradation of system (or subsystem) performance. Extended run times directly contribute to proportionally higher energy consumption, potentially negating efforts by applications, or the HPC system, to optimize energy consumption using low-level control techniques, such as dynamic voltage and frequency scaling (DVFS). Therefore, successful systemic management of application run time performance can result in less wasted energy, or even energy savings. We have been studying run time variability in terms of communication time, from the perspective of the application, focusing on the interconnection network. More recently, our focus has shifted to developing a more complete understanding of the effects of HPC subsystem interactions on parallel applications. In this context, the set of executing applications on the HPC system is treated as a subsystem, along with more traditional subsystems like the communication subsystem, storage subsystem, etc. To gain insight into the run time variability problem, our earlier work developed a framework to emulate parallel applications (PACE) that stresses the communication subsystem. Evaluation of run time sensitivity to network performance of real applications is performed with a tool called PARSE, which uses PACE. In this paper, we propose a model defining application-level behavioral attributes, that collectively describes how applications behave in terms of their run time performance, as functions of their process distribution on the system (spacial locality), and subsystem interactions (communication subsystem degradation). These subsystem interactions are produced when multiple applications execute concurrently on the same HPC system. We also revisit our evaluation framework and tools to demonstrate the flexibility of our application characterization techniques, and the ease with which attributes can be quantified. The validity of the model is demonstrated using our tools with several parallel benchmarks and application fragments. Results suggest that it is possible to articulate application-level behavioral attributes as a tuple of numeric values that describe course-grained performance behavior.

Computer Science↗
DOE OSTI2018

An automatic performance model-based scheduling tool for coupled climate system models

The prediction of the climate system is highly depended on the efficient integration of observations and simulations of the Earth. This is regarded as a canonical example of the cyber-physical system. The climate system model, the simulation engine in this cyber-physical system, is one of most challenging applications in scientific computing. It utilizes the multi-physics simulation that couples multiple components, conducts decadal to millennium simulations, and has long been an important application on supercomputers. However, current climate system models suffer from the inefficient task scheduling methods resulting in an intolerable simulation time. Take the Community Earth System Model (CESM), the most widely used climate system model, as an example, one major reason that CESM suffers from bad performances is the huge overhead to rationally distribute processes among the coupled heterogeneous components. According to the report of NCAR, every percent improvement in CESM performance frees up to the equivalent of $250,000 in computing resources in their scientific experiments. To address such challenge, our paper firstly constructs a lightweight and accurate performance model for effectively capturing and predicting the heterogeneous time-to-solution performance of end-to-end CESM components with a given simulation configuration. Then, based on the performance model, we further propose an efficient scheduling strategy based on rectangular packing method to determine the best process layout among different components, and the process numbers assigned to each component. Our evaluations show that we can achieve 58% average run time reductions on CESM comparing to the widely used sequential process layout for a scale of 144 to 480 cores on typical CPU clusters. And we can save 4 million CPU hours when we conduct one standard scientific experiment (a 2870-year simulation), which equals to save $40,089 with a charge of $0.01 per CPU hour. Meanwhile, 26% extra performance improvements could also be gained in our methods comparing to the heuristic branch and bound algorithm with the guidance of the known curve-fitting performance model.

58 GEOSCIENCES↗
DOE OSTI2016

Development in corrosion resistance by microstructural refinement in Zr-16 SS 304 alloy using suction casting technique

Highlights: • Grain refinement was made in Zr–16 wt.% SS alloy while prepared by suction casting process. • Distribution of Laves phase, e.g., Zr{sub 2}(Fe, Cr) was raised in suction cast (SC) Zr–16 wt.% SS. • Corrosion resistance was improved in SC alloy compared to that of arc-melt-cast alloy. • Grain refinement in SC alloy assisted for an increase in its corrosion resistance. - Abstract: Zirconium (Zr)-stainless steel (SS) hybrid alloys are being considered as baseline alloys for developing metallic-waste-form (MWF) with the motivation of disposing of Zr and SS base nuclear metallic wastes. Zr–16 wt.% SS, a MWF alloy optimized from previous studies, exhibit significant grain refinement and changes in phase assemblages (soft phase: Zr{sub 2}(Fe, Cr)/α-Zr vs. hard phase: Zr{sub 3}(Fe, Ni)) when prepared by suction casting (SC) technique in comparison to arc-cast-melt (AMC) route. Variation in Cr-distribution among different phases are found to be low in suction cast alloy, which along with grain refinement restricted Cr-depletion at the Zr{sub 2}(Fe, Cr)/Zr interfaces, prone to localized attack. Hence, SC alloy, compared to AMC alloy, showed lower current density, higher potential at the breakdown of passivity and higher corrosion potential during polarization experiments (carried out under possible geological repository environments, viz., pH 8, 5 and 1) indicating its superior corrosion resistance.

36 MATERIALS SCIENCE↗
DOE OSTI2011

The message logging system for NOvA experiment

The message logging system provides the infrastructure for all of the distributed processes in the data acquisition (DAQ) to report status messages of various severities in a consistent manner to a central location, as well as providing the tools for displaying and archiving the messages. The message logging system has been developed over a decade, and has been run successfully on CDF and CMS experiments. The most recent work to the message logging system is to build it as a stand-alone package with the name MessageFacility which works for any generic framework or applications, with NO A as the first driving user. System designs and architectures, as well as the efforts of making it a generic library will be discussed. We also present new features that have been added.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2012

Monitoring techniques and alarm procedures for CMS services and sites in WLCG

The CMS offline computing system is composed of roughly 80 sites (including most experienced T3s) and a number of central services to distribute, process and analyze data worldwide. A high level of stability and reliability is required from the underlying infrastructure and services, partially covered by local or automated monitoring and alarming systems such as Lemon and SLS, the former collects metrics from sensors installed on computing nodes and triggers alarms when values are out of range, the latter measures the quality of service and warns managers when service is affected. CMS has established computing shift procedures with personnel operating worldwide from remote Computing Centers, under the supervision of the Computing Run Coordinator at CERN. This dedicated 24/7 computing shift personnel is contributing to detect and react timely on any unexpected error and hence ensure that CMS workflows are carried out efficiently and in a sustained manner. Synergy among all the involved actors is exploited to ensure the 24/7 monitoring, alarming and troubleshooting of the CMS computing sites and services. We review the deployment of the monitoring and alarming procedures, and report on the experience gained throughout the first two years of LHC operation. We describe the efficiency of the communication tools employed, the coherent monitoring framework, the proactive alarming systems and the proficient troubleshooting procedures that helped the CMS Computing facilities and infrastructure to operate at high reliability levels.

Molina-Perez, J.↗