Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data formats”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39

eqtools: Modular, extensible, open-source, cross-machine Python tools for working with magnetic equilibria

As plasma physics research for fusion energy transitions to an increasing emphasis on cross-machine collaboration and numerical simulation, it becomes increasingly important that portable tools be developed to enable data from diverse sources to be analyzed in a consistent manner. This paper presents eqtools, a modular, extensible, open-source toolkit implemented in the Python programming language for handling magnetic equilibria and associated data from tokamaks. eqtools provides a single interface for working with magnetic equilibrium data, both for handling derived quantities and mapping between coordinate systems, extensible to function with data from different experiments, data formats, and magnetic reconstruction codes, replacing the diverse, non-portable solutions currently in use. Moreover, while the open-source Python programming language offers a number of advantages as a scripting language for research purposes, the lack of basic tokamak-specific functionality has impeded the adoption of the language for regular use. Implementing equilibrium-mapping tools in Python removes a substantial barrier to new development in and porting legacy code into Python. In this paper, we introduce the design of the eqtools package and detail the workflow for usage and expansion to additional devices. Additionally, the implementation of a novel three-dimensional spline solution (in two spatial dimensions and in time) is also detailed. Finally, verification and benchmarking for accuracy and speed against existing tools are detailed. Wider deployment of these tools will enable efficient sharing of data and software between institutions and machines as well as self-consistent analysis of the shared data.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Bay Area Regional Energy: Network Integrated Commercial Retrofits (BRICR) Project. Final Report

The BRICR project applied large-scale building energy modeling concepts with the aim of reducing the cost of energy efficiency targeting, design, and project development, and measurement of energy savings for energy efficiency programs implemented by local governments that serve small and medium commercial buildings (SMB). The project leveraged the services and resources of existing local government energy programs serving disadvantaged and hard-to-reach SMB customers. In contrast to programs run by utilities, local government programs generally do not have direct access to energy billing records for an entire class of customers in a geographic area, which prior research demonstrated useful for large-scale building energy model baseline development and calibration. , However, local governments are rich in public records that offer important clues about physical attributes and uses that, along with behavior, determine energy use. Relying only on public records, BRICR demonstrated development of credible baseline energy models for 3,792 office, retail, and hotel buildings. Publicly disclosed annual energy use data from a local energy benchmarking program and anonymized data from the Building Performance Database, the nation’s largest dataset about energy-related characteristics of buildings, were utilized to validate and calibrate energy models via an innovative method comparing distributions of energy intensity by fuel type for portfolios of buildings of similar size, vintage, and use. Portfolio calibration does not provide certainty that an energy model fits an individual building; the method is useful when billing data is not accessible – a common situation for researchers, energy service providers and ESCOs, local governments, and any party other than a utility. A software component was developed, the BRICR gem, which automates simulation when relevant data is added or edited by the user to a file saved in the standardized BuildingSync XML schema for energy audit data. The component was demonstrated as a simplified means to generate a mass of energy models corresponding to public records containing basic attributes such as building scale, location, use, year built, and aspect ratio in combination with building energy code prototype data corresponding to use and vintage. The component was also demonstrated as a simplified means to automate energy simulation when attributes are revised; the intention was to enable iterative improvement of the baseline model and energy savings estimates for common energy conservation measures as users revise relevant attributes based on their observations. In the context of institutional change and uncertainty for the participating local government energy programs, 13 whole building retrofits were completed. Impacts were measured by applying the CalTRACK2.0 methods to standardize measurement of normalized metered energy consumption. The GRIDMeter methods of stratified sampling and individual load shape analysis were applied to adjust for impacts of the effect of COVID-19 on retrofitted buildings in the context of all local buildings of similar size and use. Excluding impacts of the pandemic, retrofitted buildings demonstrated between 1.6% and 25.1% reduction in energy use. The project contributed use cases and feedback that helped inform evolution of the software tools and data formats that were combined for the first time in the BRICR project.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Nimbus-7 Total Ozone Mapping Spectrometer (TOMS) Data Products User's Guide

Two data products from the Total Ozone Mapping Spectrometer (TOMS) onboard Nimbus-7 have been archived at the Distributed Active Archive Center, in the form of Hierarchical Data Format files. The instrument measures backscattered Earth radiance and incoming solar irradiance; their ratio is used in ozone retrievals. Changes in the instrument sensitivity are monitored by a spectral discrimination technique using measurements of the intrinsically stable wavelength dependence of derived surface reflectivity. The algorithm to retrieve total column ozone compares measured Earth radiances at sets of three wavelengths with radiances calculated for different total ozone values, solar zenith angles, and optical paths. The initial error in the absolute scale for TOMS total ozone is 3 percent, the one standard deviation random error is 2 percent, and drift is less than 1.0 percent per decade. The Level-2 product contains the measured radiances, the derived total ozone amount, and reflectivity information for each scan position. The Level-3 product contains daily total ozone amount and reflectivity in a I - degree latitude by 1.25 degrees longitude grid. The Level-3 product also is available on CD-ROM. Detailed descriptions of both HDF data files and the CD-ROM product are provided.

McPeters, Richard D.↗

Solving seismological problems using sgraph program: II-waveform modeling

One of the seismological programs to manipulate seismic data is SGRAPH program. It consists of integrated tools to perform advanced seismological techniques. SGRAPH is considered a new system for maintaining and analyze seismic waveform data in a stand-alone Windows-based application that manipulate a wide range of data formats. SGRAPH was described in detail in the first part of this paper. In this part, I discuss the advanced techniques including in the program and its applications in seismology. Because of the numerous tools included in the program, only SGRAPH is sufficient to perform the basic waveform analysis and to solve advanced seismological problems. In the first part of this paper, the application of the source parameters estimation and hypocentral location was given. Here, I discuss SGRAPH waveform modeling tools. This paper exhibits examples of how to apply the SGRAPH tools to perform waveform modeling for estimating the focal mechanism and crustal structure of local earthquakes.

58 GEOSCIENCES↗

Open Source Release of NJOY2016 and NJOY21

NJOY has been the standard for nuclear data processing for many decades. NJOY is a comprehensive computer code used to produce point-wise and multigroup nuclear data application libraries from cross sections and related quantities obtained from evaluated nuclear data in the Evaluated Nuclear Data Format (ENDF). NJOY was first publicly released in 1977 and distributed by what is now the Radiation Safety Information Computational Center (RSICC) at Oak Ridge National Laboratory and the National Energy Software Center (NESC) at Argonne National Laboratory. From that time until the release of NJOY2012, NJOY has been distributed by RSICC and made available to researchers outside the United States by agreements between RSICC and the NEA Databank. NJOY2012 was distributed by the Feynman Center at Los Alamos National Laboratory which did not have agreements with the NEA Databank, thus making it more difficult for non US citizens to obtain copies of NJOY2012. Furthermore, NJOY has had export control restrictions preventing some researchers from obtaining NJOY at all. Two new versions of NJOY have been released, NJOY2016 and NJOY21, in January of 2017. Both NJOY2016 and NJOY21 have been released with no export control restrictions and are distributed with an open-source license. This is a big departure from previous releases and impacts not only who has access to NJOY, but also how updates are made available to users and how collaboration is facilitated among interested developers. In addition to distributing NJOY2016 and NJOY21 with an open-source license, these two new versions provide new capabilities for NJOY which will maintain NJOY as the standard for nuclear data processing into the foreseeable future. We report in this summary about the changes that come with these two new versions of NJOY and how this impacts the users and long-time contributors to NJOY.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Incident Management Preparedness and Coordination Toolkit

As with many professions, safety planners and first responders tend to be specialists in certain areas. To be truly useful, tools should be tailored to meet their specific needs. Thus, general software suites aimed at the professional geographic information system (GIS) community might not be the best solution for a first responder with little training in GIS terminology and techniques. On the other hand, commonly used web-based map viewers may not have the capability to be customized for the planning, response, and recovery (PR&R) mission. Data formats should be open and foster easy information flow among local, state, and federal partners. Tools should be free or low-cost to address real-world budget constraints at the local level. They also need to work both with and without a network connection to be robust. The Incident Management Preparedness and Coordination Toolkit (IMPACT) can satisfy many of these needs while working in harmony with established systems at the local, state, and federal levels. The IMPACT software framework, termed the Geospatial Integrated Problem Solving Environment (GIPSE), organizes tasks, tools, and resources for the end user. It uses the concept of software wizards to both customize and extend its functionality. On the Tasks panel are a number of buttons used to initiate various operations. Similar to macros, these task buttons launch scripts that utilize the full functionality of the underlying foundational components such as the SQL spatial database and ORNL-developed map editor. The user is presented with a series of instruction pages which are implemented with HTML for interactivity. On each page are links which initiate specific actions such as creating a map showing various features. Additional tasks may be quickly programmed and added to the panel. The end user can customize the graphical interface to faciltate its use during an emergency. One of the major components of IMPACT is the ORNL Geospatial Viewer (OGV). It is used to provide various map displays for the Tasks and Maps panels. Unlike typical maps however, the OGV display is highly interactive and can be customized, automated, and animated to meet the needs of the user. It is also a means to both import and export geospatial data from or to existing GIS systems. The built-in simulation capability can be used as the basis of table-top exercises.

Koch, Daniel B.↗

Supporting ARPA-E Power Grid Optimization (Final Report)

Pacific Northwest National Laboratory (PNNL), Arizona State University (ASU), Georgia Institute of Technology (Georgia Tech), Los Alamos National Laboratory (LANL), National Renewable Energy Laboratory (NREL), Texas A&M University (TAMU), The University of Texas at Austin (UT), and the University of Wisconsin-Madison (UW-M) supported the ARPA-E Grid Optimization (GO) Competition by providing a common problem formulation, data format, datasets, evaluation mechanism, scoring, rules, and results that resulted in the awarding of $\$9.24$ million dollars to teams from academia, industry, and national labs for solving three sets of increasingly difficult non-linear, security- constrained AC Optimal Powerflow (AC-OPF) optimization problems in order to increase the efficiency of the US Electric Grid. It is estimated that a 1% increase in efficiency can save $\$1$ billion. Current industry practices typically use a linear DC model (DC-OPF) in order solve the OPF problem within the time constraints of the operation schedule. The GO Competition challenges the best power engineers, mathematicians, and computer scientists to make possible operational decisions based on accurate physical models. To accomplish this, the GO Competition created a series of Challenges and funded teams to produce the best solver. Challenge 1 was to solve the security constrained Alternating Current Optimal Power Flow (ACOPF) problem. Challenge 2 extended that to by adding adjustable transformer tap ratios, phase shifting transformers, switchable shunts, price-responsive demand, ramp rate constrained generators and loads, and fast-start unit commitment (UC). Furthermore, Challenge 2 was a maximization problem while Challenge 1 was a minimization problem. While Challenge 3 was being developed, the entrants were invited to find better solutions to the Challenge 2 synthetic datasets with no restrictions on time, hardware, or algorithms. The Challenge 2 solutions turned out to be very good. Challenge 3 expanded the Challenge 2 problem further by using multiperiod dynamic markets, including advisory models for extreme weather events, day-ahead markets, and the real-time markets with an extended look-ahead. These problems included active bid-in demand and topology optimization. Together the Challenges used nearly 30 million CPU hours. Since each team was working on the same problem, using the same data, and running on the same hardware, fair comparisons could be drawn as to the best solver. The datasets were varied enough, however, that the best solver for one dataset was not necessarily the best at another, so cumulative scores were used. The process was managed by the PNNL maintained website https://GOCompetition.energy.gov, where Entrants could find information about the problem, the data, the rules, submit their solver for evaluation, and see the scores of all the competing teams on a Leaderboard. Interest was world-wide but only American teams were eligible for prizes. The Competition has produced 34 journal articles 115 papers and been cited over 500 times in the literature, including 12 dissertations (4 from foreign countries; Columbia (2), Germany, and Italy) and 3 from the DOE ExaScale project. Software developed by Pearl Street Technologies for Challenges 1 and 2 is now deployed by Southwest Power Pool (SPP) and Midcontinent Independent Service Operator (MISO). Other teams have received inquiries from venture capitalists. Google DeepMind has thanked the Competition for making the datasets developed for the Competition public. They are using it to train machine learning models. The larger datasets have billions of unknowns to be solved for, but only a small percent matter in the final solution. Knowing what unknowns are important can dramatically speedup the solution.

24 POWER TRANSMISSION AND DISTRIBUTION↗

An Open Source Extensible Smart Energy Framework

Aggregated distributed energy resources are the subject of much interest in the energy industry and are expected to play an important role in meeting our future energy needs by changing how we use, distribute and generate electricity. This energy future includes an increased amount of energy from renewable resources, load management techniques to improve resiliency and reliability, and distributed energy storage and generation capabilities that can be managed to meet the needs of the grid as well as individual customers. These energy assets are commonly referred to as Distributed Energy Resources (DER). DERs rely on a means to communicate information between an energy provider and multitudes of devices. Today DER control systems are typically vendor-specific, using custom hardware and software solutions. As a result, customers are locked into communication transport protocols, applications, tools, and data formats. Today’s systems are often difficult to extend to meet new application requirements, resulting in stranded assets when business requirements or energy management models evolve. By partnering with industry advisors and researchers, an implementation DER research platform was developed called the Smart Energy Framework (SEF). The hypothesis of this research was that an open source Internet of Things (IoT) framework could play a role in creating a commodity-based eco-system for DER assets that would reduce costs and provide interoperable products. SEF is based on the AllJoynTM IoT open source framework. The demonstration system incorporated DER assets, specifically batteries and smart water heaters. To verify the behavior of the distributed system, models of water heaters and batteries were also developed. An IoT interface for communicating between the assets and a control server was defined. This interface supports a series of “events” and telemetry reporting, similar to those defined by current smart grid communication standards. The results of this effort demonstrated the feasibility and application potential of using IoT frameworks for the creation of commodity-based DER systems. All of the identified commodity-based system requirements were met by the AllJoyn framework. By having commodity solutions, small vendors can enter the market and the cost of implementation for all parties is reduced. Utilities and aggregators can choose from multiple interoperable products reducing the risk of stranded assets. Based on this research it is recommended that interfaces based on existing smart grid communication protocol standards be created for these emerging IoT frameworks. These interfaces should be standardized as part of the IoT framework allowing for interoperability testing and certification. Similarly, IoT frameworks are introducing application level security. This type of security is needed for protecting application and platforms and will be important moving forward. Recommendations are that along with DER-based data model interfaces, platform and application security requirements also be prescribed when IoT devices support DER applications.

25 ENERGY STORAGE↗

Vector Topographic Map Data over the BOREAS NSA and SSA in SIF Format

This data set contains vector contours and other features of individual topographic map sheets from the National Topographic Series (NTS). The map sheet files were received in Standard Interchange Format (SIF) and cover the BOReal Ecosystem-Atmosphere Study (BOREAS) Northern Study Area (NSA) and Southern Study Area (SSA) at scales of 1:50,000 and 1:250,000. The individual files are stored in compressed Unix tar archives.

Knapp, David↗

Tomo2Mesh: Fast Reconstruction and Visualization of Tomography Data in Mesh Format

Tomo2Mesh is an open-source project targeted towards real-time reconstruction, segmentation, and visualization of computed tomography (CT) data in mesh format. The CT reconstruction scheme is based on filtered back-projection of voxel subsets. The segmentation scheme uses a 3D convolutional neural network. To allow for fast, real-time reconstruction, voxel subsets are first identified by coarse reconstruction. The detail in specific regions of interest is improved through full reconstruction of voxel subsets in that region. Data structures are implemented to store and process voxel subsets. Finally, voxel data is labeled using connected components to detect disconnected regions such as voids whose morphological attributes (e.g., Feret diameter, principal axis orientation, local number density, size, etc.) can be measured also in real-time. Finally, a fast marching cubes implementation processes labeled voxel data into a triangular face mesh (vertices and faces) in .ply format for visualization in Paraview or other mesh visualization tools. The code provides a simple programming interface for detecting, classifying, and visualizing regions of interest based on morphology. For example, detected voids can be classified as round pores or extended cracks. Highly porous neighborhoods can be identified based on local number density. The mesh texture (or color) is assigned based these morphological attributes to allow smart visualization scenarios in real-time (e.g., show only long cracks). At the time of first release (July 2022), extraction of face mesh for visualization for raw CT data from a 2 megapixel camera would take between 1-5 minutes for most scenarios.

TEKAWADE, ANIKET↗

BRE‐X Emissions Database for End‐of‐Life Scenarios of Selective Building Construction Materials to Enable Circular Economy in Construction

In the United States, construction and demolition debris predominately end up in landfills with minimal end‐of‐life Re‐X (recover, recycle, reuse, etc.) scenarios, resulting in large environmental impacts and lost opportunities for material recovery. Except for concrete and metals, which seem to have a few well‐defined end‐of‐life pathways, there seems to be a lack of well‐documented end‐of‐life scenarios for other construction materials, let alone their emissions data. Hence, there is a need for documented end‐of‐life Re‐X scenarios and end‐of‐life data of more building materials to motivate widespread use of Re‐X strategies in building design. This paper outlines the efforts of the National Renewable Energy Laboratory, Carbon Leadership Forum, Building Transparency, and Skidmore, Owings & Merrill to (a) create an open‐access BRE‐X (Building Re‐X) end‐of‐life emissions database consisting of greenhouse gas emissions data associated with various end‐of‐life scenarios for a select list of high‐impact building construction materials, and (b) integrate the BRE‐X end‐of‐life emissions database with CAD/BIM/LCA tools for evaluating various end‐of‐life scenarios. The paper also presents a few existing life cycle inventory databases that contain sparse amounts of end‐of‐life data for a few construction materials and their limitations in terms of scaling and data consolidation. Finally, a sample of how the collected data can be ingested into whole‐building LCA tools using open data formats and a public access link to the BRE‐X end‐of‐life emissions database is also included.

36 MATERIALS SCIENCE↗

MDSplus quality improvement project

MDSplus is a data acquisition and analysis system used worldwide predominantly in the fusion research community. Development began 29 years ago on the OpenVMS operating system. Since that time there have been many new features added and the code has been ported to many different operating systems. There have been contributions to the MDSplus development from the fusion community in the way of feature suggestions, feature implementations, documentation and porting to different operating systems. The bulk of the development and support of MDSplus, however, has been provided by a relatively small core developer group of three or four members. Given the size of the development team and the large number of users much more effort was focused on providing new features for the community than on keeping the underlying code and documentation up to date with the evolving software development standards. To ensure that MDSplus will continue to provide the needs of the community in the future, the MDSplus development team along with other members of the MDSplus user community has commenced on a major quality improvement project. The planned improvements include changes to software build scripts to better use GNU Autoconf and Automake tools, refactoring many of the source code modules using new language features available in modern compilers, using GNU MinGW-w64 to create MS Windows distributions, migrating to a more modern source code management system, improvement of source documentation as well as improvements to the www.mdsplus.org web site documentation and layout, and the addition of more comprehensive test suites to apply to MDSplus code builds prior to releasing installation kits to the community. This paper should lead to a much more robust product and establish a framework to maintain stability as more enhancements and features are added. Finally, this paper will describe these efforts that are either in progress or planned for the near future.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

MDSplus yesterday, today and tomorrow

MDSplus is a data acquisition and analysis system used worldwide predominantly in the fusion research community. Development began 31 years ago by a collaboration of software developers who were charged with providing a data acquisition system for three new fusion experiments under construction: CMOD at MIT, ZTH at LANL and RFX at Padova, Italy. The design of MDSplus combined the functionality of MDS (MIT/Model Data System developed at MIT for the Alcator and Tara fusion experiments) with new features suggested by the developers from the other laboratories. The development of MDSplus used a RAD (rapid application development) approach before RAD became a mainstream methodology. MDSplus was implemented and ready for the initial operation of CMOD in 1991. Since that time, many other fusion facilities started using MDSplus for data acquisition and/or for exporting their data to other sites. Today MDSplus is still used around the world for fusion energy research, space exploration and other fields of science and technology. Work on MDSplus continues to enhance its capabilities, support more platforms, and improve its reliability. It is anticipated that MDSplus will continue to provide valuable tools for the fusion energy research community. In conclusion, this paper describes some of the history of the MDSplus software, the work that is currently underway, and the plans to enable MDSplus to continue to be available and supported long into the future.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

HIDRA-MAT liquid metal droplet injector for liquid metal applications in HIDRA

A liquid metal droplet injector was explicitly designed for the Hybrid Illinois Device for Research and Applications Material Analysis Test-stand (HIDRA-MAT) at the University of Illinois Urbana-Champaign to prepare liquid metal plasma-facing components (PFCs) for plasma exposure. The design goals were to create a compact, reliable, and robust design that could apply liquid metal droplets in-vacuo to a variety of samples. The injector was designed for liquid metal use, and results discussed pertain to lithium. The injector can produce lithium droplets of consistent size by utilizing programmable piston movement. Here, droplet formation data is presented for four different piston step sizes (0.125 mm, 0.25 mm, 0.5 mm, and 1 mm) at three different nozzle temperatures (185 °C, 230 °C, and 270 °C). Droplets are formed on the injector nozzle tip and a linear shift mechanism vertically translates the droplet to the substrate for application. This design prevents the droplet from inadvertently detaching off the nozzle and falling to the sample. Application of the liquid metal droplet ensures placement consistency on the substrate and helps avoid damage to components in HIDRA-MAT that should not interact with liquid metals. A description of the liquid lithium droplet creation and application results is given and provides additional insight into the cooling and oxidation of lithium droplets under vacuum conditions. A second nozzle was fabricated and demonstrated repeatability in droplet diameter creation having a variance of ±0.14 mm for droplets created from the same number of piston steps. The end-use for the injector on HIDRA-MAT is to apply liquid metals to PFCs and expose them to HIDRA's plasma with subsequent intershot in-vacuo surface analysis in HIDRA-MAT for plasma-material interaction studies.

42 ENGINEERING↗

An investigation into positron emission tomography contouring methods across two treatment planning systems

Positron emission tomography (PET) imaging has been used to provide additional information regarding patient tumor location, size, and staging for radiotherapy treatment planning purposes. This additional information reduces interobserver variability and produces more consistent contouring. It is well recognized that different contouring methodology for PET data results in different contoured volumes. The goal of this study was to compare the difference in PET contouring methods for 2 different treatment planning systems using a phantom dataset and a series of patient datasets. Contouring methodology was compared on the ADAC Pinnacle Treatment Planning System and the CMS XiO Treatment Planning System. Contours were completed on the phantom and patient datasets using a number of PET contouring methods—the standardized uptake value 2.5 method, 30%, 40%, and 50% of the maximum uptake method and the signal to background ratio method. Differences of >15% were observed for PET-contoured volumes between the different treatment planning systems for the same data and the same PET contouring methodology. Contoured volume differences between treatment planning systems were caused by differences in data formatting and display and the different contouring tools available. Differences in treatment planning system as well as contouring methodology should be considered carefully in dose-volume contouring and reporting, especially between centers that may use different treatment planning systems or those that have several different treatment planning systems.

61 RADIATION PROTECTION AND DOSIMETRY↗

Recursive Factorization of the Inverse Overlap Matrix in Linear Scaling Quantum Molecular Dynamics Simulations

We present a reduced complexity algorithm to compute the inverse overlap factors required to solve the generalized eigenvalue problem in a quantum-based molecular dynamics (MD) simulation. Our method is based on the recursive iterative re nement of an initial guess Z of the inverse overlap matrix S. The initial guess of Z is obtained beforehand either by using an approximate divide and conquer technique or dynamically, propagated within an extended Lagrangian dynamics from previous MD time steps. With this formulation, we achieve long-term stability and energy conservation even under incomplete approximate iterative re nement of Z. Linear scaling performance is obtained using numerically thresholded sparse matrix algebra based on the ELLPACK-R sparse matrix data format, which also enables e cient shared memory parallelization. As we show in this article using selfconsistent density functional based tight-binding MD, our approach is faster than conventional methods based on the direct diagonalization of the overlap matrix S for systems as small as a few hundred atoms, substantially accelerating quantum-based simulations even for molecular structures of intermediate size. For a 4,158 atom water-solvated polyalanine system we nd an average speedup factor of 122 for the computation of Z in each MD step.

36 MATERIALS SCIENCE↗

I/O performance studies of analysis workloads on production and dedicated resources at CERN

The recent evolutions of the analysis frameworks and physics data formats of the LHC experiments provide the opportunity of using central analysis facilities with a strong focus on interactivity and short turnaround times, to complement the more common distributed analysis on the Grid. In order to plan for such facilities, it is essential to know in detail the performance of the combination of a given analysis framework, of a specific analysis and of the installed computing and storage resources. This contribution describes performance studies performed at CERN, using the EOS disk-based storage, either directly or through an XCache instance, from both batch resources and highperformance compute nodes which could be used to build an analysis facility. A variety of benchmarks, both synthetic and based on real-world physics analyses and their corresponding input datasets, are utilized. In particular, the RNTuple format from the ROOT project is put to the test and compared to the latest version of the TTree format, and the impact of caches is assessed. In addition, we assessed the difference in performance between the use of storage system specific protocols, like XRootd, and FUSE. The results of this study are intended to be a valuable input in the design of analysis facilities, at CERN and elsewhere.

Sciabà, Andrea↗