Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “FILES”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,639 records · Page 91

Exploration Gap Assessment (FY13 Update)

This submission contains an update to the previous Exploration Gap Assessment funded in 2012, which identify high potential hydrothermal areas where critical data are needed (gap analysis on exploration data). The uploaded data are contained in two data files for each data category: A shape (SHP) file containing the grid, and a data file (CSV) containing the individual layers that intersected with the grid. This CSV can be joined with the map to retrieve a list of datasets that are available at any given site. A grid of the contiguous U.S. was created with 88,000 10-km by 10-km grid cells, and each cell was populated with the status of data availability corresponding to five data types: 1. well data 2. geologic maps 3. fault maps 4. geochemistry data 5. geophysical data

15 GEOTHERMAL ENERGY↗

Task-parallel message passing interface implementation of Autodock4 for docking of very large databases of compounds using high-performance super-computers

A message passing interface (MPI)-based implementation (Autodock4.lga.MPI) of the grid-based docking program Autodock4 has been developed to allow simultaneous and independent docking of multiple compounds on up to thousands of central processing units (CPUs) using the Lamarkian genetic algorithm. The MPI version reads a single binary file containing precalculated grids that represent the protein-ligand interactions, i.e., van der Waals, electrostatic, and desolvation potentials, and needs only two input parameter files for the entire docking run. In comparison, the serial version of Autodock4 reads ASCII grid files and requires one parameter file per compound. The modifications performed result in significantly reduced input/output activity compared with the serial version. Autodock4.lga.MPI scales up to 8192 CPUs with a maximal overhead of 16.3%, of which two thirds is due to input/output operations and one third originates from MPI operations. The optimal docking strategy, which minimizes docking CPU time without lowering the quality of the database enrichments, comprises the docking of ligands preordered from the most to the least flexible and the assignment of the number of energy evaluations as a function of the number of rotatable bounds. In 24 h, on 8192 high-performance computing CPUs, the present MPI version would allow docking to a rigid protein of about 300K small flexible compounds or 11 million rigid compounds.

59 BASIC BIOLOGICAL SCIENCES↗

Transforming Dermatologic Imaging for the Digital Era: Metadata and Standards

Imaging is increasingly being used in dermatology for documentation, diagnosis, and management of cutaneous disease. The lack of standards for dermatologic imaging is an impediment to clinical uptake. Standardization can occur in image acquisition, terminology, interoperability, and metadata. This paper presents the International Skin Imaging Collaboration position on standardization of metadata for dermatologic imaging. Metadata is essential to ensure that dermatologic images are properly managed and interpreted. There are two standards-based approaches to recording and storing metadata in dermatologic imaging. The first uses standard consumer image file formats, and the second is the file format and metadata model developed for the Digital Imaging and Communication in Medicine (DICOM) standard. DICOM would appear to provide an advantage over using consumer image file formats for metadata as it includes all the patient, study, and technical metadata necessary to use images clinically. Whereas, consumer image file formats only include technical metadata and need to be used in conjunction with another actor—for example, an electronic medical record—to supply the patient and study metadata. The use of DICOM may have some ancillary benefits in dermatologic imaging including leveraging DICOM network and workflow services, interoperability of images and metadata, leveraging existing enterprise imaging infrastructure, greater patient safety, and better compliance to legislative requirements for image retention.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Optimization of SAMtools sorting using OpenMP tasks

SAMtools is a widely-used genomics application for post-processing high-throughput sequence alignment data. Such sequence alignment data are commonly sorted to make downstream analysis more efficient. However, this sorting process itself can be computationally- and I/O-intensive: high-throughput sequence alignment files in the de facto standard binary alignment/map (BAM) format can be many gigabytes in size, and may need to be decompressed before sorting and compressed afterwards. As a result, BAM-file sorting can be a bottleneck in genomics workflows. Here, this paper describes a case study on the performance analysis and optimization of SAMtools for sorting large BAM files. OpenMP task parallelism and memory optimization techniques resulted in a speedup of 5.9X versus the upstream SAMtools 1.3.1 for an internal (in-memory) sort of 24.6 GiB of compressed BAM data (102.6 GiB uncompressed) with 32 processor cores, while a 1.98X speedup was achieved for an external (out-of-core) sort of a 271.4 GiB BAM file.

97 MATHEMATICS AND COMPUTING↗

CALPHAD Uncertainty Quantification and TDBX

CALPHAD uncertainty quantification (UQ) is the foundation of materials design with quantified confidence. We report a framework and software packages to enable CALPHAD UQ assessment and calculation using commercial CALPHAD software (Thermo-Calc). This Bayesian inference framework is coupled with a Markov chain Monte Carlo algorithm to establish uncertainty traces with a given thermodynamic database file (TDB) and corresponding experimental data points. This general framework is demonstrated with the Ni–Cr binary system. The algorithm is firstly validated on synthetic data with known ground truth. Then it is applied to real experimental data to generate posterior traces. We develop a file format named TDBX, which provides a single source of truth by combining the original TDB content and the traces for each assessed Gibbs energy parameter. CALPHAD UQ calculations are performed based on the TDBX file, from which uncertainties for phase boundaries, enthalpy curves, and solidification range are collected as examples of basic design parameters. This TDBX file with corresponding scripts are made open-source. Finally, the combination of CALPHAD UQ assessments and calculations connected by TDBX supports uncertainty-assisted modeling, enabling the integrated application of modern design with uncertainty methodologies to computational materials design.

36 MATERIALS SCIENCE↗

The Electrolyte Genome project: A big data approach in battery materials discovery

We present a high-throughput infrastructure for the automated calculation of molecular properties with a focus on battery electrolytes. The infrastructure is largely open-source and handles both practical aspects (input file generation, output file parsing, and information management) as well as more complex problems (structure matching, salt complex generation, and failure recovery). Using this infrastructure, we have computed the ionization potential (IP) and electron affinities (EA) of 4830 molecules relevant to battery electrolytes (encompassing almost 55,000 quantum mechanics calculations) at the B3LYP/6-31+G(*) level. We describe automated workflows for computing redox potential, dissociation constant, and salt-molecule binding complex structure generation. We present routines for automatic recovery from calculation errors, which brings the failure rate from 9.2% to 0.8% for the QChem DFT code. Automated algorithms to check duplication between two arbitrary molecules and structures are described. We present benchmark data on basis sets and functionals on the G2-97 test set; one finding is that a IP/EA calculation method that combines PBE geometry optimization and B3LYP energy evaluation requires less computational cost and yields nearly identical results as compared to a full B3LYP calculation, and could be suitable for the calculation of large molecules. Our data indicates that among the 8 functionals tested, XYGJ-OS and B3LYP are the two best functionals to predict IP/EA with an RMSE of 0.12 and 0.27 eV, respectively. Application of our automated workflow to a large set of quinoxaline derivative molecules shows that functional group effect and substitution position effect can be separated for IP/EA of quinoxaline derivatives, and the most sensitive position is different for IP and EA. Published by Elsevier B.V

Battery↗

ENDFtk: A robust tool for reading and writing ENDF-formatted nuclear data

ENDFtk is a recently developed C++ and Python interface to interact with ENDF-6 formatted nuclear data files. It provides a robust and complete interface, allowing the reading and writing of all formats currently part of the ENDF-6 formats manual, as well as some non-ENDF formats used by the NJOY processing code. It provides an interface that mimics the names in the ENDF-6 formats manual as well as an equivalent interface using human-readable attribute names. It is robust and powerful enogh for nuclear data experts to develop complex applications, while also simple enough to be used non-experts to retrieve and manipulate evaluated nuclear data. ENDFtk offers the ability to easily interrogate and manipulate data either in large-scale code projects or in simple Python scripts. Here, in this paper, a brief overview of the interface is given, as well as more substantial examples demonstrating plotting simple data, interacting with more complex data, and writing new data to files. ENDFtk is open source and available for download via GitHub (https://github.com/njoy/ENDFtk).

97 MATHEMATICS AND COMPUTING↗

An Anatomically Validated Brachial Plexus Contouring Method for Intensity Modulated Radiation Therapy Planning

Purpose: To develop contouring guidelines for the brachial plexus (BP) using anatomically validated cadaver datasets. Magnetic resonance imaging (MRI) and computed tomography (CT) were used to obtain detailed visualizations of the BP region, with the goal of achieving maximal inclusion of the actual BP in a small contoured volume while also accommodating for anatomic variations. Methods and Materials: CT and MRI were obtained for 8 cadavers positioned for intensity modulated radiation therapy. 3-dimensional reconstructions of soft tissue (from MRI) and bone (from CT) were combined to create 8 separate enhanced CT project files. Dissection of the corresponding cadavers anatomically validated the reconstructions created. Seven enhanced CT project files were then automatically fitted, separately in different regions, to obtain a single dataset of superimposed BP regions that incorporated anatomic variations. From this dataset, improved BP contouring guidelines were developed. These guidelines were then applied to the 7 original CT project files and also to 1 additional file, left out from the superimposing procedure. The percentage of BP inclusion was compared with the published guidelines. Results: The anatomic validation procedure showed a high level of conformity for the BP regions examined between the 3-dimensional reconstructions generated and the dissected counterparts. Accurate and detailed BP contouring guidelines were developed, which provided corresponding guidance for each level in a clinical dataset. An average margin of 4.7 mm around the anatomically validated BP contour is sufficient to accommodate for anatomic variations. Using the new guidelines, 100% inclusion of the BP was achieved, compared with a mean inclusion of 37.75% when published guidelines were applied. Conclusion: Improved guidelines for BP delineation were developed using combined MRI and CT imaging with validation by anatomic dissection.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Evaluation of Tungsten Nuclear Reaction Data with Covariances

As a follow-up of the work presented at the ND-2007 conference in Nice, additional fast reactor benchmarks were analyzed. Adjustment to the cross sections in the keV region was necessary. Evaluated neutron cross section data files for {sup 180,182,183,184,186}W isotopes were produced. Covariances were generated for all isotopes except {sup 180}W. In the resonance range the retro-active method was used. Above the resolved resonance range the covariance prior was generated by the Monte Carlo technique from nuclear model calculations with the Empire-II code. Experimental data were taken into account through the GANDR system using the generalized least-squares technique. Introducing experimental data results in relatively small changes in the cross sections, but greatly constrains the uncertainties. The covariance files are currently undergoing testing.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Nuclear Data Sheets for A=170☆

Nuclear structure data pertaining to all nuclei with mass number A=170 have been evaluated, and incorporated into the ENSDF data file. This evaluation supersedes the previous evaluation (Coral M. Baglin), Nuclear Data Sheets 96, 611 (2002), (cutoff date 15 August 2002) and subsequent updates of the ENSDF file for Yb to Dy (cutoff 15 April 2010), Lu (cutoff 8 October 1998), Pt (cutoff 17 February 1999) and Ta, Os (cutoff 7 April 1998). Since the previous evaluation, {sup 170}Gd has been observed and the ground state T{sub 1/2} measured. New excited states have also been observed in the neutron rich {sup 170}Dy. The ground state T{sub 1/2} of {sup 170}Tb has been determined, however, there are currently still no excited states observed in this nucleus. There remains some uncertainty in the ground state and longer-lived isomer in {sup 170}Ir which could warrent further experimental investigation.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Data Processing for the SNS EQ-SANS Diffractometer

The scattering data on the Extended Q-Range Small Angle Scattering Diffractometer at the Spallation Neutron Source are stored as neutron events. Each event has two parts: the time of fight of the detected neutron, and an identifier for the neutron's location on the detector. In addition, each event is associated with the source pulse from which the neutron is originated. Due to the high neutron beam intensity at the SNS, the size of the stored data files can easily exceed several Gigabytes. Processing and reducing these data can be very time consuming. The standard way of handling the data at the SNS has been to first transform the data into histograms. On the EQ-SANS, these histograms are further processed into intensity I vs. neutron momentum transfer Q files, I(Q). For a large data, the whole process can take many minutes to hours to complete, which severely limits the turnaround time of an experiment. Experimenters often want to reduce the data quickly and depending on the result, make decisions on the setup of next measurements. They may also need to reduce the data using different inputs and conditions. In this work, we present the implementation of a fast data processing scheme for the EQ-SANS diffractometer. The scattering data are processed directly from neutron event files into I(Q) or other desired formats, greatly reducing the needed computation time. Further speed gains are obtained by the implementation of parallel computing under the Message Passing Interface framework. Most data on the EQ-SANS can now be handled in mere seconds. The process is highly automated. All relevant experimental information that is stored in instrument configuration files are read and processed. In addition, the new implementation is highly flexible. All the data corrections and handlings can be controlled transparently through user inputs. Finally, software implementation enables the slicing of neutron event, which can be used for the study of time dependent phenomena.

Data Processing↗

dCache - Joining the noWORM Storage Club

For over a decade, dCache.ORG has provided robust software, called dCache, that is used at more than 80 universities and research institutes around the world, allowing these sites to provide reliable storage services for the WLCG experiments and many other scientific communities. The flexible architecture of dCache allows running it in a wide variety of configurations and platforms - from all-in-one Raspberry-Pi up to hundreds of nodes in multi-petabyte infrastructures. The life cycle of scientific data is well defined - collected, processed, archived and finally deleted, when it’s not needed anymore. Moreover, during all those stages the data is never modified: either the original data is used, or new derived data is produced. With this knowledge, dCache was designed to handle immutable files as efficiently as possible. Data replication, HSM connectivity and data-server independent operations are only possible due to the immutable nature of stored data. Nowadays many commercial vendors provide such write-once-read-many or WORM storage systems, as they become more and more demanded with grown demand of audio, photo and video content in the web. On the other hand by providing standard NFSv4.1 interface dCache is often used as a general-purpose file-system, especially by new communities, like photon scientists or microbiologists. Although many users are aware of data immutability, some applications and use cases still require in-place updates of stored files. To satisfy new requirements some fundamental changes have to be applied to dCache’s core design. However, new developments must not compromise any aspect of existing functionality. In this presentation we will show new developments in dCache to turn it into a regular file system. We will discuss the challenges to build a distributed storage system, ‘life’ with POSIX compliance, handling of multiple replicas and backward compatibility by providing WORM and noWORM capabilities within the same storage system.

Mkrtchyan, Tigran↗

Parallel Event Selection on HPC Systems

In their recent measurement of the neutrino oscillation parameters,NOvA uses a sample of approximately 25 million reconstructed spills to searchfor electron-neutrino appearance events. These events are stored in an n-tupleformat, in 250 thousand ROOT files. File sizes range from a few hundred KiB toa few MiB; the full dataset is approximately 1.4 TiB. These millions of eventsare reduced to a few tens of events by the application of strict event selectioncriteria, and then summarized by a handful of numbers each, which are used inthe extraction of the neutrino oscillation parameters.The NOvA event selection code is currently a serial C++ program that readsthese n-tuples. The current table data format and organization and the selection/reduction processing involved provides us with an opportunity to explorealternate approaches to represent the data and implement the processing. Werepresent our n-tuple data in HDF5 format that is optimized for the HPC environmentand which allows us to use the machine’s high-performance parallelI/O capabilities. We use MPI, numpy and h5py to implement our approach andcompare the performance with the existing approach. We study the performanceimplications of using thousands of small files of different sizes as comparedwith one large file using HPC resources. This work has been done as part of theSciDAC project, “HEP analytics on HPC” in collaboration with the ASCR teamsat ANL and LBNL.

Paterno, Marc↗

A fully unprivileged CernVM-FS

The CernVM File System provides the software and container distribution backbone for most High Energy and Nuclear Physics experiments. It is implemented as a file system in user-space (Fuse) module, which permits its execution without any elevated privileges. Yet, mounting the file system in the first place is handled by a privileged suid helper program that is installed by the Fuse package on most systems. The privileged nature of the mount system call is a serious hindrance to running CernVM-FS on opportunistic resource and supercomputers. Fortunately, recent developments in the Linux kernel and in the Fuse user-space libraries enabled fully unprivileged mounting for Fuse file systems (as of RHEL 8), or at least outsourcing the privileged mount system call to a custom, external process. This opens the door to several, very appealing new ways to use CernVM-FS, such as a generally usable “super pilot” consisting of the pilot code bundled with Singularity and CernVM-FS, or the on-demand instantiation of unprivileged, ephemeral containers to publish new CernVM-FS content from anywhere. In this contribution, we discuss the integration of these new Linux features with CernVM-FS and show some of its most promising, new applications.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Scale tests of the new DUNE data pipeline

In preparation for the second runs of the ProtoDUNE detectors at CERN (NP02 and NP04)[1], DUNE has established a new data pipeline for bringing the data from the EHN-1 experimental hall at CERN to primary tape storage at Fermilab and CERN, and then spreading it out to a distributed disk data store at many locations around the world. This system includes a new Ingest Daemon and a new Declaration Daemon. The Rucio[2] replica catalog, and FTS3 transport are used to transport all files. All file metadata is declared to the new MetaCat[3] metadata service. All of these new components have been successfully tested at a scale equal to the expected output of the detector data acquisition system (~2-4 GB/s), and the expected network bandwidth out of the experimental hall. We present the procedure that was used to test and the results of the test.

Timm, Steven↗

Darshan for HEP applications

Modern HEP workflows must manage increasingly large and complex data collections. HPC facilities may be employed to help meet these workflows’ growing data processing needs. However, a better understanding of the I/O patterns and underlying bottlenecks of these workflows is necessary to meet the performance expectations of HPC systems.Darshan is a lightweight I/O characterization tool that captures concise views of HPC application I/O behavior. It intercepts application I/O calls at runtime, records file access statistics for each process, and generates log files detailing application I/O access patterns.Typical HEP workflows include event generation, detector simulation, event reconstruction, and subsequent analysis stages. A study of the I/O behavior of the ATLAS simulation and filtering stage, and the CMS simulation workflow using Darshan is presented, including insights into the I/O operations and data access size.

Wang, Rui↗

The git based ATLAS data acquisition configuration service in LHC Run 3

The ATLAS experiment at the LHC at CERN uses a large, distributed trigger and data acquisition system composed of many computing nodes, networks, and hardware modules. Its configuration service is used to provide descriptions of control, monitoring, diagnostic, recovery, dataflow and data quality configurations, interconnections, and parameters for modules, chips, and channels of various online systems, detectors, and the whole ATLAS experiment. Those descriptions have historically been stored in more than one thousand interconnected XML files, which are updated by various experts many times per day. Maintaining error-free and consistent sets of such files and providing reliable and fast access to current and historical configurations is a major challenge. This paper gives details of the configuration service upgrade on the modern Git version control system backend for LHC Run 3 and its exploitation experience. It may be interesting for developers using human-readable file formats, where consistency of the files, performance, access control, traceability of modifications, and effective archiving are key requirements.

Soloviev, Igor [Univ. of California, Irvine, CA (U↗