Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Metadata”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,189 records · Page 66
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Aerial imagery dataset of lost oil wells

Orphaned wells are wells for which the operator is unknown or insolvent. The location of hundreds of thousands of these wells remain unknown in the United States alone. Cost-effective techniques are essential to locate orphaned wells to address environmental problems. In this paper, we present a dataset consisting of 120,948 aerial images of recently documented orphan wells. Each of these 512 × 512 images is paired with segmentation masks that indicate the presence or absence of such well. These images, sourced from the National Agriculture Imagery Program, cover the continental United States with spatial resolutions ranging from 30 centimeters to 1 meter. Additionally, we included negative examples by selecting locations uniformly across the United States. Accompanying metadata includes the IDs and spatial resolution of the original images, which are available for free through the United States Geological Survey, and the pixel coordinates of documented orphaned wells identified in these images. This dataset is intended to support the development of deep-learning models that can help locating undocumented orphan wells from such imagery, thereby blunting the environmental damage they do.

Climate-change mitigation↗
DOE OSTI2025

International database of reference gamma spectra for nuclear safeguards applications

Nuclear safeguards missions use gamma spectroscopy as a non-destructive measurement technique for examining nuclear materials. Despite advances in the development of detection equipment as well as software codes, one of the concerns is the lack of well-documented spectra needed to test and validate isotopic analysis codes for their applicability. To address this need, this work introduces IDB, an international database of the reference gamma spectra of uranium, plutonium and mixed oxide nuclear material samples. IDB provides access to well-characterized sets of gamma spectra described by rich metadata, including information on the sample, measurement configuration and detector specifications. These spectra are accessible in different formats, also compatible with analysis code standards, thus promoting their sustainability and maintenance.

None, Dipti [International Atomic Energy Agency (I↗
DOE OSTI2025

A global database of soil microbial phospholipid fatty acids and enzyme activities

Abstract Soil microbes drive ecosystem function and play a critical role in how ecosystems respond to global change. Research surrounding soil microbial communities has rapidly increased in recent decades, and substantial data relating to phospholipid fatty acids (PLFAs) and potential enzyme activity have been collected and analysed. However, studies have mostly been restricted to local and regional scales, and their accuracy and usefulness are limited by the extent of accessible data. Here we aim to improve data availability by collating a global database of soil PLFA and potential enzyme activity measurements from 12,258 georeferenced samples located across all continents, 5.1% of which have not previously been published. The database contains data relating to 113 PLFAs and 26 enzyme activities, and includes metadata such as sampling date, sample depth, and soil pH, total carbon, and total nitrogen. This database will help researchers in conducting both global- and local-scale studies to better understand soil microbial biomass and function.

Science & Technology - Other Topics↗
DOE OSTI2014

A comprehensive collection of systems biology data characterizing the host response to viral infection

The Systems Biology for Infectious Diseases Research program was established by the U.S. National Institute of Allergy and Infectious Diseases to investigate host-pathogen interactions at a systems level. This program generated 47 transcriptomic and proteomic datasets from 30 studies that investigate in vivo and in vitro host responses to viral infections. Human pathogens in the Orthomyxoviridae and Coronaviridae families, especially pandemic H1N1 and avian H5N1 influenza A viruses and severe acute respiratory syndrome coronavirus (SARS-CoV), were investigated. Study validation was demonstrated via experimental quality control measures and meta-analysis of independent experiments performed under similar conditions. Primary assay results are archived at the GEO and PeptideAtlas public repositories, while processed statistical results together with standardized metadata are publically available at the Influenza Research Database (www.fludb.org) and the Virus Pathogen Resource (www.viprbrc.org). As a result, by comparing data from mutant versus wild-type virus and host strains, RNA versus protein differential expression, and infection with genetically similar strains, these data can be used to further investigate genetic and physiological determinants of host responses to viral infection.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Transitioning CMS to Rucio Data Managment

Following a thorough review in 2018, the CMS experiment at the CERN LHC decided to adopt Rucio as its new data management system. Rucio is emerging as a community software project and will replace an aging CMSonly system before the start-up of LHC Run 3 in 2021. Rucio was chosen after an evaluation determined that Rucio could meet the technical and scale needs of CMS. The data management system for CMS needs to manage the current data sample of approximately 200 PB of data with 1 PB of transfers per day. The data management system must also have a development path suitable for LHC Run 4 (2026) data rates, which are expected to be 50 times larger.This contribution details the ongoing CMS adoption process as we replace our legacy system with Rucio, focusing on the challenges of integrating Rucio into an established set of experimental tools and procedures. This will include the migration of metadata, the construction of interfaces to the rest of the CMS computing tools, scale tests, operations, monitoring, and the plan to gradually turn over primary responsibility for the management of data to Rucio. A description of CMS user data management with Rucio will also be provided.

Vaandering, Eric↗
DOE OSTI2020

ServiceX A Distributed, Caching, Columnar Data Delivery Service

We will describe a component of the Intelligent Data Delivery Service being developed in collaboration with IRIS-HEP and the LHC experiments. ServiceX is an experiment-agnostic service to enable on-demand data delivery specifically tailored for nearly-interactive vectorized analysis. This work is motivated by the data engineering challenges posed by HL-LHC data volumes and the increasing popularity of python and Spark-based analysis workflows.ServiceX gives analyzers the ability to query events by dataset metadata. It uses containerized transformations to extract just the data required for the analysis. This operation is colocated with the data to avoid transferring unnecessary branches over the WAN. Simple filtering operations are supported to further reduce the amount of data transferred.Transformed events are cached in a columnar datastore to accelerate delivery of subsequent similar requests. ServiceX will learn commonly related columns and automatically include them in the transformation to increase the potential for cache hits by other users.Selected events are streamed to the analysis system using an efficient wire protocol that can be readily consumed by a variety of computational frameworks. This reduces time-to-insight for physics analysis by delegating to ServiceX the complexity of event selection, slimming, reformatting, and streaming.

Galewsky, B.↗
DOE OSTI2025

dCache project status and update

The dCache project delivers an open-source, massively scalable, distributed storage system deployed internationally to satisfy today’s scientists’ ever-demanding storage requirements. Its multifaceted approach supports different use cases with the same storage, from high throughput data ingest, data sharing over wide area networks, efficient access from HPC clusters, and longterm data persistence on tertiary storage. Even though dCache was initially developed for HEP experiments, today, it is used by various scientific communities, including astrophysics, biomed, and life science, each with their specific requirements. To match the needs of these new communities and keep up with the scaling demands of existing experiments, dCache is permanently evolving. With this contribution, we would like to highlight the recent developments in dCache regarding integration with CERN Tape Archive (CTA), advanced metadata handling, token-based authorization support, bulk API for QoS transitions, REST API to control interaction with the tape system, and future development directions.

Mkrtchyan, Tigran [DESY]↗
DOE OSTI2025

Label-based Virtual Directories In dCache

Traditional filesystems organize data in directories. These directories are typically a collection of files whose grouping is based on a single criterion, e.g., the starting date of an experiment, experiment name, beamline ID, measurement device, or instrument. However, each file in a directory can belong to several logical groups, such as a special event type, experiment condition, or a part of a selected dataset. dCache is a storage system developed to store large amounts of scientific data, used by many HEP and Photon Science experiments. With recent developments in dCache, we have introduced a concept of file tagging, which dynamically groups files with the same label into virtual directories. The file labels can be added, removed, renamed, and deleted through the admin interface or via REST API. The files in virtual directories are exposed through all protocols supported by dCache. This contribution will describe the details of the implementation for file tagging in dCache and present our future development plans on automatic metadata extractions, a feature that will significantly simplify data management. Additionally, we are exploring the future use of virtual directories as a way to translate scientific data catalogs into filesystem views for direct data analysis.

Sahakyan, Marina [DESY]↗
DOE OSTI2023

Templates of expected measurement uncertainties for prompt fission neutron spectra

In this paper, we provide templates of uncertainty sources expected to appear for three measurement types of prompt fission neutron spectra (PFNS): (1) shape measurements, (2) clean-ratio shape, that is the monitor PFNS are measured in nearly exactly the same surrounding as the PFNS of interest, and (3) indirect ratios, where the detector efficiency is backed out from PFNS monitor measurements. Information is also listed that is needed to faithfully include PFNS in nuclear data evaluations to guide experimenters on how to best report data and metadata for their measurements. These templates also suggest a typical range of pertinent uncertainty values and their correlations in case realistic uncertainties cannot be estimated from information on the measurement itself. The templates were based on a literature review, information found in EXFOR for 252 Cf, 235, 238 U, and 239 Pu PFNS, and enhanced by expertise from experimenters contributing to these PFNS templates.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2017

An HDF5-Based Framework for the Distribution and Analysis of Ultrasonic Concrete Data

There are many commercial ultrasonic tomography devices (UTDs) available for use in nondestructive evaluation (NDE) of reinforced concrete structures. These devices emit, measure, and store ultrasonic signals typically in the 25 kHz to 5 MHz frequency range. UTDs are characterized by a composition of multiple transducers, also known as a transducer array or phased array. Often, UTDs data are in a proprietary format. Consequently, NDE research data is limited to those who have prior non-disclosure agreements or the appropriate licenses. Thus, there is a need for a proper universal data framework to exist such that proprietary file datasets for different concrete specimens can be converted, organized, and stored with relative metadata for individual or collaborative NDE research. Building upon the Hierarchical Data Format (HDF5) model, we have developed a UTD data management framework and Graphic User Interface (GUI) to promote the algorithmic reconstruction of ultrasonic data in a controlled environment for easily reproducible and publishable results.

concrete NDE↗
DOE OSTI2018

Time filtering of event based neutron scattering data: A pathway to study the dynamic structural responses of materials

Time-resolved diffraction has become a vital tool for probing dynamic responses to an applied stimulus. Such experiments traditionally use hardware solutions to histogram measured data into their respective bin. We will show that a major advantage of event-based data acquisition, which time-stamps measured diffraction data with 100 ns accuracy, is much preferred over hardware histogramming of the data by enabling postprocessing for advanced custom binning using a software solution. This approach is made even more powerful by coupling measured diffraction data with metadata about the applied stimuli and material response. In this work, we present a time-filter approach that leverages the power of event-based diffraction collection to reduce stroboscopic data measured over many hours into equally weighted segments that represent subsets of the response to a single cycle of the applied stimulus. We demonstrate this approach by observing ferroelectric/ferroelastic domain wall motion during electric field cycling of BaTiO 3 . Furthermore, the developed approach can readily be expanded to investigate other dynamic phenomena using complex sample environments.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

The fast camera (Fastcam) imaging diagnostic systems on the DIII-D tokamak

Two camera systems are installed on the DIII-D tokamak at the toroidal positions of 90° (90° system) and 225° (225° system), respectively. The cameras have two types of relay optics, namely, a coherent optical fiber bundle and a periscope system. The periscope system provides absolute intensity calibration stability while sacrificing resolution (10 lp/mm), while the fiber system provides high resolution (16 lp/mm) while sacrificing calibration stability. The periscope is available only for the 90° system. The optics of the 225° system were designed for view stability, repeatability, and easy maintenance. The cameras are located inside optimized neutron, x ray and magnetic shielding in order to reduce electronics damage, reboots, and magnetic and neutron interference, increasing the overall system reliability. An automated filter wheel, providing remote filter change, allows for remote wavelength selection. A software suite automates camera acquisition and data storage, allowing for remote operation and reduced operator involvement. System metadata is used to streamline the data analysis workflow, particularly for intensity calibration. Here, the spatial calibration uses multiple observable wall features, resulting in a reconstruction accuracy ≤2 cm.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Fermionic quantum processing with programmable neutral atom arrays

Simulating the properties of many-body fermionic systems is an outstanding computational challenge relevant to material science, quantum chemistry, and particle physics.-5.4pc]Please note that the spelling of the following author names in the manuscript differs from the spelling provided in the article metadata: D. González-Cuadra, D. Bluvstein, M. Kalinowski, R. Kaubruegger, N. Maskara, P. Naldesi, T. V. Zache, A. M. Kaufman, M. D. Lukin, H. Pichler, B. Vermersch, Jun Ye, and P. Zoller. The spelling provided in the manuscript has been retained; please confirm. Although qubit-based quantum computers can potentially tackle this problem more efficiently than classical devices, encoding nonlocal fermionic statistics introduces an overhead in the required resources, limiting their applicability on near-term architectures. In this work, we present a fermionic quantum processor, where fermionic models are locally encoded in a fermionic register and simulated in a hardware-efficient manner using fermionic gates. We consider in particular fermionic atoms in programmable tweezer arrays and develop different protocols to implement nonlocal gates, guaranteeing Fermi statistics at the hardware level. We use this gate set, together with Rydberg-mediated interaction gates, to find efficient circuit decompositions for digital and variational quantum simulation algorithms, illustrated here for molecular energy estimation. Finally, we consider a combined fermion-qubit architecture, where both the motional and internal degrees of freedom of the atoms are harnessed to efficiently implement quantum phase estimation as well as to simulate lattice gauge theory dynamics.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2016

Analysis of soil eDNA functional genes: potential to increase profitability and sustainability of pastoral agriculture

Management of soil biological resources to optimise plant production, efficiency of nutrient inputs, and system sustainability is an emerging opportunity for pastoral agriculture. To attain these goals, suitable tools that can assess the functional state of the soil ecosystem must be developed and standardised approaches to their application adopted. Towards this end, we have undertaken comprehensive, high-density functional-gene microarray analysis (GeoChip5) of environmental DNA (eDNA) extracted from 50 pastoral soils. When combined with soil, environmental and management metadata, the information can be used to provide insights into soil biological processes spanning greenhouse gas emissions, through to natural suppression of plant root diseases. To give an example of a structured workflow of analysis in a pastoral system context, we analysed the GeoChip data using a combination of approaches spanning routine univariate methods through to more complex multivariate and network-based analysis. Analyses were restricted to comparing effects of land-use (dairy or ‘other’ farming systems), and exploring relationships of the GeoChip data with the soil properties from each sample. These exemplar analyses introduce a pathway for the application of eDNA approaches (GeoChip or others) to deliver outcomes for pastoral agricultural in New Zealand.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2014

Qualities of sequential chromospheric brightenings observed in Hα and UV images

Chromospheric flare ribbons observed in Hα appear well-organized when first examined: ribbons impulsively brighten, morphologically evolve, and exponentially decay back to pre-flare levels. Upon closer inspection of the Hα flares, there is often a significant number of compact areas brightening in concert with the flare eruption but are spatially separated from the evolving flare ribbon. One class of these brightenings is known as sequential chromospheric brightenings (SCBs). SCBs are often observed in the immediate vicinity of erupting flares and are associated with coronal mass ejections. In the past decade there have been several previous investigations of SCBs. These studies have exclusively relied upon Hα images to discover and analyze these ephemeral brightenings. This work employs the automated detection algorithm of Kirk et al. to extract the physical qualities of SCBs in observations of ground-based Hα images and complementary Atmospheric Imaging Assembly images in He II, C IV, and 1700 Å. The metadata produced in this tracking process are then culled using complementary Doppler velocities to isolate three distinguishable types of SCBs. From a statistical analysis, we find that the SCBs at the chromospheric Hα layer appear earlier and last longer than their corresponding signatures measured in AIA. From this multi-layer analysis, we infer that SCBs are spatially constrained to the mid-chromosphere. We also derive an energy budget to explain SCBs which have a postulated energy of not more than 0.01% of the total flare energy.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Extending the distributed computing infrastructure of the CMS experiment with HPC resources

Particle accelerators are an important tool to study the fundamental properties of elementary particles. Currently the highest energy accelerator is the LHC at CERN, in Geneva, Switzerland. Each of its four major detectors, such as the CMS detector, produces dozens of Petabytes of data per year to be analyzed by a large international collaboration. The processing is carried out on the Worldwide LHC Computing Grid, that spans over more than 170 compute centers around the world and is used by a number of particle physics experiments. Recently the LHC experiments were encouraged to make increasing use of HPC resources. While Grid resources are homogeneous with respect to the used Grid middleware, HPC installations can be very different in their setup. In order to integrate HPC resources into the highly automatized processing setups of the CMS experiment a number of challenges need to be addressed. For processing, access to primary data and metadata as well as access to the software is required. At Grid sites all this is achieved via a number of services that are provided by each center. However at HPC sites many of these capabilities cannot be easily provided and have to be enabled in the user space or enabled by other means. At HPC centers there are often restrictions regarding network access to remote services, which is again a severe limitation. The paper discusses a number of solutions and recent experiences by the CMS experiment to include HPC resources in processing campaigns.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2014

A Data Handling System for Modern and Future Fermilab Experiments

Current and future Fermilab experiments such as Minerva, NOνA, and MicroBoone are now using an improved version of the Fermilab SAM data handling system. SAM was originally used by the CDF and D0 experiments for Run II of the Fermilab Tevatron to provide file metadata and location cataloguing, uploading of new files to tape storage, dataset management, file transfers between global processing sites, and processing history tracking. However SAM was heavily tailored to the Run II environment and required complex and hard to deploy client software, which made it hard to adapt to new experiments. The Fermilab Computing Sector has progressively updated SAM to use modern, standardized, technologies in order to more easily deploy it for current and upcoming Fermilab experiments, and to support the data preservation efforts of the Run II experiments.

Illingworth, R. A.↗
DOE OSTI2014

CMS Space Monitoring

During the first LHC run, CMS stored about one hundred petabytes of data. Storage accounting and monitoring help to meet the challenges of storage management, such as efficient space utilization, fair share between users and groups and resource planning. We present a newly developed CMS space monitoring system based on the storage metadata dumps produced at the sites. The information extracted from the storage dumps is aggregated and uploaded to a central database. A web based data service is provided to retrieve the information for a given time interval and a range of sites, so it can be further aggregated and presented in the desired format. The system has been designed based on the analysis of CMS monitoring requirements and experiences of the other LHC experiments. In this paper, we demonstrate how the existing software components of the CMS data placement system, PhEDEx, have been re-used, dramatically reducing the development effort.

Ratnikova, N.↗