Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Metadata”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Omics Metadata Management Software v. 1 (OMMS)

Our application, the Omics Metadata Management Software (OMMS), answers both needs, empowering experimentalists to generate intuitive, consistent metadata, and to perform bioinformatics analyses and information management tasks via a simple and intuitive web-based interface. Several use cases with short-read sequence datasets are provided to showcase the full functionality of the OMMS, from metadata curation tasks, to bioinformatics analyses and results management and downloading. The OMMS can be implemented as a stand alone-package for individual laboratories, or can be configured for web-based deployment supporting geographically dispersed research teams. Our software was developed with open-source bundles, is flexible, extensible and easily installed and run by operators with general system administration and scripting language literacy.

Powell, Amy↗

ESS-DIVE File Level Metadata Extractor

The File-level Metadata (FLMD) parser is a lightweight python script that can produce a summary of fields based on column headers within a Comma-separated Values (CSV) file. The script enables a way to extract certain metadata fields from CSV data files. An optional configuration file can be used to select specific column headers to extract a smaller subset of values. The script can be used on a single file, or it can be given a directory to process groups of files in bulk. The FLMD parser outputs the metadata as a JSON and can be configured to updated the source file or produce an external companion file.

McNelis, Jack↗

ESS-DIVE reporting format for leaf-level gas exchange data and metadata

Here we present documentation of the ESS-DIVE reporting format for leaf-level gas exchange data and metadata. This reporting format provides guidance to data contributors on how to store data to maximize their discoverability, facilitate their efficient reuse, and add value to individual datasets. For data users, the reporting format will better allow data repositories to optimize data search and extraction, and more readily integrate similar data into harmonized synthesis products. The reporting format specifies data table variable naming and unit conventions, as well as metadata characterizing experimental conditions and protocols. For common data types that were the focus of this initial version of the reporting format, i.e., survey measurements, dark respiration, carbon dioxide and light response curves, and parameters derived from those measurements, we took a further step of defining required additional data that would maximize the potential reuse of those data types. To aid data contributors and the development of data ingest tools by data repositories we provided a translation table comparing the outputs of common gas exchange instruments. The reporting format presented here is intended to form a foundation for future development that will incorporate additional data types and variables as gas exchange systems and measurement approaches advance in the future. The reporting format documentation is maintained and updated on the ESS-DIVE Community Space GitHub. This data package is the first published version of this reporting format, and comprises a zip file of the complete content of https://github.com/ess-dive-community/essdive-leaf-gas-exchange v1.0. The zip contains the reporting format description, guidelines, variable tables and instructions in GitHub markdown language (*.md) and 2 metadata templates as spreadsheets with drop down options (*.xlsx files, also function in GoogleSheets).

54 ENVIRONMENTAL SCIENCES↗

NEPATEC v2.0: Standardized Metadata and Text Corpus of National Environmental Policy Act Documents

The National Environmental Policy Act of 1969, as amended (NEPA), is a major environmental law in the United States, requiring Federal agencies to consider and document potential environmental impacts before deciding on a proposed action. Modernization of NEPA and permitting processes faces significant challenges due to the lack of standardized formats and interoperable systems for organizing and sharing NEPA-related information across agencies. Much of the information gathered during NEPA reviews is written into documents such as categorical exclusions, environmental assessments, and environmental impact statements, then filed in predominately independent agency file stores that may or may not be publicly accessible. The application of metadata and data standards, such as those recommended by the Council on Environmental Quality (CEQ), to NEPA documents offers a shared vocabulary and structure for key entities like projects, processes, and documents that can streamline information exchange and enhance collaboration across systems. In this work, we publicly release NEPATEC2.0, an expanded corpus of NEPA documents with associated metadata. NEPATEC2.0 encompasses approximately 120,000 documents from 60,000 projects prepared by more than 60 different agencies. Modeled to align with CEQ metadata standards, NEPATEC2.0 promotes consistency in environmental reviews and supports the ongoing effort to modernize permitting technologies by facilitating more transparent, efficient, and data-driven decision-making. Importantly, NEPATEC2.0 demonstrates the possibilities and limitations of large language model-based prompting to extract information from NEPA documents at scale.

54 ENVIRONMENTAL SCIENCES↗

DICOMs, Missiles, and Metadata: The U.S. Nuclear Weapons Program Leverages a Medical Standard

Digital Imagine and Communications in Medicine (DICOM) images, although most commonly used in medical settings, have been widely adopted by the United States Department of Energy (DOE) for capturing images of the internal components in a nuclear weapon. DICOMs, a lesser-known file format that combine nested metadata structures with complex image data-including multiple planes, frames, and high resolution-require the creation of access copies to support usability within the DOE. Used primarily for ensuring the safety, security, and reliability of the U.S. nuclear stockpile, the Los Alamos National Laboratory (LANL)’s DICOM images and corresponding image metadata must be accessible to scientists and researchers via our institutional centralized databases. This paper describes the author's creation of a Python script that converts DICOM images into accessible, archive-friendly TIFF files while preserving key image data and metadata.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Data Catalog Project - A Browsable, Searchable, Metadata System

Modern experiments are typically conducted by large, extended, where researchers rely on other team members to produce much of the data they use. The experiments record very large numbers of measurements which can be difficult for users to find, access and understand. We are developing a system for users to annotate their data products with structured metadata, providing data consumers with a discoverable, browsable data index. Machine understandable metadata captures the underlying semantics of the recorded data, which can then be consumed by both programs, and interactively by users. Collaborators can use these metadata to select and understand recorded measurements.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Omics Metadata Management Software v. 1 (OMMS)

Our application, the Omics Metadata Management Software (OMMS), answers both needs, empowering experimentalists to generate intuitive, consistent metadata, and to perform bioinformatics analyses and information management tasks via a simple and intuitive web-based interface. Several use cases with short-read sequence datasets are provided to showcase the full functionality of the OMMS, from metadata curation tasks, to bioinformatics analyses and results management and downloading. The OMMS can be implemented as a stand alone-package for individual laboratories, or can be configured for web-based deployment supporting geographically dispersed research teams. Our software was developed with open-source bundles, is flexible, extensible and easily installed and run by operators with general system administration and scripting language literacy.

Source record↗

CSV to ISO XML metadata transformation tool

Django app for converting CSV records to XML metadata documents. This transformation from the metadata content model to parsed ISO XML documents allows for metadata integration into NGDS.

Source record↗

Parallel file system with metadata distributed across partitioned key-value store c

Improved techniques are provided for storing metadata associated with a plurality of sub-files associated with a single shared file in a parallel file system. The shared file is generated by a plurality of applications executing on a plurality of compute nodes. A compute node implements a Parallel Log Structured File System (PLFS) library to store at least one portion of the shared file generated by an application executing on the compute node and metadata for the at least one portion of the shared file on one or more object storage servers. The compute node is also configured to implement a partitioned data store for storing a partition of the metadata for the shared file, wherein the partitioned data store communicates with partitioned data stores on other compute nodes using a message passing interface. The partitioned data store can be implemented, for example, using Multidimensional Data Hashing Indexing Middleware (MDHIM).

97 MATHEMATICS AND COMPUTING↗

Computing erasure metadata and data layout prior to storage using a processing platform

Techniques are provided for computing data and metadata layout prior to storage in a storage system using a processing platform. An exemplary processing platform comprises one or more of a compute node and a burst buffer appliance. The processing platform communicates with a plurality of the compute nodes over a network, wherein a plurality of applications executing on the plurality of compute nodes generate a plurality of data objects; computes erasure metadata for one or more of the data objects on at least one of the compute nodes; and provides the erasure metadata with the corresponding one or more data objects to a storage system. The processing platform optionally determines a full set of the data objects to be stored and queries the storage system to determine an anticipated layout of the full set of the data objects to be stored. The anticipated layout allows special handling, for example, for small files and large files that are identified based on predefined criteria.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Scalable filesystem enumeration and metadata operations

Systems, apparatus, and methods are disclosed for performing scalable operations in a file system, including POSIX-like file systems. Metadata entries in a namespace or directory tree are sharded across multiple file metadata servers. An enumeration operation, such as listing a directory, is parallelized across the multiple file metadata servers, while retaining standard functionality transparently to clients. Other enumeration operations include no-output operations such as changing file attributes or deleting a file, and cumulative operations such as counting total disk space usage. The parallelization is compatible with tree-level parallelization and storage-level parallelization. Disclosed technologies can be applied to other fields requiring scalable enumeration, such as database and network applications.

Grider, Gary A.↗

Scalable augmented enumeration and metadata operations for large filesystems

Systems, apparatus, and methods are disclosed for performing scalable operations in a file system. Metadata entries in a namespace or directory tree are sharded across multiple file metadata servers. An augmented enumeration operation, such as listing a directory, is parallelized across the multiple file metadata servers, transparently to clients. Exemplary augmentation features can include filtering and sorting. Augmentation features can be executed concurrently with enumeration, prior to enumeration, after enumeration, or as a combination of these, and can utilize pre-built index structures or holding structures for intermediate results. Augmented enumeration operations can also include no-output operations such as changing file attributes or deleting a file, and cumulative operations such as counting total disk space usage. The parallelization is compatible with tree-level parallelization and storage-level parallelization. Disclosed technologies can be applied to other fields requiring scalable enumeration, such as database and network applications.

Grider, Gary A.↗

Unified architecture for data-driven metadata tagging of building automation systems

This article presents a Unified Architecture (UA) for automated point tagging of Building Automation System (BAS) data, based on a combination of data-driven approaches. Advanced energy analytics applications—including fault detection and diagnostics and supervisory control—have emerged as a significant opportunity for improving the performance of our built environment. Effective application of these analytics depends on harnessing structured data from the various building control and monitoring systems, but typical BAS implementations do not employ any standardized metadata schema. While standards such as Project Haystack and Brick Schema have been developed to address this issue, the process of structuring the data, i.e., tagging the points to apply a standard metadata schema, has, to date, been a manual process. This process is typically costly, labor-intensive, and error-prone. In this work we address this gap by proposing a UA that automates the process of point tagging by leveraging the data accessible through connection to the BAS, including time-series data and the raw point names. The UA intertwines supervised classification and unsupervised clustering techniques from machine learning and leverages both their deterministic and probabilistic outputs to inform the point tagging process. Furthermore, we extend the UA to embed additional input and output data-processing modules that are designed to address the challenges associated with the real-time deployment of this automation solution. We test the UA on two datasets for real-life buildings: (i) commercial retail buildings and (ii) office buildings from the National Renewable Energy Laboratory (NREL) campus. We report the proposed methodology correctly applied 85–90% and 70–75% of the tags in each of these test scenarios, respectively for two significantly different building types used for testing UA's fully-functional prototype. The proposed UA, therefore, offers promising approach for automatically tagging BAS data as it reaches close to 90% accuracy. Further building upon this framework to algorithmically identify the equipment type and their relationships is an apt future research direction to pursue.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

The Genomes OnLine Database (GOLD) v.5: a metadata management system based on a four level (meta)genome project classification

The Genomes OnLine Database (GOLD; http://www.genomesonline.org) is a comprehensive online resource to catalog and monitor genetic studies worldwide. GOLD provides up-to-date status on complete and ongoing sequencing projects along with a broad array of curated metadata. Within this paper, we report version 5 (v.5) of the database. The newly designed database schema and web user interface supports several new features including the implementation of a four level (meta)genome project classification system and a simplified intuitive web interface to access reports and launch search tools. The database currently hosts information for about 19 200 studies, 56 000 Biosamples, 56 000 sequencing projects and 39 400 analysis projects. More than just a catalog of worldwide genome projects, GOLD is a manually curated, quality-controlled metadata warehouse. The problems encountered in integrating disparate and varying quality data into GOLD are briefly highlighted. Lastly, GOLD fully supports and follows the Genomic Standards Consortium (GSC) Minimum Information standards.

59 BASIC BIOLOGICAL SCIENCES↗

Brady's Geothermal Field Nodal Seismometers Metadata

Metadata for the nodal seismometer array deployed at the POROTOMO's Natural Laboratory in Brady Hot Spring, Nevada during the March 2016 testing. Metadata includes location and timing for each instrument as well as file lists of data to be uploaded in a separate submission.

15 GEOTHERMAL ENERGY↗

Sample Collection Metadata for Soil Cores from the East River Watershed, Colorado collected in 2017

This data package contains sample collection metadata for soil cores from the East River Watershed in Colorado used in biogeochemical analyses by the Watershed Function SFA. Soil cores were collected seasonally during autumn, winter, snowmelt, and spring at a predominately montane meadow, high altitude field site. Information provided for each soil core sample assigned an International Geo Sample Number (IGSN) includes: sample name, IGSN, material, description, collection method, core depth, location, field program, collectors, and date. The full list of 53 IGSN's is listed in the Methods section. There is also a complete core sample list and metadata for the set of baseline data.

54 ENVIRONMENTAL SCIENCES↗

Best Practices for Energy Management Information Systems Metadata Schemas

The Federal Energy Management Program (FEMP) promotes best practices for impactful utilization of Energy Management Information Systems (EMIS) at federal facilities. This best practice document is part of a series of fact sheets created to help accelerate the market adoption and use of EMIS in the federal sector. The use of standard naming conventions and a metadata schema, which may be referred to as 'data tags,' improves the ability of the EMIS to consistently analyze, visualize, and derive value from operational data. To help address these issues, this fact sheet highlights best practices for use of metadata in an EMIS stack, which can significantly reduce the initial time required to deploy an EMIS.

EMIS↗

Best Practices for Energy Management Information Systems Metadata Schemas

Fact sheet highlights best practices for metadata schemas and standard naming conventions, which improve the ability of an energy management information system (EMIS) to consistently analyze, visualize, and derive value from operational data. This best practice document is part of a series of fact sheets created to help accelerate the market adoption and use of EMIS in the federal sector. It provides an overview of best practices for using metadata in an EMIS stack, which can significantly reduce the initial time required to deploy an EMIS.

data tags↗