Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “metadata quality”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

PVAnalytics: A Python Package for Automated Processing of Solar Time Series Data

Multiple publicly available software packages exist that analyze solar time series data, including RdTools and Solar Data Tools, among others. Several of these packages contain their own unique quality assurance (QA) and feature recognition algorithms. The python PVAnalytics package was developed to offer an internally consistent source for these analysis tools, making it easier for the end user to deploy these routines on his or her solar data. The PVAnalytics package currently contains routines for outlier detection, inverter clipping detection, irradiance and temperature checks, orientation checks, and data shift detection, among other functions. These functions have been aggregated from various sources including Solar Forecast Arbiter, RdTools, and the QA process developed by NREL's PV Fleets Initiative. We are continuously adding new functionality to the package, including documentation, examples and algorithms. By bundling QA functionality into a single software package, we hope to make PVAnalytics a comprehensive software library to support analysis of solar metadata and time series data.

data cleaning↗

NASA'S Earth Science Data Stewardship Activities

NASA has been collecting Earth observation data for over 50 years using instruments on board satellites, aircraft and ground-based systems. With the inception of the Earth Observing System (EOS) Program in 1990, NASA established the Earth Science Data and Information System (ESDIS) Project and initiated development of the Earth Observing System Data and Information System (EOSDIS). A set of Distributed Active Archive Centers (DAACs) was established at locations based on science discipline expertise. Today, EOSDIS consists of 12 DAACs and 12 Science Investigator-led Processing Systems (SIPS), processing data from the EOS missions, as well as the Suomi National Polar Orbiting Partnership mission, and other satellite and airborne missions. The DAACs archive and distribute the vast majority of data from NASA’s Earth science missions, with data holdings exceeding 12 petabytes The data held by EOSDIS are available to all users consistent with NASA’s free and open data policy, which has been in effect since 1990. The EOSDIS archives consist of raw instrument data counts (level 0 data), as well as higher level standard products (e.g., geophysical parameters, products mapped to standard spatio-temporal grids, results of Earth system models using multi-instrument observations, and long time series of Earth System Data Records resulting from multiple satellite observations of a given type of phenomenon). EOSDIS data stewardship responsibilities include ensuring that the data and information content are reliable, of high quality, easily accessible, and usable for as long as they are considered to be of value.

metadata↗

Spaceflight Biospecimen Sharing in Support of Science Discovery and Exploration

For decades, NASA and international partners have flown non-human biological experiments in space to understand the effects of spaceflight and address potential biological hazards. Sending organisms into space is a costly endeavor which makes space-flown biological specimens a valuable resource. To enable maximum scientific return, samples not required by the Principal Investigators are harvested and collected mostly by NASA’s Space Biology Biospecimen Sharing Program. These specimens are collected according to well-established SOPs that maintain quality and integrity. The specimens are then preserved, archived, and made available to the international scientific community through NASA’s Institutional Scientific Collection (ISC) at Ames Research Center (ARC). The ISC-ARC biospecimens and descriptive metadata are findable and accessible for request through the Life Sciences Data Archive (LSDA). The NASA ISC-ARC currently stores over 32,000 specimens from Shuttle, International Space Station, and ground-based investigations (spaceflight analog experiments involving either hindlimb unloading, centrifugation, or partial weight-bearing study designs). Tissues are predominantly from mice and rats, though samples are also available from bacteria and quail. The specimens include tissues from many physiological systems including musculoskeletal, neurosensory, reproductive, respiratory, circulatory, and digestive. Tissues are stored at -80°C, -20°C, +4°C, or ambient and preserved in various fixatives. Descriptive metadata is available for all samples. Historically, these tissues have been used for a wide range of analyses, including histology, genomics, and transcriptomics. Plans are underway to expand the ISC-ARC beyond the mostly-rodent contents, to include a space-relevant microbial culture collection including bacteria, fungi, and yeast. This expansion of the ISC-ARC will now involve identifying and standardizing best practices for microbial curations. To ensure safe long-term storage of microbial isolates, a microbiology laboratory will be dedicated for identification, cell culture, and lyophilization. Awarding of tissue to public science investigators has resulted in 33 publications since 2011, with 48 requests being submitted since 2016. Of note, NASA GeneLab has been awarded ISC-ARC biospecimens in the past few years. GeneLab processes the biospecimens to generate various levels of ‘omics’ data, which are published on GeneLab’s open access online platform for bioinformatics analysis and visualization. This has helped a systems biology community grow around the processed-biospecimens’ datasets, resulting in many new publications and insights. Websites: https://www.nasa.gov/ames/research/space-biosciences/isc-bsp ; https://lsda.jsc.nasa.gov/Biospecimen

Ryan T. Scott↗

Vis-SAGA: Visual Analytics for Situational Awareness of Grid Anomalies: Preprint

We describe supporting near real-time situational awareness of the electric distribution system by visualizing novel data from voltage sensors deployed on existing broadband cable television network equipment. Our scalable web-based visual analytics platform supports interactive geospatial exploration, time-series analysis, and summarization of grid behavior during potentially anomalous events. The broadband cable television sensor network provides observability of the electrical distribution system at a higher local spatial resolution than is typically available to most utilities, revealing the operational state of the network and aiding in the detection of abnormal behaviors or deviations from expected patterns, particularly across electric utility service areas. We outline the design and development of interactive geospatial and time-series visualization components and the scalable data services that supply metadata, historical, and real-time streams of sensor data across the network. We evaluate our platform during periods of extreme weather, demonstrating its ability to assist in detecting patterns of operation that affect power availability, quality, resiliency, and service restoration.

cable television↗

XRootD popularity on hadoop clusters

Performance data and metadata of the computing operations at the CMS experiment are collected through a distributed monitoring infrastructure, currently relying on a traditional Oracle database system. This paper shows how to harness Big Data architectures in order to improve the throughput and the efficiency of such monitoring. A large set of operational data - user activities, job submissions, resources, file transfers, site efficiencies, software releases, network traffic, machine logs - is being injected into a readily available Hadoop cluster, via several data streamers. The collected metadata is further organized running fast arbitrary queries; this offers the ability to test several Map&Reduce-based frameworks and measure the system speed-up when compared to the original database infrastructure. By leveraging a quality Hadoop data store and enabling an analytics framework on top, it is possible to design a mining platform to predict dataset popularity and discover patterns and correlations.

97 MATHEMATICS AND COMPUTING↗

From models to reality: a systematic review on simulated and measured residential heat pump energy savings

High-performance HVAC solutions are central to residential energy management. A substantial share of these are electric, reversible-cycle systems, with heat pumps representing the largest portion of current and near-term adoption. This review synthesizes peer-reviewed and grey literature on residential space heating and cooling heat pumps. The academic literature is dominated by modeling (73.8%), with limited field measurement (13.1%). Grey literature from United States serve as a supplemental resource providing measured savings. Conversions from electric-resistance heating consistently show the largest site energy reductions, while oil/propane baselines yield moderate savings, and gas baseline scenario often deliver small and region-dependent savings. This study cross-checks the grey literature measured data with simulation data filtered from the ResStock dataset. The comparison indicates a discrepancy between simulations and measured data: simulated site EUIs are typically lower than measured EUIs, but percentage energy savings fall in similar ranges, implying simulations capture directional effects while underestimating energy use. Factors associated with variability and model–measurement differences include system characterization and control representation (e.g., backup heat engagement, thermostat/setpoint strategies, commissioning/installation quality), occupant behavior, weather normalization, metering scope, and envelope characterization. This paper also outlines the proposed methodology for comparing simulation and measured data for heat pumps. It emphasizes the metrics used for comparison and units harmonization, building characteristics matching, and compact metadata are needed for simulations to match measured data. The proposed methodology is expected to improve the credibility of simulated savings as measured evidence grows.

Yu, Lili↗

Data for Kim et al., "Variations in the optical and molecular composition of dissolved organic matter exported from coastal wetlands"

Knowledge about sources and composition of marsh-derived dissolved organic matter (DOM) is critical for understanding the role of marshes in coastal biogeochemical cycling and the fate of marsh-derived DOM in the ocean. To investigate tidal variability in composition of marsh-derived DOM, Kim et al. examined the optical and molecular characteristics of hourly surface water samples at three tidal creeks in the Chesapeake Bay. Groundwater samples along the terrestrial landscape gradient as well as estuarine water from the adjacent estuary at each site were also collected to help resolve sources of surface water DOM. Samples were collected in summer 2024 at three sites – SWH: Sweet Hall Marsh, GCW: Kirkpatrick Marsh, and GWI: Goodwin Islands – which are part of synoptic sites in the Chesapeake Bay region of the COMPASS-FME (Coastal Observations, Mechanisms, and Predictions Across Systems and Scales - Field, Measurements, and Experiments) project. Surface water samples were collected hourly over a 48-hour period at each site. Groundwater and estuarine water samples were collected once. This dataset includes- Surface water depth and salinity- Dissolved organic carbon (DOC) and total dissolved nitrogen (TDN) concentrations- Optical indices and relative composition of parallel factor analysis (PARAFAC) components- High resolution mass spectrometry data.

54 ENVIRONMENTAL SCIENCES↗

Global transpiration data from sap flow measurements: the SAPFLUXNET database

Plant transpiration links physiological responses of vegetation to water supply and demand with hydrological, energy, and carbon budgets at the land–atmosphere interface. However, despite being the main land evaporative flux at the global scale, transpiration and its response to environmental drivers are currently not well constrained by observations. Here we introduce the first global compilation of whole-plant transpiration data from sap flow measurements (SAPFLUXNET, https://sapfluxnet.creaf.cat/, last access: 8 June 2021). We harmonized and quality-controlled individual datasets supplied by contributors worldwide in a semi-automatic data workflow implemented in the R programming language. Datasets include sub-daily time series of sap flow and hydrometeorological drivers for one or more growing seasons, as well as metadata on the stand characteristics, plant attributes, and technical details of the measurements. SAPFLUXNET contains 202 globally distributed datasets with sap flow time series for 2714 plants, mostly trees, of 174 species. SAPFLUXNET has a broad bioclimatic coverage, with woodland/shrubland and temperate forest biomes especially well represented (80 % of the datasets). The measurements cover a wide variety of stand structural characteristics and plant sizes. The datasets encompass the period between 1995 and 2018, with 50 % of the datasets being at least 3 years long. Accompanying radiation and vapour pressure deficit data are available for most of the datasets, while on-site soil water content is available for 56 % of the datasets. Many datasets contain data for species that make up 90 % or more of the total stand basal area, allowing the estimation of stand transpiration in diverse ecological settings. SAPFLUXNET adds to existing plant trait datasets, ecosystem flux networks, and remote sensing products to help increase our understanding of plant water use, plant responses to drought, and ecohydrological processes. SAPFLUXNET version 0.1.5 is freely available from the Zenodo repository (https://doi.org/10.5281/zenodo.3971689; Poyatos et al., 2020a). The “sapfluxnetr” R package – designed to access, visualize, and process SAPFLUXNET data – is available from CRAN.

58 GEOSCIENCES↗

Introduction

This report provides a comprehensive overview of metadata to describe sensor signals in wastewater treatment plants and methods to obtain such metadata. In this introduction, we explain the original motivation behind the MetaCO task group. This includes a description of historical challenges (data volume, data velocity) for which mature technology is now available, and newer challenges, which relate to data structure (data variety) and data quality (veracity). We conclude the chapter with an expression of gratitude to all involved.

Aguado, Daniel↗

Final Report: Archiving Data to Support Data Synthesis of DOE Sponsored Elevated CO 2 Experiments

Over the last three decades DOE made a large investment in field-scale experiments in order to understand the role of terrestrial ecosystems in the global carbon cycle, and forecast how carbon cycling will change over the next century. The Smithsonian Environmental Research Center received one of the first awards in this program and managed two long-term studies (25 years and 10 years) with a total of approximately $10 million of support from DOE, and many more millions leveraged from the Smithsonian Institution and agencies such as NSF. The present DOE grant was based on the premise that such a large investment demands a proper synthesis effort so that the full potential of these experiments are realized through data analysis and modeling. The goal of the this grant was to archive legacy data from two major elevated carbon dioxide experiments in DOE databases, and to engage in synthesis activities using these data. Both goals were met. All datasets deemed a high priority for data synthesis and modeling were prepared for archiving and analysis. Many of these datasets were deposited in DOE’s CDIAC, while others are being held at the Oak Ridge National Lab and the Smithsonian Institution until they can be received by DOE’s new ESS-DIVE system at Berkeley Lab. Most of the effort was invested in researching and re-constituting high-quality data sets from a 30-year elevated CO 2 experiment. Using these data, the grant produced products that are already benefiting climate change science, including the publication of new coastal wetland allometry equations based on 9,771 observations, public posting of dozens of datasets, metadata and supporting codes from long-term experiments at the Global Change Research Wetland, and publication of two synthetic data papers on scrub oak forest responses to elevated CO 2 . In addition, three papers are in review or nearing submission reporting unexpected long-term patterns in ecosystem responses to elevated CO 2 and nitrogen in a coastal wetland.

54 ENVIRONMENTAL SCIENCES↗

SPRUCE Advanced Molecular Techniques Provide a Rigorous Method for Characterizing Organic Matter Quality in Complex Systems: Supporting Data

This dataset contains Fourier transform ion cyclotron resonance mass spectrometry-based measurements (FTICRMS), gas chromatography, and liquid chromatography data from the Spruce and Peatland Responses Under Changing Environments (SPRUCE) experimental site within the Marcell Experimental Forest in northern Minnesota, USA from 2011 to 2016. Measurement results demonstrate that optical characterization of peatland dissolved organic matter (DOM) may not fully capture classically identified chemical characteristics and may, therefore, not be the best measure of organic matter quality. Analyses show poor regression fits between optical characteristics and Fourier transform ion cyclotron resonance mass spectrometry-based measurements (FTICRMS). These data were used in Wilson and Tfaily (2018) and files are organized by which figure they were used to produce. Data underlying those regression analyses, as shown in Figures 1a and 1b of Wilson and Tfaily (2018), are provided in data files ds01 and ds02, respectively. ds01: Optically-derived specific ultraviolet absorbance method at 254 nm (SUVA254) values and Fourier transform ion cyclotron resonance mass spectrometry (FTICRMS) -derived double bond equivalences. ds02: Optically-derived spectral slopes and FTICRMS-derived average molecular weights. Results of analyses of a representative sample by FTICRMS and additional analytical techniques provide a broader picture of the compounds occurring in peatland dissolved organic matter (DOM) as shown in Figure 2 of Wilson and Tfaily (2018). The data underlying Figure 2 in are provided in data files ds03-ds07, organized by the respective analytical technique used to generate the data. ds03: Nitrogen containing compounds identified via FTICRMS. ds04: Compounds identified via FTICRMS which did not have nitrogen. ds05: Nuclear magnetic resonance (NMR) results. ds06: Gas chromatography mass spectrometry (GCMS) results. ds07: Liquid chromatography mass spectrometry (LCMS) results. This dataset contains seven data files in comma-separate values (*.csv) format. Additional metadata are provided: seven data dictionaries and a file-level metadata file in comma-separate values (.csv) format and a user guide in PDF (*.pdf) format.

54 ENVIRONMENTAL SCIENCES↗

A High-Throughput Computing Infrastructure to Generate Custom, Open Community Geothermal Datasets

The most significant challenge facing geothermal research, development, and deployment is a lack of comprehensive datasets describing the geological and economical properties of North America. Automated knowledge base construction, the process of designing algorithms to analyze text and images to programmatically build new datasets, is one possible solution to this problem. The xDD library of full-text scientific articles (https://xdd.wisc.edu) is one of the largest collections of open and controlled-access scientific documents available for knowledge base construction in the world, but it has been underutilized by experts in geothermal research. The xDD development team attributed the lack of engagement by software developers and geothermal researchers to two perceived shortcomings of the system. First, the workflow for obtaining data from xDD for local development and testing of data mining applications was unnecessarily abstruse and required significant manual intervention by xDD systems administrators. Second, although xDD already held articles from a broad cross-section of scientific literature with an emphasis on the geosciences, it did not have an explicit set of geothermal research documents that could serve as the nucleus of a geothermal data mining application. To address these issues, the Automated Data Extraction PlaTform (ADEPT) was proposed to extend the data distribution capabilities of the xDD system. The ADEPT extension added the following four key features to xDD: 1) integration of National Geothermal Data System (NGDS) documents into the xDD library to provide an explicitly geothermally-themed collection; 2) improved RESTful (i.e., https-protocol driven) web services for external partners to access xDD data for machine learning application development; 3) a web platform for end-users and xDD administrators to coordinate the development of data mining applications from the initial step of browsing available documents to the final stage of deploying a production-quality machine learning application on high-throughput computing infrastructure; and 4) the development of demonstration data mining applications to illustrate the new workflow to potential collaborators. A total of 21,674 geothermal documents from NGDS were fully ingested into the xDD library and the associated metadata is publicly available through the xDD web services; furthermore, the ADEPT web platform is now publicly accessible and fully live at https://xdd.wisc.edu/adept/.

15 GEOTHERMAL ENERGY↗

Dataset Lifecycle Policy

The presentation focused on describing a new dataset lifecycle policy that the NASA Physical Oceanography DAAC (PO.DAAC) has implemented for its new and current datasets to foster improved stewardship and consistency across its archive. The overarching goal is to implement this dataset lifecycle policy for all new GHRSST GDS2 datasets and bridge the mission statements from the GHRSST Project Office and PO.DAAC to provide the best quality SST data in a cost-effective, efficient manner, preserving its integrity so that it will be available and usable to a wide audience.

best practices↗

DAISY: A Rapid Approach to Evaluating Marine Energy Converter Sound (Final Technical Report)

This project’s objective was to improve the quality of acoustic information about marine energy converters that could be collected from groups of drifting hydrophones, while reducing the costs of deployment and data analysis. This was achieved through technology development addressing four focus areas: (1) minimizing flow-noise and self-noise, (2) integrating metadata streams into a single data acquisition system, (3) developing post-processing routines to facilitate rapid data review, and (4) enabling objective identification of marine energy converter sound against a backdrop of ambient noise using time-delay-of-arrival localization.

16 TIDAL AND WAVE POWER↗

FAIR Data Meets FAIR Software

Modern scientific research is increasingly defined by the interplay between data, software, and the workflows that connect them. Yet while the FAIR (Findable, Accessible, Interoperable, Reusable) principles have become foundational for scientific data stewardship, the same level of structure and expectation has only recently begun to extend to research software. This talk covers why and how FAIR principles are being applied to data and software to support data reuse. It outlines the gaps in current sharing norms, the growing federal emphasis on persistent identifiers and public access, and the opportunities created when datasets, computational workflows, code, and models are linked through rich, standardized metadata. Practical implementation pathways for the EIC and JLab communities are described, including datacards for structured dataset documentation and provenance-aware workflows. By aligning data lifecycle management with FAIR-aligned software practices, the scientific community can advance toward autonomous knowledge graphs, generative workflows, and high-quality, AI-ready scientific datasets.

McSpadden, Diana [Thomas Jefferson National Accele↗

Community Involvement in Enhancing the Global Change Master Directory (GCMD) Controlled Vocabularies (Keywords)

NASA's Global Change Master Directory (GCMD) develops and expands a hierarchical set of controlled vocabularies (keywords) covering the Earth sciences and associated information (data centers, projects, platforms, instruments, etc.). The purpose of the keywords is to describe Earth science data and services in a consistent and comprehensive manner, allowing for the precise searching of metadata and subsequent retrieval of data and services. The keywords are accessible in a standardized SKOSRDFOWL representation and are used as an authoritative taxonomy, as a source for developing ontologies, and to search and access Earth Science data within online metadata catalogues. The keyword development approach involves: (1) receiving community suggestions, (2) triaging community suggestions, (3) evaluating the keywords against a set of criteria coordinated by the NASA ESDIS Standards Office, and (4) publication/notification of the keyword changes. This approach emphasizes community input, which helps ensure a high quality, normalized, and relevant keyword structure that will evolve with users changing needs. The Keyword Community Forum, which promotes a responsive, open, and transparent processes, is an area where users can discuss keyword topics and make suggestions for new keywords. The formalized approach could potentially be used as a model for keyword development.

governance↗

Groundwater table elevation and temperature from 2015 to 2024 at the Lower Montane site in the East River Watershed, Colorado.

This groundwater level elevation and temperature data package is aimed at improving the predictive understanding of hydro-biogeochemical processes at the lower montane site in the East River Watershed, Colorado. The dataset is obtained using pressure transducers placed in shallow wells in the floodplain. This dataset contains data from wells with Location ID's ER-DOW (alias DO1West), ER-DOE (alias DO2East), ER-MBA1 (alias M1Bend1), ER-MBA2 (alias M1Bend2), ER-UPW (alias UP1West), ER-UPM (alias UP2), ER-UPE (alias UP3East). Another dataset contains the data from wells with Location ID's ER-CPA1 to ER-CPA6. Each file contains the water level elevation and the water temperature. Water level elevation has been obtained using the barometric pressure from the pressure transducer (Hobos sensor) in the well, barometric pressure from a sensor in air located at the same site (lower montane), depth from top-of-casing (TOC) to sensor measurement point, and TOC elevation. Data have been checked with a few measurements of water table depths. A real-time kinematic (RTK) global positioning system (GPS) has been used to survey the TOC (data in file Well_Location.csv). The water level elevation is given in UTM13N Geoid2012AB. While depth to water level is not present in the data files, it can be easily calculated with the TOC and distance to ground provided in the GPS coordinate file. The dataset quality is discussed in Collection/Analysis section of the methods. Time-series of measurements were initially added to the archive for the period 2015 to 2019, and later updated with time-series until 2024 (end of data collection). The dataset contains 8 *.csv data files, and 3 *.csv metadata files. Feel free to contact the author with any questions or collaboration interests. The publication year was updated from "2020" to "2025" to reflect the revised version of this dataset.

54 ENVIRONMENTAL SCIENCES↗

Groundwater table elevation and temperature from 2015 to 2024 across Meander C at the Lower Montane site in the East River Watershed, Colorado.

This groundwater level elevation and temperature data package is aimed at improving the predictive understanding of hydro-biogeochemical processes at the lower montane site in the East River Watershed, Colorado. The dataset is obtained using pressure transducers placed in shallow wells in the floodplain. This dataset contains data from wells ER-CPA1 to ER-CPA6. Another dataset contains the data from wells at nearby Locations. Each file contains the water level elevation and the water temperature. Water level elevation has been obtained using the barometric pressure from the pressure transducer (Hobos sensor) in the well, barometric pressure from a sensor in air located at the same site (lower montane), depth from top-of-casing (TOC) to sensor measurement point, and TOC elevation. Data have been checked with a few measurements of water table depths. A real-time kinematic (RTK) global positioning system (GPS) has been used to survey the TOC (data in file Well_Location.csv). The water level elevation is given in UTM13N Geoid2012AB. While depth to water level is not present in the data files, it can be easily calculated with the TOC and distance to ground provided in the GPS coordinate file. The dataset quality is discussed in Collection/Analysis section of the methods. Time-series of measurements were initially added to the archive for the period 2015 to 2019, and later updated with time-series until 2024 (end of data collection). The dataset contains 7 *.csv data files, and 3 *.csv metadata files. Feel free to contact the author with any questions or collaboration interests. The publication year was updated from "2020" to "2025" to reflect the revised version of this dataset.

54 ENVIRONMENTAL SCIENCES↗