Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data catalog”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Empowering Scientific Discovery Through Computing at the Advanced Photon Source

This paper explores the challenges and solutions for managing and processing the vast amount of data generated by the Advanced Photon Source (APS), a synchrotron light source facility producing ultra-bright x-rays for diverse scientific domains. With 68 experimental beamlines covering materials research, biology, and more, the APS serves a wide user base across academia, government, and industry. The ongoing upgrade of the APS storage ring and installation of new instruments will amplify data generation and processing demands. This paper discusses the approach to address these demands through automated data processing using standardized workflows that produce faster scientific insights. The APS Data Management System coordinates various data related tasks to manage storage, data transfer, metadata cataloging, data processing, and interfaces with tools provided by Globus. Through integration with the Argonne Leadership Computing Facility (ALCF), APS users can efficiently access high-performance computing resources. Standardized workflows have led to reduced computational burdens on scientists and greater accessibility of high performance computing resources. We demonstrate how standardization and collaboration enable scientists to rapidly convert raw data into meaningful scientific results, establishing a streamlined path from data collection to analysis and ultimately to publication.

Parraga, Hannah↗
DOE OSTI2024

Spatial Seal Database for Prospective Storage Resources in the USA

The goal of the Spatial Seal Database for Prospective Storage Resources in the USA is to provide relevant information and spatial extents of caprock and seal rocks. A lack of aggregated information is readily available that focuses on the caprock and seal units within sedimentary basins. The EPA class VI permit requires an assessment of the confining zone as part of submitting a permit. The data catalog of seal unit names and relevant properties with the seal spatial extent database aims to help provided important data for carbon storage based assessments. The data catalog and database are designed to show what seal data is available in a sedimentary basin and guide stakeholders to the original data source for those datasets.

Pantaleone, Scott↗
DOE OSTI2025

DESI 2024 III: baryon acoustic oscillations from galaxies and quasars

We present the DESI 2024 galaxy and quasar baryon acoustic oscillations (BAO) measurements using over 5.7 million unique galaxy and quasar redshifts in the range 0.1 < z < 2.1. Divided by tracer type, we utilize 300,017 galaxies from the magnitude-limited Bright Galaxy Survey with 0.1 < z < 0.4, 2,138,600 Luminous Red Galaxies with 0.4 < z < 1.1, 2,432,022 Emission Line Galaxies with 0.8 < z < 1.6, and 856,652 quasars with 0.8 < z < 2.1, over a ∼ 7,500 square degree footprint. The analysis was blinded at the catalog-level to avoid confirmation bias. All fiducial choices of the BAO fitting and reconstruction methodology, as well as the size of the systematic errors, were determined on the basis of the tests with mock catalogs and the blinded data catalogs. We present several improvements to the BAO analysis pipeline, including enhancing the BAO fitting and reconstruction methods in a more physically-motivated direction, and also present results using combinations of tracers. We employ a unified BAO analysis method across all tracers. We present a re-analysis of SDSS BOSS and eBOSS results applying the improved DESI methodology and find scatter consistent with the level of the quoted SDSS theoretical systematic uncertainties. With the total effective survey volume of ∼ 18 Gpc3, the combined precision of the BAO measurements across the six different redshift bins is ∼0.52%, marking a 1.2-fold improvement over the previous state-of-the-art results using only first-year data. We detect the BAO in all of these six redshift bins. The highest significance of BAO detection is 9.1σ at the effective redshift of 0.93, with a constraint of 0.86% placed on the BAO scale. We find that our observed BAO scales are systematically larger than the prediction of the Planck 2018-ΛCDM at z < 0.8. We translate the results into transverse comoving distance and radial Hubble distance measurements, which are used to constrain cosmological models in our companion paper.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

MetaCat - metadata catalog for data management systems

Metadata management is one of three major areas and parts of functionality of scientific data management along with replica management and workflow management. Metadata is the information describing the data stored in a data item, a file or an object. It includes the data item provenance, recording conditions, format and other attributes. MetaCat is a metadata management database designed and developed for High Energy Physics experiments. As a component of a data management system, it’s main objectives are to provide efficient metadata storage and management and fast data items selection functionality. MetaCat is supposed to work on the scale of 100 million files (or objects) and beyond. The article will discuss the functionality of MetaCat and technological solutions used to implement the product.

Mandrichenko, Igor↗
DOE OSTI2025

Community Requirements Meta-Analysis: Characterizing Needs and Opportunities for HPDF

This High Performance Data Facility (HPDF) Project is creating a new scientific user facility to provide advanced infrastructure for data-intensive science, supporting the DOE’s Office of Science (SC) community. HPDF’s mission is to enable and accelerate scientific discovery by delivering state-of-the-art data management infrastructure, capabilities, and tools. This meta-analysis examines the needs of the breadth of the SC community, captured in publicly available community reports or mission documents. The meta-analysis identifies and provides initial characterization of fifteen core requirements for the HPDF Project team to consider during the conceptual design phase. The fifteen requirements illustrate how scientific work among SC communities requires modern, seamless user experiences across the ASCR Ecosystem to advance the use of large volumes of heterogeneous data. The scientific community requires support for the missing middle of compute between local and HPC to interactively and collaboratively use growing datasets. Data producers and end users will benefit from enhanced data catalogs and portals that improve data access through advanced search of well curated data. The fifteen requirements are examined here organized across five themes for discussion. Examples in each theme illustrate the array of scientific needs that convey the important role that the fully realized and operational High Performance Data Facility will be able to play as an integral part of the evolving ASCR Ecosystem. Our amalgamated data tables from ESnet reports demonstrate ranges to the volumes of data HPDF must be concerned with, but limitations are inherent to this meta-analysis (see Key Challenges & Limitations). Feedback and validation of these requirements along with additional details and emergent community requirements will be gathered through user research and design activities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Label-based Virtual Directories In dCache

Traditional filesystems organize data in directories. These directories are typically a collection of files whose grouping is based on a single criterion, e.g., the starting date of an experiment, experiment name, beamline ID, measurement device, or instrument. However, each file in a directory can belong to several logical groups, such as a special event type, experiment condition, or a part of a selected dataset. dCache is a storage system developed to store large amounts of scientific data, used by many HEP and Photon Science experiments. With recent developments in dCache, we have introduced a concept of file tagging, which dynamically groups files with the same label into virtual directories. The file labels can be added, removed, renamed, and deleted through the admin interface or via REST API. The files in virtual directories are exposed through all protocols supported by dCache. This contribution will describe the details of the implementation for file tagging in dCache and present our future development plans on automatic metadata extractions, a feature that will significantly simplify data management. Additionally, we are exploring the future use of virtual directories as a way to translate scientific data catalogs into filesystem views for direct data analysis.

Sahakyan, Marina [DESY]↗
DOE OSTI2022

Utah FORGE Well 16A(78)-32 Stimulation DFN Fracture Plane Evaluation and Data

This dataset includes files used to fit planar fractures through the preliminary earthquake catalogs of the three stages of the April 2022 well 16A(78)-32 stimulation which is linked bellow. These planar features have been used to update the FORGE reference Discrete Fracture Network (DFN) model. The files are provided to encourage other modelers to use additional workflows to find additional/alternative features. To this end, the dataset includes the cleaned earthquake catalog data translated to the FORGE reference model global reference frame, the well trajectory of 16A(78)-32 in those same coordinates, the fit 15 planar features in csv format, and a pdf file with slides illustrating the process used to fit the features. A recorded presentation of this material is available from the October 2022 FORGE Modeling and Simulation Forum which is also linked below.

15 GEOTHERMAL ENERGY↗
DOE OSTI2022

Characterization of Acoustic Emissions From Analogue Rocks Using Sparse Regression‐DMDc

Abstract Moisture loss in rock is known to generate acoustic emissions (AE). Phenomena that result in AE during drying are related to the movement of fluids through the pores and induced‐cracks that arise from differential mineral shrinkage, especially in clay‐bearing rock. AE from the movement of fluids occurs from the reconfiguration of fluid interfaces during drying, while AE from mineral shrinkage involves the debonding within or between minerals. Here, analogue rock samples were used to examine the differences in the AE signatures when one or both AE source‐types are present. An unsupervised sparse regression model, Dynamic Mode Decomposition with control, that extends Dynamic Mode Decomposition is used to characterize the AE signals recorded during the drying of porous analogue rock samples fabricated with ordinary Portland cement, with and without clay. This method can effectively and accurately reconstruct acoustic signals emitted from samples that only experience moisture loss without cracking. However, the method struggles to reconstruct signals from samples with intricate crack networks that formed during drying because AE generating mechanisms can emit contemporaneously, and the resulting waves propagate through drying‐induced cracks that can lead to multiple internal reflections. Thus, the differential reconstruction accuracy of time series generated by different underlying physical processes provides a robust filter for reducing large data catalogs. In general, both dynamics and sparse initiating events are learned directly from data and this method exposes a data hierarchy based on the complexity of the intrinsic dynamics.

58 GEOSCIENCES↗
DOE OSTI2023

A Multimodal Event Catalog and Waveform Data Set That Supports Explosion Monitoring from Nevada, U.S.A.

Multimodal, curated data sets and nuisance event catalogs remain rare in the explosion monitoring community relative to curated seismic data sets. The source of this relative absence is the difficultly in deploying multimodal receivers that sense the seismic, acoustic, and other modalities from multiphysics sources. We provide such a data set in this study that delivers seismic, infrasound, and electromagnetic (magnetometer) sensor records collected over a two–week period, within 255 km of a 10 ton buried chemical explosion called DAG–4 that was located at 37.1146°, –116.0693° on 22 June 2019 21:06:19.88 UTC. This catalog includes 485 seismic, seismoacoustic, and infrasound–only events that an expert analyst manually built by reviewing waveforms from 29 seismic and infrasound sensors. Our data release includes waveforms from these 29 seismic, infrasound, and seismoacoustic stations and two magnetometer stations and their station metadata. We deliver these waveforms in NNSA KB Core CSS.w format (i4) with a corresponding wfdisc table that provides the header information. Here, we expect that this data set will provide a valuable, benchmark resource to develop signal processing algorithms and explosion monitoring methods against manual, human observations.

58 GEOSCIENCES↗
DOE OSTI2025

DeepLynx Ecosystem 2025

Poor data integration and governance continue to plague complex engineering projects, resulting in missed cost, schedule, and performance targets. Departments operate in isolated systems with manual data exchange, creating fragmented information that compounds errors and leads to significant delays and cost overruns. The DeepLynx ecosystem addresses these challenges through an open-source, modular data management platform that transforms fragmented project data into an integrated digital thread. Built on a federated microservice architecture, the ecosystem comprises seven specialized tools centered around DeepLynx Nexus, a unified data catalog with hierarchical organization and graph-based navigation capabilities. The ecosystem includes: DeepLynx Stream for real-time timeseries data ingestion from industrial sources; DeepLynx Ingest for governed data uploads with formal review workflows; DeepLynx Lattice for ontology-based entity and relationship extraction; DeepLynx Run for workflow orchestration and secure AI/ML compute; DeepLynx Visualize for 3D digital twin visualization; and DeepLynx Insight for AI-assisted document analysis with traceable, grounded responses. Deployable in cloud, on-premise, or hybrid environments using containerized Docker applications and Helm charts, the DeepLynx ecosystem provides flexible infrastructure that adapts to organizational requirements. By consolidating project data into a unified data lake with role-based access controls and OAuth2 authentication, DeepLynx enables digital thread and digital twin capabilities that improve decision-making, reduce risk, and support complex engineering workflows throughout the project lifecycle.

42 - ENGINEERING↗
DOE OSTI2025

Geoanalytical Evaluation of Saline Storage (GEESS) Geodatabase v2.0

The Geoanalytical Economic Evaluation of Saline Storage (GEESS) geodatabase was developed to support the United States Department of Energy (DOE) and National Energy Technology Laboratory (NETL) in their geologic carbon storage efforts by characterizing saline geologic formations present in the FECM/NETL CO2 Saline Storage Cost Model (CO2_S_COM) [1]. Using publicly available literature and data, the GEESS geodatabase characterizes 57 geologic formations across the lower-48 U.S. states in what are called Fully Integrated Geodatabases (FIGs). The FIG is a vector polygon feature containing thousands or tens of thousands of individual polygons, which each contain discrete geologic parameter values. A list of the critical geologic parameters that are characterized in the GEESS geodatabase are described in the “Processing Steps and Workflow” part of the ReadMe file, as well as the Data Catalog accompanying the GEESS geodatabase. The FIG is the basis of the GEESS geodatabase and is the direct representation of the collected geologic data. In addition to the FIG, the GEESS system contains grid files. Due to the complexity of the FIGs, grids are used to sample the geologic data so they can be exercised within CO2_S_COM. The grid files contain the geologic data sampled from the FIG, as well as estimates of “Plume Uncertainty Diameter” and “First-year Break-even Price of CO2” derived from CO2_S_COM based on the GEESS grid data.

carbon↗
DOE OSTI2023

A Quick Look at the 3 GHz Radio Sky. II. Hunting for DRAGNs in the VLA Sky Survey

Active galactic nuclei (AGNs) can often be identified in radio images as two lobes, sometimes connected to a core by a radio jet. This multicomponent morphology unfortunately creates difficulties for source finders, leading to components that are (a) separate parts of a wider whole, and (b) offset from the multiwavelength cross identification of the host galaxy. In this work we define an algorithm, DRAGN HUNTER , for identifying double radio sources associated with AGNs (DRAGNs) from component catalog data in the first epoch Quick Look images of the high-resolution (≈3" beam size) Very Large Array Sky Survey (VLASS). We use DRAGN HUNTER to construct a catalog of >17,000 DRAGNs in VLASS for which contamination from spurious sources is estimated at ≈11%. A “high-fidelity” sample consisting of 90% of our catalog is identified for which contamination is <3%. Host galaxies are found for ≈13,000 DRAGNs as well as for an additional 234,000 single-component radio sources. Using these data, we explore the properties of our DRAGNs, finding them to be typically consistent with Fanaroff–Riley class II sources and to allow us to report the discovery of 31 new giant radio galaxies identified using VLASS.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Carbon Storage Planning Inquiry Tool (CS PlanIT)

The CS PlanIT provides stakeholders with easy access to geographically explore, query, and evaluate multiple data layers for a user-defined area of interest, with the goal to accelerate carbon storage resource, feasibility assessments, and planning efforts. The data within CS PlanIT are arranged and categorized to assess each stage of the Carbon Capture and Storage supply chain, from CO2 (Carbon Dioxide) emission sources, critical energy infrastructure (including transport), to CO2 geologic storage and sink locations. Furthermore, stakeholders can gain insights into community and environmental impacts, as CS PlanIT leverages Environmental Justice and Social Justice (EJSJ) and natural hazard datasets. Additionally, documentation in the form of an interactive ArcGIS story map and data catalog are included within the EDX publication to provide user’s information to better understand and explain CS PlanIT’s layout, functions, and associated data sets.

Justman, Devin M.↗
DOE OSTI2026

Sharing the Sun: Community Solar Deployment and Subscriptions (As of January 2026)

The community solar market analysis presented here is based primarily on data collected through Sharing the Sun, an initiative of the National Community Solar Partnership+ (NCSP+). Sharing the Sun data collection and analysis are conducted by the National Laboratory of the Rockies (NLR) as part of its support for implementation of NCSP+. NLR first released a dataset of community solar projects in 2018 and updates it biannually. The January 2026 dataset, data collection methodology, and all the previous datasets are available from NLR's Data Catalog: https://data.nlr.gov/submissions/244. The dataset presents project-level information including location, capacity, operating utility, and year of interconnection. The dataset is created from multiple data sources such as utility data, public utility commissions, project developer websites, media releases, primary data collection by NLR, and data provided by developers under nondisclosure agreements. This presentation builds on a previous analysis of the community solar project dataset, Sharing the Sun: Community Solar Deployment and Subscriptions (as of June 2024). Dr. Gabriel Chan and his team at the University of Minnesota contribute to this effort. NCSP+ is led and funded by U.S. Department of Energy's Integrated Energy Systems Office (IESO).

14 SOLAR ENERGY↗
DOE OSTI2020

Sandia National Laboratories Ecosystem for Open Science: Metadata Schema v0.2 Description.

The Ecosystem for Open Science (eOS) initiative was established in 2019. Its objective is improving openness and sharing of data and information across Defense Nuclear Nonproliferation (DNN) Research and Development (R&D) activities. To support this initiative, the eOS team at Sandia National Laboratories (SNL) developed metadata and data standards and proposed a machine-readable metadata schema. The nuclear explosion monitoring field was selected as a focus area due to its the wide range of pertinent phenomenologies.We developed the DCAT-eOS-AP metadata schema extending the Data Catalog Vocabulary version 2 (DCATv2) standard using an application profile (AP), to fit the needs of multi-disciplinary NA-22 projects. The DCAT-eOS-AP metadata schema describes data at different levels of granularity ranging from general descriptions to more domain-specific granular metadata. Its implementation and serialization is flexible with the ability to include new file or data types. Thus, it will scale with the ever-increasing data management needs of government research. Due to the multitude of phenomenologies represented in the DCAT-eOS-AP schema, we anticipate that it will be easily extensible to various projects across many DOE mission areas. This document describes data management challenges faced within the DNN R&D portfolio and provides insight on how metadata and data standards/guidelines combined with a comprehensive metadata schema can add value to programs throughout the Department of Energy (DOE). It reviews the importance of metadata standards, FAIR (Findability, Accessibility, Interoperability, and Reusability) data principles, and metadata schemas. Additionally, it summarizes input from subject matter experts (SME) at SNL and other National Laboratories that resulted in metadata and data standards/guidelines encompassing domains relevant to NA-22 projects. Finally, we discuss the DCAT-eOS-AP metadata development. Implementation recommendations and future development directions are included for those keen on adopting the DCAT-eOS-AP metadata schema.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2022

Flat-spectrum Radio Quasars and BL Lacs Dominate the Anisotropy of the Unresolved Gamma-Ray Background

We analyze the angular power spectrum (APS) of the unresolved gamma-ray background (UGRB) emission and combine it with the measured properties of the resolved gamma-ray sources of the Fermi-LAT 4FGL catalog. Our goals are to dissect the composition of the gamma-ray sky and to establish the relevance of different classes of source populations of active galactic nuclei in determining the observed size of the UGRB anisotropy, especially at low energies. We find that, under physical assumptions for the spectral energy distribution, i.e., by using the 4FGL catalog data as a prior, two populations are required to fit the APS data, namely flat-spectrum radio quasars at low energies and BL Lacs at higher energies. The inferred luminosity functions agree well with the extrapolation of the flat-spectrum radio quasar and BL Lac ones obtained from the 4FLG catalog. We use these luminosity functions to calculate the UGRB intensity from blazars, finding a contribution of 20% at 1 GeV and 30% above 10 GeV. Finally, bounds on an additional gamma-ray emission due to annihilating dark matter are also derived.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Carbon Storage Open Database

The Carbon Storage Open Database is a collection of spatial data obtained from publicly available sources published by several NATCARB Partnerships and other organizations. The carbon storage open database was collected from open-source data on ArcREST servers and websites in 2018, 2019, 2021, and 2022. The original database was published on the former GeoCube, which is now EDX Spatial, in July 2020, and has since been updated with additional data resources from the Energy Data eXchange (EDX) and external public data resources. The shapefile geodatabase is available in total, and has also been split up into multiple databases based on the maps produced for EDX spatial. These are topical map categories that describe the type of data, and sometimes the region for which the data relates. The data is separated in case there is only a specific area or data type that is of interest for download. In addition to the geodatabases, this submission contains: 1. A ReadMe file describing the processing steps completed to collect and curate the data. 2. A data catalog of all feature layers within the database. Additional published resources are available that describe the work done to produce the geodatabase: Morkner, P., Bauer, J., Creason, C., Sabbatino, M., Wingo, P., Greenburg, R., Walker, S., Yeates, D., Rose, K. 2022. Distilling Data to Drive Carbon Storage Insights. Computers & Geosciences. https://doi.org/10.1016/j.cageo.2021.104945 Morkner, P., Bauer, J., Shay, J., Sabbatino, M., and Rose, K. An Updated Carbon Storage Open Database - Geospatial Data Aggregation to Support Scaling -Up Carbon Capture and Storage. United States: N. p., 2022. Web. https://www.osti.gov/biblio/1890730 Morkner, P., Rose, K., Bauer, J., Rowan, C., Barkhurst, A., Baker, D.V., Sabbatino, M., Bean, A., Creason, C.G., Wingo, P., and Greenburg, R. Tools for Data Collection, Curation, and Discovery to Support Carbon Sequestration Insights. United States: N. p., 2020. Web. https://www.osti.gov/biblio/1777195 Disclaimer: This project was funded by the United States Department of Energy, National Energy Technology Laboratory, in part, through a site support contract. Neither the United States Government nor any agency thereof, nor any of their employees, nor the support contractor, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.

carbon storage↗