Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data virtualization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

ESS-DIVE Reporting Format for Sample-based Water and Soil Chemistry Measurements

The ESS-DIVE (Environmental Systems Science Data Infrastructure for a Virtual Ecosystem) reporting format for sample-based water and soil chemistry measurements are written guidelines and spreadsheet templates, that facilitate archiving water/soil/sediment chemistry data. The templates and terminology lists provided as part of the reporting format can help organize data and potentially enable data reuse.The reporting format consists of a general instructions file (instructions.md) as well as more detailed instructions files for each template (files beginning “Detailed_Instructions_*.md). The “examples” folder includes an example of each data template with just the limited set of required fields filled out as well as other examples with both required and optional fields complete.The 'templates' folder contains CSV templates for data, methods, and terminology file templates. Similar to the examples folder, the blank templates are provided with versions for required fields only as well as required and optional fields. Lastly, the term list folder provides common terminology used in each of the templates as well as a definition and any constraints for the fields.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

ESS-DIVE Reporting Format for Dataset Package Metadata

ESS-DIVE’s (Environmental Systems Science Data Infrastructure for a Virtual Ecosystem) dataset metadata reporting format is intended to compile information about a dataset (e.g., title, description, funding sources) that can enable reuse of data submitted to the ESS-DIVE data repository. The files contained in this dataset include instructions (dataset_metadata_guide.md and README.md) that can be used to understand the types of metadata ESS-DIVE collects. The data dictionary (dd.csv) follows ESS-DIVE’s file-level metadata reporting format and includes brief descriptions about each element of the dataset metadata reporting format. This dataset also includes a terminology crosswalk (dataset_metadata_crosswalk.csv) that shows how ESS-DIVE’s metadata reporting format maps onto other existing metadata standards and reporting formats.Data contributors to ESS-DIVE can provide this metadata by manual entry using a web form or programmatically via ESS-DIVE’s API (Application Programming Interface). A metadata template (dataset_metadata_template.docx or dataset_metadata_template.pdf) can be used to collaboratively compile metadata before providing it to ESS-DIVE.Since being incorporated into ESS-DIVE’s data submission user interface, ESS-DIVE’s dataset metadata reporting format, has enabled features like automated metadata quality checks, and dissemination of ESS-DIVE datasets onto other data platforms including Google Dataset Search and DataCite.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

SCILLA Scanning Electrical Mobility Spectrometer (SEMS) airborne data

The Scanning Electrical Mobility Spectrometer (SEMS) provides measurements of bulk aerosol size distributions, which are crucial for studying aerosol-cloud interactions. The instrument sampled through two different inlets: an isokinetic inlet and a counter-flow virtual impactor (CVI) inlet. Data were collected to help improve climate and earth systems models, characterize atmospheric conditions, and validate satellite observations and climate simulations.

Scanning Electrical Mobility Spectrometer↗
DOE OSTI2025

Machine Learning-Assisted Recovery of Delicate Kinetic Information from Transient Reactor Experiments

Identifying active sites and their roles in chemical reaction steps remains a vital challenge in heterogeneous catalysis. Transient experiments offer a unique way to probe active sites and distinguish subtle kinetic features. Although physics-based analysis methods may be well-developed, they can be highly susceptible to experimental noise, and smoothing methods may erase or even distort important features; a smooth curve is not always the best curve. We demonstrate a new workflow for the direct interpretation of intrinsic kinetic information from exit flux curves measured in transient reactor experiments. This workflow contains three artificial neural networks (ANNs), including a noise reducer, a concentration predictor, and a rate predictor to analyze experimental data, followed by the virtual TAP (VTAP) physics-based reactor model and density functional theory (DFT) calculations of adsorption energies on specific sites. We use this workflow to analyze the data from experiments titrating Pt/Al 2 O 3 and Pt/SiO 2 catalysts with carbon monoxide (CO) in the temporal analysis of products (TAP) reactor. Our workflow separates the time-evolving chemical reaction and mass transfer information contained in the TAP pulse response. The existence of strong- and weak-binding sites on the Pt/Al 2 O 3 catalyst is observed in the catalyst titration experiment in the transient reactor. The structures of the strong- and weak-binding sites are then identified by using DFT calculations. We find that the Pt/SiO 2 catalyst has only strong-binding sites, which aligns with the inactive support effect of SiO 2 . We demonstrate how machine learning methods provide unique insights with high-resolution data analysis that cannot be achieved by using state-of-the-art physics-based methods.

Adsorption↗
DOE OSTI2020

ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems

Scientific applications at exascale generate and analyze massive amounts of data. A critical requirement of these applications is the capability to access and manage this data efficiently on exascale systems. As such, parallel I/O, the key technology enables moving data between compute nodes and storage, faces monumental challenges from new applications, memory, and storage architectures considered in the designs of exascale systems. As the storage hierarchy is expanding to include node-local persistent memory, burst buffers, etc., as well as disk-based storage, data movement among these layers must be efficient. Parallel I/O libraries of the future should be capable of handling file sizes of many terabytes and beyond. In this paper, we describe new capabilities we have developed in Hierarchical Data Format version 5 (HDF5), the most popular parallel I/O library for scientific applications. HDF5 is one of the most used libraries at the leadership computing facilities for performing parallel I/O on existing HPC systems. The state-of-the-art features we describe include: Virtual Object Layer (VOL), Data Elevator, asynchronous I/O, full-featured single-writer and multiple-reader (Full SWMR), and parallel querying. In this paper, we introduce these features, their implementations, and the performance and feature benefits to applications and other libraries.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Label-based Virtual Directories In dCache

Traditional filesystems organize data in directories. These directories are typically a collection of files whose grouping is based on a single criterion, e.g., the starting date of an experiment, experiment name, beamline ID, measurement device, or instrument. However, each file in a directory can belong to several logical groups, such as a special event type, experiment condition, or a part of a selected dataset. dCache is a storage system developed to store large amounts of scientific data, used by many HEP and Photon Science experiments. With recent developments in dCache, we have introduced a concept of file tagging, which dynamically groups files with the same label into virtual directories. The file labels can be added, removed, renamed, and deleted through the admin interface or via REST API. The files in virtual directories are exposed through all protocols supported by dCache. This contribution will describe the details of the implementation for file tagging in dCache and present our future development plans on automatic metadata extractions, a feature that will significantly simplify data management. Additionally, we are exploring the future use of virtual directories as a way to translate scientific data catalogs into filesystem views for direct data analysis.

Sahakyan, Marina [DESY]↗
DOE OSTI2024

Geospatial Data Platform for All

Spatiotemporal data has evolved in scale due to augmented use in cross-domain applications. Simultaneously, there is substantial growth in the availability of Geographic Information Systems (GIS) data provided by the United States Geological Survey (USGS) along with other federal, state, county, or local agencies through open-data portals and public access APIs. However, data availability does not equate with accessibility. Large-scale analyses and applications require robust, performant data management with co-location of data storage and computing. The insufficiency of data management infrastructure compels researchers to adopt ad hoc project- specific GIS data storage solutions (e.g., copying data to High-Performance computer file systems). As an ad hoc storage strategy does not scale, it hampers cross-domain analyses causing difficulty in data reuse and utilizing existing code bases. Furthermore, GIS data is complex and requires expertise to analyze and manipulate due to its intricate data structures and data-specific projection transformations. Despite the challenges, we recognize that derived GIS data products, e.g., satellite or LIDAR-based images, can be used in downstream applications such as AI by domain, but non-GIS experts. To address the data needs and overcome the challenges, we are working towards a GIS Data Platform focused on efficient data storage, data discovery and access, and an API to enable common workflows. We propose a knowledge-graph (KG) approach for data discovery, whereby datasets are semantically linked to higher- level constructs such as projects and research areas. The semantic data links enable researchers to explore datasets in a top-down approach by specifying relevant and meaningful terms (assists in finding hidden data). An advantage is that the nodes and edges in a knowledge graph create built-in semantic documentation. Deeper spatiotemporal connections between data sources can be encoded via Graph Neural Networks (GNN) (Zhang et al., 2021). The KG approach can be extended to integrate the data itself in a Virtual KG (VKG). Our work will derive inspiration from large-scale VKG efforts that have been undertaken or are currently underway as part of the OpenStreetMap project (Ding et al., 2021). For DOE Data Days, we share the proposed geospatial data platform hybrid (cloud/on-prem) architecture, our work-to-date on storing, retrieving, and transforming LiDAR and raster data relevant to two important NREL use-cases, including the Renewable Energy Potential (reV) Model, and present our proposal for a KG based data discovery engine.

data platform↗
DOE OSTI2020

A systematic feature extraction and selection framework for data-driven whole-building automated fault detection and diagnostics in commercial buildings

In data-driven automated fault detection and diagnostics (AFDD) modeling for building energy systems, feature engineering is a critical process of extracting information from high-dimensional and noisy sensor measurement and turning it into informative and representative inputs or features for data-driven modeling. However, few studies specifically discuss the feature engineering, especially the interactions between feature extraction and feature selection in whole-building AFDD. We developed a systematic feature extraction and selection framework for whole-building AFDD. In this framework, features are aggressively extracted from raw sensor data using statistical feature extraction techniques with various window sizes and statistics. With many features extracted, a hybrid feature selection algorithm that combines the filter and wrapper method then selects the best feature set. The framework considers diversity in the duration of fault behavior among fault types in whole-building AFDD, thus achieving high model generalization. We implemented our developed framework in a virtual testbed calibrated with measured data from Oak Ridge National Laboratory's Flexible Research Platform designed to mimic the operation of a typical small commercial building. The AFDD model is trained by the simulation data generated from the virtual testbed. The results show that (1) the developed framework improves the generalization of the AFDD model by 10.7% compared with literature-reported feature extraction and selection methods and (2) features with diverse window sizes and statistics are selected, providing insight into physical systems beyond the current understanding of buildings and faults and improving the detection and diagnostics of multiple fault types.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Virtual Volumes

SAND2022-14692 O Virtual Volumes is a CAD2VR plugin that gives users a powerful and intuitive way of exploring volumetric data in a three-dimensional (3D) virtual reality (VR) environment. It enables viewing of and interaction with volumetric and voxel data derived from sources such computed tomography (CT) scans. Current 2D software solutions for viewing CT scans and other volumetric data forces the user to look through "slices" of their data across anatomical planes. In Virtual Volumes, the user can intuitively interact, scale, crop, colorize, and threshold their data in a 3D VR environment. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Krukar, JohnA.↗
DOE OSTI2020

Fast address translation for virtual machines

A host machine uses a range-based address translation system rather than a conventional page-based system. This enables address translation to be performed with improved efficiency, particularly when nest virtual machines are used. A data processing system utilizes range-based address translation to provide fast address translation for virtual machines that use virtual address space.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

RCSB Protein Data Bank (RCSB.org): delivery of experimentally-determined PDB structures alongside one million computed structure models of proteins from artificial intelligence/machine learning

Abstract The Research Collaboratory for Structural Bioinformatics Protein Data Bank (RCSB PDB), founding member of the Worldwide Protein Data Bank (wwPDB), is the US data center for the open-access PDB archive. As wwPDB-designated Archive Keeper, RCSB PDB is also responsible for PDB data security. Annually, RCSB PDB serves >10 000 depositors of three-dimensional (3D) biostructures working on all permanently inhabited continents. RCSB PDB delivers data from its research-focused RCSB.org web portal to many millions of PDB data consumers based in virtually every United Nations-recognized country, territory, etc. This Database Issue contribution describes upgrades to the research-focused RCSB.org web portal that created a one-stop-shop for open access to ∼200 000 experimentally-determined PDB structures of biological macromolecules alongside >1 000 000 incorporated Computed Structure Models (CSMs) predicted using artificial intelligence/machine learning methods. RCSB.org is a ‘living data resource.’ Every PDB structure and CSM is integrated weekly with related functional annotations from external biodata resources, providing up-to-date information for the entire corpus of 3D biostructure data freely available from RCSB.org with no usage limitations. Within RCSB.org, PDB structures and the CSMs are clearly identified as to their provenance and reliability. Both are fully searchable, and can be analyzed and visualized using the full complement of RCSB.org web portal capabilities.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Detection of surface water temperature variations of Mongolian lakes benefiting from the spatially and temporally gap-filled MODIS data

Lakes provide critical water resources for human activities and ecosystems, particularly in the Mongolian Plateau (MP), which is characterized by a dry climate and a harsh environment. As a region that is sensitive to anthropogenic warming, tracking lake surface water temperature (LSWT) changes in Mongolian lakes is crucial for understanding the consequences of a warming climate on lake ecosystems. However, the long-term monitoring of LSWT is restricted by the spatiotemporal gaps in the raw imagery of remote sensing-based land surface temperature (LST), e.g., the commonly used Moderate Resolution Imaging Spectroradiometer (MODIS) LST products. This study applied an improved gap-filling method by utilizing the discrete cosine transform-based penalized least squares (DCT-PLS) strategy in the spatial domain combined with the linear interpolation (LI) algorithm in the temporal domain. The method was applied to fill gaps in the LSWT imagery of 12 representative lakes across MP. The randomly sampled high-quality MODIS LSWT values in the spatial and temporal domains were excavated as false data gaps and considered “virtual true” validation datasets. The spatial validation results showed that the estimated LSWT for all the lake cases were comparable with the “virtual true” LSWT values, with the average values of the coefficient of determination, mean absolute error, mean square error, and root mean square error being 0.98, 0.38 °C, 0.45 °C, and 0.59 °C, respectively. Meanwhile, the error of nighttime LSWT results was relatively lower than that of daytime LSWT. For temporal interpolation validation, the LI algorithm exhibited relatively better performance and could more objectively indicate the variation in LSWT. Benefiting from the spatially and temporally well-constrained data, we analyzed the interannual and intra-annual change characteristics of the LSWTs of the 12 lakes. The long-term variations of annual and seasonal mean LSWTs in the 12 selected lakes exhibited no evident trends in 2000–2020, while presented apparent interannual fluctuations. The slight changes in the average LSWTs of the 12 selected lakes were in excellent synchronization with the surrounding LST derived from the reanalysis datasets, confirming the widely reported phenomenon of “global warming hiatus” that occurred in the early 21st century. This study improves the understanding of the LSWT variations in Mongolian lakes in response to global climate change. It has the potential to provide an effective approach for monitoring LSWT changes in other large-scale studies.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

EPCAPE-PT-LANL Measurements: Ground based counterflow virtual impactor

Coastal cities offer a unique environment for studying aerosol-cloud interactions and the effects of urban emissions on cloud properties. As part of the Eastern Pacific Cloud Aerosol Precipitation Experiment (EPCAPE), the Partitioning Thrust by Los Alamos National Laboratory (EPCAPE-PT-LANL) was conducted. Our campaign focused on measuring the optical and chemical properties of aerosols and their interactions within marine stratocumulus clouds in La Jolla, California. EPCAPE-PT-LANL enhances the primary goals of EPCAPE through innovative observations of vapor-phase transitions between aerosols and cloud droplets, the impact of black carbon on aerosol-cloud dynamics, and the effects of cloud processing on aerosol optical properties. Instrument: Ground based counter flow virtual impactor (Brechtel Inc) Data Notes: A factor of 6.7 needs to be applied to all cloud droplet residual concentration to correct the enhancement of the concentration because all the residual samples collected at 100 lpm were delivered into the 15 lpm of CVI sample flow. [https://amt.copernicus.org/articles/5/1259/2012/amt-5-1259-2012.html] Header: - Visibility[m]: The atmospheric visibility at the time of measurement, expressed in meters. - QualityControl_Flag[bool]: A boolean flag indicating whether the measurement passed quality control checks. - Temperature[C]: The ambient temperature at the time of the measurement, expressed in degrees Celsius. - CVI_Flag[bool]: A boolean flag indicating whether the Counterflow Virtual Impactor (CVI) was active (true) or inactive (false) during the measurement. - RelativeHumidity[%]: The relative humidity at the time of the measurement, expressed as a percentage

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Cybersecurity and Digital Components: Supply Chain Deep Dive Assessment

The report “America’s Strategy to Secure the Supply Chain for a Robust Clean Energy Transition” lays out the challenges and opportunities faced by the United States in the energy supply chain as well as the federal government plans to address these challenges and opportunities. It is accompanied by several issue-specific deep dive assessments, including this one, in response to Executive Order 14017 “America’s Supply Chains,” which directs the Secretary of Energy to submit a report on supply chains for the energy sector industrial base. The Executive Order is helping the federal government to build more secure and diverse U.S. supply chains, including energy supply chains. As the energy sector has become more globalized and increasingly complex, digitized, and even virtualized, its supply chain risk for digital components – the software, virtual platforms and services, and data – in energy systems has evolved and expanded. All digital components in U.S. energy sector systems are vulnerable and may be subject to cyber supply cha in risks stemming from a variety of threats, vulnerabilities, and impacts. This includes digital components in all systems within the ESIB, namely those systems operated by asset owners across different energy subsectors (e.g., electricity, oil and natural gas, and renewables) and the systems operated by a worldwide industrial complex with capabilities to perform research and development and design, produce, operate, and maintain energy sector systems, subsystems, components, or parts to meet U.S. energy requirements. Supply chain risks for digital components including software, virtual platforms and services, and data have grown in recent years as increasingly sophisticated cyber adversaries have targeted exploiting vulnerabilities in these digital assets. Supply chain risks for digital components in energy sector systems will continue to evolve and likely increase as these systems are increasingly interconnected, digitized, and remotely operated.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2023

Coupling of CTF and RELAP5-3D Within an Enhanced Fidelity Nuclear Power Plant Simulator

A robust and accurate multiphysics engineering simulator is being developed to model the core behavior and system response of pressurized water reactors. This simulator relies on the NESTLE and CTF computer codes to model the neutronics and thermal hydraulics (TH), respectively, inside the core on a nodal scale and on the Reactor Excursion and Leak Analysis Program—Three Dimensional (RELAP5-3D) to model the entire nuclear steam supply system. The RELAP5-3D model includes highly detailed nodalization and multidimensional flow modeling throughout the vessel. Previously, pin-resolved data generated via the Virtual Environment for Reactor Analysis core simulator were used to improve the accuracy of the NESTLE core predictions. The engineering simulator being developed as part of this work uses the 3KEYMASTER platform to couple the enhanced NESTLE model to a nodal-fidelity CTF model to balance run time with accuracy; NESTLE provides node-dependent powers to CTF, and CTF provides node-dependent coolant densities and fuel temperatures to NESTLE.An overlapping domain approach is used for the core TH in which RELAP5-3D provides core boundary conditions based on the system response and CTF provides a node-dependent coolant heating rate to the RELAP5-3D core solution. In the preliminary TH demonstration discussed in this paper, CTF and RELAP5-3D provided similar steady-state core predictions, indicating the hydraulic compatibility between the codes, as well as reasonable and expected behavior under hypothetical transient conditions. This provides an initial step in ongoing efforts toward a robust, multiscale TH/neutronics engineering simulator capability.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2023

Shedding light on shadow generalized parton distributions

The feasibility of extracting generalized parton distributions (GPDs) from deeply virtual Compton scattering (DVCS) data has recently been questioned because of the existence of an infinite set of so-called “shadow GPDs” (SGPDs). These SGPDs depend on the process and manifest as multiple solutions (at a fixed scale 𝑄 2 ) to the inverse problem that needs to be solved to infer GPDs from DVCS data. SGPDs therefore pose a significant challenge for extracting GPDs from DVCS data. With this motivation we study the extent to which QCD evolution can provide constraints on SGPDs. This is possible because the known classes of SGPDs begin to contribute to observables after evolution, and can then be constrained (at the input scale 𝑄$^{2}_{0}$) by data that has a finite 𝑄 2 range. The impact that SGPDs could have on determining the total angular momentum, pressure and sheer force distributions, and tomography is also discussed. We find that 𝑄 2 evolution, given empirical data over a wide range of skewness (𝜉 ) and 𝑄 2 scales, constrains the SGPDs studied at low 𝑥 and low 𝜉. If this finding generalizes to all SGPDs—and we have not been able to find any SGPDs with qualitatively different behavior—then this potentially makes extraction of GPDs from DVCS possible over a limited range of 𝑥 and 𝜉.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Gravitational form factors and mechanical properties of quarks in protons: A basis light-front quantization approach

We compute the gravitational form factors (GFFs) and study their applications for the description of the mechanical properties such as the pressure, shear force distributions, and the mechanical radius of the proton from its light-front wave functions (LFWFs) based on basis light-front quantization (BLFQ). The LFWFs of the proton are given by the lowest eigenvector of a light-front effective Hamiltonian that incorporates a three-dimensional confining potential and a one-gluon exchange interaction with fixed coupling between the constituent quarks solved in the valence Fock sector. We find acceptable agreement between our BLFQ computations and the lattice QCD for the GFFs. Our D -term form factor also agrees well with the extracted data from the deeply virtual Compton scattering experiments at Jefferson Lab, and the results of different phenomenological models. The distributions of pressures and shear forces are similar to those from different models. Published by the American Physical Society 2024

Astronomy & Astrophysics↗