Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,333 records · Page 74
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

SEEDling [SWR-20-99]

SEEDling is intended to simplify the merging of multiple data sources prior to importing into the SEED platform. Geographic information about buildings and taxlots is often obtained from various data sources, and the process of synthesizing this information for many buildings across a city can be burdensome for city data managers. SEEDling uses GIS tools to automate and simplify this process, as well as apply criteria for SEED inclusion (ex: sqft > 50k) and assign UBIDs, to create records that can be easily imported into the SEED platform.

Long, Nicholas↗
DOE OSTI2020

MIQS v0.6

In this research, we propose a novel Metadata Indexing and Querying for Self-describing formats (MIQS), which removes the external DBMS and utilizes in-memory index to achieve efficient metadata searching. MIQS follows the self-contained data management paradigm and provides portable and schema-free metadata indexing and querying functionalities for self-describing file formats. We have evaluated MIQS with the state-of-the-art MongoDB-based metadata indexing solution. MIQS achieved up to 99% time reduction in index construction and up to 172,000X search performance improvement with up to 75% reduction in memory footprint.

Byna, Suren↗
DOE OSTI2020

MIQS v0.6

In this research, we propose a novel Metadata Indexing and Querying for Self-describing formats (MIQS), which removes the external DBMS and utilizes in-memory index to achieve efficient metadata searching. MIQS follows the self-contained data management paradigm and provides portable and schema-free metadata indexing and querying functionalities for self-describing file formats. We have evaluated MIQS with the state-of-the-art MongoDB-based metadata indexing solution. MIQS achieved up to 99% time reduction in index construction and up to 172,000X search performance improvement with up to 75% reduction in memory footprint.

Byna, Suren↗
DOE OSTI2021

CORAL Prototype Implementation (CORAL) v0.1

This is prototype code that implements many of the concepts in our invention, "The CORAL Framework for FAIR Data Management and Analysis" (ROI 2019-123, attached). The code includes a back end module that interfaces with third party open source software (ArangoDB), and a front end user interface (UI) that allows user interaction with the software through a web interface.

Chandonia, John-Marc↗
DOE OSTI2020

CORAL Prototype Implementation (CORAL) v0.1

This is prototype code that implements many of the concepts in our invention, "The CORAL Framework for FAIR Data Management and Analysis" (ROI 2019-123, attached). The code includes a back end module that interfaces with third party open source software (ArangoDB), and a front end user interface (UI) that allows user interaction with the software through a web interface.

Chandonia, John-Marc↗
DOE OSTI2021

sycl-reduction benchmark v1.0.0

Some different SYCL implementations of the reduction operation. The parallel reduction kernel aggregates vector elements into a single result. It consists of four ways to implement the reduction operation using two different data management strategies, which we could not find elsewhere as a single package.

Dufek, Amanda↗
DOE OSTI2021

Vulcan Digital Engineering Application

The application is being developed with the goal of preventing the entropy of all the different single purpose applications. This application utilizes a framework that enables different file formats to be read and understood through plugins dynamically loaded to the main application containing the framework. The main application provides interoperability to all the different plugins, this allows each file format to convert to every other file format that has been read. There is no theoretical limit to the number of plugins that can be used by the application. Even though the plugins may be individually written and serve a purpose like single use scripts, the plugins can only be used with the main application, this forces the application and plugins to be linked. In theory this will force better data management, or at least the bundling of the main application with the plugins.

Lehmer, JacobP↗
DOE OSTI2020

kokkosSZ (kSZ) A Portable Cellerator Implementation of SZ using Kokkos Programming Model

kSZ is a Kokkos-based implementation of the world-widely used SZ lossy compressor. We use Kokkos because it provides abstractions for both parallel execution of code and data management, which can be used to support portable implementation across different accelerator technologies. Kokkos can support OpenMP/OpenMPTarget, oneAPI, Pthreads, and CUDA as backend programming models.

ECP↗
DOE OSTI2022

ornladios/ADIOS2

ADIOS 2: The Adaptable Input Output (I/O) System version 2 is an open-source framework that addresses scientific data management challenges, e.g. scalable parallel I/O, as we approach the exascale era in high-performance computing (HPC). ADIOS 2 bindings are available in C++, C, Fortran, Python and can be used on supercomputers, personal computers, and cloud systems running on Linux, macOS and Windows. ADIOS 2 has out-of-the-box support for MPI and serial environments.

ECP↗
DOE OSTI2022

Web application for machining parameter recommendation using visual analytics

The computer codes create a web application including multiple user interfaces, visualizations, interaction modules between the user, and visualizations. The visualizations will be interactive, allowing users to query the data directly, and they will be arranged in a dashboard. When selections are made in one visualization, filters will be applied to all other visualizations providing a coordinated system for asking questions about the data. The visualization can be utilized for manufacturing data management, analysis, and recommendation.

Chae, Junghoon↗
DOE OSTI2021

Development of Distributed Microgrid Controllers Using HELICS [SWR-21-94]

Development of Distributed Microgrid Controllers Using HELICS uses HELICS framework develop distributed microgrid controllers (DMCs) for coordinated grid service. Development of Distributed Microgrid Controllers Using HELICS includes thirteen agents, including OpenDSS agent, data manager agent, and eleven DMC agents. Development of Distributed Microgrid Controllers Using HELICS can be used as a standard framework to develop fully distributed control solutions that dispatches simulated DERs in simulation platform.

Wang, Jing↗
DOE OSTI2024

MODAQ (Modular Offshore Data Acquisition System) [SWR-24-58]

The National Renewable Energy Laboratory (NREL) has developed a data acquisition system to support laboratory (component) and in-water validation and testing of Marine Renewable Energy (MRE) technologies. The Modular Ocean Data AcQuisition System (MODAQ) is a robust system built on industrial grade hardware that can acquire high-quality and summary data from many sensor and instrument types. MODAQ includes modules to perform real-time QC screening, system health monitoring, and send alerts. MODAQ is also part of a larger data management, processing, visualization, and archiving system.

Raye, Robert↗
DOE OSTI2024

MAPSTER

MAPSTER Observer is a light-weight software package that can be installed on field deployed computers which automatically manages data and creates metadata packages designed to organize large datasets, making them easy to retrieve and share.

Larson, MatthewD↗
DOE OSTI2024

Verifiable Fuel Cycle Simulation

The Verifiable Fuel Cycle Simulation (VISION) is a system-dynamics model of the entire nuclear fuel cycle, from mining to disposal, using user-defined deployment scenarios, fuel recipes, and recycling parameters. It is written in the PowerSim simulation environment and utilized MS Excel to manage data inputs and outputs.

Hays, RossD (0000000184587539)↗
DOE OSTI2025

sam-web-client

Python client and CLI for SAM data managment system

Illingworth, Robert [Fermilab]↗
DOE OSTI2021

Demystifying asynchronous I/O Interference in HPC applications

With increasing complexity of HPC workflows, data management services need to perform expensive I/O operations asynchronously in the background, aiming to overlap the I/O with the application runtime. However, this may cause interference due to competition for resources: CPU, memory/network bandwidth. The advent of multi-core architectures has exacerbated this problem, as many I/O operations are issued concurrently, thereby competing not only with the application but also among themselves. Furthermore, the interference patterns can dynamically change as a response to variations in application behavior and I/O subsystems (e.g. multiple users sharing a parallel file system). Without a thorough understanding, I/O operations may perform suboptimally, potentially even worse than in the blocking case. To fill this gap, here we investigate the causes and consequences of interference due to asynchronous I/O on HPC systems. Specifically, we focus on multi-core CPUs and memory bandwidth, isolating the interference due to each resource. Then, we perform an in-depth study to explain the interplay and contention in a variety of resource sharing scenarios such as varying priority and number of background I/O threads and different I/O strategies: sendfile, read/write, mmap/write underlining trade-offs. The insights from this study are important both to enable guided optimizations of existing background I/O, as well as to open new opportunities to design advanced asynchronous I/O strategies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Ten simple rules for getting and giving credit for data

This article attempts to summarize current best practices that support the movement towards enabling researchers to cite and receive credit for their data. The authors are a small representation of the people and organizations trying to make this happen, and we acknowledge that it is not possible to capture all efforts behind this endeavor in 10 Simple Rules. We encourage interested readers to dive deeper by providing related resources along the way.

59 BASIC BIOLOGICAL SCIENCES↗