Engineering Papers⌕ Search

Engineering topics

Fredian, Thomas

Publications and source records attributed to Fredian, Thomas.

Data Catalog Project - A Browsable, Searchable, Metadata System

Modern experiments are typically conducted by large, extended, where researchers rely on other team members to produce much of the data they use. The experiments record very large numbers of measurements which can be difficult for users to find, access and understand. We are developing a system for users to annotate their data products with structured metadata, providing data consumers with a discoverable, browsable data index. Machine understandable metadata captures the underlying semantics of the recorded data, which can then be consumed by both programs, and interactively by users. Collaborators can use these metadata to select and understand recorded measurements.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Exploring MDSplus data-acquisition software and custom devices

MDSplus is a software tool designed for data acquisition, storage, and analysis of complex scientific experiments. Over the years, MDSplus has primarily been used for data management for fusion experiments. This paper demonstrates that MDSplus can be used for a much wider variety of systems and experiments. We present a step-by-step tutorial describing how to create a simple experiment, manage the data, and analyze it using MDSplus and Python. To this end, a custom example device was developed to be used as the data source. This device was built on an opensource electronic hardware platform, and it consists of a microcontroller and two sensors. We read data from these sensors, store it in MDSplus, and use JupyterLab to visualize and process it. This project and code demo are available on the GitHub site at this URL: https://github.com/santorofer/MDSplusAndCustomeDevices

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Exploring MDSplus data-acquisition software and custom devices

MDSplus is a software tool designed for data acquisition, storage, and analysis of complex scientific experiments. Over the years, MDSplus has primarily been used for data management for fusion experiments. This paper demonstrates that MDSplus can be used for a much wider variety of systems and experiments. We present a step-by-step tutorial describing how to create a simple experiment, manage the data, and analyze it using MDSplus and Python. To this end, a custom example device was developed to be used as the data source. This device was built on an open-source electronic hardware platform, and it consists of a microcontroller and two sensors. Finally, we read data from these sensors, store it in MDSplus, and use JupyterLab to visualize and process it.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Data catalog project—A browsable, searchable, metadata system

Modern experiments are typically conducted by large, extended groups, where researchers rely on other team members to produce much of the data they use. The experiments record very large numbers of measurements that can be difficult for users to find, access and understand. Here. we are developing a system for users to annotate their data products with structured metadata, providing data consumers with a discoverable, browsable data index. Machine understandable metadata captures the underlying semantics of the recorded data, which can then be consumed by both programs, and interactively by users. Collaborators can use these metadata to select and understand recorded measurements. The data catalog project is a data dictionary and index which enables users to record general descriptive metadata, use cases and rendering information as well as providing them a transparent data access mechanism (URI). Users describe their diagnostic including references, text descriptions, units, labels, example data instances, author contact information and data access URIs. The list of possible attribute labels is extensible, but limiting the vocabulary of names increases the utility of the system. The data catalog is focused on the data products and complements process-based systems like the Metadata Ontology Provenance project [Greenwald, 2012; Schissel, 2015]. This system can be coupled with MDSplus to provide a simple platform for data driven display and analysis programs. Sites which use MDSplus can describe tree branches, and if desired create ‘processed data trees’ with homogeneous node structures for measurements. Sites not currently using MDSplus can either use the database to reference local data stores, or construct an MDSplus tree whose leaves reference the local data store. A data catalog system can provide a useful roadmap of data acquired from experiments or simulations making it easier for researchers to find and access important data and understand the meaning of the data and how it was obtained. This is particularly useful in research facilities that study the results of many different experiments or simulations and may not know the intricacies of the data organization in use where the data was generated. It is also possible to store a local copy of key data items in local MDSplus trees and then add processed data to the local catalog.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗