Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “metadata”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,549 records · Page 86
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Final Report for File System Support for Burst Buffers on HPC Systems

Distributed burst buffers are a promising storage architecture for handling I/O workloads for exascale computing. As they are being deployed on more supercomputers, a file system that efficiently manages these burst buffers for fast I/O operations carries great consequence. Over the past year, FSU team has undertaken several efforts to design, prototype and evaluate distributed file systems for burst buffers on HPC systems. These include MetaKV: a Key-Value Store for Metadata Management of Distributed Burst Buffers, a user-level file system with multiple backends, and a specialized file system for large datasets of deep neural networks. Our progress for these respective efforts are elaborated further in this report.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Analysis Report for Exascale Storage Requirements for Scientific Data.

Over the next 10 years, the Department of Energy will be transitioning from Petascale to Exascale Computing resulting in data storage, networking, and infrastructure requirements to increase by three orders of magnitude. The technologies and best practices used today are the result of a relatively slow evolution of ancestral technologies developed in the 1950s and 1960s. These include magnetic tape, magnetic disk, networking, databases, file systems, and operating systems. These technologies will continue to evolve over the next 10 to 15 years on a reasonably predictable path. Experience with the challenges involved in transitioning these fundamental technologies from Terascale to Petascale computing systems has raised questions about how these will scale another 3 or 4 orders of magnitude to meet the requirements imposed by Exascale computing systems. This report is focused on the most concerning scaling issues with data storage systems as they relate to High Performance Computing- and presents options for a path forward. Given the ability to store exponentially increasing amounts of data, far more advanced concepts and use of metadata will be critical to managing data in Exascale computing systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Advanced Energy System Design (AESD): Technical Manual for the Records API

The Records API (application program interface) for Advanced Energy System Design (AESD) enables software that serves multidimensional record-oriented data to interoperate with software than uses such data. In the context of the Records API, multidimensional data records are simply tuples of real numbers, integers, and character strings, where each data value is tagged by a variable name, according to a pre-defined schema, and each record is assigned a unique integer identifier. Conceptually, these records are isomorphic to rows in a relational database, JSON objects, or key-value maps. Records servers might supply static datasets, sensor measurements that periodically update as new telemetry become available, or the results of simulations as the simulations generate new output. Records client software might display or analyze the data, but in the case of simulations the client request the creation of new ensembles for specified input parameters. It is also possible to chain records clients and servers together so that a client consuming data from a server might transform that data and serve it to further clients. This minimalist API avoids imposing burdensome metadata, structural, or implementation requirements on developers by relying on open-source technologies that is readily available for common programming languages. In particular, the API has been designed to place the smallest possible burden on services that provide data. This document defines the message format for the Records API, a transport mechanism for communicating the data, and semantics for interpreting it. The message format is specified as Google Protocol Buffers and the transport mechanism uses WebSockets. We discuss three major use cases for serving and consuming records data: (i) static data, (ii) dynamically augmented data, (iii) on-demand simulations, (iv) with filters, and (v) with bookmarks. Separate implementations of the API exist in C++, Haskell, JavaScript, Python, and R.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Dynamic Networks Experiment FY18 - Quality Control Focus Area Efforts

The availability of data whose quality is fittingly labeled is of tantamount importance to signal detection and all other downstream signal processing operations. Without identification, the goals of low yield nuclear monitoring cannot reach their full potential. A fundamental question that remains is the knowledge of the capability gaps that prevent us from using open network data. With these new channels, we will need better understanding of the frequency, type, severity, and impact of artifacts when compared to conventional networks. Moreover, beyond a simple quality label, thresholds for acceptable use of the data will need to be established, likely on a use case by use case basis. Finally, orders of magnitude more data with an at-best static analyst pool size means that quality artifacts must be identified with as little human interaction as possible. Automated methodologies must be developed to account for the scale of the problem when ad hoc networks are candidates for processing. At the same time that “data wealth” provides more opportunity for signal exploitation, the introduction of ad hoc networks introduces an increased risk quality issues in both sensor value and metadata issues. The artifacts shown below are examples of some sensor-based quality issues that impair downstream processing. The machine learning model used in this project was able to identify the artifacts that produce these spurious correlations, significantly improving analyst productivity.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

WCATS User's Guide

This manual has been written to provide a general reference of the usage of the Waste Compliance and Tracking System (WCATS). Because WCATS is designed to be used at many different kinds of facilities, much of the usage of the application is driven by the metadata configuration of the system. General usage of the system and the different types of tasks provided is included in this document, but custom usage and an exhaustive list of configuration options will not be provided. This manual is not intended to be a replacement for detailed work procedures or desk instructions, and it will not include configuration details such as service unit names, waste stream numbers for selection, etc. Any use of such information in this document is only provided as an example.

42 ENGINEERING↗
DOE OSTI2015

Inferring Organizational Structure from Behavior

In this project, we researched new techniques for detecting hidden networks of individuals and institutions by introducing the use of temporal correlations among behaviors, leveraging both information sources and metadata. We validated the algorithms using the Wikipedia edit history. The rapid increase in crowd-sourced applications like Wikipedia is providing a rich set of data with both a record of behaviors and a set of direct interactions among individuals. Data sets with network ground truth are needed to develop and validate models, before applying them to national security settings where content and meta-data alone are available.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2019

Publication of LLNL Genomics Data to ESGF: a Technical Report for DREAM Project

We present the process used to set up several example genomic data sets within an ESGF framework. The process begins with designing a project, staging the data, and handling the necessary configuration. We describe the publication process used to ingest the datasets, in essence, all required metadata. Finally, we describe the data web service that we introduce in the DREAM project which makes these datasets available for download or exploration.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2019

Molten Salt Thermophysical Properties Database Development: 2019 Update

This report provides an update on the molten salt reactor thermophysical properties database that is being developed as part of the U.S. Department of Energy Advanced Reactor Technology Program Molten Salt Reactor (MSR) campaign. A summary of recent MSR-relevant literature and a discussion of data gaps are provided. The structure of the new MSR thermophysical properties database is outlined and three datasets are reviewed in detail to provide examples of the data and metadata that will be included in the database. Despite the availability of some high quality MSR-relevant data-sets, there remains a need for a thermophysical properties experimental program to generate data focused on addressing the design and licensing needs of current MSR concept developers. This report represents the work product milestone M4AT-19AN040601045 for work package Salt Chemistry work package AT-19AN04060104 of the Advanced Reactor Technology Program’s MSR campaign.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2017

Simons Foundation Data Management

The Simons Foundation is a non-profit organization that funds scientific research. Their Autism Research is generating significant amounts of genomic data, which they need to both archive and make available for researchers around the globe. The Foundation already stores data at Fermilab, as per agreement FRA-2015-0006. With this CRADA, the Fermilab Scientific Computing Division has developed a prototype solution for the data management needs of the Foundation, modifying services developed for High Energy Physics. The tools allow researchers to select datasets from a metadata catalogue of Foundation data, transfer selected files or datasets from Fermilab to remote storage, and enable the Foundation to manage access and authorization of the files. The prototypical solutions developed through this CRADA is considered the Phase I of a body of work that can be expanded and transitioned to production quality in a subsequent phase II, to be determined with the Foundation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Eastern North Atlantic (ENA) Aerosol Supplementary Site (S1) Data Analysis Report

We present strategies to identify time periods when local aerosol sources impact the data collected at the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) user facility in the Eastern North Atlantic (ENA). Data from the aerosol observing system (AOS) at the Central Facility and a temporary aerosol Supplementary Site (S1) were evaluated. S1 instrumentation was deployed within 0.75 km of the ENA Central Facility (C1) for approximately one year. Two time periods, one during the summer and one during the winter, were examined in support of the Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) intensive operating periods (IOPs) in 2017 and 2018. As the criteria for identifying regionally representative data varies based on the specific needs of the user, we present and discuss different strategies using measured aerosol properties and their associated meteorological parameters to isolate periods in AOS data that were impacted by local aerosol at ENA. We identify high-concentration aerosol events at ENA using submicron number concentrations, size distributions, high-time-resolution measurements, and diurnal profiles. An improved understanding of local aerosol to select the optimal aerosol mask was obtained with an understanding of local meteorology during the different seasons. We developed a mathematical algorithm that employs AOS data to mask high-concentration aerosol events at ENA. We validated the ENA Aerosol Mask with aerosol data from C1 and S1 and associated meteorological data. The Supplementary Site was ideal to understand and constrain local aerosol sources within the region. In its absence, collocated data from the ENA AOS could be used to select appropriate parameters to develop an aerosol mask that would limit over- or underestimating the regional baseline aerosol data. When available, associated metadata, such as airport flight logs and visual identification with AOS cameras should also be used to validate the application of an aerosol mask at a new site. Our results confirm the importance of optimizing an aerosol mask prior its application. In the future, the application of an aerosol mask at ARM observatories and during mobile deployments could be used to increase the data quality of continuous AOS data and to reduce uncertainties when parameterizing regional aerosol processes and impacts on clouds for incorporation in models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

FY19Q3 LL17-GEOVIS-PD3SS

NFIX has been licensed to ORNL, LANL, SNL and LBNL for use on the MINOS venture. We continue to meet with the MINOS data team to receive feedback and suggestions for additional features. We are accommodating feature requests within our resources. • A new feature was added to NFIX that lists out sensor metadata files (json files) that NFIX is not able to ingest due to formatting errors. This feature has greatly eased the effort required by the MINOS team to identify errant json files. Our summer student used this new feature to determine the number of sensor data metafiles that NFIX was not ingesting (a small number) and found the root cause to be a particular set of formatting problems. The MINOS team is using this information to correct the mis-formatted files. Our summer student also developed a tutorial for the NFIX users. She was in a unique position to develop a tutorial as she had to come up to speed on the tool in order to use it for our data ingest study. The tutorial is planned to appear as an appendix in the project final report. A white paper on using GeoVisipedia technology for detecting changes at portals using satellite imagery was submitted to NA21. The technique uses GeoVisipedia’s model projection technology as a view-invariant segment technique thereby greatly enhancing the robustness of detecting changes. We demonstrated NFIX to our program manager and his technical advisor in April. Per headquarters direction we were not required to present at a DNN R&D Joint Program Review this year.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Facility Informatics with GeoVisipedia (Final Report)

Characterization and understanding of nuclear fuel cycle facilities is of considerable interest to DNN R&D and the nonproliferation community due to their importance in nuclear weapons manufacture. Satellite imagery is a key source of information about these facilities, particularly in denied areas. Such imagery can be useful in its own right but annotations applied to the imagery vastly expand the value of the imagery and its usefulness by providing object descriptions, context and potentially linkages to other relevant information. GeoVisipedia (Geospatial Visual Wikipedia) is a new and novel approach to collecting, organizing, sharing and preserving knowledge about nuclear facilities to support nonproliferation efforts. Our approach is unique in that it emphasizes geospatial visual information about nuclear facilities rather than focusing exclusively on textual information. GeoVisipedia users, such as all-source analysts and decision makers, have access to richly annotated nuclear facility imagery with hyperlinks to visual and textual information that provides users with details and context about the facility. A major accomplishment of this project was to develop technology that allows annotations from one satellite image of a facility to be propagated to other satellite imagery of the facility. New satellite imagery of the facility is automatically populated with all previous annotations of the facility. This technology breakthrough provides users with the annotations, and hence knowledge and insight, from all the Subject Matter Experts (SMEs) who have previously annotated imagery of the facility. This capability was demonstrated in a GeoVisipedia prototype, awarded a patent and is described in several publications. In the latter part of the project an informal collaboration was established with the DNN R&D Multi Informatics for Nuclear Operations Scenarios (MINOS) venture. A second prototype of GeoVisipedia was developed that directly supports the MINOS venture. This prototype is called the Nuclear Facility Information Explorer (NFIX) and is tailored to the High Flux Isotope Reactor (HFIR) facility, a testbed for the MINOS venture. NFIX downloads sensor metadata and operator logs, parses and displays this information. NFIX has been licensed and deployed to several laboratories participating in the MINOS venture.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

NA84 Auto Aiming Tool (Final Report)

The Microsoft Hololens is capable of bi-directional communication between units. This allows many users to share a common visual experience and is the primary capability of interest due to the collaborative scenarios it may enable. One scenario pursued in the Automated Aiming Tool project is shown in figure 1 which conceptualizes the reciprocal workflow between remote team members wearing these devices. In this scenario, the down-range team member leverages the depth-scanning capability of the Hololens to scan and texture map the environment and any objects of interest. This data and any associated metadata are then packaged and transmitted to remote team members where it is then unpackaged and visualized. The remote team member can then insert computer generated imagery – a type of “virtual indicator” – within the visualized environment and that object is then rendered in the field of view of the down-range operator in real time. These virtual objects can be indicators for the placement of instrumentation, items of interest, areas from which data has been collected, etc. Pursuant to the above scenario, the primary objective has been to assess the feasibility of its implementation with current v1 hardware. Certain limitations were found to limit the degree to which the above scenario can currently be implemented. However, the future potential of this approach is clear for revolutionizing the speed and precision of multi-object registration at a single site, instrumentation placement and orientation (aiming), and for full-scene communication and coordination among two or more sites.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Open Source Scalable Data Services and Data Fusion for Biological and Environmental Sciences (SBIR Phase I Final Scientific/ Technical Report)

The overarching goal of the project is to develop an integrated open-source scientific data management system (Apache V2 license), ResonantEco, that meets the need of biological and environmental researchers and developers for data management, curation, and data processing for analyses with a wide range of scale and complexity. ResonantEco will provide web enabled data services with features such as unified data interfaces and federated views of data and metadata for heterogeneous data sources with an interactive web client for data exploration. Our use of the term fusion is taken from geospatial (GIS) domain where data fusion is often synonymous with data integration. In particular, data integration in ResonantEco involves combining data residing in different sources and providing users with a unified view of them.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2020

Waste Compliance and Tracking System (WCATS) User Guide (Release 2)

This manual has been written to provide a general reference of the usage of WCATS. Because WCATS is designed to be used at many different kinds of facilities, much of the usage of the application is driven by the metadata configuration of the system. General usage of the system and the different types of tasks provided is included in this document, but custom usage and an exhaustive list of configuration options will not be provided. This manual is not intended to be a replacement for detailed work procedures or desk instructions, and it will not include configuration details such as service unit names, waste stream numbers for selection, etc. Any use of such information in this document is only provided as an example.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

Application of the Recharge Estimation Tool (RET) to Prepare Spatially and Temporally Variable Recharge Boundary Conditions for Hanford Site Composite Analysis Vadose Zone Models

This environmental calculation file (ECF) describes the development of a tool for translating recharge estimates into readable input for STOMP© (Subsurface Transport Over Multiple Phases) vadose zone models primarily supporting the vadose zone (VZ) facets of the updated Hanford Site Composite Analysis (CA) and the Hanford Site Cumulative Impact Evaluation (CIE). The recharge estimates are spatiotemporally variable and are produced by the Recharge Evolution Tool (RET) described in Hanford Site-wide Natural Recharge Boundary Condition for Groundwater Models (ECF-HANFORD-15-0019). Outputs from the RET are given in Esri’s™ feature class format with yearly estimates and associated metadata encapsulated in file geodatabase objects. For STOMP models, the translated output is a text file in the format of an input boundary condition card, consistent with STOMP software requirements. The text file contains assimilated spatiotemporal recharge estimates produced by the RET. The tool discussed in this document will be referred to as the “RET2STOMP” tool.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

New Architecture to Support Integration and Processing of Seismic Data from Heterogeneous Sources

The Geophysical Monitoring Program (GMP) at Lawrence Livermore National Lab (LLNL) maintains a database and supporting infrastructure for geophysical data used in support of the Nuclear Detonation Detection mission. This database includes data from multiple sources, many of which do not distribute data to the public or for which there is no automated means of access. For example, Figure 1 shows (left) the distribution of waveform data in our database by source. The Incorporated Research Institutions for Seismology Data Management Center (IRISDMC) is our major source of waveform data and those data may be retrieved at will using the Federated Digital Seismograph Networks FDSN web Application Programming Interface (API). However, the next 6 most important sources of waveform data have no or only limited automated access to waveforms. As Figure 1 (right) shows, it is very common for waveform records associate with an event in our database to come from two or more sources, and in some cases data come from 10 sources. This diversity of data sources drives our need for efficient and correct integration of metadata, parametric data, and waveform data.

58 GEOSCIENCES↗
DOE OSTI2020

DOE Office of Scientific and Technical Information (OSTI) Artificial Intelligence and Machine Learning

The Department of Energy (DOE) Office of Scientific and Technical Information (OSTI) established its artificial intelligence (AI) team in the summer of 2019. The AI Team's work and research in this space are new endeavors for OSTI; identifying the appropriate areas of research and investigation are priorities for the team and will ensure results and products that support OSTI and the collection, preservation, and dissemination of R&D results. To support OSTI’s strategic plan, the AI Team has started an assessment of the current R&D results corpus (e.g., metadata and full text) collected through ingest products such as E-Link and DOE CODE and disseminated through OSTI.GOV and other discovery applications. This presentation will present applied AI and Machine Learning (ML) approaches to assess and address data challenges and discuss how these data challenges are being evaluated to establish a comprehensive corpus of R&D results, support the reuse of R&D results and its data, and extend these findings to the broader DOE community. This presentation can be presented live or via a recorded presentation.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗