Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Metadata”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,549 records · Page 86

EPICS Version 4 - Implementing Complex Data Types

Through phase 1 and phase 2 SBIR grants, s fully functional I/O Controller and communication protocol for version 4 of EPICS is completed. This new software architecture provides a flexible and extendible architecture. Version 4 is implemented fully in Java. The performance metrics look promising. The final portion of phase 2 is to optimize the communication mechanisms. Subsequent work on different aspects of this are required to provide a viable solutions in various areas. Version 3 of EPICS is able to provide a platform for implementing channel based control, because the channel and attributes for time stamping, alarm, display and control were narrow, well defined, and complete. To extend EPICS functionality beyond this, it is necessary to define attributes needed for archive data, array, image data, and directory services. The proper handling of several array types enables the development of middle layer servers such as orbit and bump control in accelerators. Phase 1 should produce a well defined, reviewed, and agreed upon definition of the metadata required for these services. A Phase 2 grant would provide tools that implemented archiving, general array, imaging, and directory applications.

43 PARTICLE ACCELERATORS↗

National Computational Infrastructure for Lattice Gauge Theory SciDAC-2 Closeout Report

As part of this project work, researchers from Vanderbilt University, Fermi National Laboratory and Illinois Institute of technology developed a real-time cluster fault-tolerant cluster monitoring framework. This framework is open source and is available for download upon request. This work has also been used at Fermi Laboratory, Vanderbilt University and Mississippi State University across projects other than LQCD. The goal for the scientific workflow project is to investigate and develop domain-specific workflow tools for LQCD to help effectively orchestrate, in parallel, computational campaigns consisting of many loosely-coupled batch processing jobs. Major requirements for an LQCD workflow system include: a system to manage input metadata, e.g. physics parameters such as masses, a system to manage and permit the reuse of templates describing workflows, a system to capture data provenance information, a systems to manage produced data, a means of monitoring workflow progress and status, a means of resuming or extending a stopped workflow, fault tolerance features to enhance the reliability of running workflows. Requirements for an LQCD workflow system are available in documentation.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Establishment of the Northeast Coastal Watershed Geospatial Data Network (NECWGDN)

The goals of NECWGDN were to establish integrated geospatial databases that interfaced with existing open-source (water.html) environmental data server technologies (e.g., HydroDesktop) and included ecological and human data to enable evaluation, prediction, and adaptation in coastal environments to climate- and human-induced threats to the coastal marine resources within the Gulf of Maine. We have completed the development and testing of a "test bed" architecture that is compatible with HydroDesktop and have identified key metadata structures that will enable seamless integration and delivery of environmental, ecological, and human data as well as models to predict threats to end-users. Uniquely this database integrates point as well as model data and so offers capacities to end-users that are unique among databases. Future efforts will focus on the development of integrated environmental-human dimension models that can serve, in near real time, visualizations of threats to coastal resources and habitats.

54 ENVIRONMENTAL SCIENCES↗

Tracking Provenance in ORNL's Flexible Research Platforms

Provenance is dened as information about the origin of objects, a concept that applies to both physical and digital objects and often overlaps both. The use of provenance in systems designed for research is an important but forgotten feature. Provenance allows for proper and exact tracking of information, its use, its lineage, its derivations and other metadata that are important for correctly adhering to the scien- tic method. In our project's prescribed use of provenance, researchers can determine detailed information about the use of sensor data in their experiments on ORNL's Flexible Research Platforms (FRPs). Our project's provenance system, Provenance Data Management System (ProvDMS), tracks information starting with the creation of information by an FRP sensor. The system determines station information, sensor information, and sensor channel information. The system allows researchers to derive generations of experiments from the sensor data and tracks their hierarchical flow. Key points can be seen in the history of the information as part of the information's workflow. The concept of provenance and its usage in science is relatively new and while used in other cases around the world, our project's provenance diers in a key area. To keep track of provenance, most systems must be designed or redesigned around the new provenance system. Our system is designed as a cohesive but sepa- rate entity and allows for researchers to continue using their own methods of analysis without being constrained in their ways in order to track the provenance. We have designed ProvDMS using a lightweight provenance library, Core Provenance Library (CPL) v.6 In addition to keeping track of sensor data experiments and its provenance, ProvDMS also provides a web-enabled visualization of the inheritance.

provenance↗

Preliminary Assessment of Tecplot Chorus for Analyzing Ensemble of CTH Simulations

The exploration of large parameter spaces in search of problem solution and uncertainty quantifcation produces very large ensembles of data. Processing ensemble data will continue to require more resources as simulation complexity and HPC platform throughput increase. More tools are needed to help provide rapid insight into these data sets to decrease manual processing time by the analyst and to increase knowledge the data can provide. One such tool is Tecplot Chorus, whose strengths are visualizing ensemble metadata and linked images. This report contains the analysis and conclusions from evaluating Tecplot Chorus with an example problem that is relevant to Sandia National Laboratories. This report documents a preliminary evaluation of Tecplot Chorus for analyzing ensemble data from CTH simulations. The project that funded this report and evaluation is also evaluating and guiding development with SNL’s Slycat. Slycat and Tecplot Chorus each have their strengths, weaknesses, and overlapping capabilities. It is quite likely that, as the scale of ensemble data increases, both of these tools (and possibly others) will be needed for different processing goals. This report will focus on Tecplot Chorus and its application to an example ensemble of data supplied by David J. Peterson and John P. Korbin; this example is of a flyer plate impact and weld study henceforth referred to as CTH Impact Example. This evaluation also defines a workflow for analysts that can help reduce the time and resources for processing ensemble data.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Guidelines for Provision and Interchange of Geothermal Data Assets

The US Department of Energy Office of Geothermal Technologies (OGT) is funding and overseeing the development of the National Geothermal Data System (NGDS), a distributed information system providing access to integrated data in support of, and generated in, all phases of geothermal development. NGDS is being built in an open paradigm and will employ state-of-the-art informatics approaches and capabilities to advance the state of geothermal knowledge in the US. This document presents guidelines related to provision and interchange of data assets in the context of the National Geothermal Data System. It identifies general specifications for NGDS catalog metadata and data content, and provides specific instructions for preparation and submission of data assets by OGT-funded projects.

geothermal data↗

Neutron and X-ray Detectors

The Basic Energy Sciences (BES) X-ray and neutron user facilities attract more than 12,000 researchers each year to perform cutting-edge science at these state-of-the-art sources. While impressive breakthroughs in X-ray and neutron sources give us the powerful illumination needed to peer into the nano- to mesoscale world, a stumbling block continues to be the distinct lag in detector development, which is slowing progress toward data collection and analysis. Urgently needed detector improvements would reveal chemical composition and bonding in 3-D and in real time, allow researchers to watch “movies” of essential life processes as they happen, and make much more efficient use of every X-ray and neutron produced by the source The immense scientific potential that will come from better detectors has triggered worldwide activity in this area. Europe in particular has made impressive strides, outpacing the United States on several fronts. Maintaining a vital U.S. leadership in this key research endeavor will require targeted investments in detector R&D and infrastructure. To clarify the gap between detector development and source advances, and to identify opportunities to maximize the scientific impact of BES user facilities, a workshop on Neutron and X-ray Detectors was held August 1-3, 2012, in Gaithersburg, Maryland. Participants from universities, national laboratories, and commercial organizations from the United States and around the globe participated in plenary sessions, breakout groups, and joint open-discussion summary sessions. Sources have become immensely more powerful and are now brighter (more particles focused onto the sample per second) and more precise (higher spatial, spectral, and temporal resolution). To fully utilize these source advances, detectors must become faster, more efficient, and more discriminating. In supporting the mission of today’s cutting-edge neutron and X-ray sources, the workshop identified six detector research challenges (and two computing hurdles that result from the corresponding increase in data volume) for the detector community to overcome in order to realize the full potential of BES neutron and X-ray facilities. Resolving these detector impediments will improve scientific productivity both by enabling new types of experiments, which will expand the scientific breadth at the X-ray and neutron facilities, and by potentially reducing the beam time required for a given experiment. These research priorities are summarized in the table below. Note that multiple, simultaneous detector improvements are often required to take full advantage of brighter sources. High-efficiency hard X-ray sensors: The fraction of incident particles that are actually detected defines detector efficiency. Silicon, the most common direct-detection X-ray sensor material, is (for typical sensor thicknesses) 100% efficient at 8 keV, 25%efficient at 20 keV, and only 3% efficient at 50 keV. Other materials are needed for hard X-rays. Replacement for 3He for neutron detectors: 3He has long been the neutron detection medium of choice because of its high cross section over a wide neutron energy range for the reaction 3He + n —> 3H + 1H + 0.764 MeV. 3He stockpiles are rapidly dwindling, and what is available can be had only at prohibitively high prices. Doped scintillators hold promise as ways to capture neutrons and convert them into light, although work is needed on brighter, more efficient scintillator solutions. Neutron detectors also require advances in speed and resolution. Fast-framing X-ray detectors: Today’s brighter X-ray sources make time-resolved studies possible. For example, hybrid X-ray pixel detectors, initially developed for particle physics, are becoming fairly mature X-ray detectors, with considerable development in Europe. To truly enable time-resolved studies, higher frame rates and dynamic range are required, and smaller pixel sizes are desirable. High-speed spectroscopic X-ray detectors: Improvements in the readout speed and energy resolution of X-ray detectors are essential to enable chemically sensitive microscopies. Advances would make it possible to take images with simultaneous spatial and chemical information. Very high-energy-resolution X-ray detectors: The energy resolution of semiconductor detectors, while suitable for a wide range of applications, is far less than what can be achieved with X-ray optics. A direct detector that could rival the energy resolution of optics could dramatically improve the efficiency of a multitude of experiments, as experiments are often repeated at a number of different energies. Very high-energy-resolution detectors could make these experiments parallel, rather than serial. Low-background, high-spatial-resolution neutron detectors: Low-background detectors would significantly improve experiments that probe excitations (phonons, spin excitations, rotation, and diffusion in polymers and molecular substances, etc.) in condensed matter. Improved spatial resolution would greatly benefit radiography, tomography, phase-contrast imaging, and holography. Improved acquisition and visualization tools: In the past, with the limited variety of slow detectors, it was straightforward to visualize data as it was being acquired (and adjust experimental conditions accordingly) to create a compact data set that the user could easily transport. As detector complexity and data rates explode, this becomes much more challenging. Three goals were identified as important for coping with the growing data volume from high-speed detectors: Facilitate better algorithm development. In particular, algorithms that can minimize the quantity of data stored. Improve community-driven mechanisms to reduce data protocols and enhance quantitative, interactive visualization tools. Develop and distribute community-developed, detector-specific simulation tools. Aim for parallelization to take advantage of high-performance analysis platforms. Improved analysis work flows: Standardize the format of metadata that accompanies detector data and describes the experimental setup and conditions. Develop a standardized user interface and software framework for analysis and data management. The diversity of detector improvements required is necessarily as broad as the range of scientific experimentation at BES facilities. This workshop identified a variety of avenues by which detector R&D can enable enhanced science at BES facilities. The Research Directions listed above will be addressed by focused R&D and detector engineering, both of which require specialized infrastructure and skills. While U.S. leadership in neutron and X-ray detectors lags behind other countries in several areas, significant talent exists across the complex. A forum of technical experts, facilities management, and BES could be a venue to provide further definition.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

The ARES High-level Intermediate Representation

The LLVM intermediate representation (IR) lacks semantic constructs for depicting common high-performance operations such as parallel and concurrent execution, communication and synchronization. Currently, representing such semantics in LLVM requires either extending the intermediate form (a signi cant undertaking) or the use of ad hoc indirect means such as encoding them as intrinsics and/or the use of metadata constructs. In this paper we discuss a work in progress to explore the design and implementation of a new compilation stage and associated high-level intermediate form that is placed between the abstract syntax tree and when it is lowered to LLVM's IR. This highlevel representation is a superset of LLVM IR and supports the direct representation of these common parallel computing constructs along with the infrastructure for supporting analysis and transformation passes on this representation.

97 MATHEMATICS AND COMPUTING↗

Report on the Status of the SFWST Campaign International Activities in Disposal Research at SNL

The following summaries are provided as fulfillment of milestone M4SF-17SN080305022 and represent international coordination activities in disposal research funded by the US DOE Spent Fuel and Waste Storage and Technologies (SFWST) Campaign during Fiscal Year 2017: SFWST funded bi-lateral interactions with Taiwan, OECD-NEA Repository Metadata (RepMet) project, SFWST funded bi-lateral interactions with the Republic of Korea.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Final Report: Archiving Data to Support Data Synthesis of DOE Sponsored Elevated CO 2 Experiments

Over the last three decades DOE made a large investment in field-scale experiments in order to understand the role of terrestrial ecosystems in the global carbon cycle, and forecast how carbon cycling will change over the next century. The Smithsonian Environmental Research Center received one of the first awards in this program and managed two long-term studies (25 years and 10 years) with a total of approximately $10 million of support from DOE, and many more millions leveraged from the Smithsonian Institution and agencies such as NSF. The present DOE grant was based on the premise that such a large investment demands a proper synthesis effort so that the full potential of these experiments are realized through data analysis and modeling. The goal of the this grant was to archive legacy data from two major elevated carbon dioxide experiments in DOE databases, and to engage in synthesis activities using these data. Both goals were met. All datasets deemed a high priority for data synthesis and modeling were prepared for archiving and analysis. Many of these datasets were deposited in DOE’s CDIAC, while others are being held at the Oak Ridge National Lab and the Smithsonian Institution until they can be received by DOE’s new ESS-DIVE system at Berkeley Lab. Most of the effort was invested in researching and re-constituting high-quality data sets from a 30-year elevated CO 2 experiment. Using these data, the grant produced products that are already benefiting climate change science, including the publication of new coastal wetland allometry equations based on 9,771 observations, public posting of dozens of datasets, metadata and supporting codes from long-term experiments at the Global Change Research Wetland, and publication of two synthetic data papers on scrub oak forest responses to elevated CO 2 . In addition, three papers are in review or nearing submission reporting unexpected long-term patterns in ecosystem responses to elevated CO 2 and nitrogen in a coastal wetland.

54 ENVIRONMENTAL SCIENCES↗

Final Report for File System Support for Burst Buffers on HPC Systems

Distributed burst buffers are a promising storage architecture for handling I/O workloads for exascale computing. As they are being deployed on more supercomputers, a file system that efficiently manages these burst buffers for fast I/O operations carries great consequence. Over the past year, FSU team has undertaken several efforts to design, prototype and evaluate distributed file systems for burst buffers on HPC systems. These include MetaKV: a Key-Value Store for Metadata Management of Distributed Burst Buffers, a user-level file system with multiple backends, and a specialized file system for large datasets of deep neural networks. Our progress for these respective efforts are elaborated further in this report.

97 MATHEMATICS AND COMPUTING↗

Analysis Report for Exascale Storage Requirements for Scientific Data.

Over the next 10 years, the Department of Energy will be transitioning from Petascale to Exascale Computing resulting in data storage, networking, and infrastructure requirements to increase by three orders of magnitude. The technologies and best practices used today are the result of a relatively slow evolution of ancestral technologies developed in the 1950s and 1960s. These include magnetic tape, magnetic disk, networking, databases, file systems, and operating systems. These technologies will continue to evolve over the next 10 to 15 years on a reasonably predictable path. Experience with the challenges involved in transitioning these fundamental technologies from Terascale to Petascale computing systems has raised questions about how these will scale another 3 or 4 orders of magnitude to meet the requirements imposed by Exascale computing systems. This report is focused on the most concerning scaling issues with data storage systems as they relate to High Performance Computing- and presents options for a path forward. Given the ability to store exponentially increasing amounts of data, far more advanced concepts and use of metadata will be critical to managing data in Exascale computing systems.

97 MATHEMATICS AND COMPUTING↗

Advanced Energy System Design (AESD): Technical Manual for the Records API

The Records API (application program interface) for Advanced Energy System Design (AESD) enables software that serves multidimensional record-oriented data to interoperate with software than uses such data. In the context of the Records API, multidimensional data records are simply tuples of real numbers, integers, and character strings, where each data value is tagged by a variable name, according to a pre-defined schema, and each record is assigned a unique integer identifier. Conceptually, these records are isomorphic to rows in a relational database, JSON objects, or key-value maps. Records servers might supply static datasets, sensor measurements that periodically update as new telemetry become available, or the results of simulations as the simulations generate new output. Records client software might display or analyze the data, but in the case of simulations the client request the creation of new ensembles for specified input parameters. It is also possible to chain records clients and servers together so that a client consuming data from a server might transform that data and serve it to further clients. This minimalist API avoids imposing burdensome metadata, structural, or implementation requirements on developers by relying on open-source technologies that is readily available for common programming languages. In particular, the API has been designed to place the smallest possible burden on services that provide data. This document defines the message format for the Records API, a transport mechanism for communicating the data, and semantics for interpreting it. The message format is specified as Google Protocol Buffers and the transport mechanism uses WebSockets. We discuss three major use cases for serving and consuming records data: (i) static data, (ii) dynamically augmented data, (iii) on-demand simulations, (iv) with filters, and (v) with bookmarks. Separate implementations of the API exist in C++, Haskell, JavaScript, Python, and R.

97 MATHEMATICS AND COMPUTING↗

Dynamic Networks Experiment FY18 - Quality Control Focus Area Efforts

The availability of data whose quality is fittingly labeled is of tantamount importance to signal detection and all other downstream signal processing operations. Without identification, the goals of low yield nuclear monitoring cannot reach their full potential. A fundamental question that remains is the knowledge of the capability gaps that prevent us from using open network data. With these new channels, we will need better understanding of the frequency, type, severity, and impact of artifacts when compared to conventional networks. Moreover, beyond a simple quality label, thresholds for acceptable use of the data will need to be established, likely on a use case by use case basis. Finally, orders of magnitude more data with an at-best static analyst pool size means that quality artifacts must be identified with as little human interaction as possible. Automated methodologies must be developed to account for the scale of the problem when ad hoc networks are candidates for processing. At the same time that “data wealth” provides more opportunity for signal exploitation, the introduction of ad hoc networks introduces an increased risk quality issues in both sensor value and metadata issues. The artifacts shown below are examples of some sensor-based quality issues that impair downstream processing. The machine learning model used in this project was able to identify the artifacts that produce these spurious correlations, significantly improving analyst productivity.

97 MATHEMATICS AND COMPUTING↗

WCATS User's Guide

This manual has been written to provide a general reference of the usage of the Waste Compliance and Tracking System (WCATS). Because WCATS is designed to be used at many different kinds of facilities, much of the usage of the application is driven by the metadata configuration of the system. General usage of the system and the different types of tasks provided is included in this document, but custom usage and an exhaustive list of configuration options will not be provided. This manual is not intended to be a replacement for detailed work procedures or desk instructions, and it will not include configuration details such as service unit names, waste stream numbers for selection, etc. Any use of such information in this document is only provided as an example.

42 ENGINEERING↗

Inferring Organizational Structure from Behavior

In this project, we researched new techniques for detecting hidden networks of individuals and institutions by introducing the use of temporal correlations among behaviors, leveraging both information sources and metadata. We validated the algorithms using the Wikipedia edit history. The rapid increase in crowd-sourced applications like Wikipedia is providing a rich set of data with both a record of behaviors and a set of direct interactions among individuals. Data sets with network ground truth are needed to develop and validate models, before applying them to national security settings where content and meta-data alone are available.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Publication of LLNL Genomics Data to ESGF: a Technical Report for DREAM Project

We present the process used to set up several example genomic data sets within an ESGF framework. The process begins with designing a project, staging the data, and handling the necessary configuration. We describe the publication process used to ingest the datasets, in essence, all required metadata. Finally, we describe the data web service that we introduce in the DREAM project which makes these datasets available for download or exploration.

59 BASIC BIOLOGICAL SCIENCES↗