Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data storage data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

NASDA's earth observation satellite data archive policy for the earth observation data and information system (EOIS)

NASDA's new Advanced Earth Observing Satellite (ADEOS) is scheduled for launch in August, 1996. ADEOS carries 8 sensors to observe earth environmental phenomena and sends their data to NASDA, NASA, and other foreign ground stations around the world. The downlink data bit rate for ADEOS is 126 MB/s and the total volume of data is about 100 GB per day. To archive and manage such a large quantity of data with high reliability and easy accessibility it was necessary to develop a new mass storage system with a catalogue information database using advanced database management technology. The data will be archived and maintained in the Master Data Storage Subsystem (MDSS) which is one subsystem in NASDA's new Earth Observation data and Information System (EOIS). The MDSS is based on a SONY ID1 digital tape robotics system. This paper provides an overview of the EOIS system, with a focus on the Master Data Storage Subsystem and the NASDA Earth Observation Center (EOC) archive policy for earth observation satellite data.

Sobue, Shin-ichi↗

Evolution of the satellite telemetry data processing facility at the Goddard Space Flight Center.

Data from scientific and application satellites managed by the NASA Goddard Space Flight Center are acquired through the world-wide Spacecraft Tracking and Data Network (STDN). These data are forwarded to a central telemetry data processing facility whose primary objective is the timely provision of the data to users in a form suitable for their analysis. In a successful evolution, a satellite data processing facility must adapt to changing support requirements while operationally supporting active spacecraft. Advances in the technology of data transmission, data storage systems, and file management make it feasible to implement a third-generation system which will be able to satisfy experimenter requirements for the next ten years.

Keipert, F. A.↗

NDMAS: Data storage, visualization, analysis, delivery, and more

Slides representing Nuclear Data Management and Analysis System (NDMAS) Data for properly preserving our publicly funded data, including DOE Public Access Plan and the users of NDMAS. From Capture, Storage, Delivery & Analysis, Archival, and Qualification of Fuel fabrication, PIE, ART Experiment monitoring and operations, and processes for recording work and for looking ahead.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Comprehensive Monitoring for Heterogeneous Geographically Distributed Storage

Storage capacity at CMS Tier-1 and Tier-2 sites reached over 100 Petabytes in 2014, and will be substantially increased during Run 2 data taking. The allocation of storage for the individual users analysis data, which is not accounted as a centrally managed storage space, will be increased to up to 40%. For comprehensive tracking and monitoring of the storage utilization across all participating sites, CMS developed a space monitoring system, which provides a central view of the geographically dispersed heterogeneous storage systems. The first prototype was deployed at pilot sites in summer 2014, and has been substantially reworked since then. In this paper we discuss the functionality and our experience of system deployment and operation on the full CMS scale.

Ratnikova, N.↗

Enabling a Decentralized Smart Grid Using Autonomous Edge Control Devices

As a large number of distributed devices are connected to the modern smart grid, the traditional centralized connectivity models fail to provide economic value. These models have relied on sending data to the cloud for processing and receiving commands to exert control actions, resulting in an ‘on-demand system’ with high bandwidth, low latency and an overload of data on the cloud. For realizing a decentralized system, there is a strong need to embed intelligence at the ‘edge of the network’. These intelligent devices, capable of sensing, local data processing and exerting control actions, report only actionable information to the cloud, acting as an edge control node. The system can then function autonomously, without constant cloud inputs, tolerating longer delays in communication, making the overall system ultra-low cost. The Global Asset Monitoring, Management and Analytics (GAMMA) Platform is a novel ultralow- cost, secure platform that operates through a Bluetooth based delay tolerant network. It relies on so-called ‘data mules’ to bridge the last mile connectivity gap in an inherently secure way. Due to this model, the platform requires no in-country certifications, does not rely on a dedicated backhaul technology and is immune to technology migration. This architecture also addresses some gaps identified in traditional IoT-based solutions in remote areas and sparse connectivity. A functional unit of the edge computing node has been built, taking into account various constraints like costs, customizations, data storage, cybersecurity and power management. Here, the platform has been built, deployed and has demonstrated distributed smart grid applications like power quality sensing, automated metering infrastructure and utility asset monitoring.

42 ENGINEERING↗

Use of HSM with Relational Databases

Hierarchical storage management (HSM) systems have evolved to become a critical component of large information storage operations. They are built on the concept of using a hierarchy of storage technologies to provide a balance in performance and cost. In general, they migrate data from expensive high performance storage to inexpensive low performance storage based on frequency of use. The predominant usage characteristic is that frequency of use is reduced with age and in most cases quite rapidly. The result is that HSM provides an economical means for managing and storing massive volumes of data. Inherent in HSM systems is system managed storage, where the system performs most of the work with minimum operations personnel involvement. This automation is generally extended to include: backup and recovery, data duplexing to provide high availability, and catastrophic recovery through use of off-site storage.

Breeden, Randall↗

Advanced optical disk storage technology

There is a growing need within the Air Force for more and better data storage solutions. Rome Laboratory, the Air Force's Center of Excellence for C3I technology, has sponsored the development of a number of operational prototypes to deal with this growing problem. This paper will briefly summarize the various prototype developments with examples of full mil-spec and best commercial practice. These prototypes have successfully operated under severe space, airborne and tactical field environments. From a technical perspective these prototypes have included rewritable optical media ranging from a 5.25-inch diameter format up to the 14-inch diameter disk format. Implementations include an airborne sensor recorder, a deployable optical jukebox and a parallel array of optical disk drives. They include stand-alone peripheral devices to centralized, hierarchical storage management systems for distributed data processing applications.

Haritatos, Fred N.↗

Integration of end-user Cloud storage for CMS analysis

End-user Cloud storage is increasing rapidly in popularity in research communities thanks to the collaboration capabilities it offers, namely synchronisation and sharing. CERN IT has implemented a model of such storage named, CERNBox, integrated with the CERN AuthN and AuthZ services. To exploit the use of the end-user Cloud storage for the distributed data analysis activity, the CMS experiment has started the integration of CERNBox as a Grid resource. This will allow CMS users to make use of their own storage in the Cloud for their analysis activities as well as to benefit from synchronisation and sharing capabilities to achieve results faster and more effectively. It will provide an integration model of Cloud storages in the Grid, which is implemented and commissioned over the world’s largest computing Grid infrastructure, Worldwide LHC Computing Grid (WLCG). In this paper, we present the integration strategy and infrastructure changes needed in order to transparently integrate end-user Cloud storage with the CMS distributed computing model. We describe the new challenges faced in data management between Grid and Cloud and how they were addressed, along with details of the support for Cloud storage recently introduced into the WLCG data movement middleware, FTS3. Finally, the commissioning experience of CERNBox for the distributed data analysis activity is also presented.

97 MATHEMATICS AND COMPUTING↗

Enabling Earth Science: The Facilities and People of the NCCS

The NCCS's mass data storage system allows scientists to store and manage the vast amounts of data generated by these computations, and its high-speed network connections allow the data to be accessed quickly from the NCCS archives. Some NCCS users perform studies that are directly related to their ability to run computationally expensive and data-intensive simulations. Because the number and type of questions scientists research often are limited by computing power, the NCCS continually pursues the latest technologies in computing, mass storage, and networking technologies. Just as important as the processors, tapes, and routers of the NCCS are the personnel who administer this hardware, create and manage accounts, maintain security, and assist the scientists, often working one on one with them.

Source record↗

Fifth NASA Goddard Conference on Mass Storage Systems and Technologies.

This document contains copies of those technical papers received in time for publication prior to the Fifth Goddard Conference on Mass Storage Systems and Technologies. As one of an ongoing series, this conference continues to serve as a unique medium for the exchange of information on topics relating to the ingestion and management of substantial amounts of data and the attendant problems involved. This year's discussion topics include storage architecture, database management, data distribution, file system performance and modeling, and optical recording technology. There will also be a paper on Application Programming Interfaces (API) for a Physical Volume Repository (PVR) defined in Version 5 of the Institute of Electrical and Electronics Engineers (IEEE) Reference Model (RM). In addition, there are papers on specific archives and storage products.

Kobler, Benjamin↗

Distributed data management on Belle II

The Belle II experiment started taking physics data in April 2018 with an estimated total volume of all files including raw events, Monte-Carlo and skim statistics of 340 petabytes expected by the end of operations in the late-2020s. Originally designed as a fully integrated component of the BelleDIRAC production system, the Belle II distributed data management (DDM) software needs to manage data across about 29 storage elements worldwide for a collaboration of nearly 1000 physicists. By late 2018, this software required significant performance improvements to meet the requirements of physics data taking and was seriously lacking in automation. Rucio, the DDM solution created by ATLAS, was an obvious alternative but required tight integration with BelleDIRAC and a seamless yet non-trivial migration. This contribution describes the work done on both DDM options, the current status of the software running successfully in production and the problems associated with trying to balance long-term operations cost against short term risk. .

97 MATHEMATICS AND COMPUTING↗

Data Crosscutting Requirements Review

In April 2013, a diverse group of researchers from the U.S. Department of Energy (DOE) scientific community assembled to assess data requirements associated with DOE-sponsored scientific facilities and large-scale experiments. Participants in the review included facilities staff, program managers, and scientific experts from the offices of Basic Energy Sciences, Biological and Environmental Research, High Energy Physics, and Advanced Scientific Computing Research. As part of the meeting, review participants discussed key issues associated with three distinct aspects of the data challenge: 1) processing, 2) management, and 3) analysis. These discussions identified commonalities and differences among the needs of varied scientific communities. They also helped to articulate gaps between current approaches and future needs, as well as the research advances that will be required to close these gaps. Moreover, the review provided a rare opportunity for experts from across the Office of Science to learn about their collective expertise, challenges, and opportunities. The "Data Crosscutting Requirements Review" generated specific findings and recommendations for addressing large-scale data crosscutting requirements.

ASCR↗

Specification of Fenix MPI Fault Tolerance library version 1.0.1

This document provides a specification of Fenix, a software library compatible with the Message Passing Interface (MPI) to support fault recovery without application shutdown. The library consists of two modules. The first, termed process recovery , restores an application to a consistent state after it has suffered a loss of one or more MPI processes (ranks). The second specifies functions the user can invoke to store application data in Fenix managed redundant storage, and to retrieve it from that storage after process recovery.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Specification of Fenix MPI Fault Tolerance library (V. 1.0)

This document provides a specification of Fenix, a software library compatible with the Message Passing Interface (MPI) to support fault recovery without application shutdown. The library consists of two modules. The first, termed process recovery, restores an application to a consistent state after it has suffered a loss of one or more MPI processes (ranks). The second specifies functions the user can invoke to store application data in Fenix managed redundant storage, and to retrieve it from that storage after process recovery.

97 MATHEMATICS AND COMPUTING↗

Scientific Data Compression for Large Scale Computational Fluid Dynamics (CFD) Simulations

This Cooperative Research and Development Agreement (CRADA) between Oak Ridge National Laboratory (ORNL) and General Electric (GE) investigated methods for reducing the size of large computational fluid dynamics (CFD) simulation datasets using scientific data compression techniques. The work focused on adapting the MultiGrid Adaptive Reduction of Data (MGARD) compression framework and integrating it with high-performance I/O and visualization tools used in CFD workflows. MGARD uses hierarchical multilevel decomposition to enable error-controlled compression of floating-point scientific data while preserving quantities of interest. During the project, MGARD compression was integrated with the ADIOS I/O framework and visualization tools such as ParaView to enable efficient storage, transfer, and analysis of simulation data. The collaboration also explored approaches for improving compression performance for CFD data defined on unstructured meshes. Results demonstrate that scientific data compression can significantly reduce storage requirements and improve data management for large-scale CFD simulations.

97 MATHEMATICS AND COMPUTING↗

A review of hypertext in a NASA project management context

The principles of data storage, the comparative strengths of data bases, and the evolution of hypertext within this context are discussed. A classification schema of indexing and of hypertext document structures is provided. Issues associated with hypertext implementation are also discussed and potential areas for further research are indicated.

Bell, Christopher J.↗

Visualization Techniques in Space and Atmospheric Sciences

Unprecedented volumes of data will be generated by research programs that investigate the Earth as a system and the origin of the universe, which will in turn require analysis and interpretation that will lead to meaningful scientific insight. Providing a widely distributed research community with the ability to access, manipulate, analyze, and visualize these complex, multidimensional data sets depends on a wide range of computer science and technology topics. Data storage and compression, data base management, computational methods and algorithms, artificial intelligence, telecommunications, and high-resolution display are just a few of the topics addressed. A unifying theme throughout the papers with regards to advanced data handling and visualization is the need for interactivity, speed, user-friendliness, and extensibility.

Szuszczewicz, E. P.↗

Flight Computer Design for the Space Technology 5 (ST-5) Mission

As part of NASA's New Millennium Program, the Space Technology 5 mission will validate a variety of technologies for nano-satellite and constellation mission applications. Included are: a miniaturized and low power X-band transponder, a constellation communication and navigation transceiver, a cold gas micro-thruster, two different variable emittance (thermal) controllers, flex cables for solar array power collection, autonomous groundbased constellation management tools, and a new CMOS ultra low-power, radiation-tolerant, +0.5 volt logic technology. The ST-5 focus is on small and low-power. A single-processor, multi-function flight computer will implement direct digital and analog interfaces to all of the other spacecraft subsystems and components. There will not be a distributed data system that uses a standardized serial bus such as MIL-STD-1553 or MIL-STD-1773. The flight software running on the single processor will be responsible for all real-time processing associated with: guidance, navigation and control, command and data handling (C&DH) including uplink/downlink, power switching and battery charge management, science data analysis and storage, intra-constellation communications, and housekeeping data collection and logging. As a nanosatellite trail-blazer for future constellations of up to 100 separate space vehicles, ST-5 will demonstrate a compact (single board), low power (5.5 watts) solution to the data acquisition, control, communications, processing and storage requirements that have traditionally required an entire network of separate circuit boards and/or avionics boxes. In addition to the New Millennium technologies, other major spacecraft subsystems include the power system electronics, a lithium-ion battery, triple-junction solar cell arrays, a science-grade magnetometer, a miniature spinning sun sensor, and a propulsion system.

Speer, David↗