Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data storage data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27

Integrating UniTree with the data migration API

The Data Migration Application Programming Interface (DMAPI) has the potential to allow developers of open systems Hierarchical Storage Management (HSM) products to virtualize native file systems without the requirement to make changes to the underlying operating system. This paper describes advantages of virtualizing native file systems in hierarchical storage management systems, the DMAPI at a high level, what the goals are for the interface, and the integration of the Convex UniTree+HSM with DMAPI along with some of the benefits derived in the resulting product.

Schrodel, David G.↗

Mass Storage Performance Information System

The purpose of this task is to develop a data warehouse to enable system administrators and their managers to gather information by querying the data logs of the MDSDS. Currently detailed logs capture the activity of the MDSDS internal to the different systems. The elements to be included in the data warehouse are requirements analysis, data cleansing, database design, database population, hardware/software acquisition, data transformation, query and report generation, and data mining.

Scheuermann, Peter↗

Space-based Science Operations Grid Prototype

Grid technology is the up and coming technology that is enabling widely disparate services to be offered to users that is very economical, easy to use and not available on a wide basis. Under the Grid concept disparate organizations generally defined as "virtual organizations" can share services i.e. sharing discipline specific computer applications, required to accomplish the specific scientific and engineering organizational goals and objectives. Grids are emerging as the new technology of the future. Grid technology has been enabled by the evolution of increasingly high speed networking. Without the evolution of high speed networking Grid technology would not have emerged. NASA/Marshall Space Flight Center's (MSFC) Flight Projects Directorate, Ground Systems Department is developing a Space-based Science Operations Grid prototype to provide to scientists and engineers the tools necessary to operate space-based science payloads/experiments and for scientists to conduct public and educational outreach. In addition Grid technology can provide new services not currently available to users. These services include mission voice and video, application sharing, telemetry management and display, payload and experiment commanding, data mining, high order data processing, discipline specific application sharing and data storage, all from a single grid portal. The Prototype will provide most of these services in a first step demonstration of integrated Grid and space-based science operations technologies. It will initially be based on the International Space Station science operational services located at the Payload Operations Integration Center at MSFC, but can be applied to many NASA projects including free flying satellites and future projects. The Prototype will use the Internet2 Abilene Research and Education Network that is currently a 10 Gb backbone network to reach the University of Alabama at Huntsville and several other, as yet unidentified, Space Station based science experimenters. There is an international aspect to the Grid involving the America's Pathway (AMPath) network, the Chilean REUNA Research and Education Network and the University of Chile in Santiago that will further demonstrate how extensive these services can be used. From the user's perspective, the Prototype will provide a single interface and logon to these varied services without the complexity of knowing the where's and how's of each service. There is a separate and deliberate emphasis on security. Security will be addressed by specifically outlining the different approaches and tools used. Grid technology, unlike the Internet, is being designed with security in mind. In addition we will show the locations, configurations and network paths associated with each service and virtual organization. We will discuss the separate virtual organizations that we define for the varied user communities. These will include certain, as yet undetermined, space-based science functions and/or processes and will include specific virtual organizations required for public and educational outreach and science and engineering collaboration. We will also discuss the Grid Prototype performance and the potential for further Grid applications both space-based and ground based projects and processes. In this paper and presentation we will detail each service and how they are integrated using Grid

Bradford, Robert N.↗

Intelligent Integrated System Health Management

Intelligent Integrated System Health Management (ISHM) is the management of data, information, and knowledge (DIaK) with the purposeful objective of determining the health of a system (Management: storage, distribution, sharing, maintenance, processing, reasoning, and presentation). Presentation discusses: (1) ISHM Capability Development. (1a) ISHM Knowledge Model. (1b) Standards for ISHM Implementation. (1c) ISHM Domain Models (ISHM-DM's). (1d) Intelligent Sensors and Components. (2) ISHM in Systems Design, Engineering, and Integration. (3) Intelligent Control for ISHM-Enabled Systems

Figueroa, Fernando↗

A Components Database Design and Implementation for Accelerators and Detectors

Many accelerator and detector systems being fabricated for the PEP-II Accelerator and BABAR Detector needed configuration control and calibration measurements tracked for their components. Instead of building a database for each distinct system, a Components Database was designed and implemented that can encompass any type of component and any type of measurement. In this paper we describe this database design that is especially suited for the engineering and fabrication processes of the accelerator and detector environments where there are thousands of unique component types. We give examples of information stored in the Components Database, which includes accelerator configuration, calibration measurements, fabrication history, design specifications, inventory, etc. The World Wide Web interface is used to access the data, and templates are available for international collaborations to collect data off-line.

43 PARTICLE ACCELERATORS↗

Slycat v 1.0 Beta

Slycat is a web-based system for analysis of large, high-dimensional data such as that produced by High Performance Computing (HPC) platforms. The Slycat server integrates data ingestion, scalable analysis, data management, and visualization with commodity web clients using a multi-tiered hierarchy of data and model storage. Analysis inputs are pushed into Slycat, where models are generated and analysis artifacts are stored in a project database. These artifacts are the basis for visualizations that are delivered to users' desktops through ordinary web browsers, eliminating the need to build and deploy platform-specific client applications. Slycat currently provides two types of analysis: canonical correlation analysis (CCA) to model relationships between inputs and output metrics, and time series analysis featuring clustering and comparative visualization of waveforms.

Source record↗

EpiK: A Knowledge Base for Epidemiological Modeling and Analytics of Infectious Diseases

Computational epidemiology seeks to develop computational methods to study the distribution and determinants of health-related states or events (including disease), and the application of this study to the control of diseases and other health problems. Recent advances in computing and data sciences have led to the development of innovative modeling environments to support this important goal. The datasets used to drive the dynamic models as well as the data produced by these models presents unique challenges owing to their size, heterogeneity and diversity. These datasets form the basis of effective and easy to use decision support and analytical environments. As a result, it is important to develop scalable data management systems to store, manage and integrate these datasets. In this paper, we develop EpiK —a knowledge base that facilitates the development of decision support and analytical environments to support epidemic science. An important goal is to develop a framework that links the input as well as output datasets to facilitate effective spatio-temporal and social reasoning that is critical in planning and intervention analysis before and during an epidemic. The data management framework links modeling workflow data and its metadata using a controlled vocabulary. The metadata captures information about storage, the mapping between the linked model and the physical layout, and relationships to support services. EpiK is designed to support agent-based modeling and analytics frameworks—aggregate models can be seen as special cases and are thus supported. We use semantic web technologies to create a representation of the datasets that encapsulates both the location and the schema heterogeneity. The choice of RDF as a representation language is motivated by the diversity and growth of the datasets that need to be integrated. A query bank is developed—the queries capture a broad range of questions that can be posed and answered during a typical case study pertaining to disease outbreaks. The queries are constructed using SPARQL Protocol and RDF Query Language (SPARQL) over the EpiK. EpiK can hide schema and location heterogeneity while efficiently supporting queries that span the computational epidemiology modeling pipeline: from model construction to simulation output. As a result, we show that the performance of benchmark queries varies significantly with respect to the choice of hardware underlying the database and resource description framework (RDF) engine.

60 APPLIED LIFE SCIENCES↗

Data storage accounting and verification at LHC experiments

All major experiments at the Large Hadron Collider (LHC) need to measure real storage usage at the Grid sites. This information is equally important for resource management, planning, and operations. To verify the consistency of central catalogs, experiments are asking sites to provide a full list of the files they have on storage, including size, checksum, and other file attributes. Such storage dumps, provided at regular intervals, give a realistic view of the storage resource usage by the experiments. Regular monitoring of the space usage and data verification serve as additional internal checks of the system integrity and performance. Both the importance and the complexity of these tasks increase with the constant growth of the total data volumes during the active data taking period at the LHC. The use of common solutions helps to reduce the maintenance costs, both at the large Tier1 facilities supporting multiple virtual organizations and at the small sites that often lack manpower. We discuss requirements and solutions to the common tasks of data storage accounting and verification, and present experiment-specific strategies and implementations used within the LHC experiments according to their computing models.

Huang, C. H.↗

Next-generation building energy management systems and implications for electricity markets.

The U.S. national electric grid is facing significant changes due to aggressive federal and state targets to decrease emissions while improving grid efficiency and reliability. Additional challenges include supply/demand imbalances, transmission constraints, and aging infrastructure. A significant number of technologies are emerging under this environment including renewable generation, distributed storage, and energy management systems. In this paper, we claim that predictive energy management systems can play a significant role in achieving federal and state targets. These systems can merge sensor data and predictive statistical models, thereby allowing for a more proactive modulation of building energy usage as external weather and market signals change. A key observation is that these predictive capabilities, coupled with the fast responsiveness of air handling units and storage devices, can enable participation in several markets such as the day-ahead and real-time pricing markets, demand and reserves markets, and ancillary services markets. Participation in these markets has implications for both market prices and reliability and can help balance the integration of intermittent renewable resources. In addition, these emerging predictive energy management systems are inexpensive and easy to deploy, allowing for broad building participation in utility centric programs.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Landsat Pathfinder tropical forest information management system

A Tropical Forest Information Management System_(TFIMS) has been designed to fulfill the needs of HTFIP in such a way that it tracks all aspects of the generation and analysis of the raw satellite data and the derived deforestation dataset. The system is broken down into four components: satellite image selection, processing, data management and archive management. However, as we began to think of how the TFIMS could also be used to make the data readily accessible to all user communities we realized that the initial system was too project oriented and could only be accessed locally. The new system needed development in the areas of data ingest and storage, while at the same time being implemented on a server environment with a network interface accessible via Internet. This paper summarizes the overall design of the existing prototype (version 0) information management system and then presents the design of the new system (version 1). The development of version 1 of the TFIMS is ongoing. There are no current plans for a gradual transition from version 0 to version 1 because the significant changes are in how the data within the HTFIP will be made accessible to the extended community of scientists, policy makers, educators, and students and not in the functionality of the basic system.

Salas, W.↗

A Flexible Online Metadata Editing and Management System

A metadata editing and management system is being developed employing state of the art XML technologies. A modular and distributed design was chosen for scalability, flexibility, options for customizations, and the possibility to add more functionality at a later stage. The system consists of a desktop design tool or schema walker used to generate code for the actual online editor, a native XML database, and an online user access management application. The design tool is a Java Swing application that reads an XML schema, provides the designer with options to combine input fields into online forms and give the fields user friendly tags. Based on design decisions, the tool generates code for the online metadata editor. The code generated is an implementation of the XForms standard using the Orbeon Framework. The design tool fulfills two requirements: First, data entry forms based on one schema may be customized at design time and second data entry applications may be generated for any valid XML schema without relying on custom information in the schema. However, the customized information generated at design time is saved in a configuration file which may be re-used and changed again in the design tool. Future developments will add functionality to the design tool to integrate help text, tool tips, project specific keyword lists, and thesaurus services. Additional styling of the finished editor is accomplished via cascading style sheets which may be further customized and different look-and-feels may be accumulated through the community process. The customized editor produces XML files in compliance with the original schema, however, data from the current page is saved into a native XML database whenever the user moves to the next screen or pushes the save button independently of validity. Currently the system uses the open source XML database eXist for storage and management, which comes with third party online and desktop management tools. However, access to metadata files in the application introduced here is managed in a custom online module, using a MySQL backend accessed by a simple Java Server Faces front end. A flexible system with three grouping options, organization, group and single editing access is provided. Three levels were chosen to distribute administrative responsibilities and handle the common situation of an information manager entering the bulk of the metadata but leave specifics to the actual data provider.

99 GENERAL AND MISCELLANEOUS↗

Science and Applications Space Platform (SASP) End-to-End Data System Study

The capability of present technology and the Tracking and Data Relay Satellite System (TDRSS) to accommodate Science and Applications Space Platforms (SASP) payload user's requirements, maximum service to the user through optimization of the SASP Onboard Command and Data Management System, and the ability and availability of new technology to accommodate the evolution of SASP payloads were assessed. Key technology items identified to accommodate payloads on a SASP were onboard storage devices, multiplexers, and onboard data processors. The primary driver is the limited access to TDRSS for single access channels due to sharing with all the low Earth orbit spacecraft plus shuttle. Advantages of onboard data processing include long term storage of processed data until TRDSS is accessible, thus reducing the loss of data, eliminating large data processing tasks at the ground stations, and providing a more timely access to the data.

Crawford, P. R.↗

OECD NEA radioactive waste repository metadata management (RepMet) initiative (2014-2018) - 15614

The OECD/NEA Radioactive Waste Repository Metadata Management (RepMet) initiative aims to bring about a better understanding of the identification and administration of metadata - a key aspect of data management - to support national programmes in managing their radioactive waste repository data, information and records in a way that is both harmonised internationally and suitable for long-term management and use. At this point in the project, the main focus lies on 'waste packages at their storage sites ready for geological disposal' but excluding spent fuel and high level waste (HLW). Later work will deal with metadata collected on spent fuel and HLW as well as with different phases of repository development: design and construction, and operation. In order to develop metadata, RepMet members have provided lists of data items they currently collect and manage for packaged waste. The lists, albeit it non comprehensive, suggest that the number and type of data items varies importantly from organisation to organisation. At term, RepMet plans to generate a comprehensive list of such data items and their metadata. In parallel, RepMet is developing a document which examines a selected number of existing metadata standards. Because of the specialist nature of radioactive waste management, including waste processing facilities, disposal facilities and waste packages, existing metadata standards do not seem to be transferable as defined. Rather, RepMet will explore such standards to identify relevant principles for possible adaptation to the radioactive waste management community. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Transforming Energy Through Sustainable Mobility: Expanding Low-Carbon Transportation R&D Solutions

As the nation's premier laboratory for cutting-edge transportation decarbonization research and development solutions, the National Renewable Energy Laboratory (NREL) pioneers the creation and deployment of sustainable mobility technologies and strategies, with a focus on slashing transportation sector greenhouse gas emissions and combatting climate change. Trucks, planes, cargo ships, and other difficult-to-decarbonize vehicles are part of this essential transition for the transportation sector, which is currently the nation's largest source of the greenhouse gas emissions. NREL provides the scientific building blocks needed to spur innovation through multifaceted analysis, research, and engineering. This work acts as a catalyst to help industry bring affordable, high-performance, energy-efficient, and low-emission modes of transport and related infrastructure to market sooner. Our researchers collaborate closely with academic, government, and industry partners to design better batteries, drivetrains, and engines. They develop technologies for high-power charging, thermal management, energy storage, and power electronics. They are also reimagining fuels and combustion while creating sustainable lightweight materials. Unbiased expert guidance - backed by credible data and analysis, tools, and scientific rigor - empowers partners to make informed decisions about sustainable transportation. NREL recognizes that communities with limited mobility options face reduced access to employment opportunities, health care, and education, lowering overall quality of life. Alongside partners, NREL experts are creating transportation solutions that meet community-identified needs and increase mobility equity in historically underserved and overburdened communities. Rather than providing a one-size-fits-all solution, we take an interdisciplinary approach to mobility equity that considers the needs and challenges of diverse groups, maximizing benefits at the individual, community, and societal levels.

ADVANCED PROPULSION SYSTEMS↗

(ODIN): An Open Source, Low-Latency Data Integration & Visualization Framework for the NASA System Wide Safety Project's Disaster Response Safety Demonstration Series

The Open Data Integration Framework (ODIN) is an open source, low latency data integration and visualization framework (https://github.com/NASARace/race-odin) developed under NASA’s System WideSafety Program to demonstrate new safety capabilities designed to improve US airspace operations. Safety demonstrations are a set of increasingly complex (from public safety perspective) disaster response scenarios under which air systems must operate with increased capacity and include: 1) Wildland fire response, 2) Hurricane relief and recovery, 4) Emergency medical delivery via UAS and 4) Urban disaster relief. To accommodate disaster response, ODIN is field deployable and can scale on one or more multi-core, commodity laptops operating with full to limited or intermittent internet connectivity, conditions likely encountered during operations. ODIN runs as webserver with local, persistent data storage to serve either public or a secured, ad hoc network (e.g., an incident command post). The current released ODIN, ODIN-Fire is tailored for wildland fire management incorporating information on satellite overpasses with links to the near real-time data and imagery from the respective agencies. Included are winds data, an important variable for emergency responders and airspace operations, and high-resolution wind forecasts generated by super-computing resources and ingested into ODIN. As an open-source project, ODIN has attracted interest from multiple entities. We will show how 1) a commercial field instrument and data provider uses ODIN to help users visualize, publish and integrate their in-situ sensor network data and 2) ODIN’s capabilities to ingest, integrate and display near-real time satellite data with air traffic and a USFS winds forecast model used in fire response and post-fire assessment. Within NASA ODIN demonstrated novel, near terminal airspace safety capabilities for a project close-out event and previously it monitored the national airspace in real-time to meet an agency milestone. ODIN is presently under development for the anticipated hurricane relief and response demonstration notionally scheduled for the 2025-27 time frame and is available from NASA's github at the above link.

Aeronautics↗

The Future of NASA Earth Science in the Commercial Cloud: Challenges and Opportunities

NASA produces a large volume and variety of data products that are used every day to support research, decision making, and education. The widespread use of NASA’s Earth Science data is enabled by NASA’s Earth Science Data System (ESDS) program, which oversees the archiving and distribution of these data and invests in the development of new data systems and tools. However, NASA’s current approach to Earth Science data distribution — based on distributed institutional archives with individual on-premises high-performance computing capabilities — faces some significant challenges, including massive increases in data volume from upcoming missions, a greater need for transdisciplinary science that synthesizes many different kinds of observations, and a push to make science more open, inclusive, and accessible. To address these challenges, NASA is aggressively migrating its Earth Science data and related tools and services into the commercial cloud. Migration of data into the commercial cloud can significantly improve NASA’s existing data system capabilities by (1) providing more flexible options for storage and compute (including rapid, as-needed access to state-of-the-art capabilities); (2) by centralizing and standardizing data access, which gives all of NASA’s institutional data centers access to all of each other’s datasets; and (3) by facilitating “analysis-in-place”, whereby users can bring their own computational workflows and tools to the data rather than having to maintain their own copies of NASA datasets. However, migration to the commercial cloud also poses some significant challenges, including (1) managing costs under a “pay-as-you-go” model; (2) incompatibility with existing tools and data formats with object-based storage and network access; (3) vendor lock-in; (4) challenges with data access for workflows that mix on-premise and cloud computing; and (5) standardization for highly diverse data as is present in NASA’s data archive. I conclude with two examples of recent NASA activities showcasing capabilities enabled by the commercial cloud: An interactive analysis and development platform for analyzing airborne imaging spectroscopy data, and a new collection of tools and services for data discovery, analysis, publication, and data-driven storytelling (Visualization, Exploration, and Data Analysis, VEDA).

Alexey N Shiklomanov↗

Data compression: The end-to-end information systems perspective for NASA space science missions

The unique characteristics of compressed data have important implications to the design of space science data systems, science applications, and data compression techniques. The sequential nature or data dependence between each of the sample values within a block of compressed data introduces an error multiplication or propagation factor which compounds the effects of communication errors. The data communication characteristics of the onboard data acquisition, storage, and telecommunication channels may influence the size of the compressed blocks and the frequency of included re-initialization points. The organization of the compressed data are continually changing depending on the entropy of the input data. This also results in a variable output rate from the instrument which may require buffering to interface with the spacecraft data system. On the ground, there exist key tradeoff issues associated with the distribution and management of the science data products when data compression techniques are applied in order to alleviate the constraints imposed by ground communication bandwidth and data storage capacity.

Tai, Wallace↗

Biological and Environmental Research Network Requirements

The Energy Sciences Network (ESnet) is the primary provider of network connectivity for the U.S. Department of Energy (DOE) Office of Science (SC), the single largest supporter of basic research in the physical sciences in the United States. In support of SC programs, ESnet regularly updates and refreshes its understanding of the networking requirements of the instruments, facilities, scientists, and science programs that it serves. This focus has helped ESnet be a highly successful enabler of scientific discovery for over 25 years. In November 2012, ESnet and the Office of Biological and Environmental Research (BER) of the DOE SC organized a review to characterize the networking requirements of the programs funded by the BER program office. Several key findings resulted from the review. Among them: 1) The scale of data sets available to science collaborations continues to increase exponentially. This has broad impact, both on the network and on the computational and storage systems connected to the network. 2) Many science collaborations require assistance to cope with the systems and network engineering challenges inherent in managing the rapid growth in data scale. 3) Several science domains operate distributed facilities that rely on high-performance networking for success. Key examples illustrated in this report include the Earth System Grid Federation (ESGF) and the Systems Biology Knowledgebase (KBase). This report expands on these points, and addresses others as well. The report contains a findings section as well as the text of the case studies discussed at the review.

09 BIOMASS FUELS↗