Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

Photovoltaics Informatics: Harnessing Energy Science via Data-Driven Approaches

We discuss our current research focus on photovoltaic (PV) informatics, which is dedicated to functionality enhancement of solar materials through data management and data mining-aided, integrated computational materials engineering (ICME) for rapid screening and identification of multi-scale processing/structure/property/performance relationships. Our current PV informatics research ranges from transparent conducting oxides (TCO) to solar absorber materials. As a test bed, we report on examples of our current data management system for PV research and advanced data mining to improve the performance of solar cells such as CuInxGa1-xSe2 (CIGS) aiming at low-cost and high-rate processes. For the PV data management, we show recent developments of a strategy for data modeling, collection and aggregation methods, and construction of data interfaces, which enable proper archiving and data handling for data mining. For scientific data mining, the value of high-dimensional visualizations and non-linear dimensionality reduction is demonstrated to quantitatively assess how process conditions or properties are interconnected in the context of the development of Al-doped ZnO (AZO) thin films as the TCO layers for CIGS devices. Such relationships between processing and property of TCOs lead to optimal process design toward enhanced performance of CIGS cells/devices.

CONSTRUCTION↗

AOIPS data base management systems support for GARP data sets

A data base management system is identified, developed to provide flexible access to data sets produced by GARP during its data systems tests. The content and coverage of the data base are defined and a computer-aided, interactive information storage and retrieval system, implemented to facilitate access to user specified data subsets, is described. The computer programs developed to provide the capability were implemented on the highly interactive, minicomputer-based AOIPS and are referred to as the data retrieval system (DRS). Implemented as a user interactive but menu guided system, the DRS permits users to inventory the data tape library and create duplicate or subset data sets based on a user selected window defined by time and latitude/longitude boundaries. The DRS permits users to select, display, or produce formatted hard copy of individual data items contained within the data records.

Gary, J. P.↗

A method for data base management and analysis for wind tunnel data

To respond to the need for improved data base management and analysis capabilities for wind-tunnel data at the Langley 16-Foot Transonic Tunnel, research was conducted into current methods of managing wind-tunnel data and a method was developed as a solution to this need. This paper describes the development of the data base management and analysis method for wind-tunnel data. The design and implementation of the software system are discussed and examples of its use are shown.

Biser, Aileen O.↗

Experience with Rucio in the wider HEP community

Managing the data of scientific projects is an increasingly complicated challenge, which was historically met by developing experiment-specific solutions. However, the ever-growing data rates and requirements of even small experiments make this approach very difficult, if not prohibitive. In recent years, the scientific data management system Rucio has evolved into a successful open-source project that is now being used by many scientific communities and organisations. Rucio is incorporating the contributions and expertise of many scientific projects and is offering common features useful to a diverse re-search community. This article describes the recent experiences in operating Rucio, as well as contributions to the project, by ATLAS, Belle II, CMS, ES-CAPE, IGWN, LDMX, Folding@Home, and the UK's Science and Technology Facilities Council (STFC).

97 MATHEMATICS AND COMPUTING↗

RIM as the data base management system for a material properties data base

Relational Information Management (RIM) was selected as the data base management system for a prototype engineering materials data base. The data base provides a central repository for engineering material properties data, which facilitates their control. Numerous RIM capabilities are exploited to satisfy prototype data base requirements. Numerical, text, tabular, and graphical data and references are being stored for five material types. Data retrieval will be accomplished both interactively and through a FORTRAN interface. The experience gained in creating and exercising the prototype will be used in specifying requirements for a production system.

Karr, P. H.↗

Managing Mobile/Satellite Propagation Data

"Data Management System for Mobile Satellite Propagation" software package collection of FORTRAN programs and UNIX shell scripts designed to handle huge amounts of data resulting from mobile/satellite radio-propagation experiments. Data from experiments converted into standard and more useful forms. Software package contains program to convert binary format of data into standard ASCII format suitable for use with wide variety of computing-machine architectures. Written in either FORTRAN 77 or UNIX shell scripts.

Kantak, Anil V.↗

Approach to Managing MeaSURES Data at the GSFC Earth Science Data and Information Services Center (GES DISC)

A major need stated by the NASA Earth science research strategy is to develop long-term, consistent, and calibrated data and products that are valid across multiple missions and satellite sensors. (NASA Solicitation for Making Earth System data records for Use in Research Environments (MEaSUREs) 2006-2010) Selected projects create long term records of a given parameter, called Earth Science Data Records (ESDRs), based on mature algorithms that bring together continuous multi-sensor data. ESDRs, associated algorithms, vetted by the appropriate community, are archived at a NASA affiliated data center for archive, stewardship, and distribution. See http://measures-projects.gsfc.nasa.gov/ for more details. This presentation describes the NASA GSFC Earth Science Data and Information Services Center (GES DISC) approach to managing the MEaSUREs ESDR datasets assigned to GES DISC. (Energy/water cycle related and atmospheric composition ESDRs) GES DISC will utilize its experience to integrate existing and proven reusable data management components to accommodate the new ESDRs. Components include a data archive system (S4PA), a data discovery and access system (Mirador), and various web services for data access. In addition, if determined to be useful to the user community, the Giovanni data exploration tool will be made available to ESDRs. The GES DISC data integration methodology to be used for the MEaSUREs datasets is presented. The goals of this presentation are to share an approach to ESDR integration, and initiate discussions amongst the data centers, data managers and data providers for the purpose of gaining efficiencies in data management for MEaSUREs projects.

Vollmer, Bruce↗

The Open Data Repositorys Data Publisher

Data management and data publication are becoming increasingly important components of researcher's workflows. The complexity of managing data, publishing data online, and archiving data has not decreased significantly even as computing access and power has greatly increased. The Open Data Repository's Data Publisher software strives to make data archiving, management, and publication a standard part of a researcher's workflow using simple, web-based tools and commodity server hardware. The publication engine allows for uploading, searching, and display of data with graphing capabilities and downloadable files. Access is controlled through a robust permissions system that can control publication at the field level and can be granted to the general public or protected so that only registered users at various permission levels receive access. Data Publisher also allows researchers to subscribe to meta-data standards through a plugin system, embargo data publication at their discretion, and collaborate with other researchers through various levels of data sharing. As the software matures, semantic data standards will be implemented to facilitate machine reading of data and each database will provide a REST application programming interface for programmatic access. Additionally, a citation system will allow snapshots of any data set to be archived and cited for publication while the data itself can remain living and continuously evolve beyond the snapshot date. The software runs on a traditional LAMP (Linux, Apache, MySQL, PHP) server and is available on GitHub (http://github.com/opendatarepository) under a GPLv2 open source license. The goal of the Open Data Repository is to lower the cost and training barrier to entry so that any researcher can easily publish their data and ensure it is archived for posterity.

Astrobiology data↗

Studying the Past for the Future: Managing Modern Biodiversity from Historic and Prehistoric Data

Management of biodiversity and anthropogenic effects on the environment are some of the most difficult challenges facing humans today. In this paper, we describe how archaeological, historical, and ecological data can be combined to identify metrics of ecological change. Archaeological surveys, historic witness tree data, analysis of ancient pollen, and recent satellite imagery are combined to create indicators of change that are useable by modem environmental managers at Fort Benning Military Reservation in Georgia and Alabama in the United States. Archaeological data were collected at 30 meter intervals, historic vegetation data were collected from early nineteenth century survey maps, and ancient pollen and charcoal were collected from soil cores in lakes. We show how data sets collected with disparate goals from anthropology and ecology can be applied to make policy decisions about environmental management.

54 ENVIRONMENTAL SCIENCES↗

StreamNet Project : Annual Report Fiscal Year 2008.

Fiscal Year 2008 (FY-08) represents a transitional year for the StreamNet project. While the project continued to acquire/update, standardize, georeference and disseminate fish-related data for the state, some tribal and one federal fisheries agencies, it also took on several new initiatives and is anticipating new regional guidance on data needs. Passage of the Columbia Basin Accords caused an administrative change within the project, separating the work done by the Columbia River Inter-Tribal Fish Commission (CRITFC) out to a separate contract with BPA. This will change the structure of the StreamNet contract but not change the relationship with the StreamNet Library or data developed by CRITFC, and will likely increase the availability of tribal data to StreamNet due to increased funding for tribal data efforts. This change will take effect in FY-09. We also expect that data work will be adjusted in the future in response to executive level policy direction in the Columbia Basin based on efforts to establish priorities under a regional data management framework. Data development emphasis was shifted this year to place highest priority on data that support indicators of fish abundance for the focal species covered in the Status of the Resource (SOTR) report, as requested by the Columbia Basin Fish and Wildlife Authority (CBFWA) Data Management Framework Subcommittee. We instituted an XML based web service allowing direct access to data from the project database for CBFWA to update the SOTR report. The project also increased efforts to work with tribal fisheries managers to provide data related assistance and to include tribal data in the StreamNet database. A primary theme this year was exploring means to speed the flow of data. We had ongoing success in our strategic emphasis on increasing automation of data conversion through development of comprehensive database systems within our partner agencies, as outlined in our Vision and Strategic Plan. By assisting development of internal database systems, we will be able to automate conversion of agency data to the regionally consistent format as well as help the agency better meet their own data needs. The Idaho StreamNet project contributed to development of IDFG's Idaho Fish and Wildlife Information System (IFWIS), and this year they successfully tested automatic conversion of data to the regional exchange format. We worked with WDFW and developed draft field data input templates for collection of smolt trap and spawning ground survey data. And, we began collaborating in a project with ODFW and EcoTrust to develop an online data dissemination tool. As these and additional data systems are brought online, we expect to be able to shorten the time needed to annually update data, and hope to use the increased efficiency to free existing staff time to develop additional types of data from our partners. Another long-term theme related to expanding data coverage to estimates of productivity and/or data needed to calculate productivity. Initial investigations within our partner agencies indicated that these data are scattered, with some components like age composition of returning fish already being addressed by StreamNet, but others not yet covered. We will continue to determine how available these data are and investigate the feasibility of capturing the estimates and supporting data in the future. Routine ongoing data development of the standard data sets in StreamNet continued this year. An update and new web page for disseminating Protected Areas data was completed. Initial work was done with the CRITFC to get ready to house and disseminate data developed by the Hatchery Scientific Review Group. All database, GIS and web server systems were maintained successfully, with repairs completed as needed. Software applications were developed or maintained, as needed. All required reports, budgets and equipment inventories were submitted. The StreamNet website (www.streamnet.org), the project's primary means of disseminating fish data, was completely redesigned this year to improve the user experience and make locating and acquiring data simpler. A primary goal was to significantly reduce the sequence of steps needed. The new site was in final testing at the end of the fiscal year, and will go live in the first quarter of FY-09. Use of the website remained strong, with 941,687 total page views representing 345,855 visits from 123,684 unique visitors. There were 18,797 actual views of tabular datasets from the online query system, plus many interactive map views, not counting downloads of the entire database direct data and information requests.

13 HYDRO ENERGY↗

CO 2 Storage Site Screening Platform Development and CO 2 Storage Resource Analysis in SECARB Offshore Reservoirs Using SAS Viya

A major goal of the SECARB Offshore Partnership (DE-FE0031557) is to screen deep saline aquifers and hydrocarbon reservoirs in the central Gulf of Mexico for CO 2 sequestration and CO 2 -enhanced oil and gas recovery (EOR/EGR) and estimate the corresponding CO 2 storage resources for select reservoirs. CO 2 storage potential associated with offshore CO 2 -EOR is considerable and likely represents “low hanging fruit” for near-term CO 2 storage given the in-place infrastructure in the region. It is for these reasons that this assessment focuses on oil and gas fields. To this end, three major objectives have been completed and include (1) managing geological data derived from different sources, (2) building a reservoir screening platform for CO 2 storage, and (3) ranking the reservoirs based on the estimated CO 2 storage resources. The SAS ® Viya platform was used for data management and analytics. The Viya platform is a cloud service platform that provides data integration, data management, quick analytics, data visualization, machine learning functions, and application programming interfaces (APIs) for multi-programming languages. Different sources of data containing geologic information, reservoir properties, and EOR/EGR information were collected, cleaned, formatted, and loaded into the SAS ® Viya platform for evaluation. The major geological characteristics of both shelf and deep-water areas of the central Gulf were examined and compared to define the appropriate reservoir screening criteria. Next, a CO 2 storage site screening system was built in the SAS ® Viya platform with the pre-defined criteria. Finally, the CO 2 storage resources of the screened reservoirs were calculated and reported at the BOEM field level to identify fields with the highest estimated CO 2 storage resource. The fields with the largest total estimated CO 2 storage resource are located in the Mississippi Canyon protraction area. Due to proximity to the Mississippi Delta (indicative of less infrastructure) and large estimated CO 2 storage resources, future development activities may wish to focus efforts in the Mississippi Canyon protraction area.

02 PETROLEUM↗

Fault management for data systems

Issues related to automating the process of fault management (fault diagnosis and response) for data management systems are considered. Substantial benefits are to be gained by successful automation of this process, particularly for large, complex systems. The use of graph-based models to develop a computer assisted fault management system is advocated. The general problem is described and the motivation behind choosing graph-based models over other approaches for developing fault diagnosis computer programs is outlined. Some existing work in the area of graph-based fault diagnosis is reviewed, and a new fault management method which was developed from existing methods is offered. Our method is applied to an automatic telescope system intended as a prototype for future lunar telescope programs. Finally, an application of our method to general data management systems is described.

Boyd, Mark A.↗

Using a data base management system for modelling SSME test history data

The usefulness of a data base management system (DBMS) for modelling historical test data for the complete series of static test firings for the Space Shuttle Main Engine (SSME) was assessed. From an analysis of user data base query requirements, it became clear that a relational DMBS which included a relationally complete query language would permit a model satisfying the query requirements. Representative models and sample queries are discussed. A list of environment-particular evaluation criteria for the desired DBMS was constructed; these criteria include requirements in the areas of user-interface complexity, program independence, flexibility, modifiability, and output capability. The evaluation process included the construction of several prototype data bases for user assessement. The systems studied, representing the three major DBMS conceptual models, were: MIRADS, a hierarchical system; DMS-1100, a CODASYL-based network system; ORACLE, a relational system; and DATATRIEVE, a relational-type system.

Abernethy, K.↗

Software Defined Cyberinfrastructure

Within and across thousands of science labs, researchers and students struggle to manage data produced in experiments, simulations, and analyses. Largely manual research data lifecycle management processes mean that much time is wasted, research results are often irreproducible, and data sharing and reuse remain rare. In response, we propose a new approach to data lifecycle management in which researchers are empowered to define the actions to be performed at individual storage systems when data are created or modified: actions such as analysis, transformation, copying, and publication. We term this approach software-defined cyberinfrastructure because users can implement powerful data management policies by deploying rules to local storage systems, much as software-defined networking allows users to configure networks by deploying rules to switches.We argue that this approach can enable a new class of responsive distributed storage infrastructure that will accelerate research innovation by allowing any researcher to associate data workflows with data sources, whether local or remote, for such purposes as data ingest, characterization, indexing, and sharing. We report on early experiments with this approach in the context of experimental science, in which a simple if-trigger-then-action (IFTA) notation is used to define rules.

Foster, Ian↗

Vibroacoustic payload environment prediction system (VAPEPS): Data base management center remote access guide

A Vibroacoustic Data Base Management Center has been established at the Jet Propulsion Laboratory (JPL). The center utilizes the Vibroacoustic Payload Environment Prediction System (VAPEPS) software package to manage a data base of shuttle and expendable launch vehicle flight and ground test data. Remote terminal access over telephone lines to a dedicated VAPEPS computer system has been established to provide the payload community a convenient means of querying the global VAPEPS data base. This guide describes the functions of the JPL Data Base Management Center and contains instructions for utilizing the resources of the center.

Thomas, V. C.↗

HDF5 in the exascale era: Delivering efficient and scalable parallel I/O for exascale applications

Accurately modeling real-world systems requires scientific applications at exascale to generate massive amounts of data and manage data storage efficiently. However, parallel input and output (I/O) faces challenges due to new application workflows and the state-of-the-art memory, interconnect, and storage architectures considered in exascale designs. The storage hierarchy has expanded with node-local persistent memory, solid-state storage, and traditional disk and tape-based storage, thus requiring efficiency at each layer and much more efficient data movement among these layers. This paper discusses how the ExaHDF5 project improved the I/O performance and data management for exascale architectures by enhancing HDF5, a widely used parallel I/O library. The team developed an Asynchronous I/O Virtual Object Layer (VOL) connector that allowed overlapping I/O with computation. They also created a Cache VOL to complement asynchronous I/O by incorporating fast storage layers, such as burst buffer and node-local storage, into the parallel I/O workflow through caching and staging data. Additionally, the team enabled data aggregation and I/O at the node level by using a Subfiling Virtual File Driver (VFD). To demonstrate superior I/O performance with HDF5 at exascale, the ExaHDF5 team collaborated with several exascale applications. In this paper, we show I/O performance improvements for three applications: Cabana (a particle-based simulation library), EQSIM (a regional earthquake simulation software), and E3SM (a climate system modeling library).

Asynchronous I/Ol↗

Provenance–aware workflow for data quality management and improvement for large continuous scientific data streams

Data quality assessment, management and improvement is integral part of any big data intensive scientific research to ensure accurate, reliable and reproducible science discoveries. The task of maintaining the quality of data, however, is non-trivial and pose a challenge for a program like Department of Energy’s Atmospheric Radiation Measurement (ARM) that collects data from hundreds of instruments across the world,and distributes thousands of streaming data products that are continuously growing in near-real-time in an archive 1.7 Petabyte in size and growing. We present a computational data processing workflow to collect the data quality issue via an easy and intuitive web-based portal that allows reporting of any quality issues for any site, facility or instruments at a granularity down to individual variable in the data files. Portal allows instrument specialists and scientists to provide corrective actions in form of symbolic equation. A parallel processing framework applies the data improvement to large volume of data in efficient parallel environment, while optimizing data transfer and file I/O operations. Corrected files are systematically versioned and archived. A provenance tracking module tracks and records any change made to the data during its entire life cycle which are communicated transparently to the scientific users. Developed in Python using open source technologies, software architecture enables efficient and fast management and improvement of data in an operational data center environment.

Kumar, Jitendra↗

Dynamics Explorer Science Data Processing System

The Dynamics Explorer project has acquired the ground data processing system from the Atmosphere Explorer project to provide a central computer facility for the data processing, data management and data analysis activities of the investigators. Access to this system is via remote terminals at the investigators' facilities, which provide ready access to the data sets derived from groups of instruments on both spacecraft. The original system has been upgraded with both new hardware and enhanced software systems. These new systems include color and grey scale graphics terminals, an augmentation computer, micrographics facility, a versatile data base with a directory and data management system, and graphics display software packages.

Smith, P. H.↗