Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scientific Data Management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22

The availability of hydrogeologic data associated with areas identified by the US Geological Survey as experiencing potentially induced seismicity resulting from subsurface injection

A critical need exists for site-specific hydrogeologic data in order to determine potential hazards of induced seismicity and to manage risk. By 2015, the United States Geological Survey (USGS) had identified 17 locations in the USA that are experiencing an increase in seismicity, which may be potentially induced through industrial subsurface injection. These locations span across seven states, which vary in geological setting, industrial exposure and seismic history. Comparing the research across the 17 locations revealed patterns for addressing induced seismicity concerns, despite the differences between geographical locations. Most induced seismicity studies evaluate geologic structure and seismic data from areas experiencing changes in seismic activity levels, but the inherent triggering mechanism is the transmission of hydraulic pressure pulses. This research conducted a systematic review of whether data are available in these locations to generate accurate hydrogeologic predictions, which could aid in managing seismicity. After analyzing peer-reviewed research within the 17 locations, this research confirms a lack of site-specific hydrogeologic data availability for at-risk areas. Commonly, formation geology data are available for these sites, but hydraulic parameters for the seismically active injection and basement zones are not available to researchers conducting peer-reviewed research. Obtaining hydrogeologic data would lead to better risk management for injection areas and provide additional scientific evidential support for determining a potentially induced seismic area.

58 GEOSCIENCES↗

The Practical Obstacles of Data Transfer: Why researchers still love scp

The importance of computing facilities is heralded every six months with the announcement of the new Top500 list, showcasing the world s fastest supercomputers. Unfortu- nately, with great computing capability does not come great long-term data storage capacity, which often means users must move their data to their local site archive, to remote sites where they may be doing future computation or anal- ysis, or back to their home institution, else face the dreaded data purge that most HPC centers employ to keep utiliza- tion of large parallel filesystems low to manage performance and capacity. At HPC centers, data transfer is crucial to the scientific workflow and will increase in importance as computing systems grow in size. The Energy Sciences Net- work (ESnet) recently launched its fifth generation network, a 100 Gbps high-performance, unclassified national network connecting more than 40 DOE research sites to support scientific research and collaboration. Despite the tenfold increase in bandwidth to DOE research sites amenable to multiple data transfer streams and high throughput, in prac- tice, researchers often under-utilize the network and resort to painfully-slow single stream transfer methods such as scp to avoid the complexity of using multiple stream tools such as GridFTP and bbcp, and contend with frustration from the lack of consistency of available tools between sites. In this study we survey and assess the data transfer methods pro- vided at several DOE supported computing facilities, includ- ing both leadership-computing facilities, connected through ESnet. We present observed transfer rates, suggested opti- mizations, and discuss the obstacles the tools must overcome to receive wide-spread adoption over scp.

Nam, Hai Ah↗

HP-MDR: High-performance and Portable Data Refactoring and Progressive Retrieval with Advanced GPUs

Scientific applications produce vast amounts of data, posing grand challenges in the underlying data management and analytic tasks. Progressive compression is a promising way to address this problem, as it allows for on-demand data retrieval with significantly reduced data movement cost. However, most existing progressive methods are designed for CPUs, leaving a gap for them to unleash the power of today’s heterogeneous computing systems with GPUs.In this work, we propose HP-MDR, a high-performance and portable data refactoring and progressive retrieval framework for GPUs. Our contributions are four-fold: (1) We carefully optimize the bitplane encoding and lossless encoding, two key stages in progressive methods, to achieve high performance on GPUs; (2) We propose pipeline optimization and incorporate it with data refactoring and progressive retrieval workflows to further enhance the performance for large data process; (3) We leverage our framework to enable high-performance data retrieval with guaranteed error control for common Quantities of Interest; (4) We evaluate HP-MDR and compare it with state of the arts using five real-world datasets. Experimental results demonstrate that HP-MDR delivers an average 13.68 × and 6.31 × throughput in data refactoring and progressive retrieval tasks, respectively. It also leads to 11.22 × throughput for recomposing required data representations under Quantity-of-Interest error control and 6.04 × performance for the corresponding end-to-end data retrieval, when compared with state-of-the-art solutions.

Li, Yanliang [University of Oregon]↗

The payload connection - Approaches to onboard data processing for scientific systems on the Space Shuttle

The integration of payload instruments into the Command and Data Management Subsystem (CDMS) of the Space Shuttle Spacelab is discussed. Specific emphasis is on the distributed (self-contained processor) vs centralized (Spacelab Computer) approach to science payload utilization of these control and data handling capabilities. An overview of the Spacelab CDMS is first given. A synopsis of the integration process for Spacelab Missions 1, 2, and 3 is discussed. Three different hardware approaches to payload command and data handling are then categorized from which Spacelab implementations to date are derived. The different types of Spacelab Mission 3 payload CDMS hardware and software being used are discussed, including examples of microprocessors and data acquisition systems. Preliminary costs for Spacelab application software vs other flight software approaches are compared. A standard RAU interface that would permit a range of ECOS services to be provided, yet be implemented as a mass-produceable unit for incorporation into payloads, is proposed, in keeping with the trend toward simplifying the integration effort, standardizing CDMS interfaces, and ultimately lowering cost.

Konkel, C. R.↗

DBMS as a Tool for Project Management

Scientific objectives of crustal dynamics are listed as well as the contents of the centralized data information system for the crustal dynamics project. The system provides for project observation schedules, gives project configuration control information and project site information.

Linder, H.↗

Flood and Landslide Applications of Near Real-time Satellite Rainfall Products

Floods and associated landslides are one of the most widespread natural hazards on Earth, responsible for tens of thousands of deaths and billions of dollars in property damage every year. During 1993-2002, over 1000 of the more than 2,900 natural disasters reported were due to floods. These floods and associated landslides claimed over 90,000 lives, affected over 1.4 billion people and cost about $210 billion. The impact of these disasters is often felt most acutely in less developed regions. In many countries around the world, satellite-based precipitation estimation may be the best source of rainfall data due to lack of surface observing networks. Satellite observations can be of essential value in improving our understanding of the occurrence of hazardous events and possibly in lessening their impact on local economies and in reducing injuries, if they can be used to create reliable warning systems in cost-effective ways. This article addressed these opportunities and challenges by describing a combination of satellite-based real-time precipitation estimation with land surface characteristics as input, with empirical and numerical models to map potential of landslides and floods. In this article, a framework to detect floods and landslides related to heavy rain events in near-real-time is proposed. Key components of the framework are: a fine resolution precipitation acquisition system; a comprehensive land surface database; a hydrological modeling component; and landslide and debris flow model components. A key precipitation input dataset for the integrated applications is the NASA TRMM-based multi-satellite precipitation estimates. This dataset provides near real-time precipitation at a spatial-temporal resolution of 3 hours and 0.25deg x 0.25deg. By careful integration of remote sensing and in-situ observations, and assimilation of these observations into hydrological and landslide/debris flow models with surface topographic information, prediction of useful probabilistic maps of landslide and floods for emergency management in a timely manner is possible. Early results shows that the potential exists for successful application of satellite precipitation data in improving/developing global monitoring systems for flood/landslide disaster preparedness and management. The scientific and technological prototype can be first applied in a representative test-bed and then the information deliverables for the region can be tailored to the societal and economic needs of the represented affected countries.

Hong, Yang↗

TRUST, Trustworthiness and EOSDIS

In recent years there has been considerable attention by the international scientific research and applications community to ensure high quality of data and information management. The terms FAIR (Findable, Accessible, Interoperable, Reusable) data, TRUST (Transparency, Responsibility, User Community, Sustainability, and Technology) principles, and CARE (Collective Benefit, Authority to Control, Responsibility, and Ethics) principles have come into vogue during the last decade. NASA has been managing data and information for over 60 years. NASA’s Earth Observing System Data and Information System (EOSDIS) has been in operation for over 25 years, managing most of NASA’s Earth science data. Trustworthiness is a goal that NASA has always strived to achieve or exceed, because it: enables the success of any NASA science mission; inspires general science research and applications; justifies the cost of operations; contributes to the value of NASA’s Open Data Policy; and influences the long term, historical view for the data collection. Given the recent growth of interest in TRUST principles, it is useful to assess and show how NASA’s attention to trustworthiness maps into those principles. This presentation addresses shows how the various steps that have been taken by the Earth Science Data and Information System (ESDIS) Project in the implementation and evolution of EOSDIS map into the TRUST principles.

Remote Sensing↗

EMPRESS

EMPRESS provides metadata management capabilities intended to address annotating data within a single variable in a scientific data set rather than at the variable or entire file granularity level

Lawson, Margaret↗

NGEE Arctic Data Management - status and planned activities for 2013

The open sharing of data and information among the NGEE Team, the broader scientific community, and the public is critical to meeting the scientific goals of the Next-Generation Ecosystem Experiment (NGEE) Arctic project. In implementing the integrated approach, the project is generating diverse data sets from observations, experiments, and models across field plot, polygonal landform, regional, and global scales and is drawing in a wealth of existing data products collected and generated by other research organizations across the Arctic. Developing the data management infrastructure to support this approach is a significant challenge. Nonetheless, the NGEE Arctic project is committed to upholding a rigorous and high-quality data management strategy and the implementation of that strategy in an innovative, cost-effective data collection, management, distribution, and archival framework. The NGEE Arctic Data Portal will support the current research in Barrow and provide a consistent framework to support future research efforts as they expand to other regions.

Devarakonda, Ranjeet↗

A general scientific information system to support the study of climate-related data

The development and use of NASA's Pilot Climate Data System (PCDS) are discussed. The PCDS is used as a focal point for managing and providing access to a large collection of actively used data for the Earth, ocean and atmospheric sciences. The PCDS provides uniform data catalogs, inventories, and access methods for selected NASA and non-NASA data sets. Scientific users can preview the data sets using graphical and statistical methods. The system has evolved from its original purpose as a climate data base management system in response to a national climate program, into an extensive package of capabilities to support many types of data sets from both spaceborne and surface based measurements with flexible data selection and analysis functions.

Treinish, L. A.↗

An interactive environment for the analysis of large Earth observation and model data sets

We propose to develop an interactive environment for the analysis of large Earth science observation and model data sets. We will use a standard scientific data storage format and a large capacity (greater than 20 GB) optical disk system for data management; develop libraries for coordinate transformation and regridding of data sets; modify the NCSA X Image and X Data Slice software for typical Earth observation data sets by including map transformations and missing data handling; develop analysis tools for common mathematical and statistical operations; integrate the components described above into a system for the analysis and comparison of observations and model results; and distribute software and documentation to the scientific community.

Bowman, Kenneth P.↗

An interactive environment for the analysis of large Earth observation and model data sets

We propose to develop an interactive environment for the analysis of large Earth science observation and model data sets. We will use a standard scientific data storage format and a large capacity (greater than 20 GB) optical disk system for data management; develop libraries for coordinate transformation and regridding of data sets; modify the NCSA X Image and X DataSlice software for typical Earth observation data sets by including map transformations and missing data handling; develop analysis tools for common mathematical and statistical operations; integrate the components described above into a system for the analysis and comparison of observations and model results; and distribute software and documentation to the scientific community.

Bowman, Kenneth P.↗

NASA's computer science research program

Following a major assessment of NASA's computing technology needs, a new program of computer science research has been initiated by the Agency. The program includes work in concurrent processing, management of large scale scientific databases, software engineering, reliable computing, and artificial intelligence. The program is driven by applications requirements in computational fluid dynamics, image processing, sensor data management, real-time mission control and autonomous systems. It consists of university research, in-house NASA research, and NASA's Research Institute for Advanced Computer Science (RIACS) and Institute for Computer Applications in Science and Engineering (ICASE). The overall goal is to provide the technical foundation within NASA to exploit advancing computing technology in aerospace applications.

Larsen, R. L.↗

The future low-temperature geochemical data-scape as envisioned by the U.S. geochemical community

Data sharing benefits the researcher, the scientific community, and the public by allowing the impact of data to be generalized beyond one project and by making science more transparent. However, many scientific communities have not developed protocols or standards for publishing, citing, and versioning datasets. One community that lags in data management is that of low-temperature geochemistry (LTG). This paper resulted from an initiative from 2018 through 2020 to convene LTG and data scientists in the U.S. to strategize future management of LTG data. Through webinars, a workshop, a preprint, a townhall, and a community survey, the group of U.S. scientists discussed the landscape of data management for LTG – the data-scape. Currently this data-scape includes a “street bazaar” of data repositories. This was deemed appropriate in the same way that LTG scientists publish articles in many journals. The variety of data repositories and journals reflect that LTG scientists target many different scientific questions, produce data with extremely different structures and volumes, and utilize copious and complex metadata. Nonetheless, the group agreed that publication of LTG science must be accompanied by sharing of data in publicly accessible repositories, and, for sample-based data, registration of samples with globally unique persistent identifiers. LTG scientists should use certified data repositories that are either highly structured databases designed for specialized types of data, or unstructured generalized data systems. Recognizing the need for tools to enable search and cross-referencing across the proliferating data repositories, the group proposed that the overall data informatics paradigm in LTG should shift from “build data repository, data will come” to “publish data online, cybertools will find”. Funding agencies could also provide portals for LTG scientists to register funded projects and datasets, and forge approaches that cross national boundaries. Finally, the needed transformation of the LTG data culture requires emphasis in student education on science and management of data.

58 GEOSCIENCES↗

Accelerating Science Impact through Big Data Workflow Management and Supercomputing

The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. ATLAS, one of the largest collaborations ever assembled in the the history of science, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. To manage the workflow for all data processing on hundreds of data centers the PanDA (Production and Distributed Analysis)Workload Management System is used. An ambitious program to expand PanDA to all available computing resources, including opportunistic use of commercial and academic clouds and Leadership Computing Facilities (LCF), is realizing within BigPanDA and megaPanDA projects. These projects are now exploring how PanDA might be used for managing computing jobs that run on supercomputers including OLCF’s Titan and NRC-KI HPC2. The main idea is to reuse, as much as possible, existing components of the PanDA system that are already deployed on the LHC Grid for analysis of physics data. The next generation of PanDA will allow many data-intensive sciences employing a variety of computing platforms to benefit from ATLAS experience and proven tools in highly scalable processing.

97 MATHEMATICS AND COMPUTING↗