Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data provenance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Computational Reproducibility of Scientific Workflows at Extreme Scales

We propose an approach for improved reproducibility that includes capturing and relating provenance characteristics and performance metrics, in a hybrid queriable system, the ProvEn server. The system capabilities are illustrated on two use cases: scientific reproducibility of results in the ACME climate simulations and performance reproducibility in molecular dynamics workflows on HPC computing platforms.

provenance, Reproducibility, Climate & Earth Syste↗

A Noisy 10GB Provenance Database

Provenance of scientific data is a key piece of the metadata record for the data's ongoing discovery and reuse. Provenance collection systems capture provenance on the fly, however, the protocol between application and provenance tool may not be reliable. Consequently, the provenance record can be partial, partitioned, and simply inaccurate. We use a workflow emulator that models faults to construct a large 10GB database of provenance that we know is noisy (that is, has errors). We discuss the process of generating the provenance database, and show early results on the kinds of provenance analysis enabled by the large provenance.

97 MATHEMATICS AND COMPUTING↗

Bridging the Gap between Scientific Data Producers and Consumers: A Provenance Approach

Despite the methodical and painstaking efforts made by scientists to record their scientific findings and protocols, a knowledge gap problem continues to persist today between producers of scientific results and consumers because technology is performing the exchange of data as opposed to scientists making direct contact. Provenance is a means to formalize how this knowledge is transferred. However, for it to be meaningful to scientists, the provenance research community needs continued contributions from the scientific community to extend and leverage provenance-based vocabularies and technology from the provenance community. Going forward the provenance community must also be vigilant to meet scalability needs of data intensive science

Stephan, Eric G.↗

Addressing and Presenting Quality of Satellite Data via Web-Based Services

With the recent attention to climate change and proliferation of remote-sensing data utilization, climate model and various environmental monitoring and protection applications have begun to increasingly rely on satellite measurements. Research application users seek good quality satellite data, with uncertainties and biases provided for each data point. However, different communities address remote sensing quality issues rather inconsistently and differently. We describe our attempt to systematically characterize, capture, and provision quality and uncertainty information as it applies to the NASA MODIS Aerosol Optical Depth data product. In particular, we note the semantic differences in quality/bias/uncertainty at the pixel, granule, product, and record levels. We outline various factors contributing to uncertainty or error budget; errors. Web-based science analysis and processing tools allow users to access, analyze, and generate visualizations of data while alleviating users from having directly managing complex data processing operations. These tools provide value by streamlining the data analysis process, but usually shield users from details of the data processing steps, algorithm assumptions, caveats, etc. Correct interpretation of the final analysis requires user understanding of how data has been generated and processed and what potential biases, anomalies, or errors may have been introduced. By providing services that leverage data lineage provenance and domain-expertise, expert systems can be built to aid the user in understanding data sources, processing, and the suitability for use of products generated by the tools. We describe our experiences developing a semantic, provenance-aware, expert-knowledge advisory system applied to NASA Giovanni web-based Earth science data analysis tool as part of the ESTO AIST-funded Multi-sensor Data Synergy Advisor project.

Leptoukh, Gregory↗

Designing Rolling-Element Bearings

Bearing Analysis Tool (BAT) is a computer program for designing rolling-element bearings for cryogenic turbomachines. BAT provides a graphical user interface (GUI) that guides the entry of data to develop mathematical models of bearings. The GUI breaks model data into logical subsets that are entered through logic-driven input screens. The software generates a threedimensional graphical model of a bearing as the data are entered. Most dataentry errors become immediately obvious in the graphical model. BAT provides for storage of all the data on a shaft/bearing system, enabling the creation of a library of proven designs. Data from the library can be transferred to subsequent projects by use of simple cut-and-paste routines. BAT includes a library of temperature- dependent cryogenic bearing-material properties for use in the mathematical models. BAT implements algorithms that (1) enable the user to select combinations of design and/or operating-condition parameters, and then (2) automatically optimize the design by performing trade studies over all of the parameter combinations. This feature enables optimization over a large trade space in a fraction of the time taken when using prior bearingmodel software.

Moore, James D., Jr.↗

Shuttle Topography Data Inform Solar Power Analysis

The next time you flip on a light switch, there s a chance that you could be benefitting from data originally acquired during the Space Shuttle Program. An effort spearheaded by Jet Propulsion Laboratory (JPL) and the National Geospatial-Intelligence Agency (NGA) in 2000 put together the first near-global elevation map of the Earth ever assembled, which has found use in everything from 3D terrain maps to models that inform solar power production. For the project, called the Shuttle Radar Topography Mission (SRTM), engineers at JPL designed a 60-meter mast that was fitted onto Shuttle Endeavour. Once deployed in space, an antenna attached to the end of the mast worked in combination with another antenna on the shuttle to simultaneously collect data from two perspectives. Just as having two eyes makes depth perception possible, the SRTM data sets could be combined to form an accurate picture of the Earth s surface elevations, the first hight-detail, near-global elevation map ever assembled. What made SRTM unique was not just its surface mapping capabilities but the completeness of the data it acquired. Over the course of 11 days, the shuttle orbited the Earth nearly 180 times, covering everything between the 60deg north and 54deg south latitudes, or roughly 80 percent of the world s total landmass. Of that targeted land area, 95 percent was mapped at least twice, and 24 percent was mapped at least four times. Following several years of processing, NASA released the data to the public in partnership with NGA. Robert Crippen, a member of the SRTM science team, says that the data have proven useful in a variety of fields. "Satellites have produced vast amounts of remote sensing data, which over the years have been mostly two-dimensional. But the Earth s surface is three-dimensional. Detailed topographic data give us the means to visualize and analyze remote sensing data in their natural three-dimensional structure, facilitating a greater understanding of the features and processes taking place on Earth."

Source record↗

Identification of Plasma Current Center by Neural Network Inference in EAST

For the efficient and safe operation of experimental advanced superconducting tokamak (EAST), it is necessary to accurately identify and control the plasma current and its central position. In this article, neural network is used to identify the position of the plasma current center. The model trained by the basic back-propagation neural network can well match the relationship between the electromagnetic diagnostic signals and plasma current center positions. Both noisy simulation data and experimental data are applied to train and test the neural network inference model. Adding 0.1% noise to the training data is proven to improve the noise immunity of the inference model. Basic neural networks trained with both noisy simulation data and actual experimental data show good results with sufficient inputs; however, in both cases, the performance degrades significantly when only the poloidal field coil currents are given as inputs. For this kind of time-series problem, the dynamic neural network containing delay and feedback architecture is introduced, and an improved model requiring much fewer inputs is trained and tested for current center inference. Furthermore, some parameters of this model are compared and analyzed in this article. With suitable neural network architecture, the mapping between the controlled variables (poloidal field coil currents) and response variables (plasma current center) can be well-established.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Bridging the Gap: Enhancing Prominence and Provenance of NASA Datasets in Research Publications

Attribution of datasets that were used to generate research results described in peer-reviewed publications to the original source of these datasets (which are often archived at NASA Earth Science data centers) has been very challenging. Even though the data citation standard of citing datasets as research artifacts and citing them with Digital Object Identifiers (DOIs) was introduced over a decade ago, most authors do not properly reference the data used in their studies and merely mention them in the text. The lack of proper citations of datasets makes the peer-reviewed publication less transparent, imperils reproducibility, and impedes open science. We offer an open-source publication management methodology and a tool that can help to enhance usage-based data discovery, prominence, and provenance of the data; reproducibility of the research results; and potentially increase the return on investment on NASA-funded research.

open-source↗

Analyzing a 35-Year Hourly Data Record: Why So Difficult?

At the Goddard Distributed Active Archive Center, we have recently added a 35-Year record of output data from the North American Land Assimilation System (NLDAS) to the Giovanni web-based analysis and visualization tool. Giovanni (Geospatial Interactive Online Visualization ANd aNalysis Infrastructure) offers a variety of data summarization and visualization to users that operate at the data center, obviating the need for users to download and read the data themselves for exploratory data analysis. However, the NLDAS data has proven surprisingly resistant to application of the summarization algorithms. Algorithms that were perfectly happy analyzing 15 years of daily satellite data encountered limitations both at the algorithm and system level for 35 years of hourly data. Failures arose, sometimes unexpectedly, from command line overflows, memory overflows, internal buffer overflows, and time-outs, among others. These serve as an early warning sign for the problems likely to be encountered by the general user community as they try to scale up to Big Data analytics. Indeed, it is likely that more users will seek to perform remote web-based analysis precisely to avoid the issues, or the need to reprogram around them. We will discuss approaches to mitigating the limitations and the implications for data systems serving the user communities that try to scale up their current techniques to analyze Big Data.

computational performance↗

Position Papers for the ASCR Workshop on Cybersecurity and Privacy for Scientific Computing Ecosystems

At the request of the Department of Energy's (DOE) Office of Advanced Scientific Computing Research (ASCR), this program committee has been tasked with organizing a workshop to identify basic research needs in cybersecurity and privacy to better support DOE's science and energy mission. As part of the process, the program committee is soliciting community input in the form of position papers to help identify significant use cases, facility issues, and other barriers to enabling verifiably trustworthy computational science while preserving data confidentiality as appropriate for scientific workflows of interest to DOE. The program committee will review these position papers and based on the fit of their area of expertise and interest, selected contributors will have the opportunity to participate in the workshop currently planned as a virtual event November 3-5th, 2021. The thrust areas that will be explored by this workshop are the following: (1) Algorithms for secure, scalable, privacy-enhancing technologies and frameworks, including: Federated AI/ML, Differential privacy, Randomized algorithms, Adversarial modeling & simulation, Graph algorithms, and Formal methods; (2) Platforms to support the entire scientific-computing ecosystem, including edge computing for large-scale experiments, focusing on heterogeneous systems and distributed systems, including: Heterogeneous computing systems, Distributed computing systems, and Secure data architectures; and (3) Data workflows to allow agile use of data while preserving integrity and privacy, making the important properties verifiable either at runtime or post-computation, including: Integrity and provenance and Data management infrastructure. Topics that are out-of-scope for the workshop include discussing specific proposed solutions or areas that are clearly out of DOE's fundamental and applied-sciences mission scope, e.g., cryptography, enterprise security, and general-operations technology.

97 MATHEMATICS AND COMPUTING↗

Natural Gas Value-Chain and Network Assessments

The current expansion of natural gas (NG) development in the United States requires an understanding of how this change will affect the natural gas industry, downstream consumers, and economic growth in order to promote effective planning and policy development. The impact of this expansion may propagate through the NG system and US economy via changes in manufacturing, electric power generation, transportation, commerce, and increased exports of liquefied natural gas. We conceptualize this problem as supply shock propagation that pushes the NG system and the economy away from its current state of infrastructure development and level of natural gas use. To illustrate this, the project developed two core modeling approaches. The first is an Agent-Based Modeling (ABM) approach which addresses shock propagation throughout the existing natural gas distribution system. The second approach uses a System Dynamics-based model to illustrate the feedback mechanisms related to finding new supplies of natural gas - notably shale gas - and how those mechanisms affect exploration investments in the natural gas market with respect to proven reserves. The ABM illustrates several stylized scenarios of large liquefied natural gas (LNG) exports from the U.S. The ABM preliminary results demonstrate that such scenario is likely to have substantial effects on NG prices and on pipeline capacity utilization. Our preliminary results indicate that the price of natural gas in the U.S. may rise by about 50% when the LNG exports represent 15% of the system-wide demand. The main findings of the System Dynamics model indicate that proven reserves for coalbed methane, conventional gas and now shale gas can be adequately modeled based on a combination of geologic, economic and technology-based variables. A base case scenario matches historical proven reserves data for these three types of natural gas. An environmental scenario, based on implementing a $50/tonne CO 2 tax results in less proven reserves being developed in the coming years while demand may decrease in the absence of acceptable substitutes, incentives or changes in consumer behavior. An increase in demand of 25% increases proven reserves being developed by a very small amount by the end of the forecast period of 2025.

03 NATURAL GAS↗

Data from: "Ecophysiological variation in two provenances of Pinus flexilis seedlings across an elevation gradient from forest to alpine"

This archive contains data used to support conclusions drawn in “Ecophysiological variation in two provenances of Pinus flexilis seedlings across an elevation gradient from forest to alpine”, by Reinhardt et al., 2011. Data were collected over one summer season in plots within the Alpine Treeline Warming Experiment (ATWE), before climate manipulations began. The experiment was located on Niwot Ridge, in the Front Range of the Colorado Rocky Mountains. This data package includes five comma-separated-values (.csv) files, five Microsoft Excel (.xlsx) files, one .pdf file, and two types of geospatial files: keyhole markup language (.kml), and ESRI shapefiles (.shp). .csv files can be opened using any simple text-editing software (such as Notepad and TextEdit), R, and Microsoft Excel. .xlsx files can only be opened using Microsoft Excel. The .pdf file can be opened using Adobe Acrobat Reader or any other compatible file viewing software. The .kml file can be opened using Google Earth and Google Maps, and shapefiles can be opened using any software compatible with the file type, such as ESRI’s ArcGIS suite and QGIS.Data archived contain gas exchange and plant physiology measurements, non-structural carbohydrate data, among others. Geospatial files are also provided for additional locational context. The files and their contents in this data package are summarized under "Data Summary" in the included Data User's Guide. All files (excluding geospatial) are available in both Microsoft Excel and in .csv format, and are indicated in the Data Summary list as well.-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Climate change is predicted to cause upward shifts in forest tree distributions, which will require seedling recruitment beyond current forest boundaries. However, predicting the likelihood of successful plant establishment beyond current species’ ranges under changing climate is complicated by the interaction of genetic and environmental controls on seedling establishment. To determine how genetics and climate may interact to affect seedling establishment, we transplanted recently germinated seedlings from high- and low-elevation provenances (HI and LO, respectively) of Pinus flexilis in common gardens arrayed along an elevation and canopy gradient from subalpine forest into the alpine zone and examined differences in physiology and morphology between provenances and among sites. Plant dry mass, projected leaf area and shoot:root ratios were 12–40% greater in LO compared with HI seedlings at each elevation. There were no significant changes in these variables among sites except for decreased dry mass of LO seedlings in the alpine site. Photosynthesis, carbon balance (photosynthesis/respiration) and conductance increased >2× with elevation for both provenances, and were 35–77% greater in LO seedlings compared with HI seedlings. There were no differences in dark-adapted chlorophyll fluorescence (Fv/Fm) among sites or between provenances. Our results suggest that for P. flexilis seedlings, provenances selected for above-ground growth may outperform those selected for stress resistance in the absence of harsh climatic conditions, even well above the species’ range limits in the alpine zone. This indicates that forest genetics may be important to understanding and managing species’ range adjustments due to climate change.

54 ENVIRONMENTAL SCIENCES↗

Reusability in NASA's Earth Observation System Data and Information System (EOSDIS)

NASA's Earth Observation System Data and Information System (EOSDIS) has been operational since 1994. The term FAIR is relatively recent compared to the operational life of EOSDIS. However, given the evolutionary nature of EOSDIS in response to both the state of the art and community expectations, it is useful to assess where EOSDIS stands relative to the FAIR data management. In this presentation we evaluate EOSDIS against the Reusability principle, which calls for a clear and accessible data usage license, rich description of data, detailed provenance, and compliance with domain-relevant community standards.Achieving consistent compliance for datasets is particularly challenging with the diverse community of missions and scientists providing data to EOSDIS. To address this, EOSDIS's constituent Distributed Active Archive Centers work with data producers to utilize the applicable standards and conventions. In addition, a guidebook is under development to provide comprehensive and understandable guidance on how to construct compliant, and thus more reusable, data products.

Hampapuram (Rama) Ramapriyan↗

Use of Remote Sensing Products for the SERVIR Project

The United Nations University (UNU) estimates that floods presently impacts greater than 520 million people per year worldwide, resulting in up to 25,000 annual deaths, extensive homelessness, disaster-induced disease, crop and livestock damage, famine, and other serious harm. Meanwhile, aid agencies such as the International Federation of Red Cross and Red Crescent Societies (IFRC) are increasingly seeking better information concerning flood hazards in order to plan for and help mitigate the effects of damaging floods. There is fertile ground to continue development of better remote sensing and modeling techniques to help manage flood related disasters. Disaster management and humanitarian aid organizations need accurate and timely information for making decisions regarding deployment of relief teams and emergency supplies during major floods. Flood maps based on the use of satellite data have proven extremely valuable to such organizations for identifying the location, extent, and severity of these events. However, despite extraordinary efforts on the part of remote sensing data providers to rapidly deliver such maps, there is typically a delay of several days or even weeks from the on-set of flooding until such maps are available to the disaster management community. This paper summarizes efforts at NASA to address this problem through development of an integrated and automated process of a) flood forecasting b) flood detection, c) satellite data acquisition, d) rapid flood mapping and distribution, and e) validation of flood forecasting and detection products.

Policelli, Frederick S.↗

Application of Satellite Observations to Manage Natural Disasters in the Lake Victoria Basin

Lake Victoria, the second largest fresh water lake in the Eastern part of Africa is a vital natural resource for the economic well being and prosperity of over 30 million people located in riparian regions of Uganda, Kenya and Tanzania. It covers a large area of about 68,870 km2 and produces a GDP of about US $30 billion per year. The region is also very much prone to natural disasters such as severe floods during heavy precipitation periods in the Eastern part of Africa. In addition to floods, the precipitation also produces large infestations of mosquito larvae due to the standing water in many areas. This further causes multiple vector borne diseases such as Malaria, Rift Valley Fever and more. These problems are of serious concern and require active and aggressive surveillance and management to minimize the loss of human and animal lives and property damage. Satellite imagery and observations along with the in situ measurements provide a great tool to analyze and study this area and inform the policy makers to make calculated policy decisions which are more beneficial to the environment. Recently, NASA and USAID have joined forces with the Regional Center for Mapping of Resources for Development (RCMRD) located in Nairobi, Kenya to utilize multiple NASA sensors such as TRMM, SRTM and MODIS to develop flood potential maps for the Lake Victoria Basin. The idea is to generate a flood forecasts and "nowcasts" that can be sent to the disaster management organizations of Uganda, Kenya, and Tanzania. Post flood event satellite imagery is becoming a common tool to assess the areas inundated by flooding. However, this work is unique undertaking by utilizing land imaging and atmospheric satellites to build credible flood potential maps. At same time, we are also studying the potential occurrence and spread of Rift Valley Fever disease based on the short term climate records and precipitation data. These activities require multi-nation coordination and agreements and multiple operational agencies within each respective country. It also requires credible in situ data such as precipitation, river flow rates and lake levels to further validate the global and regional Hood models and algorithms. This also requires a considerable amount of training and capacity building for the RCMRD experts who will help us validate the model results and eventually transition it for operational use. In a final analysis, Disaster management and humanitarian aid organizations need accurate and timely information for making decisions regarding deployment of relief teams and emergency supplies during major floods. Flood maps based on the use of satellite data have proven extremely valuable to such organizations for identifying the location, extent, and severity of these events. However, despite extraordinary efforts on the part of remote sensing data providers to rapidly deliver such maps, there is typically a delay of several days or even weeks from the on-set of flooding until such maps are available to the disaster management community. This paper summarizes efforts at NASA to address this problem through development of an integrated and automated process of a) flood forecasting b) flood detection, c) satellite data acquisition, d) rapid Hood mapping and distribution, and e) validation of Hood forecasting and detection products.

Habib, Shahid↗

MetaCat - metadata catalog for data management systems

Metadata management is one of three major areas and parts of functionality of scientific data management along with replica management and workflow management. Metadata is the information describing the data stored in a data item, a file or an object. It includes the data item provenance, recording conditions, format and other attributes. MetaCat is a metadata management database designed and developed for High Energy Physics experiments. As a component of a data management system, it’s main objectives are to provide efficient metadata storage and management and fast data items selection functionality. MetaCat is supposed to work on the scale of 100 million files (or objects) and beyond. The article will discuss the functionality of MetaCat and technological solutions used to implement the product.

Mandrichenko, Igor↗

From EXOSAT to the High Energy Astrophysics Science Archive (HEASARC): X-ray Astronomy Comes of Age

In May 1983 the European Space Agency launched EXOSAT, its first X-ray astronomy observatory. Even though it lasted only 3 short years, this mission brought not only new capabilities that resulted in unexpected discoveries, but also a pioneering approach to operations and archiving that changed X-ray astronomy from observations led by small instrument teams, to an observatory approach open to the entire community through a guest observer program. The community use of the observatory was supported by a small dedicated team of scientists, the precursor to the data center activities created to support e.g. Chandra and XMM-Newton. The new science capabilities of EX OS AT included a 90 hr highly eccentric high earth orbit that allow unprecedented continuous coverage of sources as well as direct communication with the satellite that allowed real time decisions to respond to unexpected events through targets of opportunity. The advantages of this orbit demonstrated by EXOSAT resulted in Chandra and XMM-Newton selecting similar orbits. The three instruments on board the EXOSAT observatory were complementary, designed to give complete coverage over a wide energy band pass of 0.05-50 keY. An onboard processor could be programmed to give multiple data modes that could be optimized in response to science discoveries: These new capabilities resulted in many new discoveries including the first comprehensive study of AGN variability, new orbital periods in X-ray binaries and cataclysmic variables, new black holes, quasi-periodic oscillations from neutron stars and black holes and broad band X-ray spectroscopy. The EXOSAT team generated a well-organized database accessible worldwide over the nascent internet, allowing remote selection of data products, making samples and undertaking surveys from the data. The HEASARC was established by NASA at Goddard Space Flight Center in 1990 as the repository of NASA X-ray and Gamma-ray data. The proven EXOSAT database system became the core of the HEASARC infrastructure. The HEASARC pioneered many concepts now taken for granted including standardized formats using FITS files, restoring data from earlier missions, multi-mission analysis tools and a searchable archive over the world wide web.

White, Nicholas E.↗

Improving the estimate of higher-order moments from lidar observations near the top of the convective boundary layer

Abstract. Ground-based lidar data have proven extremely useful for profiling the convective boundary layer (CBL). Many groups have derived higher-order moments (e.g., variance, skewness, fluxes) from high-temporal-resolution lidar data using an autocovariance approach. However, these analyses are highly uncertain near the CBL top when the depth of the CBL (zi) is changing during the analysis period. This is because the autocovariance approach is usually applied to constant height levels and the character of the eddies is changing on either side of the changing CBL top. Here, a new approach is presented wherein the autocovariance analysis is performed on a normalized height grid, with a temporally smoothed zi. Output from a large eddy simulation model demonstrates that deriving higher-order moments from time series on a normalized height grid has better agreement with the slab-averaged quantities than the moments derived from the original height grid.

Rosenberger, Tessa E. (ORCID:0000000333205873)↗