Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “provenance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Capturing provenance as a diagnostic tool for workflow performance evaluation and optimization

In extreme-scale computing environments such as the DOE Leadership Computing Facilities scientific workflows are routinely used to coordinate software processes for the execution of complex, computational applications that perform in-silico experiments. Monitoring the performance of workflows without also simultaneously tracking provenance is not sufficient to understand variations between runs, configurations, versions of a code, and between changes in an implemented stack, and systems, i.e. the variability of performance metrics data in their historical context. We take a provenance-based approach and demonstrate that provenance is useful as a tool for evaluating and optimizing workflow performance in extreme-scale HPC environments. We present Chimbuko, a framework for the analysis and visualization of the provenance of performance. Chimbuko implements a method for the evaluation of workflow performance from multiple components that enables the exploration of performance metrics data at scale.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Characterizing Provenance in Visualization and Data Analysis: An Organizational Framework of Provenance Types and Purposes

While the primary goal of visual analytics research is to improve the quality of insights and findings, a substantial amount of research in provenance has focused on the history of changes and advances throughout the analysis process. The term, provenance, has been used in a variety of ways to describe different types of records and histories related to visualization. The existing body of provenance research has grown to a point where the consolidation of design knowledge requires cross-referencing a variety of projects and studies spanning multiple domain areas. We present an organizational framework of the different types of provenance information and purposes for why they are desired in the field of visual analytics. Our organization is intended to serve as a framework to help researchers specify types of provenance and coordinate design knowledge across projects. We also discuss the relationships between these factors and the methods used to capture provenance information. In addition, our organization can be used to guide the selection of evaluation methodology and the comparison of study outcomes in provenance research

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Provenance information as a tool for addressing engineered nanoparticle reproducibility challenges

Nanoparticles of various types are of increasing research and technological importance in biological and other applications. Difficulties in the production and delivery of nanoparticles with consistent and well defined properties appear in many forms and have a variety of causes. Among several issues are those associated with incomplete information about the history of particles involved in research studies including the synthesis method, sample history after synthesis including time and nature of storage and the detailed nature of any sample processing or modification. In addition, the tendency of particles to change with time or environmental condition suggests that the time between analysis and application is important and some type of consistency or verification process can be important. The essential history of a set of particles can be identified as provenance information tells the origin or source of a batch of nano-objects along with information related to handling and any changes that may have taken place since it was originated. A record of sample provenance information for a set of particles can play a useful role in identifying some of the sources and decreasing the extent of particle variability and the observed lack of reproducibility observed by many researchers.

Environmental Molecular Sciences Laboratory↗
DOE OSTI2020

Proven Technologies for the Solidification of Complex Liquid Radioactive Waste (LRW): Global Case Studies of Applications and Disposal Options - 20424

Legacy radioactive waste streams from the Cold War still exist and newly generated waste streams from nuclear power plants and research institutes go untreated and expose environmental hazards at many nuclear sites. The nature of the waste is diverse, depending upon the source or the process from which it originated. The most problematic waste streams include complex liquids such as organic (tri-butyl-phosphate TBP) solutions contaminated with Pu and U isotopes, mixed sludge types, high acid radioactive waste, H-3 contaminated organic and aqueous streams, etc. Technological, environmental and economic challenges exist for the treatment and disposal of such waste streams. A proven technology that has been applied to LRW on a global basis provides one option as a low-cost solution to legacy streams and small volume, highly complex LRW frequently found during decommissioning at nuclear power plants and weapons sites. The engineered polymer technology from Nochar, USA, is capable of solidifying standard and highly complex LLW and ILW waste streams for interim or final storage, or for incineration. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2016

A Scientific Data Provenance API for Distributed Applications

Data provenance has been an active area of research as a means to standardize how the origin of data, process event history, and what or who was responsible for influencing results is explained. There are two approaches to capture provenance information. The first approach is to collect observed evidence produced by an executing application using log files, event listeners, and temporary files that are used by the application or application developer. The provenance translated from these observations is an interpretation of the provided evidence. The second approach is called disclosed because the application provides a firsthand account of the provenance based on the anticipated questions on data flow, process flow, and responsible agents. Most observed provenance collection systems collect lot of provenance information during an application run or workflow execution. The common trend in capturing provenance is to collect all possible information, then attempt to find relevant information, which is not efficient. Existing disclosed provenance system APIs do not work well in distributed environment and have trouble finding where to fit the individual pieces of provenance information. This work focuses on determining more reliable solutions for provenance capture. As part of the Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows (IPPD) project, an API was developed, called Producer API (PAPI), which can disclose application targeted provenance, designed to work in distributed environments by means of unique object identification methods. The provenance disclosure approach used adds additional metadata to the provenance information to uniquely identify the pieces and connect them together. PAPI uses a common provenance model to support this provenance integration across disclosure sources. The API also provides the flexibility to let the user decide what to do with the collected provenance. The collected provenance can be sent to a triple store using REST services or it can be logged to a file.

Raju, Bibi↗
DOE OSTI2013

Tracking Provenance in ORNL's Flexible Research Platforms

Provenance is dened as information about the origin of objects, a concept that applies to both physical and digital objects and often overlaps both. The use of provenance in systems designed for research is an important but forgotten feature. Provenance allows for proper and exact tracking of information, its use, its lineage, its derivations and other metadata that are important for correctly adhering to the scien- tic method. In our project's prescribed use of provenance, researchers can determine detailed information about the use of sensor data in their experiments on ORNL's Flexible Research Platforms (FRPs). Our project's provenance system, Provenance Data Management System (ProvDMS), tracks information starting with the creation of information by an FRP sensor. The system determines station information, sensor information, and sensor channel information. The system allows researchers to derive generations of experiments from the sensor data and tracks their hierarchical flow. Key points can be seen in the history of the information as part of the information's workflow. The concept of provenance and its usage in science is relatively new and while used in other cases around the world, our project's provenance diers in a key area. To keep track of provenance, most systems must be designed or redesigned around the new provenance system. Our system is designed as a cohesive but sepa- rate entity and allows for researchers to continue using their own methods of analysis without being constrained in their ways in order to track the provenance. We have designed ProvDMS using a lightweight provenance library, Core Provenance Library (CPL) v.6 In addition to keeping track of sensor data experiments and its provenance, ProvDMS also provides a web-enabled visualization of the inheritance.

provenance↗
DOE OSTI2017

A Scientific Data Provenance Harvester for Distributed Applications

Data provenance provides a way for scientists to observe how experimental data originates, conveys process history, and explains influential factors such as experimental rationale and associated environmental factors from system metrics measured at runtime. The US Department of Energy Office of Science Integrated end-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows (IPPD) project has developed a provenance harvester that is capable of collecting observations from file based evidence typically produced by distributed applications. To achieve this, file based evidence is extracted and transformed into an intermediate data format inspired in part by W3C CSV on the Web recommendations, called the Harvester Provenance Application Interface (HAPI) syntax. This syntax provides a general means to pre-stage provenance into messages that are both human readable and capable of being written to a provenance store, Provenance Environment (ProvEn). HAPI is being applied to harvest provenance from climate ensemble runs for Accelerated Climate Modeling for Energy (ACME) project funded under the U.S. Department of Energy’s Office of Biological and Environmental Research (BER) Earth System Modeling (ESM) program. ACME informally provides provenance in a native form through configuration files, directory structures, and log files that contain success/failure indicators, code traces, and performance measurements. Because of its generic format, HAPI is also being applied to harvest tabular job management provenance from Belle II DIRAC scheduler relational database tables as well as other scientific applications that log provenance related information.

Stephan, Eric G.↗
DOE OSTI2015

Defining the “proven technology” technical criterion in the reactor technology assessment for Malaysia’s nuclear power program

Developing countries that are considering the deployment of nuclear power plants (NPPs) in the near future need to perform reactor technology assessment (RTA) in order to select the most suitable reactor design. The International Atomic Energy Agency (IAEA) reported in the Common User Considerations (CUC) document that “proven technology” is one of the most important technical criteria for newcomer countries in performing the RTA. The qualitative description of five desired features for “proven technology” is relatively broad and only provides a general guideline to its characterization. This paper proposes a methodology to define the “proven technology” term according to a specific country’s requirements using a three-stage evaluation process. The first evaluation stage screens the available technologies in the market against a predefined minimum Technology Readiness Level (TRL) derived as a condition based on national needs and policy objectives. The result is a list of technology options, which are then assessed in the second evaluation stage against quantitative definitions of CUC desired features for proven technology. The potential technology candidates produced from this evaluation is further narrowed down to obtain a list of proven technology candidates by assessing them against selected risk criteria and the established maximum allowable total score using a scoring matrix. The outcome of this methodology is the proven technology candidates selected using an accurate definition of “proven technology” that fulfills the policy objectives, national needs and risk, and country-specific CUC desired features of the country that performs this assessment. A simplified assessment for Malaysia is carried out to demonstrate and suggest the use of the proposed methodology. In this exercise, ABWR, AP1000, APR1400 and EPR designs assumed the top-ranks of proven technology candidates according to Malaysia’s definition of “proven technology”.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2012

Provenance-Enabling DOE Visualization Applications

VisTrails, Inc.’s goal main charter has been to develop provenance management solutions for software tools. As the use of computation increases, keeping track of computational processes and the data they consume and derive is essential for many applications. Provenance (also referred to as audit trail, lineage, and pedigree) captures information about the steps used to generate a given data product. Such information provides important documentation that is key to preserving the data, to determining the data’s quality and authorship, and to reproducing as well as validating the results. Given the growing complexity of the computational processes and the volume of data they manipulate, manual approaches to maintaining provenance are both tedious and error-prone. At VisTrails, Inc., we have implemented infrastructure that can be used to provenance-enable existing tools. We have developed a series of plugins for specific applications, including ParaView and VisIt, which enable these applications to capture provenance. While developing these plugins, we identified general patterns across different applications, and these were used to inform the design of a general framework: the VisTrails Provenance Software Development Kit (SDK). The SDK encapsulates a set of core functionality including methods for capture, navigation, analysis and re-use of provenance. It allows application or plugin developers to more effectively provide provenance support: they can use the core functionality and write code only to handle application specific components. The SDK includes features that can be used in any application, for example, a refined interface for browsing the intermediate and final results captured. While the application-specific plugins were written in Python, the SDK was written from scratch in C++. This has led not only to substantial gains in performance but also the ability to integrate the SDK with applications written in other languages, including Java. To ensure that the SDK supported all of the functionality in the application-specific plugins, we have rewritten some of the existing plugins using the SDK. The SDK can be downloaded from www.vistrals.com/sdk.html.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Automated metadata, provenance cataloging and navigable interfaces: ensuring the usefulness of extreme-scale data

The MPO (Metadata, Provenance, Ontology) Project successfully addressed the goal of improving the usefulness and traceability of scientific data by building a system that could capture and display all steps in the process of creating, analyzing and disseminating that data. Throughout history, scientists have generated handwritten logbooks to keep track of data, their hypotheses, assumptions, experimental setup, and computational processes as well as reflections on observations and issues encountered. Over the last several decades, with the growth of personal computers, handheld devices, and the World Wide Web, the handwritten logbook has begun to be replaced by electronic logbooks. This transition has brought increased capability such as supporting multi-media, hypertext, and fast searching. However, content creation and metadata (a set of data that describes and gives information about other data) capturing has for the most part remained a manual activity just as it was with handwritten logbooks. This has led to a fragmentation of data, processing, and annotation that has only accelerated as scientific workflows continue to increase in complexity. From a scientific perspective, it is very important to be able to understand the lineage of any piece of data: who, what, when, how, and why. This is typically referred to as data provenance. The fragmentation discussed previously often means that data provenance is lost. As scientific workflows move to powerful computers and become more complex, the ability to track all of the steps involved in creating a piece of data become even more difficult. It was the goal of the MPO (Metadata, Provenance, Ontology) Project to create a system (the MPO System) that allows for automatic provenance and metadata capturing in such a way to allow easy searching and browsing. This goal needed to be accomplished in a general way so that it may be used across a broad range of scientific domains, yet allow the addition of vocabulary (Ontology) that is domain specific as is required for intelligent searching and browsing in the scientific context. Through the creation and deployment of the MPO system, the goals of the project were achieved. An enhanced metadata, provenance, and ontology storage system was created. This was combined with innovative methodologies for navigating and exploring these data using a web browser for both experimental and simulation-based scientific research. In addition, a system to allow scientists to instrument their existing workflows for automatic metadata and provenance is part of the MPO system. In that way, a scientist can continue to use their existing methodology yet easily document their work. Workflows and data provenance can be displayed either graphically or in an electronic notebook format and support advanced search features including via ontology. The MPO system was successfully used in both Climate and Magnetic Fusion Energy Research. The software for the MPO system is located at https://github.com/MPO-Group/MPO and is open source distributed under the Revised BSD License. A demonstration site of the MPO system is open to the public and is available at https://mpo.psfc.mit.edu/. A Docker container release of the command line client is available for public download using the command docker pull jcwright/mpo-cli at https://hub.docker.com/r/jcwright/mpo-cli.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2008

Provenance in Data Interoperability for Multi-Sensor Intercomparison

As our inventory of Earth science data sets grows, the ability to compare, merge and fuse multiple datasets grows in importance. This requires a deeper data interoperability than we have now. Efforts such as Open Geospatial Consortium and OPeNDAP (Open-source Project for a Network Data Access Protocol) have broken down format barriers to interoperability; the next challenge is the semantic aspects of the data. Consider the issues when satellite data are merged, cross-calibrated, validated, inter-compared and fused. We must match up data sets that are related, yet different in significant ways: the phenomenon being measured, measurement technique, location in space-time or quality of the measurements. If subtle distinctions between similar measurements are not clear to the user, results can be meaningless or lead to an incorrect interpretation of the data. Most of these distinctions trace to how the data came to be: sensors, processing and quality assessment. For example, monthly averages of satellite-based aerosol measurements often show significant discrepancies, which might be due to differences in spatio- temporal aggregation, sampling issues, sensor biases, algorithm differences or calibration issues. Provenance information must be captured in a semantic framework that allows data inter-use tools to incorporate it and aid in the intervention of comparison or merged products. Semantic web technology allows us to encode our knowledge of measurement characteristics, phenomena measured, space-time representation, and data quality attributes in a well-structured, machine-readable ontology and rulesets. An analysis tool can use this knowledge to show users the provenance-related distrintions between two variables, advising on options for further data processing and analysis. An additional problem for workflows distributed across heterogeneous systems is retrieval and transport of provenance. Provenance may be either embedded within the data payload, or transmitted from server to client in an out-of-band mechanism. The out of band mechanism is more flexible in the richness of provenance information that can be accomodated, but it relies on a persistent framework and can be difficult for legacy clients to use. We are prototyping the embedded model, incorporating provenance within metadata objects in the data payload. Thus, it always remains with the data. The downside is a limit to the size of provenance metadata that we can include, an issue that will eventually need resolution to encompass the richness of provenance information required for daata intercomparison and merging.

Lynnes, Chris↗
DOE OSTI2022

The Influence of Visual Provenance Representations on Strategies in a Collaborative Hand-off Data Analysis Scenario

Conducting data analysis tasks rarely occur in isolation. Especially in intelligence analysis scenarios where different experts contribute knowledge to a shared understanding, members must communicate how insights develop to establish common ground among collaborators. The use of provenance to communicate analytic sensemaking carries promise by describing the interactions and summarizing the steps taken to reach insights. Yet, no universal guidelines exist for communicating provenance in different settings. Our work here focuses on the presentation of provenance information and the resulting conclusions reached and strategies used by new analysts. In an open-ended, 30-minute, textual exploration scenario, we qualitatively compare how adding different types of provenance information (specifically data coverage and interaction history) affects analysts' confidence in conclusions developed, propensity to repeat work, filtering of data, identification of relevant information, and typical investigation strategies. We see that data coverage (i.e., what was interacted with) provides provenance information without limiting individual investigation freedom. On the other hand, while interaction history (i.e., when something was interacted with) does not significantly encourage more mimicry, it does take more time to comfortably understand, as represented by less confident conclusions and less relevant information-gathering behaviors. In conclusion, our results contribute empirical data towards understanding how provenance summarizations can influence analysis behaviors.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2014

Ontology Engineering in Provenance Enablement for the National Climate Assessment

The National Climate Assessment of the U.S. Global Change Research Program (USGCRP) analyzes and presents the impacts of climate change on the United States. The provenance information in the assessment is important because the assessment findings are of great public and academic concern and are used in policy and decision-making. By applying a use case-driven iterative methodology, we developed information models and ontology to represent the content structure of the recent National Climate Assessment draft report and its associated provenance information. We tested the ontology by using it in pilot systems serving information about instances of chapters, scientific findings, figures, tables, images, datasets, references, people, and organizations, etc. in the draft report, as well as interrelationships among those instances. The results successfully help users trace provenance in the draft report, such as finding all the journal articles from which a figure in the report was derived. The provenance information in our work was maintained in the context of the "Web of Data". In addition to the pilot systems we developed, other tools and services are also able to retrieve and utilize the provenance information. Our work is part of a Global Change Information System coordinated by the USGCRP that will eventually cover provenance information for the entire scope of global change research. Such a system will greatly increase understanding, credibility and trust in the global change research and foster reproducibility of scientific results and conclusions.

Ontology engineering↗
DOE OSTI2017

Variation in short-term and long-term responses of photosynthesis and isoprenoid-mediated photoprotection to soil water availability in four Douglas-fir provenances

For long-lived forest tree species, the understanding of intraspecific variation among populations and their response to water availability can reveal their ability to cope with and adapt to climate change. Dissipation of excess excitation energy, mediated by photoprotective isoprenoids, is an important defense mechanism against drought and high light when photosynthesis is hampered. We used 50-year-old Douglas-fir trees of four provenances at two common garden experiments to characterize provenance-specific variation in photosynthesis and photoprotective mechanisms mediated by essential and non-essential isoprenoids in response to soil water availability and solar radiation. All provenances revealed uniform photoprotective responses to high solar radiation, including increased de-epoxidation of photoprotective xanthophyll cycle pigments and enhanced emission of volatile monoterpenes. In contrast, we observed differences between provenances in response to drought, where provenances sustaining higher CO2 assimilation rates also revealed increased water-use efficiency, carotenoid-chlorophyll ratios, pools of xanthophyll cycle pigments, β-carotene and stored monoterpenes. Our results demonstrate that local adaptation to contrasting habitats affected chlorophyll-carotenoid ratios, pool sizes of photoprotective xanthophylls, β-carotene, and stored volatile isoprenoids. We conclude that intraspecific variation in isoprenoid-mediated photoprotective mechanisms contributes to the adaptive potential of Douglas-fir provenances to climate change.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Workflow Provenance in the Computing Continuum for Responsible, Trustworthy, and Energy-Efficient AI

As Artificial Intelligence (AI) becomes more pervasive in our society, it is crucial to develop, deploy, and assess Responsible and Trustworthy AI (RTAI) models, i.e., those that consider not only accuracy but also other aspects, such as explainability, fairness, and energy efficiency. Workflow provenance data have historically enabled critical capabilities towards RTAI. Provenance data derivation paths contribute to responsible workflows through transparency in tracking artifacts and resource consumption. Provenance data are well-known for their trustworthiness helping explainability, reproducibility, and accountability. However, there are complex challenges to achieve RTAI, which are further complicated by the heterogeneous infrastructure in the computing continuum (Edge-Cloud-HPC) used to develop and deploy models. As a result, a significant research and development gap remains between workflow provenance data management and RTAI. In this paper, we present a vision of the pivotal role of workflow provenance in supporting RTAI and discuss related challenges. We present a schematic view between RTAI and provenance, and highlight open research directions.

Santos Souza, Renan↗
DOE OSTI2021

Data from: "Ecophysiological variation in two provenances of Pinus flexilis seedlings across an elevation gradient from forest to alpine"

This archive contains data used to support conclusions drawn in “Ecophysiological variation in two provenances of Pinus flexilis seedlings across an elevation gradient from forest to alpine”, by Reinhardt et al., 2011. Data were collected over one summer season in plots within the Alpine Treeline Warming Experiment (ATWE), before climate manipulations began. The experiment was located on Niwot Ridge, in the Front Range of the Colorado Rocky Mountains. This data package includes five comma-separated-values (.csv) files, five Microsoft Excel (.xlsx) files, one .pdf file, and two types of geospatial files: keyhole markup language (.kml), and ESRI shapefiles (.shp). .csv files can be opened using any simple text-editing software (such as Notepad and TextEdit), R, and Microsoft Excel. .xlsx files can only be opened using Microsoft Excel. The .pdf file can be opened using Adobe Acrobat Reader or any other compatible file viewing software. The .kml file can be opened using Google Earth and Google Maps, and shapefiles can be opened using any software compatible with the file type, such as ESRI’s ArcGIS suite and QGIS.Data archived contain gas exchange and plant physiology measurements, non-structural carbohydrate data, among others. Geospatial files are also provided for additional locational context. The files and their contents in this data package are summarized under "Data Summary" in the included Data User's Guide. All files (excluding geospatial) are available in both Microsoft Excel and in .csv format, and are indicated in the Data Summary list as well.-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Climate change is predicted to cause upward shifts in forest tree distributions, which will require seedling recruitment beyond current forest boundaries. However, predicting the likelihood of successful plant establishment beyond current species’ ranges under changing climate is complicated by the interaction of genetic and environmental controls on seedling establishment. To determine how genetics and climate may interact to affect seedling establishment, we transplanted recently germinated seedlings from high- and low-elevation provenances (HI and LO, respectively) of Pinus flexilis in common gardens arrayed along an elevation and canopy gradient from subalpine forest into the alpine zone and examined differences in physiology and morphology between provenances and among sites. Plant dry mass, projected leaf area and shoot:root ratios were 12–40% greater in LO compared with HI seedlings at each elevation. There were no significant changes in these variables among sites except for decreased dry mass of LO seedlings in the alpine site. Photosynthesis, carbon balance (photosynthesis/respiration) and conductance increased >2× with elevation for both provenances, and were 35–77% greater in LO seedlings compared with HI seedlings. There were no differences in dark-adapted chlorophyll fluorescence (Fv/Fm) among sites or between provenances. Our results suggest that for P. flexilis seedlings, provenances selected for above-ground growth may outperform those selected for stress resistance in the absence of harsh climatic conditions, even well above the species’ range limits in the alpine zone. This indicates that forest genetics may be important to understanding and managing species’ range adjustments due to climate change.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2010

Distinguishing Provenance Equivalence of Earth Science Data

Reproducibility of scientific research relies on accurate and precise citation of data and the provenance of that data. Earth science data are often the result of applying complex data transformation and analysis workflows to vast quantities of data. Provenance information of data processing is used for a variety of purposes, including understanding the process and auditing as well as reproducibility. Certain provenance information is essential for producing scientifically equivalent data. Capturing and representing that provenance information and assigning identifiers suitable for precisely distinguishing data granules and datasets is needed for accurate comparisons. This paper discusses scientific equivalence and essential provenance for scientific reproducibility. We use the example of an operational earth science data processing system to illustrate the application of the technique of cascading digital signatures or hash chains to precisely identify sets of granules and as provenance equivalence identifiers to distinguish data made in an an equivalent manner.

Tilmes, Curt↗