Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data provenance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Crosslink V.0.11.x User Manual

CrossLink is a novel two-dimensional and three-dimensional geometry and mesh generation software package developed by the Simulation Tools team at Los Alamos National Laboratory. This software represents the third generation of topology-based mesh generation technology developed by the Department of Defense and the Department of Energy with a special focus on complex multi-material hydrodynamic applications, mesh scalability, and high-order element mesh generation. The topology-based meshing approach offered by CrossLink enables users to quickly and easily mesh complex geometries in a repeatable and robust manner. CrossLink’s topology-based meshing approach is well-suited for parametric design studies, parametric design optimization, damage scenario assessment, and iterative design modification (i.e. feature addition and/or removal). CrossLink’s python API allows workflow scripting of the geometry creation and mesh generation process for traceability, repeatability, data provenance, and version control. CrossLink consists of three main components: a graphical user interface (GUI), a geometry creation and mesh generation engine, and a python API that provides a workflow scripting interface to the geometry and meshing functions.

97 MATHEMATICS AND COMPUTING↗

Greenland and Canadian Arctic Ice Temperature Profiles Database

Here, we present a compilation of 95 ice temperature profiles from 85 boreholes from the Greenland ice sheet and peripheral ice caps, as well as local ice caps in the Canadian Arctic. Profiles from only 31 boreholes (36 %) were previously available in open-access data repositories. The remaining 54 borehole profiles (64 %) are being made digitally available here for the first time. These newly available profiles, which are associated with pre-2010 boreholes, have been submitted by community members or digitized from published graphics and/or data tables. All 95 profiles are now made available in both absolute (meters) and normalized (0 to 1 ice thickness) depth scales and are accompanied by extensive metadata. These metadata include a transparent description of data provenance. The ice temperature profiles span 70 years, with the earliest profile being from 1950 at Camp VI, West Greenland. To highlight the value of this database in evaluating ice flow simulations, we compare the ice temperature profiles from the Greenland ice sheet with an ice flow simulation by the Parallel Ice Sheet Model (PISM). We find a cold bias in modeled near-surface ice temperatures within the ablation area, a warm bias in modeled basal ice temperatures at inland cold-bedded sites, and an apparent underestimation of deformational heating in high-strain settings. These biases provide process level insight on simulated ice temperatures.

Greenland↗

Analyzing EOSDIS Dataset Research Outputs using Knowledge Graphs and Large Language Models

Datasets, unlike publications, can be updated over time, with each new version receiving a DOI but not always being linked to previous ones. This complicates tracking citations across a dataset’s lifecycle. We address this by integrating dataset versions and citations into a knowledge graph (KG), which helps trace dataset citations and analyze dataset usage in applied research. To categorize publications from various journals, we fine-tuned NASA IMPACT INDUS Large Language Model (LLM) on a labeled publication set, assigning publications to one of twenty applied research areas. By linking datasets to these research areas, we improved dataset searchability and discovery through these domains.

open-source↗

Discovering Research Areas in Dataset Applications Through Knowledge Graphs and Large Language Models

Scientific datasets are increasingly cited in peer-reviewed journal publications, facilitating easy access to research utilizing those datasets. Datasets undergo a life cycle where older versions of datasets are replaced by newer versions often due to improvements in data resolution, algorithms, and other factors. Unlike peer reviewed documents registered with a single Digital Unique Identifier (DOI), datasets can be updated over time and the newer version of the datasets are registered with a new DOI which is not necessarily linked to the previous version of the dataset. It is challenging when publications citing a dataset need to be traced over the entire life cycle of that dataset. We provide an innovative approach to link the dataset versions and publications using a knowledge graph (KG). KG can help to trace the dataset cited in publications over the entire dataset life cycle and shed light into dataset usage in various applied research areas. We fine-tuned the pretrained NASA IMPACTINDUS Large Language Model (LLM) on a set of labeled publications abstracts. Our results showed that 87% of the publications were classified into one of twenty applied research areas, while the remaining 13% were classified into non-applied research areas. By linking datasets to applied research areas through the KG and employing Global Change Master Directory(GCMD), a well-established controlled vocabulary of scientific keywords describing Earth science datasets, we contribute to a transparent and advanced search and discovery mechanism for datasets across the Earth data ecosystem. The integrated KG and LLM approach is now incorporated and operational in dataset publication management at one of NASA’s Earth science data archival centers.

data provenance↗

Enhancing Dataset Discovery and Usage Tracking in Earth Sciences: Integrating Knowledge Graphs and Large Language Models

NASA's Data Active Archive Centers (DAACs) have played a crucial role in supporting a wide range of applied research in Earth and Environmental sciences. To date, over 20,000 publications have been collected, citing more than 3,000 NASA Earth science datasets. We present an innovative approach that links datasets and collected publications through a knowledge graph (KG). This KG enables the tracking of dataset citations throughout the dataset's lifecycle, revealing patterns of dataset usage across various applied research areas. We fine-tuned the pre-trained NASA IMPACT INDUS-Base Retriever Large Language Model (LLM) using a set of labeled publication abstracts. Our results indicate that 87% of the publications were classified into one of twenty applied research areas, while the remaining 13% were categorized into non-applied research areas. The classified publications linked to datasets are used to discover datasets by users interested in specific applied research and by dataset providers to determine dataset usage for applications.

open-source↗

SuperIdentity: Fusion of Identity across Real and Cyber Domains

Under both benign and malign circumstances, people now manage a spectrum of identities across both real-world and cyber domains. Our belief, however, is that all these instances ultimately track back for an individual to reflect a single 'SuperIdentity'. This paper outlines the assumptions underpinning the SuperIdentity Project, describing the innovative use of data fusion to incorporate novel real-world and cyber cues into a rich framework appropriate for modern identity. The proposed combinatorial model will support a robust identification or authentication decision, with confidence indexed both by the level of trust in data provenance, and the diagnosticity of the identity factors being used. Additionally, the exploration of correlations between factors may underpin the more intelligent use of identity information so that known information may be used to predict previously hidden information. With modern living supporting the 'distribution of identity' across real and cyber domains, and with criminal elements operating in increasingly sophisticated ways in the hinterland between the two, this approach is suggested as a way forwards, and is discussed in terms of its impact on privacy, security, and the detection of threat.

99 GENERAL AND MISCELLANEOUS↗

Provenance Storage, Querying, and Visualization in PBase

We present PBase, a repository for scientific workflows and their corresponding provenance information that facilitates the sharing of experiments among the scientific community. PBase is interoperable since it uses ProvONE, a standard provenance model for scientific workflows. Workflows and traces are stored in RDF, and with the support of SPARQL and the tree cover encoding, the repository provides a scalable infrastructure for querying the provenance data. Furthermore, through its user interface, it is possible to: visualize workflows and execution traces; visualize reachability relations within these traces; issue SPARQL queries; and visualize query results.

Kianmajd, Parisa↗

Blockchain: Next Generation Supply Chain Security for Energy Infrastructure and NERC Critical Infrastructure Protection (CIP) Compliance

The U.S. power grid is a complex system of systems that requires a secure, reliable and trustworthy global supply chain. This is especially true for the grid’s increasing number of networked energy delivery system (EDS) and industrial control systems (ICS) and associated vendors, distributers, integrators and end users. Grid modernization has increased the use of “smart” energy devices that network, digitize, automate and increasingly converge the cyber-physical energy supply chain. In this Energy Internet of Things (EIoT) environment there is an increasing number of both critical cyber assets as well as data speed and size requirements, creating new cyber supply chain security and North American Electric Reliability Corporation (NERC) CIP compliance challenges for utilities, regulators and vendors. In realization of these goals, improvements are needed in the process and technology to better monitor, audit and realize cyber security and compliance goals in complex EIoT environments. This paper explores how blockchain technology may present a disruptive solution to facilitate NERC CIP compliance and increase the security of the electricity infrastructure supply chain through a cryptographically signed distributed ledger that provides increased data provenance, attribution and auditability.

Blockchain, Cyber security, grid cyber, NERC CIP, ↗

Blockchain for High Performance Data Integrity and Provenance

Describes the development of a test bed to evaluate the the feasibility of different Blockchain mechanisms for determining the integrity of data computed on a high performance computer while limiting the degradation of performance.

99 GENERAL AND MISCELLANEOUS↗

Data from: "Warming and provenance limit tree recruitment across and beyond the elevation range of subalpine forest"

This data package contains data used to support conclusions drawn in “Warming and provenance limit tree recruitment across and beyond the elevation range of subalpine forest”, by Kueppers et al. 2017. Data were collected in field sites within the Alpine Treeline Warming Experiment (ATWE), located on Niwot Ridge, on the eastern slope of the Colorado Rocky Mountains, USA. Files containing geospatial data are also included, to provide additional locational context.There are four document formats associated with this archive: three comma-separated values (.csv) files, three Microsoft Excel (.xlsx) files, one .pdf data user’s guide, four keyhole markup language (.kml) files, and a compressed folder containing seven ESRI shapefiles (.shp). The .csv files can be opened using any simple text-editor software, R, or Microsoft Excel. The .xlsx files can only be opened using Microsoft Excel. The .kml file can be opened by Google Earth and Google Maps, and the shapefiles can be opened with any GIS application compatible with the file type, such as ESRI’s ArcGIS, and QGIS.We provide two versions of the seedling data file: “PIEN_PIFLseedlings20150522_20150525rev12222020.csv/.xlsx” (hereafter PIEN_PIFLseedlings2015) and “PIEN_PIFLseedlings20160408rev12222020.csv/.xlsx” (hereafter PIEN_PIFLseedlings2016). PIEN_PIFLseedlings2015 contains the data we used in the paper. PIEN_PIFLseedlings2016 contains an updated version of these data that includes sampling from later years. The main differences between the two files lie in the columns titled “k[YEAR],” which describe the number of seedlings that were killed in a particular year. In PIEN_PIFLseedlings2016, there also is an additional year of data for k2015, and k2014 also has additional data input for the 2014 cohort. Additionally, in years 2010-2014, there are minor differences in the number of seedlings killed -- in as few as 0 plots (in 2011) to as many as 5 plots (in 2014) -- due to errors in data input that were rectified in later years.------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Upslope range shifts by subalpine tree species are a widely anticipated effect of climate change. Climate niche models predict subalpine forests to expand upslope, given more suitable growing conditions for adult trees. However, these models do not take into account climates required for successful seedling recruitment and establishment, an essential element for expansion. Further, localized upper treeline populations are hypothesized to contain favorable traits for colonizing the alpine. To test these expectations and to expand our knowledge of seedling recruitment under climate change, we designed a common garden, climate-warming experiment spread across an elevation gradient at Niwot Ridge in the Colorado Rocky Mountains. We focus on two widespread Western North American species, Engelmann spruce (Picea engelmannii Parry ex. Engelm) and limber pine (Pinus flexilis James), which occur at treeline. While the former is considered a late-seral species more tolerant of shade, limber pine is a shade-intolerant pioneer species able to establish on infertile sites.Every autumn, seeds of the two species were collected from high- (3370 m–3570 m) and low-provenance (2910–3240 m) sources close to the experimental sites and sown in our plots. A subset of plots were heated and another subset watered over the summer months to offset the effects of warming. Across five years, we found that seeds originating from low elevation recruited more strongly for both species, although this provenance difference diminished by the fourth year for Engelmann spruce, likely due to small sample sizes. Despite the recruitment of low-provenance seed, warming treatments decreased recruitment at all elevations. Combining this with the likeliness and availability of lower-quality, high provenance seed moving upslope at the treeline, tree migration into the alpine may be slowed. Overall, our findings suggest that the hardier limber pine is likely to become a more significant species in subalpine forest communities in the future, while the more sensitive Engelmann spruce may experience range contraction.

54 ENVIRONMENTAL SCIENCES↗

Projecting Solar Photovoltaic Efficiencies from Lab to Market

The future is uncertain and forecasting it is difficult. Yet to implement energy policies aimed at sustainable development, it is imperative to determine future energy scenarios. This requires forecasting the development of all relevant generation technologies, including the conversion efficiency of solar photovoltaics. Using historical data, a proven development formula and data on the lag between lab and market, efficiency projections up to 2050 are made for c-Si, CdTe, CIGS, 3- and 4 junction, perovskite, and quantum dot solar cells. The projected efficiencies of the first three, older technologies do not change significantly. The last four, newer technologies could make great leaps in efficiency and increase their market potential, if R&D efforts continue.

14 SOLAR ENERGY↗

Expanding Biological Repository Data Available for Sharing and Knowledge Discovery

Biology has developed next-generation data science and alternative analytical approaches with methodologies which require principal investigator (PI) experimental assay data be re-used. This new approach involves mining multiple datasets at once from various hierarchical organizations of biological complexity, while concurrently evaluating how experimental factors affect endpoints of standard assays. The purpose of the NASA Ames Life Sciences Data Archive (ALSDA) is to collect, curate, and make findable, accessible, interoperable, and reusable (FAIR) all non-human space-relevant biological data. These data include mission metadata, subject metadata, assay metadata (parameters), raw and processed assay data, assay imagery, and subject-experienced telemetry (radiation, temperature, humidity, acoustics, vibrations). ALSDA has transformed to bring current biological repository data and all future collected data into this new scientific data mining reality. It has integrated into the ‘NASA Open Science’ group of projects to facilitate a suite of new tools and workflows to improve data accessibility and reusability by implementing data management plans, automating data submission agreements, and adopting the single-point-of-entry data submission portal, originally developed by NASA GeneLab. These systems required ALSDA to develop science assay configurations for the submission portal, capturing essential assay parameters according to established norms in each sub-field within biology. The submission portal expedites data collection by enhancing ease of PI data submission, providing a user interface and specificity for which data is to be submitted. ALSDA datasets are curated to maintain rich metadata, accuracy of datasets, data transparency, provenance, and additionally ensure data are machine-readable (e.g., R and Python languages). ALSDA integration with GeneLab and its analysis portals enable higher-order physiological-level datasets be mined in conjunction with -omics datasets. As ALSDA physiological-level datasets are published (micro-computed tomography, histology, intraocular pressure, hormonal assays, immunostaining, ultrasonography), the merging of hierarchical organizations of biological complexity from spaceflight will enable new knowledge discovery approaches.

Ryan T Scott↗

Sensor Data Management, Validation, Correction, and Provenance for Building Technologies

Oak Ridge National Laboratory (ORNL) conducts research on technologies that use a wide range of sensors to develop and characterize building energy performance. The management of high-resolution sensor data, analysis, and tracing lineage of such activities is challenging. Missing or corrupt data due to sensor failure, fouling, drifting, calibration error, or data logger failure is another issue. This paper focuses on sensor data management, validation, correction, and provenance to combat these issues, ensuring complete and accurate sensor datasets for building technologies applications and research. The design and development of two integrated software products are discussed: Sensor Data Validation and Correction (SensorDVC) and the Provenance Data Management System (ProvDMS) platform.

Data management↗

Provenance–aware workflow for data quality management and improvement for large continuous scientific data streams

Data quality assessment, management and improvement is integral part of any big data intensive scientific research to ensure accurate, reliable and reproducible science discoveries. The task of maintaining the quality of data, however, is non-trivial and pose a challenge for a program like Department of Energy’s Atmospheric Radiation Measurement (ARM) that collects data from hundreds of instruments across the world,and distributes thousands of streaming data products that are continuously growing in near-real-time in an archive 1.7 Petabyte in size and growing. We present a computational data processing workflow to collect the data quality issue via an easy and intuitive web-based portal that allows reporting of any quality issues for any site, facility or instruments at a granularity down to individual variable in the data files. Portal allows instrument specialists and scientists to provide corrective actions in form of symbolic equation. A parallel processing framework applies the data improvement to large volume of data in efficient parallel environment, while optimizing data transfer and file I/O operations. Corrected files are systematically versioned and archived. A provenance tracking module tracks and records any change made to the data during its entire life cycle which are communicated transparently to the scientific users. Developed in Python using open source technologies, software architecture enables efficient and fast management and improvement of data in an operational data center environment.

Kumar, Jitendra↗

Data and Communications in Basic Energy Sciences: Creating a Pathway for Scientific Discovery

This report is based on the Department of Energy (DOE) Workshop on “Data and Communications in Basic Energy Sciences: Creating a Pathway for Scientific Discovery” that was held at the Bethesda Marriott in Maryland on October 24-25, 2011. The workshop brought together leading researchers from the Basic Energy Sciences (BES) facilities and Advanced Scientific Computing Research (ASCR). The workshop was co-sponsored by these two Offices to identify opportunities and needs for data analysis, ownership, storage, mining, provenance and data transfer at light sources, neutron sources, microscopy centers and other facilities. Their charge was to identify current and anticipated issues in the acquisition, analysis, communication and storage of experimental data that could impact the progress of scientific discovery, ascertain what knowledge, methods and tools are needed to mitigate present and projected shortcomings and to create the foundation for information exchanges and collaboration between ASCR and BES supported researchers and facilities. The workshop was organized in the context of the impending data tsunami that will be produced by DOE’s BES facilities. Current facilities, like SLAC National Accelerator Laboratory’s Linac Coherent Light Source, can produce up to 18 terabytes (TB) per day, while upgraded detectors at Lawrence Berkeley National Laboratory’s Advanced Light Source will generate ~10TB per hour. The expectation is that these rates will increase by over an order of magnitude in the coming decade. The urgency to develop new strategies and methods in order to stay ahead of this deluge and extract the most science from these facilities was recognized by all. The four focus areas addressed in this workshop were: Workflow Management - Experiment to Science: Identifying and managing the data path from experiment to publication. Theory and Algorithms: Recognizing the need for new tools for computation at scale, supporting large data sets and realistic theoretical models. Visualization and Analysis: Supporting near-real-time feedback for experiment optimization and new ways to extract and communicate critical information from large data sets. Data Processing and Management: Outlining needs in computational and communication approaches and infrastructure needed to handle unprecedented data volume and information content. It should be noted that almost all participants recognized that there were unlikely to be any turn-key solutions available due to the unique, diverse nature of the BES community, where research at adjacent beamlines at a given light source facility often span everything from biology to materials science to chemistry using scattering, imaging and/or spectroscopy. However, it was also noted that advances supported by other programs in data research, methodologies, and tool development could be implemented on reasonable time scales with modest effort. Adapting available standard file formats, robust workflows, and in-situ analysis tools for user facility needs could pay long-term dividends. Workshop participants assessed current requirements as well as future challenges and made the following recommendations in order to achieve the ultimate goal of enabling transformative science in current and future BES facilities: Theory and analysis components should be integrated seamlessly within experimental workflow. Develop new algorithms for data analysis based on common data formats and toolsets. Move analysis closer to experiment. Move the analysis closer to the experiment to enable real-time (in-situ) streaming capabilities, live visualization of the experiment and an increase of the overall experimental efficiency. Match data management access and capabilities with advancements in detectors and sources. Remove bottlenecks, provide interoperability across different facilities/beamlines and apply forefront mathematical techniques to more efficiently extract science from the experiments. This workshop report examines and reviews the status of several BES facilities and highlights the successes and shortcomings of the current data and communication pathways for scientific discovery. It then ascertains what methods and tools are needed to mitigate present and projected data bottlenecks to science over the next 10 years. The goal of this report is to create the foundation for information exchanges and collaborations among ASCR and BES supported researchers, the BES scientific user facilities, and ASCR computing and networking facilities. To jumpstart these activities, there was a strong desire to see a joint effort between ASCR and BES along the lines of the highly successful Scientific Discovery through Advanced Computing (SciDAC) program in which integrated teams of engineers, scientists and computer scientists were engaged to tackle a complete end-to-end workflow solution at one or more beamlines, to ascertain what challenges will need to be addressed in order to handle future increases in data

97 MATHEMATICS AND COMPUTING↗

Formal Provenance Representation of the Data and Information Supporting the National Climate Assessment

The Global Change Information System (GCIS) provides a framework for the formal representation of structured metadata about data and information about global change. The pilot deployment of the system supports the National Climate Assessment (NCA), a major report of the U.S. Global Change Research Program (USGCRP). A consumer of that report can use the system to browse and explore that supporting information. Additionally, capturing that information into a structured data model and presenting it in standard formats through well defined open inter- faces, including query interfaces suitable for data mining and linking with other databases, the information becomes valuable for other analytic uses as well.

Provenance↗

Application of Bayesian Classification to Content-Based Data Management

The high volume of Earth Observing System data has proven to be challenging to manage for data centers and users alike. At the Goddard Earth Sciences Distributed Active Archive Center (GES DAAC), about 1 TB of new data are archived each day. Distribution to users is also about 1 TB/day. A substantial portion of this distribution is MODIS calibrated radiance data, which has a wide variety of uses. However, much of the data is not useful for a particular user's needs: for example, ocean color users typically need oceanic pixels that are free of cloud and sun-glint. The GES DAAC is using a simple Bayesian classification scheme to rapidly classify each pixel in the scene in order to support several experimental content-based data services for near-real-time MODIS calibrated radiance products (from Direct Readout stations). Content-based subsetting would allow distribution of, say, only clear pixels to the user if desired. Content-based subscriptions would distribute data to users only when they fit the user's usability criteria in their area of interest within the scene. Content-based cache management would retain more useful data on disk for easy online access. The classification may even be exploited in an automated quality assessment of the geolocation product. Though initially to be demonstrated at the GES DAAC, these techniques have applicability in other resource-limited environments, such as spaceborne data systems.

Lynnes, Christopher↗