Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data provenance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Leveraging The Open Provenance Model as a Multi-Tier Model for Global Climate Research

Global climate researchers rely upon many forms of sensor data and analytical methods to help profile subtle changes in climate conditions. The U.S. Department of Energy's Atmospheric Radiation Measurement (ARM) program provides researchers with curated Value Added Products (VAPs) resulting from continuous sensor data streams, data fusion, and modeling. The ARM operations staff and software development teams (data producers) rely upon a number of techniques to ensure strict quality control (QC) and quality assurance (QA) standards are maintained. Climate researchers (data consumers) are highly interested in obtaining as much provenance (data quality, data pedigree) as possible to establish data trustworthiness. Currently all the provenance is not easily attainable or identifiable without significant efforts to extract and piece together information from configuration files, log files, codes, and status information from ARM databases. The need for a formalized approach to managing provenance became paramount with the planned addition of 120 new instruments, new data products, and data collection scaling to half a terabyte daily. Last year our research identified the need for a multi-tier provenance model to enable the data consumer easy access to the provenance for their data. This year we are leveraging the Open Provenance Model as a foundational construct that serves the needs of both the VAP producers and consumers, we are organizing the provenance in different tiers of granularity to model VAP lineage, causality at the component level within a VAP, and the causality for each time step as samples are being assembled within the VAP. This paper shares our implementation strategy and how the ARM operations staff and the climate research community can greatly benefit from this approach to more effectively assess and quantify VAP provenance.

54 ENVIRONMENTAL SCIENCES↗

NASA’s Data Preservation Strategies for Ensuring Ongoing Use and Access: Leveraging the Capabilities at the NASA DAACs

As NASA prepares for the follow on to EOS through the deployment of many new space and airborne-based assets envisioned under the ESO initiative, the need for ensuring user access to the very large and diverse archive of past missions remains a critical endeavor for current and future research. The ESDIS team has been working for many years on developing strategies and implementations that will make the linkage between the data held in our archives and the emerging missions possible. The focus on how best to preserve these past data has proven to be a collaborative effort, and one that may well benefit from the adoption of innovative technical tools that enable organizational efficiencies. This poster reveals both aspects of NASA’s EOS data repository strategy by discussing the collaborative community efforts of development of ISO 19165-2 – an international standard supported by a team from many countries around the world, the use by EOSDIS DAACs of a Preservation Content Specification, and ongoing work with mission teams to ensure mission data and all associated information are preserved and available for current and future use.

Francis Lindsay↗

Approach to Managing MeaSURES Data at the GSFC Earth Science Data and Information Services Center (GES DISC)

A major need stated by the NASA Earth science research strategy is to develop long-term, consistent, and calibrated data and products that are valid across multiple missions and satellite sensors. (NASA Solicitation for Making Earth System data records for Use in Research Environments (MEaSUREs) 2006-2010) Selected projects create long term records of a given parameter, called Earth Science Data Records (ESDRs), based on mature algorithms that bring together continuous multi-sensor data. ESDRs, associated algorithms, vetted by the appropriate community, are archived at a NASA affiliated data center for archive, stewardship, and distribution. See http://measures-projects.gsfc.nasa.gov/ for more details. This presentation describes the NASA GSFC Earth Science Data and Information Services Center (GES DISC) approach to managing the MEaSUREs ESDR datasets assigned to GES DISC. (Energy/water cycle related and atmospheric composition ESDRs) GES DISC will utilize its experience to integrate existing and proven reusable data management components to accommodate the new ESDRs. Components include a data archive system (S4PA), a data discovery and access system (Mirador), and various web services for data access. In addition, if determined to be useful to the user community, the Giovanni data exploration tool will be made available to ESDRs. The GES DISC data integration methodology to be used for the MEaSUREs datasets is presented. The goals of this presentation are to share an approach to ESDR integration, and initiate discussions amongst the data centers, data managers and data providers for the purpose of gaining efficiencies in data management for MEaSUREs projects.

Vollmer, Bruce↗

AiiDA: automated interactive infrastructure and database for computational science

Computational science has seen in the last decades a spectacular rise in the scope, breadth, and depth of its efforts. Notwithstanding this prevalence and impact, it is often still performed using the renaissance model of individual artisans gathered in a workshop, under the guidance of an established practitioner. Great benefits could follow instead from adopting concepts and tools coming from computer science to manage, preserve, and share these computational efforts. We illustrate here our paradigm sustaining such vision, based around the four pillars of Automation, Data, Environment, and Sharing. We then discuss its implementation in the open-source AiiDA platform (http://www.aiida.net), that has been tuned first to the demands of computational materials science. AiiDA’s design is based on directed acyclic graphs to track the provenance of data and calculations, and ensure preservation and searchability. Remote computational resources are managed transparently, and automation is coupled with data storage to ensure reproducibility. Last, complex sequences of calculations can be encoded into scientific workflows. We believe that AiiDA’s design and its sharing capabilities will encourage the creation of social ecosystems to disseminate codes, data, and scientific workflows.

Materials Science↗

Analysis of ERS 1 synthetic aperture radar data of frozen lakes in northern Montana and implications for climate studies

Lakes that freeze each winter are good indicators of regional climate change if key parameters, such as freeze-up and breakup date and maximum ice thickness, are measured over a decade-scale time frame. Synthetic aperture radar (SAR) satellite data have proven to be especially useful for measurement of climatologically significant parameters characteristic of frozen lakes. In this paper, five lakes in Glacier National Park, Montana, have been studied both in the field and using Earth Remote-Sensing Satellite (ERS) 1 SAR data during the 1992-1993 winter. The lakes are characterized by clear ice, sometimes with tubular or rounded bubbles, and often with a layer of snow ice on top of the clear ice. They are also often snow covered. Freeze-up is detected quite easily using ERS 1 SAR data as soon as a thin layer of ice forms. The effect of snow ice on the backscatter is thought to be significant but is, as yet, undetermined. On the five lakes studied, relative backscatter was found to increase with ice thickness until a maximum was reached in February. Breakup, an often ill-defined occurrence, is difficult to detect because surface water causes the SAR signal to be absorbed, thus masking the ice below. Comparison of the bubble structure of thaw lakes in northern Alaska with lakes in northern Montana has shown that the ice structure is quite different, and this difference may contribute to differential SAR signature evolution in the lakes of the two areas.

Hall, Dorothy K.↗

Enabling Modular Autonomous Feedback‐Loops in Materials Science through Hierarchical Experimental Laboratory Automation and Orchestration

Abstract Materials acceleration platforms (MAPs) operate on the paradigm of integrating combinatorial synthesis, high‐throughput characterization, automatic analysis, and machine learning. Within a MAP, one or multiple autonomous feedback loops may aim to optimize materials for certain functional properties or to generate new insights. The scope of a given experiment campaign is defined by the range of experiment and analysis actions that are integrated into the experiment framework. Herein, the authors present a method for integrating many actions within a hierarchical experimental laboratory automation and orchestration (HELAO) framework. They demonstrate the capability of orchestrating distributed research instruments that can incorporate data from experiments, simulations, and databases. HELAO interfaces laboratory hardware and software distributed across several computers and operating systems for executing experiments, data analysis, provenance tracking, and autonomous planning. Parallelization is an effective approach for accelerating knowledge generation provided that multiple instruments can be effectively coordinated, which the authors demonstrate with parallel electrochemistry experiments orchestrated by HELAO. Efficient implementation of autonomous research strategies requires device sharing, asynchronous multithreading, and full integration of data management in experimental orchestration, which to the best of the authors’ knowledge, is demonstrated for the first time herein.

36 MATERIALS SCIENCE↗

Uncertainty Quantification, Sensitivity Analysis, and Data Assimilation for Nuclear Systems Simulation

Reliable evaluation of nuclear data will play a major role in reduction of nuclear systems simulation uncertainties via the use of advanced sensitivity analysis (SA), uncertainty quantification (UQ), and data assimilation (DA) methodologies. This follows since nuclear data have proven to constitute a major source of neutronics uncertainties. This paper will overview the use of the Efficient Subspace Method (ESM), developed at NCSU, to overcome one of the main deficiencies of existing methodologies for SA/UQ/DA, that is the ability to handle codes with large input and output (I/O) data streams, where neither the forward nor the adjoint approach alone are appropriate. We demonstrate the functionality of ESM for an LWR core, a boiling water reactor, and a fast reactor benchmark experiment, the ZPR6/7A assembly. This work demonstrates the capability of adjusting cross section data thereby providing guidance to cross section evaluation efforts by identification of key cross sections and associated energy ranges that contribute the most to the propagated core attributes uncertainties.

99 GENERAL AND MISCELLANEOUS↗

BASTet (Berkeley Analysis and Storage Toolkit)

BASTet is an advanced software library written in Python. BASTet serves as the analysis and storage library for the OpenMSI project. BASTet is an integrate framework for: i) storage of spectral imaging data, ii) storage of derived analysis data, iii) provenance of analyses, iv) integration and execution of analyses via complex workflows. BASTet implements the API for the HDF5 storage format used by OpenMSI. Analyses that are developed using BASTet benefit from direct integration with storage format, automatic tracking of provenance, and direct integration with command-line and workflow execution tools. BASTet also defines interfaces to enable developers to directly integrate their analysis with OpenMSI's web-based viewing infrastruture without having to know OpenMSI. BASTet also provides numerous helper classes and tools to assist with the conversion of data files, ease parallel implementation of analysis algorithms, ease interaction with web-based functions, description methods for data reduction. BASTet also includes detailed developer documentation, user tutorials, iPython notebooks, and other supporting documents.

Bowen, Benjamin↗

Simple, Scalable, Script-Based Science Processor (S4P)

The development and deployment of data processing systems to process Earth Observing System (EOS) data has proven to be costly and prone to technical and schedule risk. Integration of science algorithms into a robust operational system has been difficult. The core processing system, based on commercial tools, has demonstrated limitations at the rates needed to produce the several terabytes per day for EOS, primarily due to job management overhead. This has motivated an evolution in the EOS Data Information System toward a more distributed one incorporating Science Investigator-led Processing Systems (SIPS). As part of this evolution, the Goddard Earth Sciences Distributed Active Archive Center (GES DAAC) has developed a simplified processing system to accommodate the increased load expected with the advent of reprocessing and launch of a second satellite. This system, the Simple, Scalable, Script-based Science Processor (S42) may also serve as a resource for future SIPS. The current EOSDIS Core System was designed to be general, resulting in a large, complex mix of commercial and custom software. In contrast, many simpler systems, such as the EROS Data Center AVHRR IKM system, rely on a simple directory structure to drive processing, with directories representing different stages of production. The system passes input data to a directory, and the output data is placed in a "downstream" directory. The GES DAAC's Simple Scalable Script-based Science Processing System is based on the latter concept, but with modifications to allow varied science algorithms and improve portability. It uses a factory assembly-line paradigm: when work orders arrive at a station, an executable is run, and output work orders are sent to downstream stations. The stations are implemented as UNIX directories, while work orders are simple ASCII files. The core S4P infrastructure consists of a Perl program called stationmaster, which detects newly arrived work orders and forks a job to run the appropriate executable (registered in a configuration file for that station). Although S4P is written in Perl, the executables associated with a station can be any program that can be run from the command line, i.e., non-interactively. An S4P instance is typically monitored using a simple Graphical User Interface. However, the reliance of S4P on UNIX files and directories also allows visibility into the state of stations and jobs using standard operating system commands, permitting remote monitor/control over low-bandwidth connections. S4P is being used as the foundation for several small- to medium-size systems for data mining, on-demand subsetting, processing of direct broadcast Moderate Resolution Imaging Spectroradiometer (MODIS) data, and Quick-Response MODIS processing. It has also been used to implement a large-scale system to process MODIS Level 1 and Level 2 Standard Products, which will ultimately process close to 2 TB/day.

Lynnes, Christopher↗

Impact of the Timing of a SAR Image Acquisition on the Calibration of a Flood Inundation Model

Synthetic Aperture Radar (SAR) data have proven to be a very useful source of information for the calibration of flood inundation models. Previous studies have focused on assigning uncertainties to SAR images in order to improve flood forecast systems (e.g. Giustarini et al. (2015) and Stephens et al. (2012)). This paper investigates whether the timing of a SAR acquisition of a flood has an important impact on the calibration of a flood inundation model. As no suitable time series of SAR data exists, we generate a sequence of consistent SAR images through the use of a synthetic framework. This framework uses two available ERS-2 SAR images of the study area, one taken during the flood event of interest, the second taken during a dry reference period. The obtained synthetic observations at different points in time during the flood event are used to calibrate the flood inundation model. The results of this study indicate that the uncertainty of the roughness parameters is lower when the model is calibrated with an image taken before rather than during or after the flood peak. The results also show that the error on the modeled extent is much lower when the model is calibrated with a pre-flood peak image than when calibrated with a near-flood peak or a post-flood peak image. It is concluded that the timing of the SAR image acquisition of the flood has a clear impact on the model calibration and consequently on the precision of the predicted flood extent.

Flood inundation↗

The Use of Sage Water Vapor Data for Investigating Climate Change Issues

SAGE water vapor data has proven valuable for addressing several of the important issues in climate change research. It has been used to investigate how the upper troposphere water vapor responds to warming and convection, a key question in understanding the water vapor feedback to anthropogenic global warming. In the case of summer versus winter differences, SAGE results showed that the upper tropospheric relative humidity remained approximately constant; this result was in general agreement with how a GCM handled the seasonal difference, and gave credence to the argument that the GCM was not overestimating the water vapor feedback associated with convection. In addition, the convection-water vapor relationship was investigated further using SAGE water vapor and ISCCP cloud data. The results showed that upper tropospheric drying did appear to occur simultaneously with deep convective events in the tropics, only to be replaced by moistening a few hours later, associated (most likely) with the reevaporation of moisture from anvil clouds. The total effect was, again, a moistening of the upper troposphere associated with convection. Calculation of the actual trends in upper tropospheric moisture is a potential goal for SAGE data usage; trends calculated with radiosonde data, or instruments calibrated with radiosonde data have the problem of the effect of changing radiosonde instruments. SAGE data have in effect been used to compare different radiosondes through comparisons, and could continue to do so. SAGE 3 should also help clarify the absolute accuracy of SAGE retrievals in the troposphere. and its consequences. Model results show that water vapor increases can help explain the observations of stratospheric cooling, along with increasing C02 and ozone reduction. SAGE has been shown to provide trends similar to those of some other satellite and in situ retrievals, with increasing water vapor over time. However, SAGE is impacted by aerosol contamination which must be removed from the data; approaches used in the past will be reviewed. The SAGE water vapor errors, if taken literally, would make it hard to justify using SAGE data for such trends, and the error bars must be investigated. Stratospheric water vapor increases, like tropospheric increases, would lead to noticeable warming at the surface, and their changes must be quantified.

Rind, D.↗

Application of Deep Learning on Integrating Prediction, Provenance, and Optimization

In this research, we investigated two approaches to detect job anoma- lies and/or contention for large scale computing efforts: 1. Preemptive job scheduling using binomial classification long short-term memory networks 2. Forecasting intra-node computing loads from the active jobs and addi- tional job(s) For approach 1, we achieved a 14% improvement in computational resources utilization and an overall classification accuracy of 85% on real tasks executed in a High Energy Physics computing workflow. For this paper, we present the preliminary results used in second approach.

Anomaly Detection, Belle II Computing, data proven↗

File level provenance tracking in CMS

The CMS off-line framework stores provenance information within CMS's standard ROOT event data files. The provenance information is used to track how each data product was constructed, including what other data products were read to do the construction. We will present how the framework gathers the provenance information, the efforts necessary to minimize the space used to store the provenance in the file and the tools that will be available to use the provenance.

99 GENERAL AND MISCELLANEOUS↗

Bicubic partially blended rational fractal surface for a constrained interpolation problem

This paper investigates some univariate and bivariate constrained interpolation problems using fractal interpolation functions. First, we obtain rational cubic fractal interpolation functions lying above a prescribed straight line. Using a transfinite interpolation via blending functions, we extend the properties of the univariate rational cubic fractal interpolation function to generate surfaces that lie above a plane. In particular, the constrained bivariate interpolation discussed herein includes a method to construct fractal interpolation surfaces that preserve positivity inherent in a prescribed data set. Uniform convergence of the bivariate fractal interpolant to the original function which generates the data is proven.

97 MATHEMATICS AND COMPUTING↗

Electromagnetic production of kaons on the nucleon

Studies of the electromagnetic production of strange quarks started in the 1950s as something of a curiosity that puzzled experimentalists and theorists alike. Eventually, a nascent understanding of these processes began to take shape through the first pioneering experiments dedicated to explore photo- and electroproduction that were carried out in the period from the 1950s to the 1980s. As the datasets increased, concomitant advances in theoretical models were realized. However, these initial studies also made clear that more precise data was essential to continue to move forward. A paradigm shift occurred in the 1990s with the development of second-generation facilities at ELSA, MAMI, SPring-8, and JLab. High-intensity, high duty-factor accelerators, coupled with novel detector systems and advances in computing and readout electronics, brought nuclear physics experiments forward by orders of magnitude in counting statistics compared to the first-generation efforts. This was an utter boon to strangeness physics investigations, and to date, more than 50 dedicated experiments in kaon photo- and electroproduction have been completed at facilities around the world, leading to a host of experimental observables that have enabled significant advances in the exploration of strongly interacting systems that decay via $s\bar{s}$ quark pair creation. These data have proven to be an essential complementary pathway to study the spectrum and structure of the excited states of the nucleon, and the search for missing and exotic baryon configurations. As well, investigations in these channels are requisite for exploring hypernuclear production as a probe of the $YN$ interaction and for studies of the electromagnetic form factors of strange mesons. This review was designed to provide the first-ever in-depth overview of both the experimental and theoretical progress in the field of the electromagnetic production of strangeness. This work looks back over 70 years of past developments, discusses ongoing work and near-term plans, and details future possibilities being considered for third-generation facilities. Extensive lists of the available datasets and theoretical models are provided, together with a comprehensive supporting bibliography of the field. Throughout this work, the primary impacts of these explorations are highlighted, along with connections to a wide range of related phenomenological applications. An important goal of this review is to provide a complete, (reasonably) self-contained guide into this field prepared at a level that is relevant for both new and seasoned scientists, whether experimentalists, phenomenologists, or theorists, to better understand what has been accomplished by so many dedicated folks-each building on what has come before-and to appreciate the exciting future potential for continued studies in this area.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Potential Foldamers Based on an ortho- Terphenyl Amino Acid

We describe the synthesis and characterization of a new class of oligomers built from a terphenyl-based amino acid. Here, these oligomeric amides are of interest because the adoption of specific conformations could potentially be driven by the coordinated formation of inter-residue hydrogen bonds and aromatic interactions. Although high-resolution structural data have proven inaccessible, circular dichroism and nuclear magnetic resonance studies suggest that the new oligomers fold concomitantly with discrete self-association in chloroform. Molecular simulations find no sign of folding as a monomer, and in fact suggest that hydrogen bond patterning is anti-cooperative, inhibiting longer helices.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A Phenological Legacy: Leafing and flowering data for lilacs and honeysuckles 1956-2014

The dataset is comprised of leafing and flowering data collected across the continental United States from 1956 to 2014 for purple common lilac (Syringa vulgaris), a cloned lilac cultivar (S. x chinensis Red Rothomagensis ) and two cloned honeysuckle cultivars (Lonicera tartarica Arnold Red and L. korolkowii Zabeli ). Applications of this observational dataset range from detecting regional weather patterns to understanding the impacts of global climate change on the onset of spring at the national scale. While minor changes in methods have occurred over time, and some documentation is lacking, outlier analyses identified fewer than 3% of records as unusually early or late. Lilac and honeysuckle phenology data have proven robust in both model development and climatic research.

54 ENVIRONMENTAL SCIENCES↗

Workflows Community Summit 2022: A Roadmap Revolution

Scientific workflows have become integral tools in broad scientific computing use cases. Science discovery is increasingly dependent on workflows to orchestrate large and complex scientific experiments that range from the execution of a cloud-based data preprocessing pipeline to multi-facility instrument-to-edge-to-HPC computational workflows. Given the changing landscape of scientific computing (often referred to as a computing continuum) and the evolving needs of emerging scientific applications, it is paramount that the development of novel scientific workflows and system functionalities seek to increase the efficiency, resilience, and pervasiveness of existing systems and applications. Specifically, the proliferation of machine learning/artificial intelligence (ML/AI) workflows, need for processing large-scale datasets produced by instruments at the edge, intensification of near real-time data processing, support for long-term experiment campaigns, and emergence of quantum computing as an adjunct to HPC, have significantly changed the functional and operational requirements of workflow systems. Workflow systems now need to, for example, support data streams from the edge-to-cloud-to-HPC, enable the management of many small-sized files, allow data reduction while ensuring high accuracy, orchestrate distributed services (workflows, instruments, data movement, provenance, publication, etc.) across computing and user facilities, among others. Further, to accelerate science, it is also necessary that these systems implement specifications/standards and APIs for seamless (horizontal and vertical) integration between systems and applications, as well as enable the publication of workflows and their associated products according to the FAIR principles.

97 MATHEMATICS AND COMPUTING↗