Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data provenance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

An Examination of Causes of Multi-Wire Bias during SLD Instrumentation Testing in the IRT

The Icing Research Tunnel (IRT) at NASA Glenn Research Center has been using the SEA Multi-Element Sensor (aka, the multi-wire) as its primary liquid water content instrument since 2011. Recent tests in the IRT with four different multi-wire probes (i.e., different serial numbers) have indicated situations that can result in biasing of the probe data. The reference data used in this report are multi-wire data that have been proven to agree well with the icing blade in low-impingement rate conditions. Data from these recent tests suggest that the presence of a canister probe located 4.45 ft axially downstream and 2.5 ft to the side may have caused multi-wire data to read around 5% low at an airspeed of 150 kts. Additionally, two probes of different serial numbers were observed to measure 5-15% lower than the reference probes, and the causes have been linked to excessive solder on the sensing element and to an out of date element calibration. For these reasons, the authors suggest the community be aware of the possible aerodynamic influences on the probe inlet and outlet, as well as the importance of regular calibration and maintenance of the probes.

aircraft icing↗

Enabling Re-executable Workflows with Near-realtime Visualization, Provenance Capture and Advanced Querying for Mass Spectrometry Data

Mass spectrometry imaging (MSI) enables simultaneous spatially-resolved analysis of numerous ionizable molecules on a sample surface. The current analysis pipeline involves analyzing the data coming off the instrument using a tool called MSI Quickview and saving the results onto a storage drive before moving on to the next experiment. While this works well for single datasets, there is a demand for more scalable, flexible workflows that are re-executable across datasets, support extensive querying and ease collaboration. Here, we present a workflow that moves data analysis from a mere desktop application for single experiments to a more general capability

Re-executable workflow↗

Hyperplane decision trees as piecewise linear surrogate models for chemical process design

Recent trends in chemical engineering research point towards an increasing reliance on data-driven modeling approaches. Neural networks, for instance, have proven to be accurate when data is plentiful and high-dimensional, but in many cases, they require computationally-intensive training procedures. Here, in this work, we describe hyperplane decision trees (HT) as a highly expressive and low-compute machine learning model architecture. These models are locally linear and have linear decision boundaries, resulting in a piecewise linear model of the data. This property allows them to be converted into mixed-integer linear constraints which can be globally optimized. Our open-source PyTorch implementation of this method is a fast, flexible, and accessible way to build accurate piecewise linear models of data.

Decision trees↗

Data from: "Lab and Field Warming Similarly Advance Germination Date and Limit Germination Rate for High and Low Elevation Provenances of Two Widespread Subalpine Conifers"

This archive contains data used to draw conclusions in “Lab and Field Warming Similarly Advance Germination Date and Limit Germination Rate for High and Low Elevation Provenances of Two Widespread Subalpine Conifers”, by Kueppers et al. 2017. Cone collection and field experiments were conducted on Niwot Ridge, Colorado, with the latter completed within the Alpine Treeline Warming Experiment (ATWE). Laboratory germination experiments were conducted at the University of California Merced. Geospatial files are also provided for additional geographical context.There are four data file formats in this package: comma-separated values (.csv), Microsoft Excel (.xlsx), keyhole markup language (.kml), and ESRI shapefile (.shp). The latter two are geospatial file formats. .csv files can be opened using any simple text editor program such as TextEdit (Mac) and Notepad (Windows), and .xlsx files can be opened using Microsoft Excel or Google Sheets. The .kml file contains coordinate data, and is compatible with Google Earth and Google Maps. ESRI shapefiles (.shp) are compatible with any geospatial software able to read the file type, such as QGIS and ESRI’s ArcGIS Suite. This data user’s guide is available in as a .pdf, and can be opened using software such as Adobe Acrobat Reader.-----------------------------------------------------------------------------------------------------------------------------------Accurately predicting upslope shifts in subalpine tree ranges with warming requires understanding how future forest populations will be affected by climate change, as these are the seed sources for new tree line and alpine populations. Early life history stages are particularly sensitive to climate and are also influenced by genetic variation among populations. We tested the climate sensitivity of germination and initial development for two widely distributed subalpine conifers, using controlled-environment growth chambers with one temperature regime from subalpine forest in the Colorado Rocky Mountains and one 5 °C warmer, and two soil moisture levels. We tracked germination rate and timing, rate of seedling development, and seedling morphology for two seed provenances separated by ~300 m elevation. Warming advanced germination timing and initial seedling development by a total of ~2 weeks, advances comparable to mean differences between provenances. Advances were similar for both provenances and species; however, warming reduced the overall germination rate, as did low soil moisture, only for Picea engelmannii. A three-year field warming and watering experiment planted with the same species and provenances yielded responses qualitatively consistent with the lab trials. Together these experiments indicate that in a warmer, drier climate, P. engelmannii germination, and thus regeneration, could decline, which could lead to declining subalpine forest populations, while Pinus flexilis forest populations could remain robust as a seed source for upslope range shifts.

54 ENVIRONMENTAL SCIENCES↗

Prescriptive provenance for streaming analysis of workflows at scale

Abstract- We extend our approach capturing and relating the provenance and performance metrics of computational workflows as a diagnostic tool for runtime optimization and placement. One important challenge is the volume of extracted data, both for performance metrics and provenance, even when specifying filters and focusing on quantities of interest in a simulation. We reduce this data by performing anomaly detection on streaming data and store provenance for the detected anomalies, an approach we call prescriptive provenance. This paper discusses the Chimbuko architecture enabling the approach. We present the use of a protein structure propagation workflow based on NWChemEx. We are testing algorithms for anomaly detection and present preliminary results here obtained with Local Outlier Factor. While scaling remains a challenge, these results show that our robust Chimbuko architecture for streaming analysis with prescriptive provenance is a promising approach.

97 MATHEMATICS AND COMPUTING↗

Fully Employing Software Inspections Data

Software inspections provide a proven approach to quality assurance for software products of all kinds, including requirements, design, code, test plans, among others. Common to all inspections is the aim of finding and fixing defects as early as possible, and thereby providing cost savings by minimizing the amount of rework necessary later in the lifecycle. Measurement data, such as the number and type of found defects and the effort spent by the inspection team, provide not only direct feedback about the software product to the project team but are also valuable for process improvement activities. In this paper, we discuss NASA's use of software inspections and the rich set of data that has resulted. In particular, we present results from analysis of inspection data that illustrate the benefits of fully utilizing that data for process improvement at several levels. Examining such data across multiple inspections or projects allows team members to monitor and trigger cross project improvements. Such improvements may focus on the software development processes of the whole organization as well as improvements to the applied inspection process itself.

Shull, Forrest↗

Using Domain Requirements to Achieve Science-Oriented Provenance

Using Domain Requirements to Achieve Science-Oriented Provenance Eric Stephan1 Todd Halter1 Terence Critchlow1 Paulo Pinheiro da Silva2 Leonardo Salayandia2 1 Pacific Northwest National Laboratory, Richland WA, USA 2 University of Texas at El Paso, El Paso TX, USA Abstract. The US Department of Energy (DOE) Atmospheric Radi- ation Measurement Program (ARM) is adopting the use of formalized provenance to support observational data products produced by ARM operations and relied upon by researchers. Because of the diversity of needs in the climate community provenance will need to be conveyed in a domain-oriented context. This paper explores a use case where semantic abstract workflows (SAW) are employed as a means to filter, aggregate, and contextually describe the historical events responsible for the ARM data product the scientist is relying upon.

54 ENVIRONMENTAL SCIENCES↗

ProvLight: Efficient Workflow Provenance Capture on the Edge-to-Cloud Continuum

Modern scientific workflows require hybrid infrastructures combining numerous decentralized resources on the IoT/Edge interconnected to Cloud/HPC systems (aka the Computing Continuum) to enable their optimized execution. Understanding and optimizing the performance of such complex Edge-to-Cloud workflows is challenging. Capturing the provenance of key performance indicators, with their related data and processes, may assist in understanding and optimizing workflow executions. However, the capture overhead can be prohibitive, particularly in resource-constrained devices, such as the ones on the IoT/Edge.To address this challenge, based on a performance analysis of existing systems, we propose ProvLight, a tool to enable efficient provenance capture on the IoT/Edge. We leverage simplified data models, data compression and grouping, and lightweight transmission protocols to reduce overheads. We further integrate ProvLight into the E2Clab framework to enable workflow provenance capture across the Edge-to-Cloud Continuum. This integration makes E2Clab a promising platform for the performance optimization of applications through reproducible experiments.We validate ProvLight at a large scale with synthetic workloads on 64 real-life IoT/Edge devices in the FIT IoT LAB testbed. Evaluations show that ProvLight outperforms state-of-the-art systems like ProvLake and DfAnalyzer in resource-constrained devices. ProvLight is 26—37x faster to capture and transmit provenance data; uses 5—7x less CPU; 2x less memory; transmits 2x less data; and consumes 2—2.5x less energy. ProvLight [1] and E2Clab [2] are available as open-source tools.

Rosendo, Daniel↗

UPC++: A PGAS extension for C++

Partitioned Global Address Space (PGAS) languages are convenient for expressing algorithms with large, random-access data, and they have proven to provide high performance and scalability through lightweight one-sided communication and locality control. While very convenient for moving data around the system, PGAS languages have taken different views on the model of computation, with the static Single Program Multiple Data (SPMD) model providing the best scalability. In this paper we present UPC++, a PGAS extension for C++ that has three main objectives: 1) to provide an object-oriented PGAS programming model in the context of the popular C++ language, 2) to add useful parallel programming idioms unavailable in UPC, such as asynchronous remote function invocation and multidimensional arrays, to support complex scientific applications, 3) to offer an easy on-ramp to PGAS programming through interoperability with other existing parallel programming systems (e.g., MPI, OpenMP, CUDA). We implement UPC++ with a 'compiler-free' approach using C++ templates and runtime libraries. We borrow heavily from previous PGAS languages and describe the design decisions that led to this particular set of language features, providing significantly more expressiveness than UPC with very similar performance characteristics. We evaluate the programmability and performance of UPC++ using five benchmarks on two representative supercomputers, demonstrating that UPC++ can deliver excellent performance at large scale up to 32K cores while offering PGAS productivity features to C++ applications.

Zheng, Y↗

Eye Safe, Visible Wavelength Lidar Systems: Design and Operational Advances, Results and Potential

In the early nineties the first of the eye safe visible wavelength lidar systems known now as Micro Pulse Lidar (MPL) became operational. The important advance of the design was a system that, unlike most existing lidar, operated at eye safe energy densities and could thus operate unattended for full time monitoring. Since that time there have been many dozens of these systems produced and applied for full time profiling of atmospheric cloud and aerosol structure. There is currently an observational network of MPL sites to support global climate research. In thc course of application of these instruments there have been significant improvements in the, design and performance of the systems. In the last half decade particularly there has been significant application and technical development of MPL systems. In this paper we review progress. The current MPL systems in use are all single wavelength systems designed for cloud and aerosol applications. For the cloud and aerosol applications, both lidar depolarization and multi wavelength measurements have significant applications. These can be accomplished with the MPL, approach. The main current challenge for the lidar network activity are in the area of the reliability, repeatability and efficiency of data processing. The network makes use of internet data downloads and automated processing. The heights of all cloud and aerosol layers are needed. The recent emphasis has been in operationally deriving aerosol extinction cross section. Future emphasis will include adding cirrus optical parameters. For operational effectiveness, improvements to simplify routine data signal calibration are being researched. Overall the MPL systems have proven very effective. A large data base of results from globally distributed sites can be easily accessed through the internet. Applications have included atmospheric model development. Validation of current global satellite observations of aerosol and clouds, including now orbital lidar observations, was a primary goal for NASA. Although sampling issues require careful consideration, results have proven useful.

Spinhirne, James↗

Deriving Earth Science Data Analytics Requirements

Data Analytics applications have made successful strides in the business world where co-analyzing extremely large sets of independent variables have proven profitable. Today, most data analytics tools and techniques, sometimes applicable to Earth science, have targeted the business industry. In fact, the literature is nearly absent of discussion about Earth science data analytics. Earth science data analytics (ESDA) is the process of examining large amounts of data from a variety of sources to uncover hidden patterns, unknown correlations, and other useful information. ESDA is most often applied to data preparation, data reduction, and data analysis. Co-analysis of increasing number and volume of Earth science data has become more prevalent ushered by the plethora of Earth science data sources generated by US programs, international programs, field experiments, ground stations, and citizen scientists.Through work associated with the Earth Science Information Partners (ESIP) Federation, ESDA types have been defined in terms of data analytics end goals. Goals of which are very different than those in business, requiring different tools and techniques. A sampling of use cases have been collected and analyzed in terms of data analytics end goal types, volume, specialized processing, and other attributes. The goal of collecting these use cases is to be able to better understand and specify requirements for data analytics tools and techniques yet to be implemented. This presentation will describe the attributes and preliminary findings of ESDA use cases, as well as provide early analysis of data analytics toolstechniques requirements that would support specific ESDA type goals. Representative existing data analytics toolstechniques relevant to ESDA will also be addressed.

data analytics↗

SEARCHING FOR COMETS ON THE WORLD WIDE WEB: THE ORBIT OF 17P/HOLMES FROM THE BEHAVIOR OF PHOTOGRAPHERS

We performed an image search for 'Comet Holmes', using the Yahoo{exclamation_point} Web search engine, on 2010 April 1. Thousands of images were returned. We astrometrically calibrated-and therefore vetted-the images using the Astrometry.net system. The calibrated image pointings form a set of data points to which we can fit a test-particle orbit in the solar system, marginalizing over image dates and detecting outliers. The approach is Bayesian and the model is, in essence, a model of how comet astrophotographers point their instruments. In this work, we do not measure the position of the comet within each image, but rather use the celestial position of the whole image to infer the orbit. We find very strong probabilistic constraints on the orbit, although slightly off the Jet Propulsion Lab ephemeris, probably due to limitations of our model. Hyperparameters of the model constrain the reliability of date meta-data and where in the image astrophotographers place the comet; we find that {approx}70% of the meta-data are correct and that the comet typically appears in the central third of the image footprint. This project demonstrates that discoveries and measurements can be made using data of extreme heterogeneity and unknown provenance. As the size and diversity of astronomical data sets continues to grow, approaches like ours will become more essential. This project also demonstrates that the Web is an enormous repository of astronomical information, and that if an object has been given a name and photographed thousands of times by observers who post their images on the Web, we can (re-)discover it and infer its dynamical properties.

79 ASTRONOMY AND ASTROPHYSICS↗

A new CDF model for data movement based on SRM

Being a large international collaboration established well before the full development of the Grid as the main computing tool for High Energy Physics, CDF has recently changed and improved its computing model, decentralizing some parts of it in order to be able to exploit the rising number of distributed resources available nowadays. Despite those efforts, while the large majority of CDF Monte Carlo production has moved to the Grid, data processing is still mainly performed in dedicated farms hosted at FNAL, requiring a centralized management of data and Monte Carlo samples needed for physics analysis. This rises the question on how to manage the transfer of produced Monte Carlo samples from remote Grid sites to FNAL in an efficient way; up to now CDF has relied on a non scalable centralized solution based on dedicated data servers accessed through rcp protocol, which has proven to be unsatisfactory. A new data transfer model has been designed that uses SRMs as local caches for remote Monte Carlo production sites, interfaces them with SAM, the experiment data catalog, and finally realizes the file movement exploiting the features provided by the data catalog transfer layer. We describe here the model and its integration within the current CDF computing architecture.

99 GENERAL AND MISCELLANEOUS↗

Packaging data products using data grid middleware for Deep Space Mission Systems

We are investigating a distributed computing infrastructure for releasing science products out of the mission pipeline. We are creating a DSMS Product Service and DSMS Data Packaging Service based on the OODT middleware, a Data Grid middleware technology that has proven successful in providing access to heterogeneous, disparate data, and federating that data using a common data model.

Deep Space Mission Systems↗

A summary of ERTS data applications in Alaska

ERTS has proven to be an exceedingly useful tool for the preparation of urgently needed resource surveys in Alaska. For this reason the wide utilization of ERTS data by federal, state and industrial agencies in Alaska is increasingly directed toward the solution of operational problems in resource inventories, environmental surveys, and land use planning. Examples of some applications are discussed in connection with surveys of potential agricultural lands; mapping of predicted archaeological sites; permafrost terrain and aufeis mapping; snow melt enhancement from Prudhoe Bay roads; geologic interpretations correlated ith possible new petroleum fields, with earthquake activity, and with plate tectonic motion along the Denali fault system; hydrology in monitoring surging glaciers and the break-up characteristics of the Chena River watershed; sea-ice morphology correlated with marine mammal distribution; and coastal sediment plume circulation patterns.

Miller, J. M.↗

Techniques for computer-aided analysis of ERTS-1 data, useful in geologic, forest and water resource surveys

Forestry, geology, and water resource applications were the focus of this study, which involved the use of computer-implemented pattern-recognition techniques to analyze ERTS-1 data. The results have proven the value of computer-aided analysis techniques, even in areas of mountainous terrain. Several analysis capabilities have been developed during these ERTS-1 investigations. A procedure to rotate, deskew, and geometrically scale the MSS data results in 1:24,000 scale printouts that can be directly overlayed on 7 1/2 minutes U.S.G.S. topographic maps. Several scales of computer-enhanced "false color-infrared" composites of MSS data can be obtained from a digital display unit, and emphasize the tremendous detail present in the ERTS-1 data. A grid can also be superimposed on the displayed data to aid in specifying areas of interest.

Hoffer, R. M.↗

Impact damage in filament wound composite bottles

Increasingly, composite materials are being used in advanced structural applications because of the significant weight savings they offer when compared to more traditional engineering materials. The higher cost of composites must be offset by the increased performance that results from reduced structural weight if these new materials are to be used effectively. At present, there is considerable interest in fabricating solid rocket motor cases out of composite materials, and capitalizing on the reduced structural weight to increase rocket performance. However, one of the difficulties that arises when composite materials are used is that composites can develop significant amounts of internal damage during low velocity impacts. Such low velocity impacts may be encountered in routine handling of a structural component like a rocket motor case. The ability to assess the reduction in structural integrity of composite motor cases that experience accidental impacts is essential if composite rocket motor cases are to be certified for manned flight. While experimental studies of the post-impact performance of filament wound composite motor cases haven been proven performed (2,3), scaling impact data from small specimens to full scale structures has proven difficult. If such a scaling methodology is to be achieved, an increased understanding of the damage processes which influence residual strength is required. The study described herein was part of an ongoing investigation of damage development and reduction of tensile strength in filament wound composites subjected to low velocity impacts. The present study, which focused on documenting the damage that develops in filament wound composites as a result of such impacts, included two distinct tasks. The first task was to experimentally assess impact damage in small, filament wound pressure bottles using x-ray radiography. The second task was to study the feasibility of using digital image processing techniques to assist in determining the 3-D distribution of damage from stereo x-ray pairs.

Highsmith, Alton L.↗

Data Compression Algorithm Architecture for Large Depth-of-Field Particle Image Velocimeters

A large depth-of-field particle image velocimeter (PIV) is designed to characterize dynamic dust environments on planetary surfaces. This instrument detects lofted dust particles, and senses the number of particles per unit volume, measuring their sizes, velocities (both speed and direction), and shape factors when the particles are large. To measure these particle characteristics in-flight, the instrument gathers two-dimensional image data at a high frame rate, typically >4,000 Hz, generating large amounts of data for every second of operation, approximately 6 GB/s. To characterize a planetary dust environment that is dynamic, the instrument would have to operate for at least several minutes during an observation period, easily producing more than a terabyte of data per observation. Given current technology, this amount of data would be very difficult to store onboard a spacecraft, and downlink to Earth. Since 2007, innovators have been developing an autonomous image analysis algorithm architecture for the PIV instrument to greatly reduce the amount of data that it has to store and downlink. The algorithm analyzes PIV images and automatically reduces the image information down to only the particle measurement data that is of interest, reducing the amount of data that is handled by more than 10(exp 3). The state of development for this innovation is now fairly mature, with a functional algorithm architecture, along with several key pieces of algorithm logic, that has been proven through field test data acquired with a proof-of-concept PIV instrument.

Bos, Brent↗