Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data provenance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21

Visual Data Comm: A Tool for Visualizing Data Communication in the Multi Sector Planner Study

Data comm is a new technology proposed in future air transport system as a potential tool to provide comprehensive data connectivity. It is a key enabler to manage 4D trajectory digitally, potentially resulting in improved flight times and increased throughput. Future concepts with data comm integration have been tested in a number of human-in-the-loop studies but analyzing the results has proven to be particularly challenging because future traffic environment in which data comm is fully enabled has assumed high traffic density, resulting in data set with large amount of information. This paper describes the motivation, design, current and potential future application of Visual Data Comm (VDC), a tool for visualizing data developed in Java using Processing library which is a tool package designed for interactive visualization programming. This paper includes an example of an application of VDC on data pertaining to the most recent Multi Sector Planner study, conducted at NASA s Airspace Operations Laboratory in 2009, in which VDC was used to visualize and interpret data comm activities

Lee, Hwasoo Eric↗

The X-windows interactive navigation data editor

A new computer program called the X-Windows Interactive Data Editor (XIDE) was developed and demonstrated as a prototype application for editing radio metric data in the orbit-determination process. The program runs on a variety of workstations and employs pull-down menus and graphical displays, which allow users to easily inspect and edit radio metric data in the orbit data files received from the Deep Space Network (DSN). The XIDE program is based on the Open Software Foundation OSF/Motif Graphical User Interface (GUI) and has proven to be an efficient tool for editing radio metric data in the navigation operations environment. It was adopted by the Magellan Navigation Team as their primary data-editing tool. Because the software was designed from the beginning to be portable, the prototype was successfully moved to new workstation environments. It was also itegrated into the design of the next-generation software tool for DSN multimission navigation interactive launch support.

Rinker, G. C.↗

Where Is the Provenance? Ethical Replicability and Reproducibility in GIScience and Its Critical Applications

As replicability and reproducibility (R&R) crises develop within emerging convergent inquiry, ethical use of provenance information is central to the establishment and preservation of trust in critical applications of GIScience and geospatial technologies. Today large volumes of geospatial data are generated at high velocity from satellite sensors and unmanned aircraft systems, citizen sensors, geolocation-based data services, global navigation satellite systems, and so on. The extensive use of these data for applications such as disaster and humanitarian response raises the issue of R&R from competing perspectives of location privacy and geospatial data quality. Although geospatial data can be integrated and linked with contextual information to identify individuals’ movements, steps taken to ensure privacy can complicate the multiuser development of high-quality geospatial workflows. Provenance information as digital records of historical (retrospective) and potential future (prospective) geospatial processes is often overlooked, misunderstood, or inadequately addressed. We explore the relationship between provenance information, location privacy, and geospatial data quality in the context of R&R with a focus on disaster analytics. Here, we argue that in the era of big data and deep learning, GIScientists and associated institutions bear greater responsibility both for geospatial workflow quality and for location privacy. Given vastly heterogenous computational landscapes, we provide practical recommendations for ethically driven provenance and R&R research and development within the GIScience community and beyond.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

US Astronomers Access to SIMBAD in Strasbourg, France

During the last year the US SIMBAD Gateway Project continued to provide services like user registration to the US users of the SIMBAD database in France. Currently there are over 4300 US users registered. We also provided user support by answering questions from users and handling requests for lost passwords when still necessary. Even though almost all users now access SIMBAD without a password, based on hostnames/IP addresses, there are still some users that need individual passwords. We continued to maintain the mirror copy of the SIMBAD database on a server at SAO. This allows much faster access for the US users. During the past year we moved this mirror to a faster server to improve access for the US users. We again supported a demonstration of the SIMBAD database at the meeting of the American Astronomical Society in January. We provided support for the demonstration activities at the SIMBAD booth. We paid part of the fee for the SIMBAD demonstration. We continued to improve the cross-linking between the SIMBAD project and the Astrophysics Data System. This cross-linking between these systems is very much appreciated by the users of both the SIMBAD database and the ADS Abstract Service. The mirror of the SIMBAD database at SAO makes this connection faster for the US astronomers. We exchange information between the ADS and SIMBAD on a daily basis. The close cooperation between the CDS in Strasbourg and SAO, facilitated by this project, is an important part of the astronomy-wide digital library initiative. It has proven to be a model in how different data centers can collaborate and enhance the value of their products by linking with other data centers. We continue this collaboration in order to provide better services to both the US and European astronomical community. This collaboration is even more important in light of the developments for the Virtual Observatory projects in the different countries.

Eichhorn, G.↗

US Gateway to SIMBAD Astronomical Database

During the last year the US SIMBAD Gateway Project continued to provide services like user registration to the US users of the SIMBAD database in France. Currently there are over 3400 US users registered. We also provide user support by answering questions from users and handling requests for lost passwords when still necessary. We have implemented in cooperation with the CDS SIMBAD project access to the SIMBAD database for US users on an Internet address basis. This allows most US users to access SIMBAD without having to enter passwords. We have maintained the mirror copy of the SIMBAD database on a server at SAO. This has allowed much faster access for the US users. We also supported a demonstration of the SIMBAD database at the meeting of the American Astronomical Society in January. We shipped computer equipment to the meeting and provided support for the demonstration activities at the SIMBAD booth. We continued to improve the cross-linking between the SIMBAD project and the Astrophysics Data System. This cross-linking between these systems is very much appreciated by the users of both the SIMBAD database and the ADS Abstract Service. The mirror of the SIMBAD database at SAO makes this connection faster for the US astronomers. We exchange information between the ADS and SIMBAD on a daily basis. The close cooperation between the CDS in Strasbourg and SAO, facilitated by this project, is an important part of the astronomy-wide digital library initiative called Urania. It has proven to be a model in how different data centers can collaborate and enhance the value of their products by linking with other data centers.

Eichhorn, G.↗

Managing and Servicing Physical Oceanographic Data at a NASA Distributed Active Archive Center

The NASA Earth Science Data Information Systems Project funds and operates 12 Distributed Active Archive Center(s) (DAAC) throughout the United States. Of these 12 centers, the Physical Oceanography DAAC (PO.DAAC) is committed to providing long term archival, distribution and stewardship for NASA physical oceanographic data, primarily derived from space-born satellite systems, but also including a growing set of recent and future in situ observations from the SPURS-1 and SPURS-2 campaigns. Notable NASA missions supported include: Seasat, TOPEX/Poseidon, NSCAT, QuikSCAT, ISS-RapidScat, Jason-1, Jason-2/OSTM, GRACE, Aquarius, GHRSST, and MODIS. The following interagency and international missions are also supported by PO.DAAC: AVHRR, Coriolis, DMSP, MetOp-A, MetOp-B, Oceansat-2. The PO.DAAC currently holds 525 datasets in public distribution, spanning the following observational parameters: sea surface temperature, sea surface salinity, ocean color, ocean surface currents, ocean surface wind speed, ocean surface wind direction, sea surface height, significant wave height, ocean water mass/thickness, and sea ice age. A hundred of these datasets are available in near-real-time. Datasets are distributed through a variety of open-source access protocols including FTP, OPeNDAP, and THREDDS. FTP will soon be phased out in favor of a recently introduced HTTPS PO.DAAC Drive interface that supports WebDAV and interoperable machine-to-machine communication. OPeNDAP supports remote data/metadata query, subset, and download. THREDDS provides the features of OPeNDAP with the additional feature of temporal aggregation. PO.DAAC also offers proprietary tools and services to further enhance the data discovery, visualization and analysis experience, including but not limited to: State of the Ocean, Web Services (data/metadata discovery and extraction), HiTIDE Level-2 subsetter, Live Access Server (LAS), Webification (w10nsci), and Rich Site Summary (RSS) Datacasting. To assist with provenance of datasets, PO.DAAC has implemented DOIs for the data it distributes so that they can be properly cited. There is a user forum and helpdesk that contains data recipes and via which users can get guidance. In summary, this presentation aims to provide a general overview of PO.DAAC’s web portal and data holdings along with a set of illustrative examples leading prospective data users into the practical utility of its tools and services.

Moroni, David F.↗

Data catalog project—A browsable, searchable, metadata system

Modern experiments are typically conducted by large, extended groups, where researchers rely on other team members to produce much of the data they use. The experiments record very large numbers of measurements that can be difficult for users to find, access and understand. Here. we are developing a system for users to annotate their data products with structured metadata, providing data consumers with a discoverable, browsable data index. Machine understandable metadata captures the underlying semantics of the recorded data, which can then be consumed by both programs, and interactively by users. Collaborators can use these metadata to select and understand recorded measurements. The data catalog project is a data dictionary and index which enables users to record general descriptive metadata, use cases and rendering information as well as providing them a transparent data access mechanism (URI). Users describe their diagnostic including references, text descriptions, units, labels, example data instances, author contact information and data access URIs. The list of possible attribute labels is extensible, but limiting the vocabulary of names increases the utility of the system. The data catalog is focused on the data products and complements process-based systems like the Metadata Ontology Provenance project [Greenwald, 2012; Schissel, 2015]. This system can be coupled with MDSplus to provide a simple platform for data driven display and analysis programs. Sites which use MDSplus can describe tree branches, and if desired create ‘processed data trees’ with homogeneous node structures for measurements. Sites not currently using MDSplus can either use the database to reference local data stores, or construct an MDSplus tree whose leaves reference the local data store. A data catalog system can provide a useful roadmap of data acquired from experiments or simulations making it easier for researchers to find and access important data and understand the meaning of the data and how it was obtained. This is particularly useful in research facilities that study the results of many different experiments or simulations and may not know the intricacies of the data organization in use where the data was generated. It is also possible to store a local copy of key data items in local MDSplus trees and then add processed data to the local catalog.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Automated Framework for Groundwater Monitoring Using DWT with LSTM and Transformers

Environmental monitoring is critical for safeguarding public health and ecological well-being. Traditional data structuring and workflow monitoring methods consume significant time and effort, hindering timely insights and effective decision-making. Our study addresses this challenge by presenting an AI framework that automates data cleaning, structuring, and modeling processes, specifically targeting applications in groundwater monitoring. By leveraging automation for data processing and model training, our framework establishes a novel and efficient paradigm for environmental monitoring, with its potential application to the vast network of over a hundred Department of Energy Environmental Management (DoE-EM) cleanup sites across the country. It analyzes data streams from a network of groundwater Internet-of-Things (IoT) sensors deployed at the Savannah River Site (SRS) for prediction modeling. This allows human experts to focus on analysis and decision-making, ultimately leading to better environmental outcomes.The framework employs multivariate time-series forecasting methods to study and model the behavior of varying chemical analytes. The continuous learning process is enabled by utilizing deep learning techniques. It allows the framework to become more nuanced in its analysis over time, adapting to the specific characteristics of the environmental site and the evolving nature of contaminant behavior. Deep learning models known for sequence modeling, LSTM, and Transformers are employed for time series forecasting. Data processing and structuring are essential components significantly impacting the final model's performance. This hypothesis was proven by presenting a comparative analysis of model performance with processed and unprocessed data. The feature engineering approach utilized was the Discrete Wavelet Transform, which works well with time series data.

Discrete Wavelet Transform (DWT)↗

Progress Towards AIRS Science Team Version-7 at SRT

The AIRS Science Team Version-6 retrieval algorithm is currently producing level-3 Climate Data Records (CDRs) from AIRS that have been proven useful to scientists in understanding climate processes. CDRs are gridded level-3 products which include all cases passing AIRS Climate QC. SRT has made significant further improvements to AIRS Version-6. At the last Science Team Meeting, we described results using SRT AIRS Version-6.22. SRT Version-6.22 is now an official build at JPL called 6.2.4. Version-6.22 results are significantly improved compared to Version-6, especially with regard to water vapor and ozone profiles. We have adapted AIRS Version-6.22 to run with CrIS/ATMS, at the Sounder SIPS which processed CrIS/ATMS data for August 2014. JPL AIRS Version-6.22 uses the Version-6 AIRS tuning coefficients. AIRS Version-6.22 has at least two limitations which must be improved before finalization of Version-7: Version-6.22 total O3 has spurious high values in the presence of Saharan dust over the ocean; and Version-6.22 retrieved upper stratospheric temperatures are very poor in polar winter. SRT Version-6.28 addresses the first concern. John Blaisdell ran the analog of AIRS Version-6.28 in his own sandbox at JPL for the 14th and 15th of every month in 2014 and all of July and October for 2014. AIRS Version-6.28a is hot off the presses and addresses the second concern.

CrIS↗

Data-Driven Selection of Tessellation Models Describing Polycrystalline Microstructures

Tessellation models have proven to be useful for the geometric description of grain microstructures in polycrystalline materials. With the use of a suitable tessellation model, the complex morphology of grains can be represented by a small number of parameters assigned to each grain, which not only entails a significant reduction in complexity, but also facilitates the investigation of certain geometric features of the microstructure. However, for a given set of microstructural data, the choice of a particular geometric model is traditionally based on researcher intuition. The model should provide a sufficiently good approximation to the data, while keeping the number of parameters small. In this paper, we discuss general aspects of the process of model selection and suggest several criteria for selecting an appropriate candidate from a certain set of tessellation models. The choice of candidate represents a trade-off between accuracy and complexity of the model. Here, the selected model is used solely to approximate given data samples, but it also provides guidance for developing stochastic tessellation models and generating virtual microstructures. Model fitting is carried out by simulated annealing, applied in a consistent manner to twelve different tessellation models.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

ProvSec: Open Cybersecurity System Provenance Analysis Benchmark Dataset with Labels

System provenance forensic analysis has been studied by a large body of research work. This area needs fine granularity data such as system calls along with event fields to track the dependencies of events. While prior work on security datasets has been proposed, we found a useful dataset of realistic attacks and details that are needed for high-quality provenance tracking is lacking. We created a new dataset of eleven vulnerable cases for system forensic analysis. It includes the full details of system calls including syscall parameters. Realistic attack scenarios with real software vulnerabilities and exploits are used. For each case, we created two sets of benign and adversary scenarios which are manually labeled for supervised machine-learning analysis. In addition, we present an algorithm to improve the data quality in the system provenance forensic analysis. We demonstrate the details of the dataset events and dependency analysis of our dataset cases.

97 MATHEMATICS AND COMPUTING↗

Techno-economic analysis of ethanol production from sugarcane bagasse using a Liquefaction plus Simultaneous Saccharification and co-Fermentation process

A techno-economic analysis was conducted for a simplified lignocellulosic ethanol production process developed and proven by the University of Florida at laboratory, pilot, and demonstration scales. Data obtained from all three scales of development were used with Aspen Plus to create models for an experimentally-proven base-case and 5 hypothetical scenarios. The model input parameters that differed among the hypothetical scenarios were time of L+SScF, enzyme loading, enzymatic conversion, solids loading, and overall process yield. The minimum ethanol selling price (MESP) varied between 50.38 and 62.72 US cents/L. The feedstock and the capital cost were the main contributors to the production cost, comprising between 23-28% and 40-49% of the MESP, respectively. A sensitivity analysis showed that overall ethanol yield had the greatest effect on the MESP. Here, these findings suggest that future efforts to increase the economic feasibility of a cellulosic ethanol process should focus on optimization for highest ethanol yield.

09 BIOMASS FUELS↗

Effects of physical aging on long-term behavior of composites

The HSCT plane, envisioned to have a lifetime of over 60,000 flight hours and to travel at speeds in excess of Mach 2, is the source of intensive study at NASA. In particular, polymer matrix composites are being strongly considered for use in primary and secondary structures due to their high strength to weight ratio and the options of property tailoring. However, an added difficulty in the use of polymer based materials is that their properties change significantly over time, especially at the elevated temperatures that will be experienced during flight, and prediction of properties based on irregular thermal and mechanical loading is extremely difficult. This study focused on one aspect of long-term polymer composite behavior: physical aging. When a polymer is cooled to below its glass transition temperature, the material is not in thermodynamic equilibrium and the free volume and enthalpy evolve over time to approach their equilibrium values. During this time, the mechanical properties change significantly and this change is termed physical aging. This work begins with a review of the concepts of physical aging on a pure polymer system. The effective time theory, which can be used to predict long term behavior based on short term data, is mathematically formalized. The effects of aging to equilibrium are proven and discussed. The theory developed for polymers is then applied first to a unidirectional composite, then to a general laminate. Comparison to experimental data is excellent. It is shown that the effects of aging on the long-term properties of composites can be counter-intuitive, stressing the importance of the development and use of a predictive theory to analyze structures.

Brinson, L. Catherine↗

SRT Status and Plans for Version-7

The AIRS Science Team Version-6 retrieval algorithm is currently producing level-3 Climate Data Records (CDRs) from AIRS that have been proven useful to scientists in understanding climate processes. CDRs are gridded level-3 products which include all cases passing AIRS Climate QC. SRT has made significant further improvements to AIRS Version-6. Research is continuing at SRT toward the development of AIRS Version-7. At the last Science Team Meeting, we described results using SRT AIRS Version-6.19. SRT Version-6.19 is now an official build at JPL called 6.2. SRTs latest version is AIRS Version-6.22. We have also adapted AIRS Version-6.22 to run with CrISATMS. AIRS Version-6.22 and CrIS Version- 6.22 both run now on JPL computers, but are not yet official builds. The main reason for finalization of Version-7, and using it in the relatively near future for the future processing and reprocessing of old AIRS data, is to produce even better CDRs for use by climate scientists. For this reason all results shown in this talk use only AIRS Climate QC.

CDRs↗

Rhenium Isotope Reconnaissance of Uranium Ore Concentrates

Exploration of natural isotopic variations of the element rhenium (Re) is in its infancy, with initial studies revealing isotopic fractionation in a variety of geological materials. Here, in this work, we investigate Re isotope variation as a new geochemical tool, given its redox-sensitive properties and affinity for organic matter and sulfides. In this work, Re abundance and isotope ratio data were collected from uranium ore concentrates (UOCs) across a variety of depositional ages, locations, geologic settings, and deposit types. Ore types from which the UOC were derived include sandstone, unconformity, and quartz-pebble (QP) conglomerate. To isolate Re from the U-rich matrix of UOCs, a new purification method utilizing DGA ion exchange resin was developed. We found that UOCs exhibit a wide range of Re isotope ratios, with sandstone ore-derived UOCs having the isotopically lightest values, QP conglomerate ore-derived UOCs having the heaviest, and unconformity ore-derived UOCs in between (with some overlap with sandstone UOCs). The Re isotope ratio range observed in UOCs extends previously reported values by more than a factor of two. Industrial processing (e.g., incomplete recovery of Re from ore, contamination, fractionation during processing) may play a role in the isotopic variability in the UOCs. However, systematic differences between ore types suggest that the depositional setting is a significant factor. For nuclear forensic investigations, Re isotopic compositions combined with data from other isotopic systems provide geochemical signatures that can aid in provenance assessment of UOCs. Regardless of the specific causes for the wide range of Re isotope ratios in UOCs, these initial data indicate Re is a promising tool for nuclear forensic investigations on samples from early in the nuclear fuel cycle.

58 GEOSCIENCES↗

Adding Data Management Services to Parallel File Systems

The objective of this project, called DAMASC for “Data Management in Scientific Computing”, is to coalesce data management with parallel file system management to present a declarative interface to scientists for managing, querying, and analyzing extremely large data sets efficiently and predictably. Managing extremely large data sets is a key challenge of exascale computing. The overhead, energy, and cost of moving massive volumes of data demand designs where computation is close to storage. In current architectures, compute/analysis clusters access data in a physically separate parallel file system and largely leave it scientist to reduce data movement. Over the past decades the high-end computing community has adopted middleware with multiple layers of abstractions and specialized file formats such as NetCDF-4 and HDF5. These abstractions provide a limited set of high-level data processing functions, but have inherent functionality and performance limitations: middleware that provides access to the highly structured contents of scientific data files stored in the (unstructured) file systems can only optimize to the extent that file system interfaces permit; the highly structured formats of these files often impedes native file system performance optimizations. We are developing Damasc, an enhanced high-performance file system with native rich data management services. Damasc will enable efficient queries and updates over files stored in their native byte-stream format while retaining the inherent performance of file system data storage via declarative queries and updates over views of underlying files. Damasc has four key benefits for the development of data-intensive scientific code: (1) applications can use important data-management services, such as declarative queries, views, and provenance tracking, that are currently available only within database systems; (2) the use of these services becomes easier, as they are provided within a familiar file-based ecosystem; (3) common optimizations, e.g., indexing and caching, are readily supported across several file formats, avoiding effort duplication; and (4) performance improves significantly, as data processing is integrated more tightly with data storage. Our key contributions are: SciHadoop which explores changes to MapReduce assumption by taking advantage of semantics of structured data while preserving MapReduce’s failure and resource management; DataMods which extends common abstractions of parallel file systems so they become programmable such that they can be extended to natively support a variety of data models and can be hooked into emerging distributed runtimes such as Stanford’s Legion; and Miso which combines Hadoop and relational data warehousing to minimize time to insight, taking into account the overhead of ingesting data into data warehousing.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

An Erlang-Based Front End Framework for Accelerator Controls

We have developed a new front-end framework for the ACNET control system in Erlang. Erlang is a functional programming language developed for real-time telecommunications applications. The primary task of the front-end software is to connect the control system with drivers collecting data from individual field bus devices. Erlang's concurrency and message passing support have proven well-suited for managing large numbers of independent ACNET client requests for front-end data. Other Erlang features which make it particularly well-suited for a front-end framework include fault-tolerance with process monitoring and restarting, real-time response,and the ability to change code in running systems. Erlang's interactive shell and dynamic typing make writing and running unit tests an easy part of the development process. Erlang includes mechanisms for distributing applications which we will use for deploying our framework to multiple front-ends, along with a configured s et of device drivers. We've developed Erlang code to use Fermilab's TCLK event distribution clock and Erlang's interface to C/C++ allows hardware-specific driver access.

43 PARTICLE ACCELERATORS↗

Architecture for interoperable software in biology

Understanding biological complexity demands a combination of high-throughput data and interdisciplinary skills. One way to bring to bear the necessary combination of data types and expertise is by encapsulating domain knowledge in software and composing that software to create a customized data analysis environment. To this end, simple flexible strategies are needed for interconnecting heterogeneous software tools and enabling data exchange between them. Drawing on our own work and that of others, we present several strategies for interoperability and their consequences, in particular, a set of simple data structures—list, matrix, network, table and tuple—that have proven sufficient to achieve a high degree of interoperability. We provide a few guidelines for the development of future software that will function as part of an interoperable community of software tools for biological data analysis and visualization.

59 BASIC BIOLOGICAL SCIENCES↗