Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Provenance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

System and method associated with expedient detection and reconstruction of cyber events in a compact scenario representation using provenance tags and customizable policy

A system associated with detecting a cyber-attack and reconstructing events associated with a cyber-attack campaign, is disclosed. The system performs various operations that include receiving an audit data stream associated with cyber events. The system identifies trustworthiness values in a portion of data associated with the cyber events and assigns provenance tags to the portion of the data based on the identified trustworthiness values. An initial visual representation is generated based on the assigned provenance tags to the portion of the data. The initial visual representation is condensed based on a backward traversal of the initial visual representation in identifying a shortest path from a suspect node to an entry point node. A scenario visual representation is generated that specifies nodes most relevant to the cyber events associated with the cyber-attack based on the identified shortest path.A corresponding method and computer-readable medium are also disclosed.

Source record↗

Towards Lightweight Data Integration Using Multi-Workflow Provenance and Data Observability

Modern large-scale scientific discovery requires multidisciplinary collaboration across diverse computing facilities, including High Performance Computing (HPC) machines and the Edge-to-Cloud continuum. Integrated data analysis plays a crucial role in scientific discovery, especially in the current AI era, by enabling Responsible AI development, FAIR, Reproducibility, and User Steering. However, the heterogeneous nature of science poses challenges such as dealing with multiple supporting tools, cross-facility environments, and efficient HPC execution. Building on data observability, adapter system design, and provenance, we propose MIDA: an approach for lightweight runtime Multi-workflow Integrated Data Analysis. MIDA defines data observability strategies and adaptability methods for various parallel systems and machine learning tools. With observability, it intercepts the dataflows in the background without requiring instrumentation while integrating domain, provenance, and telemetry data at runtime into a unified database ready for user steering queries. We conduct experiments showing end-to-end multi-workflow analysis integrating data from Dask and MLFlow in a real distributed deep learning use case for materials science that runs on multiple environments with up to 276 GPUs in parallel. We show near-zero overhead running up to 100,000 tasks on 1,680 CPU cores on the Summit supercomputer.

Santos Souza, Renan↗

A Simple, Revolutionary Approach to Waste Packaging Utilizing Proven Technology - 20201

Historically, removing hazardous waste from gloveboxes involves the use of bags as the primary containment method. This method can prove tedious, repetitive, and time consuming to ensure the process follows all safeguards required to contain the waste and mitigate any breach of containment. Layers of bags, yards of tape, and multiple filters are added to the waste stream in order to transfer the hazardous waste from the glovebox safely into the drum package. Then, finally, it can be shipped and disposed of. The process to cut, tie, and tape each bag for disposal involves manual effort by multiple operators. Following Administrative Safety Controls, good operators are able to safely perform those tasks; but, since the bag is the only confinement layer, incidents can occur. Operators, in some cases, are also at close proximity to this waste which may increase the risk of radiation exposure. Once the waste is transferred out through these bags, it is nearly impossible to remove packaged waste that may contain higher dosage levels, thus increasing drum disposal costs incurred by the facility. Exceeding the allowable radiation levels, and drums not fully filled, both contribute significantly to excess disposal costs. Central Research Laboratories (CRL), based in Red Wing Minnesota, has developed the Waste Drum Transfer System (WDTS) building upon the proven technology and containment of the Rapid Transfer Port (RTP) as an alternate, more efficient method of waste transfer. This solution consists of an RTP Alpha port mounted inside a glovebox and Beta assembly unit composed of a drum liner fitted inside a standard 55 gallon drum. The use of the Beta drum liner bypasses the typical use of bags, sleeves or tape to provide leak tight containment. The process to connect and disconnect a drum from the glovebox is achieved by the self-docking and rotating system controls of the RTP Alpha, all done with only one operator standing in an upright ergonomic position. This system also allows for greater flexibility in available Beta connection options, expanding the overall functionality of the system. This system can be disconnected and reconnected multiple times to maximize volume containment. This will minimize drum disposal costs and reduce the overall volume of waste output by more efficiently filling the drums. Radiation levels inside each drum can be monitored and adjusted more efficiently by operators utilizing the reconnect option. Minimal As Low As Reasonably Achievable (ALARA) levels are also achievable with this system. This paper will show how highly efficient packaging and disposal of hazardous waste can be accomplished in a cost-effective manner utilizing the proven RTP technology. Maximized operator productivity and safety, as well as minimized radiation dose exposure, are all attainable through the system's engineered controls. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Tracking Files Using the Kepler Provenance Framework

Workflow Management Systems (WFMS), such as Kepler, are proving to be an important tool in scientific problem solving. They can automate and manage complex processes and huge amounts of data produced by petascale simulations. Typically, the produced data need to be properly visualized and analyzed by scientists in order to achieve the desired scientific goals. Both run-time and post analysis may benefit from, even require, additional meta-data - provenance information. One of the challenges in this context is the tracking of the data files that can be produced in very large numbers during stages of the workflow, such as visualizations. The Kepler provenance framework collects all or part of the raw information flowing through the workflow graph. This information then needs to be further parsed to extract meta-data of interest. This can be done through add-on tools and algorithms. We show how to automate tracking specific information such as data files locations.

97 MATHEMATICS AND COMPUTING↗

Tracking Files in the Kepler Provenance Framework

Workflow Management Systems (WFMS), such as Kepler, are proving to be an important tool in scientific problem solving. They can automate and manage complex processes and huge amounts of data produced by petascale simulations. Typically, the produced data need to be properly visualized and analyzed by scientists in order to achieve the desired scientific goals. Both run-time and post analysis may benefit from, even require, additional meta-data - provenance information. One of the challenges in this context is the tracking of the data files that can be produced in very large numbers during stages of the workflow, such as visualizations. The Kepler provenance framework collects all or part of the raw information flowing through the workflow graph. This information then needs to be further parsed to extract meta-data of interest. This can be done through add-on tools and algorithms. We show how to automate tracking specific information such as data files locations.

99 GENERAL AND MISCELLANEOUS↗

yProv4ML: Effortless provenance tracking for machine learning systems

The rapid growth in interest in deep learning and foundation models (FMs) in particular, has attracted the attention of a diverse range of researchers thanks to their generalization ability. However, the advent of these techniques has also brought to light the lack of transparency and rigor in the way development is pursued. In particular, the inability to determine the number of epochs and other hyperparameters in advance presents challenges in identifying the best model. To address this challenge, machine learning frameworks such as MLFlow can automate the collection of this type of information. However, these tools capture data using proprietary formats and pose little attention to lineage. This paper proposes yProv4ML, a framework that captures provenance information generated during machine learning processes in PROV-JSON format, with minimal code modification.

Machine learning↗

Bridging the Gap: Enhancing Prominence and Provenance of NASA Datasets in Research Publications

Attribution of datasets that were used to generate research results described in peer-reviewed publications to the original source of these datasets (which are often archived at NASA Earth Science data centers) has been very challenging. Even though the data citation standard of citing datasets as research artifacts and citing them with Digital Object Identifiers (DOIs) was introduced over a decade ago, most authors do not properly reference the data used in their studies and merely mention them in the text. The lack of proper citations of datasets makes the peer-reviewed publication less transparent, imperils reproducibility, and impedes open science. We offer an open-source publication management methodology and a tool that can help to enhance usage-based data discovery, prominence, and provenance of the data; reproducibility of the research results; and potentially increase the return on investment on NASA-funded research.

open-source↗

An Approach to Evaluate Scientist Support in Abstract Workflows and Provenance Traces

Abstract workflows are useful to bridge the gap between scientists and technologists towards using computer systems to carry out scientific processes. Provenance traces provide evidence required to validate results and support their reuse. Assuming both technologies are based on formal semantics, a knowledge-based system that consistently merges both technologies is useful for scientists that produce data to document their data collecting and transformation processes; it is also useful for scientists that reuse data to assess scientific processes and resulting datasets produced by others. While evaluation of each technology is necessary for a given application, this work discusses their combined evaluation. The claim is that both technologies should complement each other and align consistently to a scientist’s perspective in order to be effective for science. Evaluation criteria are proposed based on lessons learned and exemplified for discussion.

Provenance↗

Functional Requirements for Information Resource Provenance on the Web

We provide a means to formally explain the relationship between HTTP URLs and the representations returned when they are requested. According to existing World Wide Web architecture, the URL serves as an identier for a semiotic referent while the document returned via HTTP serves as a representation of the same referent. This begins with two sides of a semiotic triangle; the third side is the relationship between the URL and the representation received. We complete this description by extending the library science resource model Functional Requirements for Bibliographic Resources (FRBR) with cryptographic message and content digests to create a Functional Requirements for Information Resources (FRIR). We show how applying the FRIR model to HTTP GET and POST transactions disambiguates the many relationships between a given URL and all representations received from its request, provides fine-grained explanations that are complementary to existing explanations of web resources, and integrates easily into the emerging W3C provenance standard.

Data Management↗

Production technology and provenance study of archaeological ceramics from relevant sites in the Alcantara River Valley (North-eastern Sicily, Italy)

In this paper, volcanic-rich ceramic remains from the archaeological sites of Francavilla, Naxos and Taormina (Province of Messina, North-eastern Sicily) were studied by using inclusions as main provenance marker. Technological features, such as temper choice, vitrification degree and firing temperatures, were investigated by polarizing microscopy, X-ray diffractometry (XRD) and scanning electron microscopy (SEM). Information on the production centres was obtained through the identification of the source area of raw materials used as temper. Indeed, petrochemical analysis of the volcanic inclusions within the examined ceramics displayed strong affinities with structures/textures and compositions of the locally outcropping mugearitic products, probably ascribed to the eruptive activity of an eccentric vent of Mt. Etna (Mt. Mojo). A local production for the studied pottery samples has been therefore advanced, assuming that the used volcanic temper was easily available from the alluvial deposits along the Alcantara River stream, which is connected to the lava flow of Mt. Mojo.

36 MATERIALS SCIENCE↗

Provenance and Transport Processes of Sediments along the Southeastern Brazilian Coast

Natural gamma radiation measurements of Quaternary sedimentary deposits were performed with the aim of understanding the provenance and transport processes of sediments along the coastal zone of the Brazilian Southeast. The method employs thorium, uranium and potassium as tracers of the mineralogical properties of beach sand minerals, which reflect the geological history of transport and sorting processes. A considerable positive correlation with the geological evolution of these sandy coastal deposits was observed.

54 ENVIRONMENTAL SCIENCES↗

Gold Provenance Studies for Romanian Archaeological Objects Using Micro-SR-XRF

Studies by Synchrotron Radiation--X-Ray Fluorescence (SR-XRF) for the search of the presence of trace elements like Sb, Sn, Te and Pb in archaeological metallic objects found on the territory of Romania--old coins and Bronze Age jewelry, aimed to determine the provenance of the gold used in their manufacture. The results are compared with the detailed elemental composition of alluvial or primary gold samples, obtained by the same technique. This work attempted to establish the origin of the gold used for the mint of two different types of koson coins. We found that the kosons with monogram are made of refined gold, while the one used for the kosons without monogram is mainly alluvial. The gold used in the manufacture of the Calarasi Vulchitrun-type disk and the Tauteu hair ring is also of alluvial origin.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A mechanism for proven technology foresight for emerging fast reactor designs and concepts

The assessment of emerging nuclear fast reactor designs and concepts viability requires a combination of foresight methods. A mechanism that allows for the comparison and quantification of the possibility of being a proven technology in the future, β for the existing fast reactor designs and concepts is proposed as one of the quantitative foresight method. The methodology starts with the identification at the national or regional level, of the factors that would affect β. The factors are then categorized into several groups; economic, social and technology elements. Each of the elements is proposed to be mathematically modelled before all of the elemental models can be combined. Once the overall β model is obtained, the β{sub min} is determined to benchmark the acceptance as a candidate design or concept. The β values for all the available designs and concepts are then determined and compared with the β{sub min}, resulting in a list of candidate designs that possess the β value that is larger than the β{sub min}. The proposed methodology can also be applied to purposes other than technological foresight.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

LQCD workflow execution framework: Models, provenance and fault-tolerance

Large computing clusters used for scientific processing suffer from systemic failures when operated over long continuous periods for executing workflows. Diagnosing job problems and faults leading to eventual failures in this complex environment i s difficult, specifically when the success of an entire workflow might be affected by a single job failure. In this paper, we introduce a model-based, hierarchical, reliable execution framework that encompass workflow specification, data provenance, execu tion tracking and online monitoring of each workflow task, also referred to as participants. The sequence of participants is described in an abstract parameterized view, which is translated into a concrete data dependency based sequence of participants wi th defined arguments. As participants belonging to a workflow are mapped onto machines and executed, periodic and on-demand monitoring of vital health parameters on allocated nodes is enabled according to pre-specified rules. These rules specify condition s that must be true pre-execution, during execution and post-execution. Monitoring information for each participant is propagated upwards through the reflex and healing architecture, which consists of a hierarchical network of decentralized fault manageme nt entities, called reflex engines. They are instantiated as state machines or timed automatons that change state and initiate reflexive mitigation action(s) upon occurrence of certain faults. We describe how this cluster reliability framework is combined with the workflow execution framework using formal rules and actions specified within a structure of first order predicate logic that enables a dynamic management design that reduces manual administrative workload, and increases cluster-productivity.

Piccoli, Luciano↗

Integrating prediction, provenance, and optimization into high energy workflows

We propose a novel approach for efficient execution of workflows on distributed resources. The key components of this framework include: performance modeling to quantitatively predict workflow component behavior; optimization-based scheduling such as choosing an optimal subset of resources to meet demand and assignment of tasks to resources; distributed I/O optimizations such as prefetching; and provenance methods for collecting performance data. In preliminary results, these techniques improve throughput on a small Belle II workflow by 20%.

Schram, M.↗

Provenance–aware workflow for data quality management and improvement for large continuous scientific data streams

Data quality assessment, management and improvement is integral part of any big data intensive scientific research to ensure accurate, reliable and reproducible science discoveries. The task of maintaining the quality of data, however, is non-trivial and pose a challenge for a program like Department of Energy’s Atmospheric Radiation Measurement (ARM) that collects data from hundreds of instruments across the world,and distributes thousands of streaming data products that are continuously growing in near-real-time in an archive 1.7 Petabyte in size and growing. We present a computational data processing workflow to collect the data quality issue via an easy and intuitive web-based portal that allows reporting of any quality issues for any site, facility or instruments at a granularity down to individual variable in the data files. Portal allows instrument specialists and scientists to provide corrective actions in form of symbolic equation. A parallel processing framework applies the data improvement to large volume of data in efficient parallel environment, while optimizing data transfer and file I/O operations. Corrected files are systematically versioned and archived. A provenance tracking module tracks and records any change made to the data during its entire life cycle which are communicated transparently to the scientific users. Developed in Python using open source technologies, software architecture enables efficient and fast management and improvement of data in an operational data center environment.

Kumar, Jitendra↗

A Multi-Tier Provenance Model for Global Climate Research

Global climate researchers rely upon many forms of sensor data and analytical methods to help profile subtle changes in climate conditions. The U.S. Department of Energy Atmospheric Radiation Measurement (ARM) program provides researchers with curated products called Value Added Products (VAPs) resulting from continuous instrumentation streams, data fusion, and analytical profiling. To provide these projects the ARM operations and research teams rely upon a number of techniques to ensure strict quality control and quality assurance codes are maintained. End users in the climate research community are highly interested in obtaining as much causal evidence as possible and currently either not all the evidence are easily attainable or easily identifiable without significant effort. Our research interests are to identify a provenance model that serves both the producers and consumers of the VAP maintaining the quality assurance/quality control standards and tailored to meeting the individual researcher’s needs.

54 ENVIRONMENTAL SCIENCES↗

Landscape responses to intraplate deformation in the Kalahari constrained by sediment provenance and chronology in the Okavango Basin

The structural depression that occupies the Okavango Basin in southern Africa comprises a depo-centre within the intracratonic Kalahari Basin where sediments of the Cenozoic Kalahari Group have accumulated. The Okavango Basin has been formed due to stretching and subsidence at an area of diffused deformation, southwestwards to the main East African Rift System (EARS). Sediments from two full Kalahari Group sequences, located on opposite sides of the Gumare Fault that forms a major fault within the Okavango Basin, were studied to determine their provenance and chronology. Terrestrial Cosmogenic Nuclide (TCN) 26 Al/ 10 Be burial dating was used to constrain a chronostratigraphical framework, and Pb, Sr, and Nd isotopic ratios combined with geochemical and sedimentological analyses were applied to track the source areas of the sediments.Results indicate the following sequence of basin filling: (a) Accumulation between ca. 4–3 Ma during which the currently downthrown (southern) block received a mixture of sediments mostly from the Choma-Kalomo, Ghanzi-Chobe, and Damara terranes, and possibly from the Lufilian Belt and/or Karoo basalts during earlier stages of deposition. Simultaneously, the upthrown (northern) block received sediments from more distant Archean sources in the Zimbabwe and/or Kasai cratons, (b) Hiatus in sedimentation occurred at both sites between ca. 3–2 Ma, (c) Sediments on both sides of the Gumare Fault share a similar source (Angolan Shield) with minor distinct contributions to the downthrown block from the Kasai Craton and local sources input to the upthrown block, and (d) Regional distribution of aeolian sand since at least 1 Ma. The change in source areas is attributed to rearrangements of the drainage systems that were probably linked to vertical crustal movements on the margins of the Okavango Basin. The tectonically induced morphodynamics controlled the landscape evolution of the endorheic basin where vast lakes, wetlands and salt pans have developed through time.

58 GEOSCIENCES↗