Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data provenance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

Storing and managing information artifacts collected by information analysts using a computing device

Systems and computer-implemented processes for storage and management of information artifacts collected by information analysts using a computing device. The processes and systems can capture a sequence of interactive operation elements that are performed by the information analyst, who is collecting an information artifact from at least one of the plurality of software applications. The information artifact can then be stored together with the interactive operation elements as a snippet on a memory device, which is operably connected to the processor. The snippet comprises a view from an analysis application, data contained in the view, and the sequence of interactive operation elements stored as a provenance representation comprising operation element class, timestamp, and data object attributes for each interactive operation element in the sequence.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Performance Characterization and Provenance of Distributed Task-based Workflows on HPC Platforms

Understanding performance and provenance of task-based workflows poses significant challenges, particularly in distributed configurations where resources are shared by multiple applications. Task-based workflow management systems further complicate performance predictability because of their dynamicity that subtly alters task execution order from run to run. In this paper we propose a layered characterization framework for performance and task provenance for Dask.distributed workflows running on high-performance computing (HPC) platforms. It collects data from jobs, the workflow management system, and the operating system to aid in understanding the performance of these workflows. Our approach encompasses three main contributions: first, an extension of Dask.distributed to capture high-fidelity task provenance using Mochi data services; second, the adaptation of the established HPC I/O characterization tool Darshan to gather high-fidelity I/O data, thereby enhancing the granularity of our analysis; and third, a framework to combine and process the collected data and provide helpful insights into performance characterization and reproducibility, alongside our lessons learned.

Dask↗

Collaborative simulations and experiments for a novel yield model of coal devolatilization in oxy-coal combustion conditions

Oxy-coal combustion is an emerging low-cost “clean coal” technology for emissions reduction and Carbon Capture and Sequestration (CCS). The use of Computational Fluid Dynamics (CFD) tools is crucial for the development of cost-effective oxy-fuel technologies and the minimization of environmental concerns at industrial scale. The coupling of detailed chemistry models and CFD simulations is still challenging, especially for large-scale plants, because of the high computational efforts required. The development of scale-bridging models is therefore necessary, to find a good compromise between computational efforts and the physical-chemical modeling precision. This paper presents a procedure for scale-bridging modeling of coal devolatilization, in the presence of experimental error, that puts emphasis on the thermodynamic aspect of devolatilization, namely the final volatile yield of coal, rather than kinetics. The procedure consists of an engineering approach based on dataset consistency and Bayesian methodology including Gaussian-Process Regression (GPR). Experimental data from devolatilization tests carried out in an oxy-coal entrained flow reactor were considered and CFD simulations of the reactor were performed. Jointly evaluating experiments and simulations, a novel yield model was validated against the data via consistency analysis. In parallel, a Gaussian-Process Regression was performed, to improve the understanding of the uncertainty associated to the devolatilization, based on the experimental measurements. Potential model forms that could predict yield during devolatilization were obtained. The set of model forms obtained via GPR includes the yield model that was proven to be consistent with the data. Finally, the overall procedure has resulted in a novel yield model for coal devolatilization and in a valuable evaluation of uncertainty in the data, in the model form, and in the model parameters.

01 COAL, LIGNITE, AND PEAT↗

Geochemical Fingerprinting of Coltan Ores by Machine Learning on Uneven Datasets

Two modern machine learning techniques, Linear Programming Boosting (LPBoost) and Support Vector Machines (SVMs), are introduced and applied to a geochemical dataset of niobium-tantalum ('coltan') ores from Central Africa to demonstrate how such information may be used to distinguish ore provenance, i.e., place of origin. The compositional data used include uni- and multivariate outliers and elemental distributions are not described by parametric frequency distribution functions. The 'soft margin' techniques of LPBoost and SVMs can be applied to such data. Optimization of their learning parameters results in an average accuracy of up to c. 92%, if spot measurements are assessed to estimate the provenance of ore samples originating from two geographically defined source areas. A parameterized performance measure, together with common methods for its optimization, was evaluated to account for the presence of uneven datasets. Optimization of the classification function threshold improves the performance, as class importance is shifted towards one of those classes. For this dataset, the average performance of the SVMs is significantly better compared to that of LPBoost.

42 ENGINEERING↗

A 'digital' technique for manual extraction of data from aerial photography

The interpretation procedure described uses a grid cell approach. In addition, a random point is located in each cell. The procedure required that the cell/point grid be established on a base map, and identical grids be made to precisely match the scale of the photographic frames. The grid is then positioned on the photography by visual alignment to obvious features. Several alignments on one frame are sometimes required to make a precise match of all points to be interpreted. This system inherently corrects for distortions in the photography. Interpretation is then done cell by cell. In order to meet the time constraints, first order interpretation should be maintained. The data is put onto coding forms, along with other appropriate data, if desired. This 'digital' manual interpretation technique has proven to be efficient, and time and cost effective, while meeting strict requirements for data format and accuracy.

Istvan, L. B.↗

Capturing provenance as a diagnostic tool for workflow performance evaluation and optimization

In extreme-scale computing environments such as the DOE Leadership Computing Facilities scientific workflows are routinely used to coordinate software processes for the execution of complex, computational applications that perform in-silico experiments. Monitoring the performance of workflows without also simultaneously tracking provenance is not sufficient to understand variations between runs, configurations, versions of a code, and between changes in an implemented stack, and systems, i.e. the variability of performance metrics data in their historical context. We take a provenance-based approach and demonstrate that provenance is useful as a tool for evaluating and optimizing workflow performance in extreme-scale HPC environments. We present Chimbuko, a framework for the analysis and visualization of the provenance of performance. Chimbuko implements a method for the evaluation of workflow performance from multiple components that enables the exploration of performance metrics data at scale.

97 MATHEMATICS AND COMPUTING↗

Characterizing manufacturing wastewater in the United States for the purpose of analyzing energy requirements for reuse

This paper seeks to inform an improved understanding of the energy tradeoff associated with on-site manufacturing water reuse in the United States from a lifecycle perspective, in part by developing an analytical framework for understanding when this tradeoff for reuse is beneficial. We survey the literature to assess the current state of reuse and its motives and barriers in the United States, before synthesizing information from publicly available EPA data on contaminants in US manufacturing wastewaters and technologies for treating them. Using the available data, we derive a set of “ubiquitous contaminants” among the top ten in terms of mass discharged in more than half of US manufacturing subsectors (NAICS 31–33) according to EPA permit data. We also present information on proven treatment trains and their energy requirements. We then compare water quality requirements for specific contaminants in reclaimed water to those characteristic of wastewater streams currently being discharged from manufacturing plants into surface waters to highlight sectors with reuse opportunities that could require little cost to realize, such as primary metals and, to a lesser extent, petroleum and coal products. We conclude by highlighting data limitations that need to be rectified before applying the framework more broadly and discussing how these data gaps could be filled. Better understanding the relationship between energy and water in the context of on-site manufacturing water reuse would allow manufacturers to improve resiliency by reducing regulatory, physical, and reputational risks while lessening their footprint on local watersheds.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

A Tailored Convolutional Neural Network for Nonlinear Manifold Learning of Computational Physics Data Using Unstructured Spatial Discretizations

In this work, we propose a nonlinear manifold learning technique based on deep convolutional autoencoders that is appropriate for model order reduction of physical systems in complex geometries. Convolutional neural networks have proven to be highly advantageous for compressing data arising from systems demonstrating a slow-decaying Kolmogorov n-width. However, these networks are restricted to data on structured meshes. Unstructured meshes are often required for performing analyses of real systems with complex geometry. Our custom graph convolution operators based on the available differential operators for a given spatial discretization effectively extend the application space of deep convolutional autoencoders to systems with arbitrarily complex geometry that are typically discretized using unstructured meshes. We propose sets of convolution operators based on the spatial derivative operators for the underlying spatial discretization, making the method particularly well suited to data arising from the solution of partial differential equations. We demonstrate the method using examples from heat transfer and fluid mechanics and show better than an order of magnitude improvement in accuracy over linear methods.

97 MATHEMATICS AND COMPUTING↗

A Data Science and Machine Learning Platform Supporting Large Particle Accelerator Control and Diagnostics Applications Final Report: SBIR Initial Phase II DE-SC0022583

The Machine Learning Data Platform (MLDP) is a product providing full-stack support for data science, Machine Learning, and Artificial Intelligence (ML/AI) applications at particle accelerator and large experimental physics facilities. It supports ML/AI applications from front-end, high-speed acquisition of heterogeneous, time-series data, through data archiving and management, to back-end analysis. The MLDP embodies a “data-science ready” platform for data analysis and ML/AI applications in diagnosis, modelling, control, and optimization of these facilities. It provides data scientists and applications a consistent, datacentric interface to archive data standardizing implementation and deployment of ML/AI algorithms to different operations configurations within the same facility, or between facilities. Being an open-source, public-domain project, the MLDP is intended for broadest possible impact by increasing accessibility and minimizing the required expertise for installation and operation. The MLDP can also be deployed at user facilities for experimental data collection, archiving, and analysis. It is capable of acquisition and archiving of heterogeneous data from experimental equipment (e.g., images, arrays, structures, etc.) along with system hardware configurations (e.g., scalars, tables), control system process variables, and any metadata required for provenance. Thus, the MLDP can manage experimental data through its entire lifecycle, from acquisition and archiving, through analysis and investigation, to release and final publication.

43 PARTICLE ACCELERATORS↗

Irradiation Testing of Ultrasonic Transducers

Ultrasonic technologies offer the potential for high accuracy and resolution in-pile measurement of a range of parameters, including geometry changes, temperature, crack initiation and growth, gas pressure and composition, and microstructural changes. Many Department of Energy-Office of Nuclear Energy (DOE-NE) programs are exploring the use of ultrasonic technologies to provide enhanced sensors for in-pile instrumentation during irradiation testing. For example, the ability of single, small diameter ultrasonic thermometers (UTs) to provide a temperature profile in candidate metallic and oxide fuel would provide much needed data for validating new fuel performance models. Other efforts include an ultrasonic technique to detect morphology changes (such as crack initiation and growth) and acoustic techniques to evaluate fission gas composition and pressure. These efforts are limited by the lack of existing knowledge of ultrasonic transducer material survivability under irradiation conditions. For this reason, the Pennsylvania State University (PSU) was awarded an Advanced Test Reactor National Scientific User Facility (ATR NSUF) project to evaluate promising magnetostrictive and piezoelectric transducer performance in the Massachusetts Institute of Technology Research Reactor (MITR) up to a fast fluence of at least 1021 n/cm2 (E> 0.1 MeV). The goal of this research is to characterize magnetostrictive and piezoelectric transducer survivability during irradiation, enabling the development of novel radiation tolerant ultrasonic sensors for use in Material and Test Reactors (MTRs). As such, this test will be an instrumented lead test and real-time transducer performance data will be collected along with temperature and neutron and gamma flux data. The current work bridges the gap between proven out-of-pile ultrasonic techniques and in-pile deployment of ultrasonic sensors by acquiring the data necessary to demonstrate the performance of ultrasonic transducers.

In-Pile Instrumentation, Material and Test Reactor↗

Irradiation Testing of Ultrasonic Transducers

Ultrasonic technologies offer the potential for high accuracy and resolution in-pile measurement of a range of parameters, including geometry changes, temperature, crack initiation and growth, gas pressure and composition, and microstructural changes. Many Department of Energy-Office of Nuclear Energy (DOE-NE) programs are exploring the use of ultrasonic technologies to provide enhanced sensors for in-pile instrumentation during irradiation testing. For example, the ability of single, small diameter ultrasonic thermometers (UTs) to provide a temperature profile in candidate metallic and oxide fuel would provide much needed data for validating new fuel performance models. Other efforts include an ultrasonic technique to detect morphology changes (such as crack initiation and growth) and acoustic techniques to evaluate fission gas composition and pressure. These efforts are limited by the lack of existing knowledge of ultrasonic transducer material survivability under irradiation conditions. For this reason, the Pennsylvania State University (PSU) was awarded an Advanced Test Reactor National Scientific User Facility (ATR NSUF) project to evaluate promising magnetostrictive and piezoelectric transducer performance in the Massachusetts Institute of Technology Research Reactor (MITR) up to a fast fluence of at least 1021 n/cm2 (E> 0.1 MeV). The goal of this research is to characterize magnetostrictive and piezoelectric transducer survivability during irradiation, enabling the development of novel radiation tolerant ultrasonic sensors for use in Material and Test Reactors (MTRs). As such, this test will be an instrumented lead test and real-time transducer performance data will be collected along with temperature and neutron and gamma flux data. The current work bridges the gap between proven out-of-pile ultrasonic techniques and in-pile deployment of ultrasonic sensors by acquiring the data necessary to demonstrate the performance of ultrasonic transducers.

harsh-environment sensors↗

Ultrasonic Transducer Irradiation Test Results

Ultrasonic technologies offer the potential for high-accuracy and -resolution in-pile measurement of a range of parameters, including geometry changes, temperature, crack initiation and growth, gas pressure and composition, and microstructural changes. Many Department of Energy-Office of Nuclear Energy (DOE-NE) programs are exploring the use of ultrasonic technologies to provide enhanced sensors for in-pile instrumentation during irradiation testing. For example, the ability of small diameter ultrasonic thermometers (UTs) to provide a temperature profile in candidate metallic and oxide fuel would provide much needed data for validating new fuel performance models. Other ongoing efforts include an ultrasonic technique to detect morphology changes (such as crack initiation and growth) and acoustic techniques to evaluate fission gas composition and pressure. These efforts are limited by the lack of identified ultrasonic transducer materials capable of long term performance under irradiation test conditions. For this reason, the Pennsylvania State University (PSU) was awarded an ATR NSUF project to evaluate the performance of promising magnetostrictive and piezoelectric transducers in the Massachusetts Institute of Technology Research Reactor (MITR) up to a fast fluence of at least 10 21 n/cm 2 . The goal of this research is to characterize and demonstrate magnetostrictive and piezoelectric transducer operation during irradiation, enabling the development of novel radiation-tolerant ultrasonic sensors for use in Material Testing Reactors (MTRs). As such, this test is an instrumented lead test and real-time transducer performance data is collected along with temperature and neutron and gamma flux data. The current work bridges the gap between proven out-of-pile ultrasonic techniques and in-pile deployment of ultrasonic sensors by acquiring the data necessary to demonstrate the performance of ultrasonic transducers. To date, one piezoelectric transducer and two magnetostrictive transducers have demonstrated reliable operation under irradiation. The irradiation is ongoing.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Monitor weather conditions for cloud seeding control

The author has identified the following significant results. The near real-time DCS platform data transfer to the time-share compare is a working reality. Six stations are now being automatically monitored and displayed with a system delay of 3 to 8 hours from time of data transmission to time of data accessibility on the computer. The DCS platform system has proven itself a valuable tool for near real-time monitoring of mountain precipitation. Data from Wolf Creek Pass were an important input in making the decision when to suspend seeding operations to avoid exceeding suspension criteria in that area. The DCS platforms, as deployed in this investigation, have proven themselves to be reliable weather resistant systems for winter mountain environments in the southern Colorado mountains.

Kahan, A. M.↗

Combined High Spectral Resolution Lidar and Millimeter Wavelength Radar Measurement of Ice Crystal Precipitation

The goal of this research has been to improve measurements of snowfall using a combination of millimeter-wavelength radar and High Spectral Resolution Lidar (HSRL) Observations. Snowflakes are large compared to the 532nm HSRL wavelength and small compared to the 3.2 and 8.6 mm wavelength radars used in this study. This places the particles in the optical scattering regime of the HSRL, where extinction cross-section is proportional to the projected area of the particles, and in the Rayleigh regime for the radar, where the backscatter cross-section is proportional to the mass-squared of the particles. Forming a ratio of the radar measured cross-section to the HSRL measured cross section eliminates any dependence on the number of scattering particles, yielding a quantity proportional to the average mass-squared of the snowflakes over the average area of the flakes. Using simultaneous radar measurements of particle fall velocities, which are dependent particle mass and cross-sectional area it is possible to derive the average mass of the snow flakes, and with the radar measured fall velocities compute the snowfall rate. Since this retrieval requires the optical extinction cross-section we began by considering errors this quantity. The HSRL is particularly good at measuring the backscatter cross-section. In previous studies of snowfall in the high Arctic were able to estimate the extinction cross-section directly as a fixed ratio to the backscatter cross-section. Measurements acquired in the STORMVEX experiment in Colorado showed that this approach was not valid in mid-latitude snowfalls and that direct measurement of the extinction cross-section is required. Attempts to measure the extinction directly uncovered shortcomings in thermal regulation and mechanical stability of the newly deployed DOE HSRL systems. These problems were largely mitigated by modifications installed in both of the DOE systems. We also investigated other sources of error in the HSRL direct measurement of extinction (see appendix II of this report). We also developed improved algorithms to extract extinction from the HSRL data. These have been installed in the standard HSRL data processing software and are now available to all users of HSRL data. Validation of snowfall measurements has proven difficult due to the unreliability of conventional snowfall measurements coupled with the complexity of considering the vast variety of snowflake geometries. It was difficult to tell how well the algorithm’s approach to accommodating differences in snowflakes was working without good measurements for comparison. As a result, we decided to apply this approach to the somewhat simpler, but scientifically important, problem of drizzle measurement. Here the particle shape is known and the conventional measurement are more reliable. These algorithms where successfully applied to drizzle data acquired during the ARM MAGIC study of marine stratus clouds between California and Hawaii (see Appendix I). This technique is likely to become a powerful tool for studying lifetime of the climatically important marine stratus clouds.

47 OTHER INSTRUMENTATION↗

Methods for the Automated Determination of Sustained Maximum Amplitudes in Oscillating Signals

Machine condition monitoring has been proven to reduce machine downtime and increase productivity. The state-of-the-art research uses vibration monitoring for tasks such as maintenance and tool wear prediction. A less explored aspect is how vibration monitoring might be used to monitor equipment sensitive to vibration. In a manufacturing environment, one example of where this might be needed is in monitoring the vibration of optical linear encoders used in high-precision machine tools and coordinate measuring machines. Monitoring the vibration of sensitive equipment presents a unique case for vibration monitoring because an accurate calculation of the maximum sustained vibration is needed, as opposed to extracting trends from the data. To do this, techniques for determining sustained peaks in vibration signals are needed. Here, this work fills this gap by formalizing and testing methods for determining sustained vibration amplitudes. The methods are tested on simulated signals based on experimental data. Results show that processing the signal directly with the novel Expire Timer method produces the smallest amounts of error on average under various test conditions. Additionally, this method can operate in real-time on streaming vibration data.

Industrial Internet of Things↗

Hydrocarbon characterization experiments in fully turbulent fires : results and data analysis.

As the capabilities of numerical simulations increase, decision makers are increasingly relying upon simulations rather than experiments to assess risks across a wide variety of accident scenarios including fires. There are still, however, many aspects of fires that are either not well understood or are difficult to treat from first principles due to the computational expense. For a simulation to be truly predictive and to provide decision makers with information which can be reliably used for risk assessment the remaining physical processes must be studied and suitable models developed for the effects of the physics. The model for the fuel evaporation rate in a liquid fuel pool fire is significant because in well-ventilated fires the evaporation rate largely controls the total heat release rate from the fire. This report describes a set of fuel regression rates experiments to provide data for the development and validation of models. The experiments were performed with fires in the fully turbulent scale range (> 1 m diameter) and with a number of hydrocarbon fuels ranging from lightly sooting to heavily sooting. The importance of spectral absorption in the liquid fuels and the vapor dome above the pool was investigated and the total heat flux to the pool surface was measured. The importance of convection within the liquid fuel was assessed by restricting large scale liquid motion in some tests. These data sets provide a sound, experimentally proven basis for assessing how much of the liquid fuel needs to be modeled to enable a predictive simulation of a fuel fire given the couplings between evaporation of fuel from the pool and the heat release from the fire which drives the evaporation.

99 GENERAL AND MISCELLANEOUS↗

Globus service enhancements for exascale applications and facilities

Many extreme-scale applications require the movement of large quantities of data to, from, and among leadership computing facilities, as well as other scientific facilities and the home institutions of facility users. These applications, particularly when leadership computing facilities are involved, can touch upon edge cases (e.g., terabyte files) that had not been a focus of previous Globus optimization work, which had emphasized rather the movement of many smaller (megabyte to gigabyte) files. We report here on how automated client-driven chunking can be used to accelerate both the movement of large files and the integrity checking operations that have proven to be essential for large data transfers. In conclusion, we present detailed performance studies that provide insights into the benefits of these modifications in a range of file transfer scenarios.

97 MATHEMATICS AND COMPUTING↗

An Adaptable Seismic Data Format

We present ASDF, the Adaptable Seismic Data Format, a modern and practical data format for all branches of seismology and beyond. The growing volume of freely available data coupled with ever expanding computational power opens avenues to tackle larger and more complex problems. Current bottlenecks include inefficient resource usage and insufficient data organization. Properly scaling a problem requires the resolution of both these challenges, and existing data formats are no longer up to the task. ASDF stores any number of synthetic, processed or unaltered waveforms in a single file. A key improvement compared to existing formats is the inclusion of comprehensive meta information, such as event or station information, in the same file. Additionally, it is also usable for any non-waveform data, for example, cross-correlations, adjoint sources or receiver functions. Last but not least, full provenance information can be stored alongside each item of data, thereby enhancing reproducibility and accountability. Any data set in our proposed format is self-describing and can be readily exchanged with others, facilitating collaboration. The utilization of the HDF5 container format grants efficient and parallel I/O operations, integrated compression algorithms and check sums to guard against data corruption. To not reinvent the wheel and to build upon past developments, we use existing standards like QuakeML, StationXML, W3C PROV and HDF5 wherever feasible. Usability and tool support are crucial for any new format to gain acceptance. We developed mature C/Fortran and Python based APIs coupling ASDF to the widely used SPECFEM3D_GLOBE and ObsPy toolkits.

58 GEOSCIENCES↗