Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data staging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

In-situ 4D visualization and analysis of temperature-driven creep in an oil shale propped fracture

Here, the proppant embedment due to creep in shales is a known issue affecting the useable lifetime of wells in unconventional oil and gas recovery. One of the factors influencing creep is the presence of organics, whose properties can be very sensitive to temperature. In this work we investigated for the first time the role of temperature-induced creep increase in proppant embedment in an organics-rich Green River oil shale sample via in-situ synchrotron X-ray micro-computed tomography. We observed that temperatures as low as 75 °C already induce fast creep, with a fracture aperture closing rate of 13 μm/h and a loss of fracture conductivity rate of 8.7%/h, due only to proppant embedment, in the measured interval at the first heating stage. Local displacement data analysis provided evidence for markedly plastic deformation around the proppant-shale contacts, in contrast with the brittle proppant embedment observed on more cemented and less organics-rich shales at room temperature. The results highlight how the problem of temperature-dependent mechanical behavior might be more important than previously thought, in shales with a high content in organics, and that in-situ micro-imaging techniques can play a key role in understanding the underlying mechanisms, contributing to solve creep-related problems associated with hydraulic fracturing in complex scenarios.

02 PETROLEUM↗
DOE OSTI2021

Resolving the Evolution of Atomic Layer-Deposited Thin-Film Growth by Continuous In Situ X-Ray Absorption Spectroscopy

In situ synchrotron X-ray absorption near-edge structure characterization of thin-film titania growth by atomic layer deposition (ALD) over ZnO nanowires reveals persistent low-coordinated Ti motifs leading to a new picture of ALD growth. Through the design of growth and measurement cycles, Ti K-edge spectral data are continuously recorded so as to characterize the film evolution as a function of ALD cycle number and the surface changes within the time scale of the ALD cycle. A unified set of analysis tools is developed to interpret the time-series of spectral data. A prenucleation stage of growth, a transition region, and then a steady-state growth stage are observed with distinguishable features. Multivariate curve resolution analysis, that is physically constrained, demonstrates two specific spectral components with associated, time-dependent concentrations. The bulk-film component tracks the stages of growth. The surface and interface components, present throughout the stages of growth, reveal a significant coverage of relatively isolated or loosely networked tetrahedrally coordinated Ti atomic motifs. Lastly, spectral signatures for the intra-cycle growth kinetics are reconstructed at a time resolution of ~1 s and demonstrate that the transient Ti motifs on the growing surface stabilize within a few seconds of the Ti precursor pulse.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Enhanced Frequency Support Scheme of Generic Inverter-Based Resource Models for Renewable-Dominated Power Grids

The frequency response of SG-dominated power grids is predictable ahead of an occurrence of a frequency event because the frequency response of SGs is consistent, and it can be inferred from the swing equation [1]. However, increasing the portion of IBRs in an SG-dominated power grid might make the characteristics of the conventional power grids no longer valid because this changing resource mix affects grid dynamics and controls [2]. Thus, maintaining these characteristics greatly benefits the control and operation of the power grids with high penetration of IBRs. To maintain these characteristics in IBR-dominated power grids, IBRs should have frequency response capability similar to that of an SG. The WECC modeling validation subcommittee has developed generic IBR models for large system planning [3]-[5]. These models can represent various vendors' dynamic behavior for WTG, PV, and ESS [5]. The current generic IBR models approved by WECC can provide frequency response only from droop control loops in REPC models [6], [7]. The contribution of the loops is proportional to the frequency deviation from the nominal frequency. Thus, it presents an insufficient contribution to arrest frequency variation compared to the frequency response of SGs because it allows a high ROCOF in the early stage of frequency events. This shortfall will become greater as the PL of IBRs increases in power grids. Controller enhancement for the generic IBR models is required to secure the frequency stability under high PL of IBRs as in the SG-dominated power grids. This paper proposes a control extension for the generic IBR models to enhance the frequency support capabilities and discusses the classification of frequency support for the different types of IBR considering their operating constraints. An inertial control scheme is implemented in the REPC and REEC models of the generic IBR models to achieve these objectives. The inertial control scheme includes the following stages: Control area data acquisition, inertia time constant estimation, IBR-related constraint check, IBR contribution determination, and inertial response provision. In the scheme, a REPC acquires control area data from a system operator and estimates a total inertia time constant for the control area the applicable IBR power plant belongs. Then, the estimated inertial time constant is transferred to each IBR controller—REEC—within the power plant. Each REEC checks the availability of applicable IBR for inertial response participation. If the IBR is available, the REEC amplifies the estimated inertial time constant to utilize it for inertial response provision. In this way, the proposed inertial response scheme extends the functionality of the generic IBR models to provide SG-like frequency response within their constraints. Various scenarios considering different IBR types, IBR penetration levels, and frequency control schemes were simulated and compared in an IEEE 39-bus system using PSCAD simulator to verify the effectiveness of the proposed scheme.

Kim, Jinho↗
DOE OSTI2021

Parallel Algorithms for Efficient Computation of High-Order Line Graphs of Hypergraphs

This paper considers structures of systems beyond dyadic (pairwise) interactions and investigates mathematical modeling of multi-way interactions and connections as hypergraphs, where captured relationships among system entities are set-valued. To date, in most situations, entities in a hypergraph are considered connected as long as there is at least one common ``neighbor''. However, minimal commonality sometimes discards the ``strength'' of connections and interactions among groups. To this end, considering the ``width'' of a connection, referred to as the \emph{$s$-overlap} of neighbors, provides more meaningful insights into how closely the communities or entities interact with each other. In addition, $s$-overlap computation is the fundamental kernel to construct the line graph of a hypergraph, a low-order approximation of the hypergraph which can carry significant information about the original hypergraph. Subsequent stages of a data analytics pipeline then can apply highly-tuned graph algorithms on the line graph to reveal important features. Given a hypergraph, computing the $s$-overlaps by exhaustively considering all pairwise entities can be computationally prohibitive. To tackle this challenge, we develop efficient algorithms to compute $s$-overlaps and the corresponding line graph of a hypergraph. We propose several heuristics to avoid execution of redundant work and improve performance of the $s$-overlap computation. Our parallel algorithm, combined with these heuristics, is orders of magnitude (more than $10\times$) faster than the naive algorithm in all cases and the SpGEMM algorithm with filtration in most cases (especially with large $s$ value).

hypergraph algorithms, graph algorithms, parallel ↗
DOE OSTI2020

Enhanced Tensor Completion Based Approaches for State Estimation in Distribution Systems

Grid state estimation is essential for effective control and management of distribution systems. While weighted least squares has been the conventional method for state estimation, sparsity-aware methods have become popular due to their superior performance with limited data. Matrix completion and compressed sensing-based state estimation approaches exploit the underlying smoothness in the state variables. However, classic matrix completion methods do not take into account the temporal correlation of system states. Compressed sensing methods, on the other hand, require an appropriate choice of sparsifying basis that may not be easy to identify. This paper proposes a blocktensor completion based framework which uses an alternative approach to estimate voltage phasor, power injections and branch currents. This approach utilizes the temporal correlation of the system states in a tensor trace-norm minimization formulation with power flow equations as constraints. Herein, feature scaling is introduced in the problem formulation to benefit from the improved sensitivity of the tensor trace norm to the matrix columns in the scaled unfoldings of the tensor. Weighted tensor norm is utilized to exploit the structures of the different unfoldings of the state measurement tensor to improve the voltage estimation. The estimation accuracy is further improved by alternatively estimating the tensor columns and increasing the available data at each stage in the tensor completion process. The proposed methods are evaluated on the IEEE-33, 37 test systems and a 100- node test system. The proposed methods are shown to provide significant performance gains relative to the classic matrix and tensor completion based approaches.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Architecture-Aware Models of AI Engines for High-Performance Matrix Matrix Multiplication

The AI Engine (AIE) architecture, available in systems from mobile SoCs to server-class FPGAs, aims to efficiently execute AI/ML tasks through a two-dimensional array of compute tiles. Previous work on AIEs has explored different approaches to mapping computation across spatial arrays, but the compute kernel running on each tile has not been the focus. Additionally, the AIE-ML architecture introduces memory tiles and omits programmable logic, requiring new approaches to staging and moving data throughout the array. In this work we update analytical models developed for CPUs to produce the design of high performance kernels while introducing new model considerations such as memory structure, throughput, and latency as required by the AIE hardware. We evaluate our models by developing AIE-ML kernels for matrix multiplication in low-precision data types showing performance up to 95% of compute peak for the kernel when data resides in local memory and above 90% of compute peak when data resides in main memory.

Binder, Elliott D. [Carnegie Mellon University, Pi↗
DOE OSTI2022

Hardware-Control: Instrument control and automation package

Conducting experimental research often relies on the control of laboratory instruments to, for example, control power supplies, move stages, and measure data. Tasks, such as data logging or parameter scans, often need to be automated. Being able to easily create a user interface to the hardware and to be able to reuse code is highly desirable.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

SAIL-Isotopologue (SAIL-iso) Field Campaign Report

The SAIL-iso project took place from November 19, 2022 to June 3, 2023 as part of the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility’s Surface Atmosphere Integrated Field Laboratory (SAIL) project at the main project site in Gothic, Colorado. The project sought to use measurements of the stable isotopic composition of water vapor to better understand cloud-aerosol interactions in a remote mountainous watershed. Water vapor isotopologue and humidity measurements were determined using a Los Gatos Research Triple Water Vapor Isotope Analyzer (TWVIA). The instrument was deployed in the guest van and consists of three main components: a water vapor isotope analyzer, a Los Gatos Research Water Vapor Isotope Standard Source (WVISS), and a Los Gatos Research Dry Air Source. Ambient air samples were collected through an inlet located about one meter above roof of the shipping container. Samples were then delivered to the analyzer through Teflon tubing by use of an external pump to minimize the transport time between the inlet and analyzer. The inlet tubing was surrounded by a Briskheat heat trace and insulating material to ensure ambient air samples did not fall below the dew point and result in condensation. The analyzer uses cavity ringdown spectroscopy to report isotopic ratios of ambient air samples at a frequency of 1 Hz. The calibration unit of the instrument was used in conjunction with the dry air source to periodically measure the δ values of standard waters. It uses a nebulizer to push small water droplets into a hot chamber that vaporizes the water without fractionation. This vapor was then transported to the analyzer using a built-in compressor and the dry air source, which allows each standard with known δ values to be measured at a wide range of humidity values for post-measurement calibration of ambient air samples. Multiple sources of uncertainty are introduced at different stages during the data collection and processing. These include (1) instrument precision, (2) uncertainty in the secondary standards, (3) humidity-correction uncertainty, and (4) Vienna Standard Mean Ocean Water – Standard Light Antarctic Precipitation (VSMOW-SLAP) calibration uncertainty. Uncertainty from each step is propagated in quadrature to calculate a total uncertainty of each isotopologue. Uncertainty was determined to be 2.3‰ for δD and 0.4‰ for δ 18 O.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Water vapor isotopic composition during TRACER IOP

The TRACER-Iso project took place from June 1 to September 30, 2022 as part of the intensive observation period (IOP) of the Tracking Aerosol Convection Interactions (TRACER) project at the main project site at the La Porte, Texas, municipal airport. The project sought to use measurements of the stable isotopic composition of water vapor to better understand cloud-aerosol interactions in deep convection in a polluted urban setting. Water vapor isotopologue and humidity measurements were determined using a Picarro L2130 Water Vapor Isotope Analyzer. The instrument was deployed in the guest van and consists of three main components: a water vapor isotope analyzer, and a Los Gatos Research Water Vapor Isotope Standard Source (WVISS), and a Los Gatos Research Dry Air Source. Ambient air samples were collected through an inlet located about one meter above roof of the shipping container. Samples were then delivered to the analyzer through teflon tubing by use of an external pump to minimize the transport time between the inlet and analyzer. The inlet tubing was surrounded by a Briskheat heat trace and insulating material to ensure ambient air samples did not fall below the dew point and result in condensation. The analyzer uses cavity ringdown spectroscopy to report isotopic ratios of ambient air samples at a frequency of 1 Hz. The calibration unit of the instrument was used in conjunction with the dry air source to periodically measure the δ values of standard waters. It uses a nebulizer to push small water droplets into a hot chamber that vaporizes the water without fractionation. This vapor was then transported to the analyzer using a built-in compressor and the dry air source, which allows each standard with known δ values to be measured at a wide range of humidity values for post-measurement calibration of ambient air samples. There are multiple sources of uncertainty introduced at different stages during the data collection and processing. These include (1) instrument precision, (2) uncertainty in the secondary standards, (3) humidity-correction uncertainty, and (4) VSMOW-SLAP calibration uncertainty. Uncertainty from each step is propagated in quadrature to calculate a total uncertainty of each isotopologue. Uncertainty was determined to be 2.3‰ for δD and 0.4‰ for δ18O.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

RISE: Reducing I/O Contention in Staging-based Extreme-Scale In-situ Workflows

While in-situ workflow formulations have addressed some of the data-related challenges associated with extreme-scale scientific workflows, these workflows involve complex interactions and different modes of data exchange. In the context of increasing system complexity, such workflows present significant resource management challenges, requiring complex cost-performance tradeoffs. This paper presents RISE, an intelligent staging-based data management middleware, which builds on the DataSpaces framework and performs intelligent scheduling of data management operations to reduce I/O contention. In RISE, data are always written immediately to local buffers to reduce the effect of the transfer impact upon application performance. RISE identifies applications’ data access patterns and moves data towards data consumers only when the network is expected to be idle, reducing the impact of asynchronous background data movement upon critical data read/write requests. Here, we experimentally demonstrate that RISE can take advantage of staging nodes to offload data during writes without degrading application data movement performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Fourier Neural Networks as Function Approximators and Differential Equation Solvers

We present a Fourier neural network (FNN) that can be mapped directly to the Fourier decomposition. The choice of activation and loss function yields results that replicate a Fourier series expansion closely while preserving a straightforward architecture with a single hidden layer. The simplicity of this network architecture facilitates the integration with any other higher-complexity networks, at a data pre- or postprocessing stage. We validate this FNN on naturally periodic smooth functions and on piecewise continuous periodic functions. We showcase the use of this FNN for modeling or solving partial differential equations with periodic boundary conditions. The main advantages of the current approach are the validity of the solution outside the training region, interpretability of the trained model, and simplicity of use.

Fourier decomposition↗
DOE OSTI2022

Adaptive elasticity policies for staging-based in situ visualization

In situ processing aims to alleviate the growing gap between computation and I/O capabilities by performing data processing close to the data source. In situ processing is widely used to process data generated by multiple data sources, including observation data from edge devices or scientific observational facilities and the simulation data generated by scientific computation on a high-performance computing (HPC) platform. For a scientific workflow that is run on an HPC platform and composed of a simulation program and an in situ data analytics or visualization (abbreviated as ana/vis) task, there is an implicit assumption that the computing resources assigned to the workflow keep static during the workflow execution. However, with the converging trend between the HPC and cloud computing platform, running the in situ ana/vis task in an elastic way is promising to decrease its overhead and improve its resource utilization rate. Resource elasticity represents the ability to change resource configurations such as the number of computing nodes/processes during workflow execution. An elastic job may dynamically adjust resource configurations; it may use a few resources at the beginning and more resources toward the end of the job when interesting data appear. However, it is hard to predict a priori how many computing nodes/processes need to be added/removed during the workflow execution to adapt to changing workflow needs. How to efficiently guide elasticity operations, such as growing or shrinking the number of processes used for in situ analysis during workflow execution, is an open-ended research question. In this article, we present adaptive elasticity policies that adopt workflow runtime information collected during workflow execution to predict how to trigger the addition/removal of processes in order to minimize in situ processing overhead. Taking in situ visualization tasks as an example, we integrate the presented elasticity policies into a staging-based elastic workflow and evaluate its efficiency in multiple elasticity scenarios. Compared with the situation without elasticity or with a static elasticity policy that uses a fixed number of processes for each rescaling operation, the adaptive elasticity policy can save overhead in finding a proper resource configuration and improve resource utilization efficiency. Furthermore, one experiment illustrates that the adaptive elasticity policy saves 41% of core-hours compared with the situation without the resource elasticity.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

First test results of the HGCAL concentrator ASICs: ECON-T and ECON-D

With over 6 million channels, the High Granularity Calorimeter for the CMS HL-LHC upgrade presents a unique data transmission challenge. The ECON ASICs provide a critical stage of on-detector data compression and selection for the trigger path (ECON-T) and data acquisition path (ECON-D) of the HGCAL. The ASICs, fabricated in 65 nm CMOS, are radiation tolerant up to 200 Mrad and require low power consumption: < 2.5 mW/sensor-channel per chip. Here, we report on the first functionality and radiation tests for the ECON-D-P1 full-functionality prototype. We present a comparison of single event effect (SEE) cross sections measured for different methods of triple modular redundancy using test results from the ECON-T-P1 full-functionality prototype.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

High-Throughput Field Plant Phenotyping: A Self-Supervised Sequential CNN Method to Segment Overlapping Plants

High-throughput plant phenotyping—the use of imaging and remote sensing to record plant growth dynamics—is becoming more widely used. The first step in this process is typically plant segmentation, which requires a well-labeled training dataset to enable accurate segmentation of overlapping plants. However, preparing such training data is both time and labor intensive. To solve this problem, we propose a plant image processing pipeline using a self-supervised sequential convolutional neural network method for in-field phenotyping systems. This first step uses plant pixels from greenhouse images to segment nonoverlapping in-field plants in an early growth stage and then applies the segmentation results from those early-stage images as training data for the separation of plants at later growth stages. The proposed pipeline is efficient and self-supervising in the sense that no human-labeled data are needed. We then combine this approach with functional principal components analysis to reveal the relationship between the growth dynamics of plants and genotypes. We show that the proposed pipeline can accurately separate the pixels of foreground plants and estimate their heights when foreground and background plants overlap and can thus be used to efficiently assess the impact of treatments and genotypes on plant growth in a field environment by computer vision techniques. This approach should be useful for answering important scientific questions in the area of high-throughput phenotyping.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Streaming Compression of Scientific Data via Weak-SINDy

Here, in this paper, a streaming weak-SINDy algorithm is developed specifically for compressing streaming scientific data. The production of scientific data, either via simulation or experiments, is undergoing a stage of exponential growth, which makes data compression important and often necessary for storing and utilizing large scientific data sets. As opposed to classical “offline” compression algorithms that perform compression on a readily available data set, streaming compression algorithms compress data “online” while the data generated from simulation or experiments is still flowing through the system. This feature makes streaming compression algorithms well suited for scientific data compression, where storing the full data set offline is often infeasible. This work proposes a new streaming compression algorithm, streaming weak-SINDy, which takes advantage of the underlying data characteristics during compression. The streaming weak-SINDy algorithm constructs feature matrices and target vectors in the online stage via a streaming integration method in a memory efficient manner. The feature matrices and target vectors are then used in the offline stage to build a model through a regression process that aims to recover equations that govern the evolution of the data. For compressing high-dimensional streaming data, we adopt a streaming proper orthogonal decomposition (POD) process to reduce the data dimension and then use the streaming weak-SINDy algorithm to compress the temporal data of the POD expansion. We propose modifications to the streaming weak-SINDy algorithm to accommodate the dynamically updated POD basis. By combining the built model from the streaming weak-SINDy algorithm and a small amount of data samples, the full data flow could be reconstructed accurately at a low memory cost, as shown in the numerical tests.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Data reduction considerations for the burning velocity of spherical constant volume flames of R32 (CH 2 F 2 ) with air

Here, the present work explores data reduction techniques for the measurement of the laminar burning velocities of R32(CH 2 F 2 )-air mixtures using a constant volume combustion device, in which the pressure-time history is the only measured parameter. To allow clear assessment of the accuracy of the data reduction methods, the pressure-time histories used for analysis are synthetically generated via a detailed numerical simulation employing full kinetics and with and without an optically-thin radiation model. Various data reduction models are employed, including a two-zone model and two multi-zone models, and these are compared with the results from the burning velocity obtained from the output of the numerical simulation. The data reduction schemes are shown to be accurate if the same radiation model is employed in the data reduction as was used in the flame simulation to generate the pressure trace used for post-processing. If the incorrect radiation model is employed, however, the errors can be quite large. The effects of stretch, radiation, and different data post-processing methodologies are explored and the errors quantified. Stretch is shown to be important for the early stages and the selected data range that is used for extrapolation has a significant effect on the extrapolated burning velocity. However, with an appropriate choice of data considered for extrapolation, the prediction of the unstretched burning velocity can be quite accurate.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Reviews and syntheses: The promise of big diverse soil data, moving current practices towards future potential

Abstract. In the age of big data, soil data are more available and richer than ever, but – outside of a few large soil survey resources – they remain largely unusable for informing soil management and understanding Earth system processes beyond the original study. Data science has promised a fully reusable research pipeline where data from past studies are used to contextualize new findings and reanalyzed for new insight. Yet synthesis projects encounter challenges at all steps of the data reuse pipeline, including unavailable data, labor-intensive transcription of datasets, incomplete metadata, and a lack of communication between collaborators. Here, using insights from a diversity of soil, data, and climate scientists, we summarize current practices in soil data synthesis across all stages of database creation: availability, input, harmonization, curation, and publication. We then suggest new soil-focused semantic tools to improve existing data pipelines, such as ontologies, vocabulary lists, and community practices. Our goal is to provide the soil data community with an overview of current practices in soil data and where we need to go to fully leverage big data to solve soil problems in the next century.

54 ENVIRONMENTAL SCIENCES↗