Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Transfer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

When to use rsync

We have endeavored to show, using a series of data transfer results obtained from two testbeds, when to use the popular data copying tool rsync and related tools. Tests have been conducted in local area network (LAN) and wide area network (WAN) environments. We conclude that for files in a certain size range and network latency ≦ 10 ms round trip time (RTT), rsync is still useful for data moving tasks in the category 4 of the U.S. DOE Technical Report “Data Movement Categories”. For more demanding data movement requirements, tools of different classes are suggested. Sample histograms from two DOE user facilities are provided to further support our conclusions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Modular Staged Pressurized Oxy-Combustion (SPOC) Power Plant for Coal and Biomass – Integration of Combustor Boiler and DCC

Utilities and grid operators worldwide are under significant pressure to incorporate intermittent renewable sources while strategizing on how to maintain the necessary stability and reliability of the grid. The development of a power plant which will be capable of flexible operation to meet the needs of the grid as more intermittent sources like wind and solar energy are incorporated is key to maintain the reliability of the grid. Through the use of innovative and cutting-edge technologies that improve efficiency and reduce carbon emissions, and being small compared to today's conventional utility-scale power plants, the modular Staged, Pressurized Oxy-Combustion (SPOC) process envisioned by and under development at Washington University in St. Louis (WUSTL) has the potential to achieve these goals. Specifically, the process offers: 1) a modular plant design that allows for better operational flexibility; 2) the utilization of fuel-staging and pressurized oxy-combustion, resulting in smaller plant size, improved plant efficiency, and reduced costs for pollutant and CO2 removal compared to traditional power plants with post-combustion capture technology; and 3) the use of small modular boilers and pollutant removal units that can be constructed off-site leading to reduced capital costs for the plant. WUSTL is advancing the development of the critical components for the SPOC power plant, including the integrated combustor-boiler system and the direct contact cooler. To demonstrate the boiler convective section and to obtain critical data for commercial-scale pressurized boiler design, a simulated convective heat transfer boiler test section was designed and integrated with the combustor (radiant section). A direct contact cooler (DCC) was integrated with the combustor-boiler to demonstrate the dynamic operation of the integrated system and the performance of the DCC, including the efficiency for simultaneous removal of NOx and SOx. This talk will present an update on the evaluation of the critical components for system integration, including heat transfer data from the simulated boiler, scrubbing efficiency for the DCC under different operating conditions, and CFD modeling and validation for burner and boiler development.

Magalhaes, Duarte↗
DOE OSTI2022

Understanding the Impact of Data Staging for Coupled Scientific Workflows

We report the rate of data generated by cutting-edge experimental science facilities and large-scale simulations enabled by current high-performance computing (HPC) systems has continued to grow at a far greater pace than the development of the network and storage capabilities on which these systems rely. To cope with this challenge, scientist are moving toward the creation of autonomous experiments and HPC simulations using machine learning. However, efficiently moving, storing, and processing large amounts of data away from the point of origin presents an incredible challenge. In-memory computing, in situ analysis, data staging, and data streaming are recognized viable alternatives to traditional file-based methods for transferring data between coupled workflows. However, the performance trade-offs and limitations for these methods are not fully understood when used in HPC applications. This article presents a comprehensive performance assessment of the current solutions for data staging when applied to applications that are not necessary I/O intensive which makes them not ideal candidates for these methods. Our study is based on experiments running at scale on Oak Ridge National Laboratory's Summit supercomputer using applications and simulations that cover typical computational motifs and patterns. We investigated the usability and cost/benefit trade-offs of staging algorithms for HPC applications under different scenarios and highlight opportunities for optimizing the dataflow between coupled simulation workflows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

FY 22 Project Name: Boots versus Bytes

IAEA has increasingly leveraged remote data transfer, amplifying the effectiveness of inspectors and analysts by allowing them to view data from Headquarters rather than requiring on-site activities. We propose that there may be even more opportunities to shift the international nuclear safeguards paradigm to remote activities through the implementation of enhanced data sharing and analysis.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2024

To Derive or Not to Derive: I/O Libraries Take Charge of Derived Quantities Computation

The ever-increasing volume of data produced by HPC simulations necessitates scalable methods for data exploration and knowledge extraction. Scientific data analysis often involves complex queries across distributed datasets, requiring manipulation of multiple primary variables and generating derived data that needs to be handled efficiently, creating challenges for applications that need to parse many large datasets. Relying on individual applications to handle all intermediate data generally leads to redundant computations across studies and unnecessary data transfers. In this paper, we investigate the performance of different approaches where applications define derived variables as quantities of interest (QoIs) and offload the computation and transfer of these QoIs to the I/O library. This significantly reduces redundancy and optimizes data movement across the distributed storage and processing infrastructure by allowing control over when and where derived variables are computed. We present a detailed analysis of the performance-storage trade-offs associated with different solutions and showcase results for our study on two large-scale datasets created from climate and combustion simulations.

Gainaru, Ana↗
DOE OSTI2025

University Data Management Pilot Utilizing the Nuclear Research Data System

Background In 2022, the Office of Science and Technology Policy (OSTP) issued a memo that significantly reshaped the landscape of access to federally funded research. The memo mandated that all taxpayer-funded research be made available to the public without delay upon publication, without an embargo period, superseding the 2013 OSTP public access policy. This public access policy promotes transparency and the democratization of knowledge, ensuring that the fruits of scientific endeavors funded by federal agencies could be immediately accessed and built upon by scientists, educators, students, and the public at large. To implement the requirements of the OSTP guidance and DOE Public Access Plan, the Office of Nuclear Energy (NE) has implemented public access plan guidance and has identified several areas where better data management practices would further expand public access to important nuclear energy related scientific data, reports, and other technical products. Significant NE supported efforts are already underway for data management and public access to important nuclear energy related data.1 2 To address gaps in data management practices, and improve retention and accessibility of data, NE is actively exploring enhanced data management options utilizing its high-performance computing resources administered by its Nuclear Scientific User Facility Program. A newly piloted system, the Nuclear Research Data System (NRDS) acts as a portal for data collection and dissemination. Nuclear Energy University Program Research and Development Portfolio According to Web of Science, NEUP has produced 2,345 journal publication that have been cited more than 61,000 times3 and countless conference proceedings. These publications are publicly available through OSTI.gov and in the open literature. Additional scientific and technical products including project milestones that are not publications and NEUP project final reports are vetted through OSTI.gov and released once reviewed and approved by DOE. Since 2009, NEUP has awarded close to 1,000 different R&D projects in technical areas across the NE research programs. As of June 2023, 512 NEUP reports are publicly available on OSTI. The underlying data for projects is still held at universities, and data transfer, co-location, and dissemination has not occurred in a systematic way. NEUP data is currently accessible through myriad university-based data repositories, or through direct requests to PIs. The program identified this patchwork of repositories, or often lack of publicly available data, as a significant barrier to an organized, accessible, and comprehensive solution to sharing data with the larger nuclear energy community. Approach The goal of this pilot project is to establish a pathway to a consolidated long-term repository for NEUP project data. To accomplish this goal, the pilot strives to accomplish the following objectives: Establish data collection standards, including a standard set of required supplementary information to contextualize and support raw data files. Work with the HPC group collect and upload information and to modify the NRDS system, as needed, to support a standardized approach. Resolve potential barriers to successful roll out of an expanded data collection strategy, including modifying data management plan guidelines and establishing a document and data release process that accounts for potential intellectual property and/or export control concerns. Results Overall, the pilot was successful in collecting 8,982 raw and processes data files, 220 reports, 56 calibration files, and 5,931 other supplementary documents. Supplementary documents included experimental plans, methods, journal publications and conference proceedings, milestone reports, and final reports. Figure 2 shows the number of data sets and supplementary project information provided by each project. Projects has significantly different input, depending on experimental data produced and completeness of the datasets provided.

Data collection↗
DOE OSTI2022

Source Term Analysis of Xenon (STAX): An effort focused on differentiating man-made isotope production from nuclear explosions via stack monitoring

An overview of the hardware and software developed for the Source Term Analysis of Xenon (STAX) project is presented which includes the data collection from two stack monitoring systems installed at medical isotope production facilities, infrastructure to transfer data to a central repository, and methods for sharing data from the repository with users. STAX is an experiment to collect radioxenon emission data from industrial nuclear facilities with the goal of developing a better understanding of the global radioxenon background and the effect industrial radioxenon releases have on nuclear explosion monitoring. The final goal of this work is to utilize collected data along with atmospheric transport modeling to calculate the contribution of a peak or set of peaks detected by the International Monitoring System (IMS) to provide desired discriminating information to the International Data Centre (IDC) and National Data Centers (NDCs). Types of data received from the STAX equipment are shown and collected data was used for a case study to predict radioxenon concentrations at two IMS stations closest to the Institute for RadioElements (IRE) in Belgium. The initial evaluation of results indicate that the data is very valuable to the nuclear explosion monitoring community.

07 ISOTOPE AND RADIATION SOURCES↗
DOE OSTI2026

Federated learning for 2D synchrotron x-ray diffractometry: a cross-institutional approach for phase quantification of Ti–6Al–4V alloy

High-energy Two dimensional (2D) synchrotron x-ray diffractometry provides important insights into the atomistic structure and phase evolution of materials, yet traditional analysis methods remain complex, knowledge-intensive, and computationally demanding. Deep-learning models offer a powerful alternative for automating their analysis. Institutions that hold these datasets may be unwilling to share their data due to privacy and security policies, as well as the challenges associated with large-scale data transfer. As a result, models trained on local datasets often perform well only on their own data but exhibit bias and poor generalization across different instruments or facilities. To overcome these limitations, we explore federated learning (FL) for 2D synchrotron diffractograms, enabling collaborative model training without exchanging raw data. In this study, 2D synchrotron diffractograms of Ti–6Al–4V alloy collected from two independent facilities are used to train convolutional neural networks for predicting the β-phase volume fraction. Experimental results show that federated global models significantly outperform locally trained models in terms of generalization and achieve accuracy comparable to centralized trained models. These findings demonstrate the potential of FL to enable secure, cross-institutional collaboration and enhance the scalability of deep-learning-based materials characterization.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Virtual Neuron: A Neuromorphic Approach for Encoding Numbers

Neuromorphic computers perform computations by emulating the human brain and are expected to be indispensable for energy-efficient computing in the future. They are primarily used in spiking neural network-based machine learning applications. However, neuromorphic computers are unable to preprocess data for these applications. Currently, data is preprocessed on a CPU or a GPU-this incurs a significant cost of transferring data from the CPU/GPU to the neuromorphic processor and vice versa. This cost can be avoided if preprocessing is done on the neuromorphic processor. To efficiently preprocess data on a neuromorphic processor, we first need an efficient mechanism for encoding data that can lend itself to all general-purpose preprocessing operations. Current encoding approaches have limited applicability and may not be suitable for all preprocessing operations. In this paper, we present the virtual neuron as a mechanism for encoding integers and rational numbers on neuromorphic processors. We evaluate the performance of the virtual neuron on physical and simulated neuromorphic hardware and show that it can perform an addition operation using 23 nJ of energy on average using a mixed-signal, memristor-based neuromorphic processor. The virtual neuron encoding approach is the first step in preprocessing data on a neuromorphic processor.

Date, Prasanna↗
DOE OSTI2023

Streaming Data from Experimental Facilities to Supercomputers for Real-Time Data Processing

In this paper we demonstrate direct data streaming from instruments and detectors at a large-scale experimental facility to a supercomputer for real-time data processing and feedback. Streaming data to supercomputers introduces the potential for novel scientific applications and workflow models, including the ability to provide real-time feedback from very large datasets during an experiment and the integration of real-time ML training and inference at scale. We discuss a successful demonstration for real-time processing of data from the Advanced Photon Source (APS) on the Polaris supercomputer using an EPICS-based streaming framework. We describe the capabilities of the streaming framework itself, and outline the architecture that allows us to process experimentally derived data on a supercomputer without file-based data transfers. We present throughput measurements that are indicative of system performance capable of sustaining the expected data production rates of the facility, as well as discuss some outstanding challenges and our future directions.

real-time processing↗
DOE OSTI2026

A scalable framework for efficient coupling of thermal and microstructural simulations in additive manufacturing

Predicting microstructure evolution in metal additive manufacturing (AM) is important for process optimization, but spatiotemporal scale disparities between thermal transport and microstructure evolution create significant challenges for efficient data transfer between simulation codes. To address this, we present Stork, a scalable framework for coupling thermal and microstructural simulations. Stork uses a sparse data representation to identify and store active solidification sub-volumes, enabling highly parallel quad-linear interpolation from coarse thermal grids to fine microstructure grids without large intermediate storage. We demonstrate the framework by coupling the semi-analytic heat transfer code 3DThesis with the time-parallel cellular automata code Toucan. This approach achieves over two orders of magnitude reduction in data generation time and file size compared to prior workflows. Numerical studies show that quad-linear interpolation preserves grain morphology and crystallographic texture in laser powder bed fusion (LPBF) simulations for coarsening ratios up to 16. Overall, Stork provides a scalable pathway for high-throughput, component-scale AM simulations on modern high-performance computing systems.

36 MATERIALS SCIENCE↗
DOE OSTI2025

BULKI-Store v0.3.2

BULKI-Store is a distributed object storage system optimized for high-performance computing environments. Built with a Rust core and Python bindings, it efficiently manages scientific and machine learning datasets across HPC clusters. The system employs a client-server architecture with MPI integration, enabling seamless scaling on supercomputers like Perlmutter. BULKI-Store's object-oriented approach provides intuitive data organization with rich metadata support, contrasting with traditional file-based solutions. Key optimizations include selective checkpoint loading, unified checkpoint files, and object chunking for large data transfers. For machine learning workloads, BULKI-Store offers advantages through fine-grained access patterns, dynamic data sharing between training instances, and reduced memory pressure. Memory management features include strategic Python GC calls, minimized data copies, and batch processing capabilities. The system leverages Rayon's thread pool for asynchronous data prefetching and supports multiple CPU architectures (ARM64, x86, AMD, RISC-V). By combining performance optimizations with developer-friendly APIs, BULKI-Store addresses the complex data management challenges of modern HPC applications while maintaining compatibility across heterogeneous computing environments.

Zhang, Wei [Lawrence Berkeley National Laboratory ↗
DOE OSTI2023

A COUPLED DETERMINISTIC TRANSPORT CALCULATION IN COMSOL USING PROPER ORTHOGONAL DECOMPOSITION

A coupled physics problem consisting of radiation transport and heat transfer was modeled using COMSOL Multiphysics to simulate a criticality accident. Reduced order models (ROM) were created to accelerate the radiation transport calculation through Proper Orthogonal Decomposition (POD). Additionally, on-the-fly neutron cross sections were generated from a non-linear function through a feed forward neural network. The Livelink for MATLAB module allowed for data transfer between an otherwise independent solver and in-house modules. A data driven approach to POD was developed to act as a surrogate model to couple with heat transfer, with the POD module written in MATLAB coupled to an isolated heat transfer solve in COMSOL. A coupled transient solution resulted in a good approximation of the flux and temperature, with the maximum error being 3E-2 and 2.5E-3 respectively.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2023

TomocuPy – efficient GPU-based tomographic reconstruction with asynchronous data processing

Fast 3D data analysis and steering of a tomographic experiment by changing environmental conditions or acquisition parameters require fast, close to real-time, 3D reconstruction of large data volumes. Here a performance-optimized TomocuPy package is presented as a GPU alternative to the commonly used central processing unit (CPU) based TomoPy package for tomographic reconstruction. TomocuPy utilizes modern hardware capabilities to organize a 3D asynchronous reconstruction involving parallel read/write operations with storage drives, CPU–GPU data transfers, and GPU computations. In the asynchronous reconstruction, all the operations are timely overlapped to almost fully hide all data management time. Since most cameras work with less than 16-bit digital output, the memory usage and processing speed are furthermore optimized by using 16-bit floating-point arithmetic. As a result, 3D reconstruction with TomocuPy became 20–30 times faster than its multi-threaded CPU equivalent. Full reconstruction (including read/write operations and methods initialization) of a 2048 3 tomographic volume takes less than 7 s on a single Nvidia Tesla A100 and PCIe 4.0 NVMe SSD, and scales almost linearly increasing the data size. To simplify operation at synchrotron beamlines, TomocuPy provides an easy-to-use command-line interface. Efficacy of the package was demonstrated during a tomographic experiment on gas-hydrate formation in porous samples, where a steering option was implemented as a lens-changing mechanism for zooming to regions of interest.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

High-Throughput Data Processing at FRIB Using ESnet

Real-time or nearly real-time (nearline) data processing methods are critical tools as detector technologies and data acquisition (DAQ) systems allow for higher data rates and volumes. The introduction of the energy sciences network (ESnet), a U.S. Department of Energy (DOE) supported high-speed network for scientific research, creates opportunities to leverage the computing power of DOE facilities like the National Energy Research Scientific Computing Center (NERSC). As a first step toward realizing a DOE Office of Science Integrated Research Infrastructure (IRI) pattern, an automated workflow was developed to remotely process data obtained from a nuclear physics experiment at the Facility for Rare Isotope Beams (FRIB) at NERSC with data transferred between FRIB and NERSC over ESnet. The workflow demonstrated the ability to process one week’s worth of experimental data in approximately 90 min and was used successfully for nearline analysis during a recently completed FRIB experiment. Here, a summary of the workflow development and results of recent demonstrations will be presented.

Data processing↗
DOE OSTI2021

Flow condensation heat transfer performance of natural and emerging synthetic refrigerants

There is great interest in predicting flow condensation heat transfer for lower global warming potential (GWP) fluids. This paper analyzes the efficacy of common flow condensation correlations developed for particular fluids in order to identify their suitability to predict heat transfer performance of low GWP fluids. Condensation heat transfer data were extracted from the literature, including 19 papers and 1,473 data points for natural refrigerants [i.e., ammonia (R717), CO 2 (R744), propane (R290), isobutane (R600a)] and 35 papers and 5,030 data points for synthetic refrigerants [i.e., R12, R1234yf, R1234ze(E), R1234ze(Z), R22, R32, R41, R123, R125, R134a, R142b, R152a, R161, R404A, R410A, R448A, R449A, R450A, R452B, R454C, R455A, R513A] encompassing tube diameters of 0.1–11.5 mm, mass fluxes of 55–1200 kg/m2s, and saturation temperatures of -25°C–65°C. Correlations analyzed included Akers et al. (1959), Cavallini et al. (2006, 2011), Kim and Mudawar (2013), Macdonald and Garimella (2016), Shah (1979, 2009, 2013, 2016) and Traviss et al. (1973) for smooth tubes and Chamra et al. (2005) and Kedzierski and Goncalves (1999) for enhanced tubes. Since most studies did not report wall temperature, correlations which relied on wall temperature directly or indirectly were excluded from the analysis. For synthetic refrigerants, mean average error (MAE) ranged from 6%–257%, and Cavallini et al. (2011) and Kim and Mudawar (2013) were the best predictors for emerging synthetic refrigerants. Furthermore, the Kim and Mudawar (2013) correlation was found to best predict the heat transfer performance for propane and R600a data, but most correlations did not accurately predict ammonia and CO 2 flow condensation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

A Holistic Algorithmic Approach to Improving Accuracy, Robustness, and Computational Efficiency for Atmospheric Dynamics

Atmospheric weather and climate models must perform simulations very quickly to be useful. Therefore, modelers have traditionally focused on reducing computations as much as possible. However, in our new era of increasingly compute-capable hardware, data movement is now the prohibiting expense. This study examines the computational benefits of a new algorithmic approach to modeling atmospheric dynamics on scales relevant to weather and climate simulation. Rather than minimizing computations, this new approach considers the larger problem more holistically, including spatial accuracy, temporal accuracy, robustness (i.e., oscillations), on-node efficiency, and internode data transfers together at once. Numerical experiments demonstrate how computations can be strategically increased to simultaneously address each of these constraints while reducing data movement to adapt to modern accelerated hardware. The new algorithm can achieve at times up to 80% peak floating point throughput in single precision on the Nvidia Tesla V100 GPU, where the traditional approach is shown to only achieve single-digit floating point efficiency. Further, the new algorithm is twice as fast as a standard Runge--Kutta time integrator, and high-order accuracy with Weighted Essentially Non-Oscillatory (WENO) limiting came at less than 30% additional runtime cost on a GPU, thus increasing the accuracy per degree of freedom.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Transition from wall modes to multimodality in liquid gallium magnetoconvection

Coupled laboratory-numerical experiments of Rayleigh-Bénard convection in liquid gallium subject to a vertical magnetic field are presented. The experiments are carried out in two cylindrical containers with diameter-to-height aspect ratio Γ = 1.0 and 2.0 at varying thermal forcing (Rayleigh numbers 10 5 ≲ Ra ≲ 10 8 ) and magnetic field strength (Chandrasekhar numbers 0 ≲ Ch ≲ 3 × 10 5 ). Laboratory measurements and numerical simulations confirm that magnetoconvection in our finite cylindrical tanks onsets via nondrifting wall-attached modes, in good agreement with asymptotic predictions for a semi-infinite domain. With increasing supercriticality, the experimental and numerical thermal measurements and the numerical velocity data reveal transitions between wall mode states with different azimuthal mode numbers and between wall-dominated convection to wall and interior multimodality. These transitions are also reflected in the heat transfer data, which combined with previous studies connect onset to supercritical turbulent behaviors in liquid metal magnetoconvection over a large parameter space. The gross heat transfer behaviors between magnetoconvection and rotating convection in liquid metals are compared and discussed.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗