Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data transfer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Sim-Situ: A Framework for the Faithful Simulation of in situ Processing

The amount of data generated by numerical simulations in various scientific domains led to a fundamental redesign of how the analysis and visualization of simulation outputs are performed. The throughput and capacity of storage subsystems have not evolved as fast as the computing power in extreme-scale supercomputers, making the classical post-hoc approach highly inefficient. In situ processing has then emerged as a solution in which simulation and data analysis/visualization are intertwined for better performance and greater interactivity.Determining the best allocation, i.e., how many resources to allocate to simulation and analysis respectively, mapping, i.e., where and at which frequency to run the analysis/visualization, and data transfer mode is a complex task whose performance assessment is crucial to the efficient execution of in situ processing. However, such a performance evaluation of different strategies usually relies either on directly running them on the targeted execution environments, which can rapidly become extremely time- and resource-consuming, or on resorting to simplified models of the components of an in situ application, which can lack of realism. In both cases, the validity of the performance evaluation is limited.In this paper, we present Sim-Situ, a simulation-based framework for the faithful performance evaluation of in situ processing strategies. We designed Sim-Situ to reflect the typical features of in situ processing systems. Thanks to its modular design, Sim-situ has the necessary flexibility to easily and faithfully evaluate the behavior and performance of various allocation, mapping, and data transfer strategies. We illustrate the simulation capabilities of Sim-Situ on a Molecular Dynamics use case. We study the impact of different strategies on performance and show how users can leverage Sim-Situ to determine interesting tradeoffs when adding analysis/visualization components to their application.

Honoré, Valentin↗
DOE OSTI2022

An Adaptive Geometry-Free Thermo-Mechanical Model for Directed Energy Deposition Process Modeling

This presentation describes a novel, geometry-free thermo-mechanical model with adaptive subdomain con- struction to accurately predict the thermal conditions, distortions, and residual stresses throughout the directed energy deposition (DED) process. A novel finite element workflow is designed to con- duct the numerical analysis, based on the multi-app and data transfer capabilities in the open-source Multiphysics Object-Oriented Simulation Environment (MOOSE). Unlike with traditional methods, the part geometry in this model is not predefined. Instead, it is a combined effect of the processing parameters and material properties. At each time step, the model utilizes a subdomain construction paradigm to model the material deposition. A specialized mesh adaptivity scheme is incorporated to provide an accurate prediction while reducing the overall computational cost. The results generated by the proposed model show general agreement with the experimental measurements for the single track scan with varying processing parameters and demonstrate reasonable predictions for higher material buildups.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Enabling Low-Overhead HT-HPC Workflows at Extreme Scale using GNU Parallel

GNU Parallel is a versatile and powerful tool for process parallelization widely used in scientific computing. This paper demonstrates its effective application in high-performance computing (HPC) environments, particularly focusing on its scalability and efficiency in executing large-scale high-throughput high-performance computing (HT-HPC) workflows. Through real-world examples, we highlight GNU Parallel’s performance across various HPC workloads, including GPU computing, container-based workloads, and node-local NVMe storage. Our results on two leading supercomputers, OLCF’s Frontier and NERSC’s Perlmutter, showcase GNU Parallel’s rapid process dispatching ability and its capacity to maintain low overhead even at extreme scales. We explore GNU Parallel’s application in massive parallel file transfers using a scheduled Data Transfer Node (DTN) cluster, emphasizing its broad utility in diverse scientific workflows. Beyond its direct application as a viable workflow manager, GNU Parallel can be employed in conjunction with other workflow systems as a "last-mile" parallelizing driver and as a quick prototyping tool to design and extract parallel profiles from application executions. We then argue that the potential for GNU Parallel to transform workflow management at extreme scales is substantial, paving the way for more efficient and effective scientific discoveries.

Maheshwari, Ketan↗
DOE OSTI2025

Enhancing Cluster Identification in Atom Probe Tomography Data Using Transfer Learning

Atom probe tomography (APT) has enabled the direct visualization of solute clusters, providing valuable insights into material structures. This clustering is crucial for understanding the nanoscale composition and behavior of materials, which can significantly influence their mechanical and physical properties. However, the widely used clustering methods in the APT community face challenges such as subjective parametric selection and limited applicability, particularly in dealing with overlapping clusters, nested clusters, and artifacts across different scales, such as precipitates and dislocations. To address these challenges, we present a framework based on density-based cluster analysis that aims to be less dependent on user input, reproducible, and robust.

Density-based clustering↗
DOE OSTI2025

Portable and Adaptable Neutron Diagnostics for Advancing Fusion Energy Science Addendum

Activation detectors developed at LLNL for measuring real-time neutron fluence from fusion sources are used in the broader fusion community. The recommended fluence operating range of this diagnostic is 5x10 2 – 1x10 6 n/cm2. The upper limit on this fluence range is set by the dead time caused by data transfer between the detector and data acquisition computer. Delaying the start of counting is a possible strategy to operate these detectors in higher fluences.

42 ENGINEERING↗
DOE OSTI2020

Distributed Computing for the Project 8 Experiment

The Project 8 collaboration aims to measure the absolute neutrino mass or improve on the current limit by measuring the tritium beta decay electron spectrum. We present the current distributed computing model for the Project 8 experiment. Project 8 is in its second phase of data taking with a near continuous data rate of 1Gbps. The current computing model uses DIRAC (Distributed Infrastructure with Remote Agent Control) for its workflow and data management. A detailed meta-data assignment using the DIRAC File Catalog is used to automate raw data transfers and subsequent stages of data processing. The DIRAC system is deployed on containers managed using a Kubernetes cluster to provide a scalable infrastructure. A modified DIRAC Site Director provides the ability to submit jobs using Singularity on opportunistic High-Performance Computing (HPC) sites.

Distributed Computing, Kubernetes, DIRAC, Project ↗
DOE OSTI2022

FPGA-based computing system for processing data in size, weight, and power constrained environments

Technologies that are well-suited for use in size, weight, and power (SWAP)-constrained environments are described herein. A host controller dispatches data processing instructions to hardware acceleration engines (HAEs) of one or more field programmable gate arrays (FPGAs) and further dispatches data transfer instructions to a memory controller, such that the HAEs perform processing operations on data stored in local memory devices of the HAEs in parallel with other data being transferred from external memory devices coupled to the FPGA(s) to the local memory devices.

Napier, Matthew↗
DOE OSTI2023

A novel transfer learning framework for sorghum biomass prediction using UAV-based remote sensing data and genetic markers

Yield for biofuel crops is measured in terms of biomass, so measurements throughout the growing season are crucial in breeding programs, yet traditionally time- and labor-consuming since they involve destructive sampling. Modern remote sensing platforms, such as unmanned aerial vehicles (UAVs), can carry multiple sensors and collect numerous phenotypic traits with efficient, non-invasive field surveys. However, modeling the complex relationships between the observed phenotypic traits and biomass remains a challenging task, as the ground reference data are very limited for each genotype in the breeding experiment. In this study, a Long Short-Term Memory (LSTM) based Recurrent Neural Network (RNN) model is proposed for sorghum biomass prediction. The architecture is designed to exploit the time series remote sensing and weather data, as well as static genotypic information. As a large number of features have been derived from the remote sensing data, feature importance analysis is conducted to identify and remove redundant features. A strategy to extract representative information from high-dimensional genetic markers is proposed. To enhance generalization and minimize the need for ground reference data, transfer learning strategies are proposed for selecting the most informative training samples from the target domain. Consequently, a pre-trained model can be refined with limited training samples. Field experiments were conducted over a sorghum breeding trial planted in multiple years with more than 600 testcross hybrids. The results show that the proposed LSTM-based RNN model can achieve high accuracies for single year prediction. Further, with the proposed transfer learning strategies, a pre-trained model can be refined with limited training samples from the target domain and predict biomass with an accuracy comparable to that from a trained-from-scratch model for both multiple experiments within a given year and across multiple years.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Complete and Correct Transfer of Information (CACTI)

Many distributed systems, file transfer mechanisms, and message passing systems offer reliability mechanisms such as acknowledgements, retries, and durability. While these tools may be “good enough” for their typical use cases, they may not offer sufficient coverage for the wide range of faults that impact data transfers and communication. A gap in the reliability measures may lead to some small amount of data loss. Some high-consequence systems cannot tolerate the loss or corruption of even a single record. We present seven principles that will counter a wide range of faults and protect against data loss and corruption. These principles bring together lessons learned from a wide range of technologies and can inform appropriate system design and application usage. These principles will help readers reason on how prevent data loss in a multi-hop pipeline and how to properly use tools that may have a deficiency in reliability.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Engaging the Regulatory Community to Aid Environmental Consenting/Permitting Processes for Marine Renewable Energy

Regulators involved in consenting/permitting marine renewable energy (MRE) have faced multiple challenges due to relatively new, unfamiliar technologies and uncertainty surrounding potential environmental impacts. This has resulted in slow progress for the MRE industry, including long consenting timeframes and extensive and expensive monitoring requirements, which increase financial risk for investors. OES-Environmental has surveyed regulators internationally to understand their key knowledge gaps and perspectives to support the development of the MRE industry. From the results of these surveys a data transferability process and a risk retirement pathway have been developed to assess consenting and monitoring requirements in proportion to risk. A tool for discovering existing data sets by using an online matrix has been developed, along with training materials, regulatory guidance documents, and a strategic outreach plan to engage regulators and advisers. his engagement and the application of these products should lead to a better understanding of the environmental effects of marine energy, and more efficient consenting processes.

16 TIDAL AND WAVE POWER↗
DOE OSTI2023

Leveraging History to Predict Infrequent Abnormal Transfers in Distributed Workflows

Scientific computing heavily relies on data shared by the community, especially in distributed data-intensive applications. This research focuses on predicting slow connections that create bottlenecks in distributed workflows. In this study, we analyze network traffic logs collected between January 2021 and August 2022 at the National Energy Research Scientific Computing Center (NERSC). Based on the observed patterns, we define a set of features primarily based on history for identifying low-performing data transfers. Typically, there are far fewer slow connections on well-maintained networks, which creates difficulty in learning to identify these abnormally slow connections from the normal ones. We devise several stratified sampling techniques to address the class-imbalance challenge and study how they affect the machine learning approaches. Our tests show that a relatively simple technique that undersamples the normal cases to balance the number of samples in two classes (normal and slow) is very effective for model training. This model predicts slow connections with an F1 score of 0.926.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Controlling the helicity of light by electrical magnetization switching

Controlling the intensity of emitted light and charge current is the basis of transferring and processing information. By contrast, robust information storage and magnetic random-access memories are implemented using the spin of the carrier and the associated magnetization in ferromagnets. In this study, the missing link between the respective disciplines of photonics, electronics and spintronics is to modulate the circular polarization of the emitted light, rather than its intensity, by electrically controlled magnetization. Here we demonstrate that this missing link is established at room temperature and zero applied magnetic field in light-emitting diodes through the transfer of angular momentum between photons, electrons and ferromagnets. With spin-orbit torque a charge current generates also a spin current to electrically switch the magnetization. This switching determines the spin orientation of injected carriers into semiconductors, in which the transfer of angular momentum from the electron spin to photon controls the circular polarization of the emitted light. The spin-photon conversion with the nonvolatile control of magnetization opens paths to seamlessly integrate information transfer, processing and storage. Our results provide substantial advances towards electrically controlled ultrafast modulation of circular polarization and spin injection with magnetization dynamics for the next-generation information and communication technology, including space-light data transfer. The same operating principle in scaled-down structures or using two-dimensional materials will enable transformative opportunities for quantum information processing with spin-controlled single-photon sources, as well as for implementing spin-dependent time-resolved spectroscopies.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2024

2024 OES-Environmental 2024 State of the Science Report, Chapter 8: Marine Renewable Energy Data and Information Systems

As the marine renewable energy (MRE) sector grows, large amounts of environmental and technical data and information are being collected. When these data and information are openly available, they can be used to guide research and development, inform responsible siting and consenting of projects, and increase stakeholder understanding through transparency. For example, quality environmental data collected during the siting, consenting, construction, operation, and decommissioning of MRE projects can all play key roles in better characterizing baseline conditions, developing effective monitoring and mitigation strategies, and retiring environmental risks through data transferability (see Chapter 6). Ensuring that these data and information are easily discoverable and accessible will help the MRE sector make informed decisions and coexist in an increasingly busy ocean environment.

16 TIDAL AND WAVE POWER↗
DOE OSTI2025

DoCeph: DPU-Offloaded Messaging in Ceph for Reduced Host CPU Utilization

Ceph is a widely used distributed object store, but its messenger layer imposes substantial CPU overhead on the host. To address this limitation, we propose DoCeph, a DPU-offloaded storage architecture for Ceph that disaggregates the system by offloading the communication-intensive messaging component to the DPU while retaining the storage backend on the host. The DPU efficiently manages communication, using lightweight RPC for metadata operations and DMA for data transfer. Moreover, DoCeph introduces a pipelining technique that overlaps data transmission with buffer preparation, mitigating hardware-imposed transfer size limitations. We implemented DoCeph on a Ceph cluster with NVIDIA BlueField-3 DPUs. Evaluation results indicate that DoCeph cuts host CPU usage by up to 92% while sustaining stable throughput and providing larger performance benefits for object writes over 1 MB.

Park, Kuri [Sogang University]↗
DOE OSTI2021

Pressurizer dynamic model and emulated programmable logic controllers for nuclear power plants cybersecurity investigations

This work demonstrates the functionality of the pressurizer using a fast-running three regions, non-equilibrium model and control programs of emulated PLCs. The state variables from an integrated model of primary loop in a representative PWR plant are communicated to the pressurizer’s emulated PLCs using a synchronized data transfer function. In turn, the PLCs communicate back instructions to the pressurizer model to adjust the pressure and water level to remain within preprogramed setpoints. Pressurizer model simultaneously solves the coupled mass and energy conservation equations in the saturated vapor and saturated and subcooled liquid regions using fixed step solver. Results demonstrate the response of the pressurizer model linked to an emulated pressure and water level PLCs in a simulated transient involving surge-in and surge-out events. The 50 ms response delay time for the emulated PLC insignificantly affects operation and predictions of the pressurizer model.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Applying an Oriented Divergence Theorem to Swept Face Remap

Here we present a novel oriented divergence theorem and apply the results to a swept face remap method (conservative data transfer between two meshes) in arbitrary Langrangian–Eulerian hydrodynamics. In our setting, we compute the material flux along swept regions between corresponding faces in the source and target meshes. Since the swept region may add material, subtract material, or do both when it intersects itself, we cannot apply the conventional divergence theorem without accounting for orientation and self-overlaps. In this work, we encode the swept region orientation and geometry with a map from the unit n -dimensional cube, and then apply an oriented analog of divergence theorem to compute the material flux. We present efficient implementation strategies for the presented method. We also provide numerical evidence supporting our results and discuss extensions to more general mesh topologies.

97 MATHEMATICS AND COMPUTING↗