SEARCH · Engineering Papers

Results for “data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

All sources NASA NTRS DOE OSTI

DOE OSTI2024

ENDFtk: A robust tool for reading and writing ENDF-formatted nuclear data

ENDFtk is a recently developed C++ and Python interface to interact with ENDF-6 formatted nuclear data files. It provides a robust and complete interface, allowing the reading and writing of all formats currently part of the ENDF-6 formats manual, as well as some non-ENDF formats used by the NJOY processing code. It provides an interface that mimics the names in the ENDF-6 formats manual as well as an equivalent interface using human-readable attribute names. It is robust and powerful enogh for nuclear data experts to develop complex applications, while also simple enough to be used non-experts to retrieve and manipulate evaluated nuclear data. ENDFtk offers the ability to easily interrogate and manipulate data either in large-scale code projects or in simple Python scripts. Here, in this paper, a brief overview of the interface is given, as well as more substantial examples demonstrating plotting simple data, interacting with more complex data, and writing new data to files. ENDFtk is open source and available for download via GitHub (https://github.com/njoy/ENDFtk).

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

Nuclear Data Uncertainty Quantification: Past, Present and Future

An historical overview is provided of the mathematical foundations of uncertainty quantification and the roles played in the more recent past by nuclear data uncertainties in nuclear data evaluations and nuclear applications. Significant advances that have established the mathematical framework for contemporary nuclear data evaluation methods, as well as the use of uncertainty information in nuclear data evaluation and nuclear applications, are described. This is followed by a brief examination of the current status concerning nuclear data evaluation methodology, covariance data generation, and the application of evaluated nuclear data uncertainties in contemporary nuclear technology. A few possible areas for future investigation of this subject are also suggested.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2018

Efficiency and Reliability Analyses of AC and 380V DC Distribution in Data Centers

Modern data centers consume large amounts of electricity, resulting in high operational costs. The efficiency of a data center power distribution system can be increased, and the operational cost reduced, if the number of power conversion stages can be minimized and more efficient converters, such as wide bandgap (WBG) converters, are used. This can be achieved by using DC distribution voltage at the rack level by eliminating extra conversion stages. In this paper, benchmarks for both AC and 380V DC data centers were developed and efficiency analyses were performed for an entire year. The impact of integrating photovoltaic (PV) systems into the data centers has also been analyzed in both cases in terms of efficiency. The results show that 380V DC data centers are more efficient than AC data centers with and without PV integration. Furthermore, the reliability of an AC system was compared to a 380V DC architecture with Tier-IV standard. Monte-Carlo simulations were used to perform reliability analyses for different levels of redundancy in the Uninterruptible Power Supply (UPS) system for both cases. In conclusion, the simulation results showed that the 380V DC distribution system had a higher level of reliability than the AC distribution system in data centers.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2019

Delineating Facies Spatial Distribution by Integrating Ensemble Data Assimilation and Indicator Geostatistics with Level-Set Transformation

The facies-based approach has been widely adopted to delineate an aquifer into distinct geological units with unique distributions of hydraulic, physical, and/or chemical properties. The recent development in ensemble-based data assimilation methods allows both the direct and indirect data to be used to improve facies delineation. A major difficulty in those applications is to honor the spatial continuity and avoid overfitting after data assimilation. We introduce a new facies delineation framework to integrate ensemble data assimilation with traditional transition probability-based geostatistics. A level-set concept is used to parametrize discrete facies indicators and for updating facies shape. During the iterative data assimilation process, we impose spatial continuity by conditioning facies field generation on points selected adaptively based on their sensitivity to observation data. This reconditioning step is a key step to maintain spatial continuity and overcome over-fitting problems in inversion. We selected two examples to evaluate the performance of the new framework in estimating facies-based permeability field. The first example is a two-dimensional synthetic system with transient head data induced by pumping tests used for delineating two facies. The second example is a three-dimensional case with three facies, conceptualized from a field tracer experiment within the Columbia River corridor in Washington State, USA. Both examples demonstrate that the new method can adequately capture the spatial pattern of hydrofacies with reconditioning, which leads to the improved prediction of system behaviors.

Song, Xuehang↗

DOE OSTI2019

RDMA vs. RPC for Implementing Distributed Data Structures

Distributed data structures are key to implementing scalable applications for scientific simulations and data analysis. In this paper we look at two implementation styles for distributed data structures: remote direct memory access (RDMA) and remote procedure call (RPC). We focus on operations that require individual accesses to remote portions of a distributed data structure, e.g., accessing a hash table bucket or distributed queue, rather than global operations in which all processors collectively exchange information. We look at the trade-offs between the two styles through microbenchmarks and a performance model that approximates the cost of each. The RDMA operations have direct hardware support in the network and therefore lower latency and overhead, while the RPC operations are more expressive but higher cost and can suffer from lack of attentiveness from the remote side. We also run experiments to compare the real-world performance of RDMA- and RPC-based data structure operations with the predicted performance to evaluate the accuracy of our model, and show that while the model does not always precisely predict running time, it allows us to choose the best implementation in the examples shown. We believe this analysis will assist developers in designing data structures that will perform well on current network architectures, as well as network architects in providing better support for this class of distributed data structures.

distributed data structures↗

DOE OSTI2025

Data and scripts from: “Denoising autoencoder for reconstructing sensor observation data and predicting evapotranspiration: noisy and missing values repair and uncertainty quantification”

This data package includes data and scripts from the manuscript “Denoising autoencoder for reconstructing sensor observation data and predicting evapotranspiration: noisy and missing values repair and uncertainty quantification”.The study addressed common challenges faced in environmental sensing and modeling, including uncertain input data, missing sensor observations, and high-dimensional datasets with interrelated but redundant variables. Point-scaled meteorological and soil sensor observations were perturbed with noises and missing values, and denoising autoencoder (DAE) neural networks were developed to reconstruct the perturbed data and further predict evapotranspiration. This study concluded that (1) the reconstruction quality of each variable depends on its cross-correlation and alignment to the underlying data structure, (2) uncertainties from the models were overall stronger than those from the data corruption, and (3) there was a tradeoff between reducing bias and reducing variance when evaluating the uncertainty of the machine learning models.This package includes:(1) Four ipython scripts (.ipynb): “DAE_train.ipynb” trains and evaluates DAE neural networks, “DAE_predict.ipynb” makes predictions from the trained DAE models, “ET_train.ipynb” trains and evaluates ET prediction neural networks, and “ET_predict.ipynb” makes predictions from trained ET models.(2) One python file (.py): “methods.py” includes all user-defined functions and python codes used in the ipython scripts.(3) A “sub_models” folder that includes five trained DAE neural networks (in pytorch format, .pt), which could be used to ingest input data before being fed to the downstream ET models in ‘ET_train.ipynb” or ‘ET_predict.ipynb’.(4) Two data files (.csv). Daily meteorological, vegetation, and soil data is in “df_data.csv”, where “df_meta.csv” contains the location and time information of “df_data.csv”. Each row (index) in “df_meta.csv” corresponds to each row in “df_data.csv”. These data files are formatted to follow the data structure requirements and be directly used in the ipython scripts, and they have been shuffled chronologically to train machine learning models. The meteorological and soil data was collected using point sensors between 2019-2023 at(4.a) Three shrub-dominated field sites in East River, Colorado (named “ph1”, “ph2” and “sg5” in “df_meta.csv”, where “ph1” and “ph2” were located at PumpHouse Hillslopes, and “sg5” was at Snodgrass Mountain meadow) and(4.b) One outdoor, mesoscale, and herbaceous-dominated experiment in Berkeley, California (named “tb” in “df_meta.csv”, short for Smartsoils Testbed at Lawrence Berkeley National Lab).- See "df_data_dd.csv" and "df_meta_dd.csv" for variable descriptions and the Methods section for additional data processing steps. See "flmd.csv" and "README.txt" for brief file descriptions.- All ipython scripts and python files are written in and require PYTHON language software.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

CORAL: A framework for rigorous self-validated data modeling and integrative, reproducible data analysis

Abstract Background Many organizations face challenges in managing and analyzing data, especially when relevant datasets arise from multiple sources and methods. Analyzing heterogeneous datasets and additional derived data requires rigorous tracking of their interrelationships and provenance. This task has long been a Grand Challenge of data science and has more recently been formalized in the FAIR principles: that all data objects be Findable, Accessible, Interoperable, and Reusable, both for machines and for people. Adherence to these principles is necessary for proper stewardship of information, for testing regulatory compliance, for measuring the efficiency of processes, and for facilitating reuse of data-analytical frameworks. Findings We present the Contextual Ontology-based Repository Analysis Library (CORAL), a platform that greatly facilitates adherence to all 4 of the FAIR principles, including the especially difficult challenge of making heterogeneous datasets Interoperable and Reusable across all parts of a large, long-lasting organization. To achieve this, CORAL's data model requires that data generators extensively document the context for all data, and our tools maintain that context throughout the entire analysis pipeline. CORAL also features a web interface for data generators to upload and explore data, as well as a Jupyter notebook interface for data analysts, both backed by a common API. Conclusions CORAL enables organizations to build FAIR data types on the fly as they are needed, avoiding the expense of bespoke data modeling. CORAL provides a uniquely powerful platform to enable integrative cross-dataset analyses, generating deeper insights than are possible using traditional analysis tools.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

RC-SFA Data Management Templates and Guidance for Standardized, Reusable AI-Ready Data Packages

This data package provides templates and supporting documentation developed by the River Corridor Science Focus Area (RC-SFA; https://www.pnnl.gov/projects/river-corridor) to communicate its approach to managing and publishing AI-ready data. The package is intended to help data users and data producers understand the structures, metadata practices, and quality-control approaches that support consistent, reusable, and machine-actionable data products across RC-SFA studies. Rather than focusing on a single experimental dataset, this package documents the data management framework used to make RC-SFA data easier to find, ingest, navigate, and interpret. The materials in this package reflect RC-SFA practices for standardized data package organization, including the use of a human- and machine-readable README, file-level metadata, data dictionaries, descriptive file naming, method identifiers, and automated and review-based quality assurance procedures. Together, these components illustrate how RC-SFA extends FAIR data principles toward AI-readiness by prioritizing deep metadata, consistency across data packages, and support for informed downstream reuse by both humans and computational tools. This dataset is comprised of (1) readme; (2) presentation slides with an overview of RC-SFA approach and guidance; (3) document of RC-SFA best practices; (4) data dictionary (dd); (5) file level metadata (flmd); and a subfolder containing templates for dd and flmd. All files are .csv and .pdf. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About.

AI-readiness↗

DOE OSTI2019

Recommended Nuclear Data for the Production of Selected Therapeutic Radionuclides

Increasing interest in the treatment of human disease by means of targeted radionuclide-based therapies requires an accurate understanding of achievable yields and quantification of realisable degrees of purity. Important studies within a more extensive Coordinated Research Project of the International Atomic Energy Agency (IAEA CRP) have included in-depth assessments and evaluations of all available measured cross-section data for ten nuclear reactions leading to the formation of 131 Cs, 131 Ba, 225 Ac, 225 Ra, 227 Th, 227 Ac, 230 U and 230 Pa for various on-going and proposed therapeutic applications in nuclear medicine. Equivalent studies were also undertaken of possible reactions to produce the 178W /low-spin 178m Ta generator system that has been clinically applied in diagnosis by means of SPECT - these particular cross-section studies and recommendations are included in this paper for completeness. All available measured data have been critically reviewed and assessed to produce statistically consistent datasets. Selected datasets have been fitted by a least-squares method with Padé approximations of variable order to evaluate the production cross sections and corresponding uncertainties. New measurements have been made in some instances, and reported data have been adjusted to accommodate new nuclear decay or monitor reaction data which altered the original reported quantities adversely. Insufficient measured data are available in many cases to permit a reliable evaluation of cross sections, and these instances are discussed. Finally, recommended therapeutic radionuclide production reaction data and their uncertainties are available on the Web page of the IAEA Nuclear Data Section at www-nds.iaea.org/radionuclides/ and also at the IAEA medical portal www-nds.iaea.org/medportal/.

63 RADIATION, THERMAL, AND OTHER ENVIRON. POLLUTAN↗

DOE OSTI2017

Scidac-Data: Enabling Data Driven Modeling of Exascale Computing

Here, the SciDAC-Data project is a DOE-funded initiative to analyze and exploit two decades of information and analytics that have been collected by the Fermilab data center on the organization, movement, and consumption of high energy physics (HEP) data. The project analyzes the analysis patterns and data organization that have been used by NOvA, MicroBooNE, MINERvA, CDF, D0, and other experiments to develop realistic models of HEP analysis workflows and data processing. The SciDAC-Data project aims to provide both realistic input vectors and corresponding output data that can be used to optimize and validate simulations of HEP analysis. These simulations are designed to address questions of data handling, cache optimization, and workflow structures that are the prerequisites for modern HEP analysis chains to be mapped and optimized to run on the next generation of leadership-class exascale computing facilities. We present the use of a subset of the SciDAC-Data distributions, acquired from analysis of approximately 71,000 HEP workflows run on the Fermilab data center and corresponding to over 9 million individual analysis jobs, as the input to detailed queuing simulations that model the expected data consumption and caching behaviors of the work running in high performance computing (HPC) and high throughput computing (HTC) environments. In particular we describe how the Sequential Access via Metadata (SAM) data-handling system in combination with the dCache/Enstore-based data archive facilities has been used to develop radically different models for analyzing the HEP data. We also show how the simulations may be used to assess the impact of design choices in archive facilities.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

BRAINformat: A Data Standardization Framework for Neuroscience Data

Neuroscience is entering the era of `extreme data` with little experience and few plans for the associated volume, velocity, variety, and veracity challenges. This is a serious impediment for both the sharing of data across labs, as well as the utilization of modern and high-performance computing capabilities to enable data driven discovery. Here, we introduce BRAINformat, a novel file format and model for management and storage of neuroscience data. The BRAINformat library defines application-independent design concepts and modules that together create a general framework for standardization of scientific data. We describe the formal specification of scientific data standards, which facilitates sharing and verification of data and formats. We introduce the concept of Managed Objects, enabling semantic components of data formats to be specified as self-contained units, supporting modular and reusable design of data format components and file storage. The BRAINformat is built off of HDF5, enabling portable, scalable, and self-describing data storage. We introduce the novel concept of Relationship Attributes for modeling and use of semantic relationships between data objects, and discuss the annotation of data using dedicated data annotation modules provided by the BRAINformat library. Based on these concepts we implement dedicated, application-oriented modules and design a data standard for neuroscience data.The BRAINformat software library is open source, easy-to-use, and provides detailed user and developer documentation and is freely available at: https://bitbucket.org/oruebel/brainformat.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2016

Proton radius from electron scattering data

Background: The proton charge radius extracted from recent muonic hydrogen Lamb shift measurements is significantly smaller than that extracted from atomic hydrogen and electron scattering measurements. The discrepancy has become known as the proton radius puzzle. Purpose: In an attempt to understand the discrepancy, we review high-precision electron scattering results from Mainz, Jefferson Lab, Saskatoon and Stanford. Methods: We make use of stepwise regression techniques using the F-test as well as the Akaike information criterion to systematically determine the predictive variables to use for a given set and range of electron scattering data as well as to provide multivariate error estimates. Results: Starting with the precision, low four-momentum transfer (Q 2 ) data from Mainz (1980) and Saskatoon (1974), we find that a stepwise regression of the Maclaurin series using the F-test as well as the Akaike information criterion justify using a linear extrapolation which yields a value for the proton radius that is consistent with the result obtained from muonic hydrogen measurements. Applying the same Maclaurin series and statistical criteria to the 2014 Rosenbluth results on GE from Mainz, we again find that the stepwise regression tends to favor a radius consistent with the muonic hydrogen radius but produces results that are extremely sensitive to the range of data included in the fit. Making use of the high-Q 2 data on G E to select functions which extrapolate to high Q 2 , we find that a Pad´e (N = M = 1) statistical model works remarkably well, as does a dipole function with a 0.84 fm radius, G E (Q 2 ) = (1 + Q 2 /0.66 GeV 2 ) -2 . Conclusions: Rigorous applications of stepwise regression techniques and multivariate error estimates result in the extraction of a proton charge radius that is consistent with the muonic hydrogen result of 0.84 fm; either from linear extrapolation of the extreme low-Q 2 data or by use of the Pad´e approximant for extrapolation using a larger range of data. Thus, based on a purely statistical analysis of electron scattering data, we conclude that the electron scattering result and the muonic hydrogen result are consistent. Lastly, it is the atomic hydrogen results that are the outliers.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2021

Measurements of the E-mode polarization and temperature-E-mode correlation of the CMB from SPT-3G 2018 data

We present measurements of the E -mode ( E E ) polarization power spectrum and temperature- E -mode ( T E ) cross-power spectrum of the cosmic microwave background using data collected by SPT-3G, the latest instrument installed on the South Pole Telescope. This analysis uses observations of a 1500 deg 2 region at 95, 150, and 220 GHz taken over a four-month period in 2018. We report binned values of the E E and T E power spectra over the angular multipole range 300 ≤ ℓ < 3000 , using the multifrequency data to construct six semi-independent estimates of each power spectrum and their minimum-variance combination. These measurements improve upon the previous results of SPTpol across the multipole ranges 300 ≤ ℓ ≤ 1400 for E E and 300 ≤ ℓ ≤ 1700 for T E , resulting in constraints on cosmological parameters comparable to those from other current leading ground-based experiments. We find that the SPT-3G data set is well fit by a Λ CDM cosmological model with parameter constraints consistent with those from Planck and SPTpol data. From SPT-3G data alone, we find H 0 = 68.8 ± 1.5 km s - 1 Mpc - 1 and σ 8 = 0.789 ± 0.016 , with a gravitational lensing amplitude consistent with the Λ CDM prediction ( A L = 0.98 ± 0.12 ). We combine the SPT-3G and the Planck data sets and obtain joint constraints on the Λ CDM model. The volume of the 68% confidence region in six-dimensional Λ CDM parameter space is reduced by a factor of 1.5 compared to Planck-only constraints, with no significant shifts in central values. We note that the results presented here are obtained from data collected during just half of a typical observing season with only part of the focal plane operable, and that the active detector count has since nearly doubled for observations made with SPT-3G after 2018.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2019

Provenance–aware workflow for data quality management and improvement for large continuous scientific data streams

Data quality assessment, management and improvement is integral part of any big data intensive scientific research to ensure accurate, reliable and reproducible science discoveries. The task of maintaining the quality of data, however, is non-trivial and pose a challenge for a program like Department of Energy’s Atmospheric Radiation Measurement (ARM) that collects data from hundreds of instruments across the world,and distributes thousands of streaming data products that are continuously growing in near-real-time in an archive 1.7 Petabyte in size and growing. We present a computational data processing workflow to collect the data quality issue via an easy and intuitive web-based portal that allows reporting of any quality issues for any site, facility or instruments at a granularity down to individual variable in the data files. Portal allows instrument specialists and scientists to provide corrective actions in form of symbolic equation. A parallel processing framework applies the data improvement to large volume of data in efficient parallel environment, while optimizing data transfer and file I/O operations. Corrected files are systematically versioned and archived. A provenance tracking module tracks and records any change made to the data during its entire life cycle which are communicated transparently to the scientific users. Developed in Python using open source technologies, software architecture enables efficient and fast management and improvement of data in an operational data center environment.

Kumar, Jitendra↗

DOE OSTI2020

Data-Driven Modeling and Optimization of Building Energy Consumption: a Case Study

Installing sensors and Building Automation Systems (BAS) allows controlling the facility operations while generating data that can be analyzed for model development. This work focuses on data-driven modeling of the building to optimize energy consumption. The City of Orlando aims to reduce its energy consumption so, they provided us access to their BAS for data and studying the operation of its facilities. We selected a mid-size pilot building to conduct data analysis and modeling. We develop an Application Programming Interface (API) to login to the servers and scrape data. The scraped data contains features ranging from environmental conditions to equipment activity. This dataset is a time series so, it's handled in accordance and analyzed to investigate patterns and relations between data points that help choose parameters for predictive models for building and equipment. Finally, the models are optimized to reduce the energy consumption of the facility.

Grover, Divas↗

DOE OSTI2022

Early arrival waveform inversion using data uncertainties and matching filters with application to near-surface seismic refraction data

We develop an early arrival waveform inversion (EAWI) technique for high-resolution near-surface velocity estimation by iteratively updating the P-wave velocity model to minimize the difference between the observed and calculated seismic refraction data. Traditional EAWI uses a least-squares penalty function and an acoustic forward-modeling engine. Conventional least-squares error is sensitive to data with low signal-to-noise ratio (S/N) and iterations of EAWI stop at a local-minimum data misfit or at the preassigned maximum number of iterations. These stopping criteria can result in overfitting the data. In addition, fitting the elastic field data with an acoustic modeling engine can introduce artifacts in velocity estimation, especially in land data with significant elastic effects. To overcome these challenges, we develop a robust EAWI (REAWI) method by (1) incorporating the data uncertainties into the penalty function and (2) mitigating the elastic effects using a matching filter workflow. The data uncertainties are estimated from waveform reciprocal errors. When full-waveform reciprocity is not available, trace interpolation is applied. The proposed method prevents closely fitting data with low S/N, avoids overall overfitting by stopping the iterations when a normalized chi-square ([Formula: see text]) waveform misfit of one is achieved, and is less affected by elastic effects. Numerical examples and application to near-surface refraction data at a groundwater contamination site suggest that the final REAWI models are more accurate than the corresponding EAWI models, at the same level of misfit. This is the first known application of a matching filter workflow to real land data. The final REAWI models satisfy an appropriate misfit between the real data and predicted elastic P-wave data, making this approach in this respect equivalent to elastic waveform inversion. We also develop a method to analyze model constraint by examining the energy of the wavefield Fréchet derivative thereby avoiding the influence of the data residuals in traditional Fréchet kernels.

Geochemistry & Geophysics↗

DOE OSTI2019

ASCR Workshop on In Situ Data Management: Enabling Scientific Discovery from Diverse Data Sources

In January 2019, the U.S. Department of Energy, Office of Science program in Advanced Scientific Computing Research, convened a workshop to identify priority research directions for in situ data management (ISDM). The workshop defined ISDM as the practices, capabilities, and procedures to control the organization of data and enable the coordination and communication among heterogeneous tasks, executing simultaneously in a high-performance computing system, cooperating toward a common objective. The workshop revealed two primary, interdependent motivations for processing and managing data in situ. The first motivation is that the in situ methodology enables scientific discovery from a broad range of data sources over a wide scale of computing platforms: leadership-class systems, clusters, clouds, workstations, and embedded devices at the edge. The successful development of ISDM capabilities will benefit real-time decision-making, design optimization, and data-driven scientific discovery. The second motivation is the need to decrease data volumes. ISDM can make critical contributions to managing large data volumes from computations and experiments to minimize data movement, save storage space, and boost resource efficiency, often while simultaneously increasing scientific precision. A fundamental finding of this workshop is that the methodologies used to manage data among a variety of tasks in situ can be used to facilitate scientific discovery from many different data sources—simulation, experiment, and sensors, for example—and that being able to do so at numerous computing scales will benefit real-time decision-making, design optimization, and data-driven scientific discovery across the Office of Science mission space. Applications wanting to use the in situ capabilities include those where data analysis feeds back to the simulation, decisions are made autonomously, big data or machine learning is among the tasks to be coordinated, and computations need to be completed in real time. The workshop identified six priority research directions that highlight the components and capabilities needed for ISDM to be successful for the wide variety of applications discussed: making ISDM capabilities more pervasive, controllable, composable, and transparent, with a focus on greater coordination with the software stack and a diversity of fundamentally new data algorithms.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

Minimally buffered data transfers between nodes in a data communications network

Methods, apparatus, and products for minimally buffered data transfers between nodes in a data communications network are disclosed that include: receiving, by a messaging module on an origin node, a storage identifier, a origin data type, and a target data type, the storage identifier specifying application storage containing data, the origin data type describing a data subset contained in the origin application storage, the target data type describing an arrangement of the data subset in application storage on a target node; creating, by the messaging module, origin metadata describing the origin data type; selecting, by the messaging module from the origin application storage in dependence upon the origin metadata and the storage identifier, the data subset; and transmitting, by the messaging module to the target node, the selected data subset for storing in the target application storage in dependence upon the target data type without temporarily buffering the data subset.

97 MATHEMATICS AND COMPUTING↗