Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Machine learning at the Spallation Neutron Source accelerator and target

We describe the ongoing efforts to apply Machine Learning techniques to improve the performance of our accelerator and target. Specially, we are looking to minimize halo beam losses in the absence of a proper physics model, automatically detect and log anomalies in the target support systems such as cooling, and detect and prevent errant beam pulses in the linac. We also describe the infrastructure we use to acquire and stream data to the GPU cluster for training, our code development cycle, and edge computing for model inference. To minimize halo beam losses, we use a Reinforcement Learning technique tested on a virtual accelerator. The target anomaly detection is trained on archived data using incomplete physics models and is made part of the existing target reporting system. The errant beam prevention analyzes beam current and beam phase waveforms as well as accelerator configuration data to predict errant pulses. We also develop continual learning to adapt to changes in the accelerator.

Accelerator Physics↗
DOE OSTI2022

Deep learning to estimate permeability using geophysical data

Time-lapse electrical resistivity tomography (ERT) is a popular geophysical method to estimate three-dimensional (3D) permeability fields from electrical potential difference measurements. Traditional inversion and data assimilation methods are used to ingest this ERT data into hydrogeophysical models to estimate permeability. Due to ill-posedness and the curse of dimensionality, existing inversion strategies provide poor estimates and low resolution of the 3D permeability field. Recent advances in deep learning provide us with powerful algorithms to overcome this challenge. This paper presents a deep learning (DL) framework to estimate the 3D subsurface permeability from time-lapse ERT data. To test the feasibility of the proposed framework, we train DL-enabled inverse models on simulation data. Each measurement in both synthetic and field data is standardized by removing the mean and scaling the time-series to unit variance. This pre-processing step is necessary to bring simulation data closer to field observations. Subsurface process models based on hydrogeophysics are used to generate this synthetic data. Training performed on limited simulation data resulted in the DL model over-fitting. An advanced data augmentation based on mixup is implemented to generate additional training samples to overcome this issue. This mixup technique creates weakly labeled (low-fidelity) samples from strongly labeled (high-fidelity) data. The weakly labeled training data is then used to develop DL-enabled inverse models and reduce over-fitting. As both time-lapse ERT (1133048 features/realization) and 3D permeability (585453 features/realization) data samples are from a high-dimensional space, principal component analysis (PCA) is employed to reduce dimensionality. Encoded ERT and encoded permeability are generated using the trained PCA estimators. A deep neural network is then trained to map the encoded ERT to encoded permeability. This mixup training and unsupervised learning allowed us to build a fast and reasonably accurate DL-based inverse model under limited simulation data. Results show that proposed weak supervised learning can capture salient spatial features in the 3D permeability field. Quantitatively, the average mean squared error (in terms of the natural log) on the strongly labeled training, validation, and test datasets is less than 0.5. The R 2 -score (global metric) is greater than 0.75, and the percent error in each cell (local metric) is less than 10%. Finally, an added benefit in terms of computational cost is that the proposed DL-based inverse model is at least O(10 4 ) times faster than running a forward model once it is trained. Data generation, DL model training, and hyperparameter tuning to identify optimal neural network architectures utilized high-performance computing resources while the DL inference is performed on a standard laptop. Approximately, O(10 5 ) processor hours are used for generating data and DL tuning and training. We acknowledge that the data generation and DL model development are expensive. But once a DL model is trained, it can be re-used for inversion rapidly for the given system, with set physics and domain. Note that traditional inversion may require multiple forward model simulations (e.g., in the order of 10 to 1000), which are very expensive. This computational savings ≈ O(10 5 ) – O(10 7 )) makes the proposed DL-based inverse model attractive for subsurface imaging and real-time ERT monitoring applications due to fast and yet reasonably accurate estimations of permeability field.

58 GEOSCIENCES↗
DOE OSTI2023

Detection of open cluster rotation fields from Gaia EDR3 proper motions

Context: Most stars from in groups which with time disperse, building the field population of their host galaxy. In the Milky Way, open clusters have been continuously forming in the disk up to the present time, providing it with stars spanning a broad range of ages and masses. Observations of the details of cluster dissolution are, however, scarce. One of the main difficulties is obtaining a detailed characterisation of the internal cluster kinematics, which requires very high-quality proper motions. For open clusters, which are typically loose groups with tens to hundreds of members, there is the additional difficulty of inferring kinematic structures from sparse and irregular distributions of stars. Aims: Here, we aim to analyse internal stellar kinematics of open clusters, and identify rotation, expansion, or contraction patterns. Methods: We use Gaia Early Data Release 3 (EDR3) astrometry and integrated nested Laplace approximations to perform vector-field inference and create spatio-kinematic maps of 1237 open clusters. The sample is composed of clusters for which individual stellar memberships were already known, thus minimising contamination from field stars in the velocity maps. Projection effects were corrected using EDR3 data complemented with radial velocities from Gaia Data Release 2 and other surveys. Results: We report the detection of rotation patterns in eight open clusters. Nine additional clusters display possible rotation signs. We also observe 14 expanding clusters, with 15 other objects showing possible expansion patterns. Contraction is evident in two clusters, with one additional cluster presenting a more uncertain detection. In total, 53 clusters are found to display kinematic structures. Within these, elongated spatial distributions suggesting tidal tails are found in five clusters. These results indicate that the approach developed here can recover kinematic patterns from noisy vector fields, as those from astrometric measurements of open clusters or other stellar or galactic populations, thus offering a powerful probe for exploring the internal kinematics and dynamics of these types of objects.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Solar, Atmospheric, and Volcanic Impacts on 10 Be Depositions in Greenland and Antarctica During the Last 100 Years

Cosmogenic radionuclides (e.g., 10 Be) from ice cores are a powerful tool for solar reconstructions back in time. However, superimposed on the solar signal, other factors like weather/climate and volcanic influences on 10 Be can complicate the interpretation of 10 Be data. A comprehensive study of 10 Be records over the recent period, when atmospheric 10 Be production and meteorological conditions are relatively well-known, can improve our interpretation of 10 Be records. Here we conduct a systematic study of the production and climate/volcanic signals in Antarctica and Greenland 10 Be records, including a new 10 Be record from the East GReenland Ice-core Project site. Greenland and Antarctica records show significant decreasing trends (5%–6.5%/decade) for 1900–1950, which is comparable with the expected production rate inferred from sunspot observations. By comparing 10 Be records with reanalysis data and atmospheric circulation patterns, 10 Be records from Southern/Southeastern Greenland are significantly correlated with the Scandinavia pattern. Stacking 10 Be records from different locations can enhance the production signal. However, this approach is not always straightforward as uncertainties in some records can lead to a weaker solar signal. A strategy can be employed to select records for the bipolar stack by comparing Greenland records with Antarctica records, assuming the shared signal is a production signal. Finally, we observe significant increases (36%–64%) in 10 Be depositions in Greenland related to the Agung eruption. This large increase in Greenland 10 Be records after the Agung eruption, could be partly explained by the enhanced air mass transport from mid-latitudes coinciding with the decreased precipitation en-route.

10Be↗
DOE OSTI2020

The internal structure and geodynamics of Mars inferred from a 4.2-Gyr zircon record

Combining U–Pb ages with Lu–Hf data in zircon provides insights into the magmatic history of rocky planets. The Northwest Africa (NWA) 7034/7533 meteorites are samples of the southern highlands of Mars containing zircon with ages as old as 4476.3 ± 0.9 Ma, interpreted to reflect reworking of the primordial Martian crust by impacts. We extracted a statistically significant zircon population (n= 57) from NWA 7533 that defines a temporal record spanning 4.2 Gyr. Ancient zircons record ages from 4485.5 ± 2.2 Ma to 4331.0 ± 1.4 Ma, defining a bimodal distribution with groupings at 4474 ± 10 Ma and 4442 ± 17 Ma. We interpret these to represent intense bombardment episodes at the planet’s surface, possibly triggered by the early migration of gas giant planets. The unradiogenic initial Hf-isotope composition of these zircons establishes that Mars’s igneous activity prior to ~4.3 Ga was limited to impact-related reworking of a chemically enriched, primordial crust. A group of younger detrital zircons record ages from 1548.0 ± 8.8 Ma to 299.5 ± 0.6 Ma. The only plausible sources for these grains are the temporally associated Elysium and Tharsis volcanic provinces that are the expressions of deep-seated mantle plumes. The chondritic-like Hf-isotope compositions of these zircons require the existence of a primitive and convecting mantle reservoir, indicating that Mars has been in a stagnant-lid tectonic regime for most of its history. Our results imply that zircon is ubiquitous on the Martian surface, providing a faithful record of the planet’s magmatic history.

58 GEOSCIENCES↗
DOE OSTI2020

Local Atomic Structure in Photo-isomerized Ruthenium Sulfur Dioxide Complexes Revealed by Pair Distribution Function Analysis

SO 2 linkage photoisomerization in crystalline ruthenium-based complexes has demonstrated nanophotonic phenomena such as optical switching and nano-optomechanical transduction. Molecular insights into these materials have been explored largely via the characterization of their photoinduced crystal structures via in situ single-crystal X-ray diffraction, known as photocrystallography. Photoinduced molecular disorder is present, which photocrystallography can model to the extent that it is confined within the periodic boundary of a unit cell. However, non-periodic molecular disorder is suspected to exist as well. In situ total scattering experiments were therefore carried out on finely powdered crystals of four ruthenium-sulfur dioxide complexes. Data were modeled using light-minus-dark' difference pair distribution function analysis, which afforded photoinduced structural changes exclusively. This revealed structural features that were first compared against models of photoinduced crystal structures known a priori from photocrystallography. Furthermore, statistical inference was then employed, which evidenced generally good agreement between the total scattering data and the photocrystallographic models, while revealing real differences that are indicative of a structure with only a short-range order. Overall, our findings demonstrate that in situ light-induced total scattering experiments on finely powdered crystals are able to reveal the photoinduced structure. Overall, the evidence suggests that such structure could include a short-range order as well as photocrystallographic content. Our demonstration experiment offers a pathway to develop studies that capture the short-range order in linkage photoisomers, while we have outlined the procedure for testing the validity of associated structural models.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

A biology-informed similarity metric for simulated patches of human cell membrane

Complex scientific inquiries rely increasingly upon large and autonomous multiscale simulation campaigns, which fundamentally require similarity metrics to quantify ‘sufficient’ changes among data and/or configurations. However, subject matter experts are often unable to articulate similarity precisely or in terms of well-formulated definitions, especially when new hypotheses are to be explored, making it challenging to design a meaningful metric. Furthermore, the key to practical usefulness of such metrics to enable autonomous simulations lies in in situ inference, which requires generalization to possibly substantial distributional shifts in unseen, future data. Here, we address these challenges in a cancer biology application and develop a meaningful similarity metric for ‘patches’—regions of simulated human cell membrane that express interactions between certain proteins of interest and relevant lipids. In the absence of well-defined conditions for similarity, we leverage several biology-informed notions about data and the underlying simulations to impose inductive biases on our metric learning framework, resulting in a suitable similarity metric that also generalizes well to significant distributional shifts encountered during the deployment. We combine these intuitions to organize the learned embedding space in a multiscale manner, which makes the metric robust to incomplete and even contradictory intuitions. Our approach delivers a metric that not only performs well on the conditions used for its development and other relevant criteria, but also learns key spatiotemporal relationships without ever being exposed to any such information during training.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Measuring the thermal and ionization state of the low- z IGM using likelihood free inference

ABSTRACT We present a new approach to measure the power-law temperature density relationship $T=T_0 (\rho/ \bar{\rho })^{\gamma -1}$ and the UV background photoionization rate $\Gamma _{{{{\rm H\, {\small I}}}}{}}$ of the intergalactic medium (IGM) based on the Voigt profile decomposition of the Ly α forest into a set of discrete absorption lines with Doppler parameter b and the neutral hydrogen column density $N_{\rm H\, {\small I}}$. Previous work demonstrated that the shape of the $b-N_{{{{\rm H\, {\small I}}}}{}}$ distribution is sensitive to the IGM thermal parameters T0 and γ, whereas our new inference algorithm also takes into account the normalization of the distribution, i.e. the line-density dN/dz, and we demonstrate that precise constraints can also be obtained on $\Gamma _{{{{\rm H\, {\small I}}}}{}}$. We use density-estimation likelihood-free inference (DELFI) to emulate the dependence of the $b-N_{{{{\rm H\, {\small I}}}}{}}$ distribution on IGM parameters trained on an ensemble of 624 nyx hydrodynamical simulations at z = 0.1, which we combine with a Gaussian process emulator of the normalization. To demonstrate the efficacy of this approach, we generate hundreds of realizations of realistic mock HST/COS data sets, each comprising 34 quasar sightlines, and forward model the noise and resolution to match the real data. We use this large ensemble of mocks to extensively test our inference and empirically demonstrate that our posterior distributions are robust. Our analysis shows that by applying our new approach to existing Ly α forest spectra at z ≃ 0.1, one can measure the thermal and ionization state of the IGM with very high precision ($\sigma _{\log T_0} \sim 0.08$ dex, σγ ∼ 0.06, and $\sigma _{\log \Gamma _{{{{\rm H\, {\small I}}}}{}}} \sim 0.07$ dex).

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Data Assimilative Optimization of WSA Source Surface and Interface Radii using Particle Filtering

TheWang-Sheeley-Arge (WSA) model estimates solar wind speed and interplanetary magnetic field polarity in the inner heliosphere using global photospheric magnetic field maps. WSA employs the Potential Field Source Surface (PFSS) and Schatten Current Sheet (SCS) models to determine the Sun's global coronal magnetic field. The PFSS and SCS models are connected through two radial parameters, the source surface and interface radii, which specify the overlap region between the inner SCS and outer PFSS models. Though both radii values are adjustable, they have typically been fixed to 2.5 solar radii. Our work highlights how solar wind predictions improve when the radii are allowed to vary over time. Data assimilation using particle filtering (sequential Monte Carlo) is used to infer optimal values over a fixed time window. Solar wind model predictions and satellite observations are compared with a newly developed quality-of-agreement prediction metric. The agreement metric between the model and observations is assumed to correspond to the probability of the two key WSA model parameters, the source surface and interface radii, where the highest metric value implies the optimal radii.We find that the optimal particle filter values of solar radii can perform twice as well as standard values for an exploratory period during Carrington Rotation 1901, with these values also reducing nonphysical kinking effects seen in solar magnetic field lines. Data assimilation choices of input realization and time frame have implications for variation in the solar wind over time. We present this work's theoretical context and practical applications for prediction accuracy.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

An attention-based neural ordinary differential equation framework for modeling inelastic processes

To preserve strictly conservative behavior as well as model the variety of dissipative behavior displayed by solid materials, we propose a significant enhancement to the internal state variable-neural ordinary differential equation (ISV-NODE) framework. In this data-driven, physics-constrained modeling framework internal states are inferred rather than prescribed. The ISV-NODE consists of: (a) a stress model dependent on observable deformation and inferred internal state, and (b) a model of the evolution of the internal states. The enhancements to ISV-NODE proposed in this work are multifold: (a) a partially input convex neural network stress potential provides polyconvexity in terms of observed strain while leaving the inferred state unconstrained, and (b) an internal state flow model uses common latent features to inform novel attention-based gating and drives the flow of internal state only in dissipative regimes. We demonstrated that this architecture can accurately model dissipative and conservative behavior across an isotropic, isothermal elastic-viscoelastic-elastoplastic spectrum with three exemplars, while maintaining fundamental principles by design.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Pressure falloff testing to characterize CO 2 plume and dry-out zone during CO 2 injection in saline aquifers

The fluid mobilities and the lateral extent of the CO 2 plume and the dry-out zone are among important unknowns required to effectively manage CO 2 storage in saline aquifers projects. These unknowns can be derived from pressure falloff testing which has been widely used to characterize the subsurface. This paper provides a three-region analytical solution for falloff pressure during the infinite-acting period of CO 2 injection in saline aquifers. The solution is derived using Laplace transformation technique of the governing diffusivity equations and the accompanying initial and boundary conditions. The exact solution is validated against numerical results obtained from a black-oil reservoir simulator for two cases. The solution is approximated into simple forms in real-time domain considering different time periods. Then, a graphical semi-log analysis technique based on the approximate solutions is provided as an interpretation tool of falloff tests where the fluid mobilities and the extents of dry-out zone and CO 2 plume can be inferred. The interpretation technique is applied to synthetic falloff test data to investigate its potential and reliability.

02 PETROLEUM↗
DOE OSTI2021

Systematic characterization of human gut microbiome-secreted molecules by integrated multi-omics

The human gut microbiome produces a complex mixture of biomolecules that interact with human physiology and play essential roles in health and disease. Crosstalk between micro-organisms and host cells is enabled by different direct contacts, but also by the export of molecules through secretion systems and extracellular vesicles. The resulting molecular network, comprised of various biomolecular moieties, has so far eluded systematic study. Here we present a methodological framework, optimized for the extraction of the microbiome-derived, extracellular biomolecular complement, including nucleic acids, (poly)peptides, and metabolites, from flash-frozen stool samples of healthy human individuals. Our method allows simultaneous isolation of individual biomolecular fractions from the same original stool sample, followed by specialized omic analyses. The resulting multi-omics data enable coherent data integration for the systematic characterization of this molecular complex. Our results demonstrate the distinctiveness of the different extracellular biomolecular fractions, both in terms of their taxonomic and functional composition. This highlights the challenge of inferring the extracellular biomolecular complement of the gut microbiome based on single-omic data. The developed methodological framework provides the foundation for systematically investigating mechanistic links between microbiome-secreted molecules, including those that are typically vesicle-associated, and their impact on host physiology in health and disease.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Analytic solutions for Asay foil trajectories with implications for ejecta source models and mass measurements

We consider the trajectory of an Asay foil ejecta diagnostic for scenarios where ejecta are produced at a singly shocked planar surface and fly ballistically through a perfect vacuum to the sensor. We do so by building upon a previously established mathematical framework derived for the analytic study of stationary sensors. First, we derive the momentum conservation equation for the problem, in a form amenable to accelerating sensors, in terms of a generic ejecta source model. The result is an integrodifferential equation of motion for the foil trajectory. This equation yields an easily calculable closed-form implicit solution for the foil trajectory in instant-production scenarios. From there, we derive a boundary condition that particle velocity distributions must satisfy if their associated foil trajectories are to exhibit a smooth initial acceleration, as occurs in some experiments. This condition is identical to one derived previously from a consideration of piezoelectric voltage data obtained in similar experiments. We also compare techniques for inferring accumulated ejecta masses from foil trajectories, first by deriving the exact solution, and then by quantifying the error imposed by a frequently used approximate solution (both subject to the assumption of instantaneous ejecta production). Finally, we examine the common practice of presenting inferred cumulative ejecta masses as a function of implied ejecta velocity, establishing the conditions under which this methodology is most meaningful.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Simple and statistically sound recommendations for analysing physical theories

Physical theories that depend on many parameters or are tested against data from many different experiments pose unique challenges to statistical inference. Many models in particle physics, astrophysics and cosmology fall into one or both of these categories. These issues are often sidestepped with statistically unsound ad hoc methods, involving intersection of parameter intervals estimated by multiple experiments, and random or grid sampling of model parameters. Whilst these methods are easy to apply, they exhibit pathologies even in low-dimensional parameter spaces, and quickly become problematic to use and interpret in higher dimensions. In this article we give clear guidance for going beyond these procedures, suggesting where possible simple methods for performing statistically sound inference, and recommendations of readily-available software tools and standards that can assist in doing so. Our aim is to provide any physicists lacking comprehensive statistical training with recommendations for reaching correct scientific conclusions, with only a modest increase in analysis burden. Furthermore, our examples can be reproduced with the code publicly available at Zenodo.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Quartet-based inference is statistically consistent under the unified duplication-loss-coalescence model

Abstract Motivation The classic multispecies coalescent (MSC) model provides the means for theoretical justification of incomplete lineage sorting-aware species tree inference methods. This has motivated an extensive body of work on phylogenetic methods that are statistically consistent under MSC. One such particularly popular method is ASTRAL, a quartet-based species tree inference method. Novel studies suggest that ASTRAL also performs well when given multi-locus gene trees in simulation studies. Further, Legried et al. recently demonstrated that ASTRAL is statistically consistent under the gene duplication and loss model (GDL). GDL is prevalent in evolutionary histories and is the first core process in the powerful duplication-loss-coalescence evolutionary model (DLCoal) by Rasmussen and Kellis. Results In this work, we prove that ASTRAL is statistically consistent under the general DLCoal model. Therefore, our result supports the empirical evidence from the simulation-based studies. More broadly, we prove that the quartet-based inference approach is statistically consistent under DLCoal. Supplementary information Supplementary data are available at Bioinformatics online.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

SODAs: sparse optimization for the discovery of differential and algebraic equations

Differential-algebraic equations (DAEs) integrate ordinary differential equations (ODEs) with algebraic constraints, providing a fundamental framework for developing models of dynamical systems characterized by time-scale separation, conservation laws and physical constraints. While sparse optimization has revolutionized model development by allowing data-driven discovery of parsimonious models from a library of possible equations, existing approaches for dynamical systems assume DAEs can be reduced to ODEs by eliminating variables before model discovery. This assumption limits the applicability of such methods for DAE systems with unknown constraints and time scales. We introduce sparse optimization for differential-algebraic systems (SODAs), a data-driven method for the identification of DAEs in their explicit form. By discovering the algebraic and dynamic components sequentially without prior identification of the algebraic variables, this approach leads to a sequence of convex optimization problems. It has the advantage of discovering interpretable models that preserve the structure of the underlying physical system. To this end, SODAs improves since SODAs is singular numerical stability when handling high correlations between library terms, caused by near-perfect algebraic relationships, by iteratively refining the conditioning of the candidate library. We demonstrate the performance of our method on biological, mechanical and electrical systems, showcasing its robustness to noise in both simulated time series and real-time experimental data.

DAE↗
DOE OSTI2025

Quantum optimal control of superconducting qubits based on machine-learning characterization

Implementing fast and high-fidelity quantum operations using open-loop quantum optimal control relies on having an accurate model of the quantum dynamics. Any deviations between this model and the complete dynamics of the device, such as the presence of spurious modes or pulse distortions, can degrade the performance of optimal controls in practice. Here, we propose an experimentally simple approach to realize optimal quantum controls tailored to the device parameters and environment while specifically characterizing this quantum system. Concretely, we use physics-inspired machine learning to infer an accurate model of the dynamics from experimentally available data and then optimize our experimental controls on this trained model. We show the power and feasibility of this approach by optimizing arbitrary single-qubit operations in detailed numerical simulations of a superconducting transmon qubit. Furthermore, we demonstrate that this framework produces an accurate description of the device dynamics under arbitrary controls, together with the precise pulses achieving arbitrary single-qubit gates with a high fidelity of ∼99.99%.

Artificial neural networks↗
DOE OSTI2022

A Scalable Pipeline for Gigapixel Whole Slide Imaging Analysis on Leadership Class HPC Systems

Whole Slide Imaging (WSI) captures microscopic details of a patient's histopathological features at multiple res-olutions organized across different levels. Images produced by WSI are gigapixel-sized, and saving a single image in memory requires a few gigabytes which is scarce since a complicated model occupies tens of gigabytes. Performing a simple met-ric operation on these large images is also expensive. High-performance computing (HPC) can help us quickly analyze such large images using distributed training of complex deep learning models. One popular approach in analyzing these images is to divide a WSI image into smaller tiles (patches) and then train a simpler model with these reduced-sized but large numbers of patches. However, we need to solve three pre-processing challenges efficiently for pursuing this patch-based approach. 1) Creating small patches from a high-resolution image can result in a high number (hundreds of thousands per image) of patches. Storing and processing these images can be challenging due to a large number of I/O and arithmetic operations. To reduce I/Oand memory accesses, an optimal balance between the size and number of patches must exist to reduce I/O and memory accesses. 2) WSI images may have tiny annotated regions for cancer tissue and a significant portion with normal and fatty tissues; correct patch sampling should avoid dataset imbalance. 3) storing and retrieving many patches to and from disk storage might incur I/O latency while training a deep learning model. An efficient distributed data loader should reduce I/O latency during the training and inference steps. This paper explores these three challenges and provides empirical and algorithmic solutions deployed on the Summit supercomputer hosted at the Oak Ridge Leadership Computing Facility.

Dash, Sajal↗