Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DNN Training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Development of interatomic potential for Al–Tb alloys using a deep neural network learning method

An interatomic potential for the Al–Tb alloy around the composition of Al 90 Tb 10 is developed using the deep neural network (DNN) learning method. The atomic configurations and the corresponding total potential energies and forces on each atom obtained from ab initio molecular dynamics (AIMD) simulations are collected to train a DNN model to construct the interatomic potential for the Al–Tb alloy. Here we show that the obtained DNN model can well reproduce the energies and forces calculated by AIMD simulations. Molecular dynamics (MD) simulations using the DNN interatomic potential also accurately describe the structural properties of the Al 90 Tb 10 liquid, such as partial pair correlation functions (PPCFs) and bond angle distributions, in comparison with the results from AIMD simulations. Furthermore, the developed DNN interatomic potential predicts the formation energies of the crystalline phases of the Al–Tb system with an accuracy comparable to ab initio calculations. The structure factors of the Al 90 Tb 10 metallic liquid and glass obtained by MD simulations using the developed DNN interatomic potential are also in good agreement with the experimental X-ray diffraction data. The development of short-range order (SRO) in the Al 90 Tb 10 liquid and the undercooled liquid is also analyzed and three dominant SROs, i.e., Al-centered distorted icosahedron (DISICO) and Tb-centered ‘3661’ and ‘15551’ clusters, respectively, are identified.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Computer Model Calibration with Time Series Data Using Deep Learning and Quantile Regression

Computer models play a key role in many scientific and engineering problems. One major source of uncertainty in computer model experiments is input parameter uncertainty. Computer model calibration is a formal statistical procedure to infer input parameters by combining information from model runs and observational data. The existing standard calibration framework suffers from inferential issues when the model output and observational data are high-dimensional dependent data, such as large time series, due to the difficulty in building an emulator and the nonidentifiability between effects from input parameters and data-model discrepancy. To overcome these challenges, we, in this work, propose a new calibration framework based on a deep neural network (DNN) with long short-term memory layers that directly emulates the inverse relationship between the model output and input parameters. Adopting the “learning with noise” idea, we train our DNN model to filter out the effects from data-model discrepancy on input parameter inference. We also formulate a new way to construct interval predictions for DNN using quantile regression to quantify the uncertainty in input parameter estimates. Through a simulation study and real data application with the Weather Research and Forecasting Model Hydrological modeling system (WRF-Hydro), we show our approach can yield accurate point estimates and well-calibrated interval estimates for input parameters.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Multilevel-in-width training for deep neural network regression

A common challenge in regression is that for many problems, the degrees of freedom required for a high-quality solution also allows for overfitting. Regularization is a class of strategies that seek to restrict the range of possible solutions so as to discourage overfitting while still enabling good solutions, and different regularization strategies impose different types of restrictions. In this paper, we present a multilevel regularization strategy that constructs and trains a hierarchy of neural networks, each of which has layers that are wider versions of the previous network's layers. Here, we draw intuition and techniques from the field of Algebraic Multigrid (AMG), traditionally used for solving linear and nonlinear systems of equations, and specifically adapt the Full Approximation Scheme (FAS) for nonlinear systems of equations to the problem of deep learning. Training through V-cycles then encourage the neural networks to build a hierarchical understanding of the problem. We refer to this approach as multilevel-in-width to distinguish from prior multilevel works which hierarchically alter the depth of neural networks. The resulting approach is a highly flexible framework that can be applied to a variety of layer types, which we demonstrate with both fully connected and convolutional layers. We experimentally show with PDE regression problems that our multilevel training approach is an effective regularizer, improving the generalize performance of the neural networks studied.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Thermal Experiments for Fractured Rock Characterization: Theoretical Analysis and Inverse Modeling

Abstract Field‐scale properties of fractured rocks play a crucial role in many subsurface applications, yet methodologies for identification of the statistical parameters of a discrete fracture network (DFN) are scarce. We present an inversion technique to infer two such parameters, fracture density and fractal dimension, from cross‐borehole thermal experiments data. It is based on a particle‐based heat‐transfer model, whose evaluation is accelerated with a deep neural network (DNN) surrogate that is integrated into a grid search. The DNN is trained on a small number of the heat‐transfer model runs and predicts the cumulative density function of the thermal field. The latter is used to compute fine posterior distributions of the (to be estimated) parameters. Our synthetic experiments reveal that fracture density is well constrained by data, while fractal dimension is harder to determine. Adding nonuniform prior information related to the DFN connectivity improves the inference of this parameter.

Zhou, Zitong↗
DOE OSTI2023

Deep Learning Accelerated History Matching and Forecasting in Geologic CO 2 Sequestration [Slides]

We have successfully developed a predictive workflow for the Geologic CO 2 Sequestration using a deep learning model based on the Fourier Neural Operator. The workflow has high accuracy for predicting pressure & saturation during the injection and post-injection periods and has decent accuracy for predicting water and CO 2 production rates during injection period. It is necessary to train exclusive DNN models for pressure prediction in long term GCS, but for saturation prediction, we can use a single model to predict it.

58 GEOSCIENCES↗
DOE OSTI2024

Machine Learning for Joint Quality Control

The use of lightweight material combinations has been highly demanded in manufacturing automotive structures. However, making robust dissimilar material joints of such lightweight materials is still challenging. A significant barrier to achieving high-quality and repeatable joint performance is a deficient understanding of the relationship between the welding process, joint attributes, and joint performance. In this context, welding factors refer to material, equipment, environment, and process parameters, while joint features comprise specific microstructural attributes of the weld such as nugget size, heat affected zone (HAZ) topology, intermetallic layer thickness, and sheet thickness reduction. Joint performance is quantified in terms of strength (e.g., tensile shear, coach peel, cross-tension), weld size, and hardness, among other factors. While there have been many attempts to establish this process-structure-property relationship by developing a model derived from the associated physics and first principles, the complexity of the joining processes compounded by the complex interactions with different materials in an automotive assembly line environment, has hindered the usefulness of such attempts. The complexity is further exacerbated using different stacking materials, especially comprising dissimilar material combinations. In practice, the common approach has been the laborious process of creating welds, characterizing them, and then physically testing them through experimentation. With the emergence of artificial intelligence (AI) methods, an alternative pathway to eliciting the desired process-structure-property relationship at an accelerated pace is to use a data-driven approach by employing machine-learning (ML) techniques. This approach is benefitted by the availability of large streams of data, generated through years of research and testing by original equipment manufacturers, in the form of material, process, environmental, equipment, microstructural, and bulk-scale performance information from multimodal, multiscale sensors making measurements from laboratory-scale to production-scale processes. During Phase I efforts, which ended in fiscal year (FY) 2021, the Oak Ridge National Laboratory and Pacific Northwest National Laboratory (ORNL/PNNL) team demonstrated the effectiveness of different ML/AI frameworks in modeling complex relationships between resistance spot welding (RSW) process parameters, weld attributes, and joint properties using a subset of data from General Motors (GM). In FY 2022, the project team further refined and expanded their respective ML models to analyze additional welds with new weld stack-ups and materials to enhance the ML model predictive capability. ORNL extended its unified deep neural networks (DNN) ML training and prediction framework with new data streams of process parameters, and PNNL extended its model describing RSW process parameters’ associations with weld attributes. In FY 2023, the project team completed the development of the AI/ML architecture for analyzing aluminum/steel joints manufactured by GM via RSW and transitioned into the inline welding quality monitoring task for steel/steel RSW joints provided by GM.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Physics-Informed Neural Network Method for Forward and Backward Advection-Dispersion Equations

Advection-dispersion equations (ADEs) are commonly used to describe transport phenomena in porous media. Even though mature discretization-based numerical methods for ADEs exist, some challenges still remain, especially when it comes to solving advection-dominated forward ADEs and diffusion-dominated backward ADEs. The latter problem usually arises in the source identification context and leads to numerically unstable grid-based solutions that require a form of regularization or should be treated as an inverse problem that is computationally more expensive because it requires solving the forward problem multiple times. In this study, we propose a discretization-free approach based on the physics-informed neural network (PINN) method for solving coupled ADE and Darcy flow equations with space-dependent hydraulic conductivity. In this approach, the hydraulic conductivity, hydraulic head, and concentration fields are approximated with deep neural networks (DNNs). We assume that the conductivity field is given by its values on a grid, and we use these values to train the conductivity DNN. The head and concentration DNNs are trained by minimizing the residuals of the flow equation and ADE and using the initial and boundary conditions as additional constraints. The PINN method is applied to one- and two-dimensional forward ADE problems, where its performance for various P\'{e}clet numbers ($Pe$) is compared with the analytical and numerical solutions. We find that the PINN method is accurate with errors of less than 1\% and outperforms some conventional discretization-based methods for $Pe$ larger that 100. Next, we demonstrate that the PINN method remains accurate for the backward ADEs, with the relative errors in most cases staying under 5\% compared to the reference concentration field. Finally, we show that when available, the concentration measurements can be easily incorporated in the PINN method and significantly improve (by more than 50\% in the considered cases) the accuracy of the PINN solution of the backward ADE.

He, Qizhi↗
DOE OSTI2024

UFNet: Joint U-Net and Fully Connected Neural Network to Bias Correct Precipitation Predictions from

Paper information. Shuang Yu, Indrasis Chakraborty, Gemma J. Anderson, Donald D. Lucas, Yannic Lops, and Daniel Galea. UFNet: Joint U-Net and fully connected neural network to bias correct precipitation predictions from climate models. Artificial Intelligence for the Earth Systems, 2024. Overview. This work develops the UFNet methodology to correct E3SM historical precipitation projection bias. The UFNet deep learning framework consists of a two-part architecture: a U-Net convolutional network to capture the spatiotemporal distribution of precipitation and a fully connected network to capture the distribution of higher-order statistics. The joint network, termed UFNet, can simultaneously improve the spatial structure of the modeled precipitation and capture the distribution of extreme precipitation values. Below we provide guidance for applying UFNet to correct the Energy Exascale Earth System Model (E3SM; Golaz et al. 2019) daily precipitation projection over the contiguous United States (CONUS). Getting started 1. Obtain the historical climate simulation and observation data. The E3SM historical simulation data are available through https://aims2.llnl.gov/search/cmip6/. The CPC unified gauge-based analysis of daily precipitation can be found through https://psl.noaa.gov/data/gridded/data.cpc.globalprecip.html. The ECMWF atmospheric reanalysis of the 20th century (ERA-20C) data are available through https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era-20c. The spatial resolution of E3SM and observed datasets are both regridded to a common 1° resolution grid using conservative interpolation. The regridded E3SM, CPC and ERA-20C with 1° resolution can be found throught ./data/. 2. Train the fully connected network (DNN) Python train_dnn.py 3. Train the UFNet Python train_ufnet.py 4. Evaluation and compared with the baseline Python evaluation.py

Lucas, DonaldD↗
DOE OSTI2020

A Deep Neural Network for Accurate and Robust Prediction of the Glass Transition Temperature of Polyhydroxyalkanoate Homo- and Copolymers

The purpose of this study was to develop a data-driven machine learning model to predict the performance properties of polyhydroxyalkanoates (PHAs), a group of biosourced polyesters featuring excellent performance, to guide future design and synthesis experiments. A deep neural network (DNN) machine learning model was built for predicting the glass transition temperature, Tg, of PHA homo- and copolymers. Molecular fingerprints were used to capture the structural and atomic information of PHA monomers. The other input variables included the molecular weight, the polydispersity index, and the percentage of each monomer in the homo- and copolymers. The results indicate that the DNN model achieves high accuracy in estimation of the glass transition temperature of PHAs. In addition, the symmetry of the DNN model is ensured by incorporating symmetry data in the training process. The DNN model achieved better performance than the support vector machine (SVD), a nonlinear ML model and least absolute shrinkage and selection operator (LASSO), a sparse linear regression model. The relative importance of factors affecting the DNN model prediction were analyzed. Sensitivity of the DNN model, including strategies to deal with missing data, were also investigated. Compared with commonly used machine learning models incorporating quantitative structure–property (QSPR) relationships, it does not require an explicit descriptor selection step but shows a comparable performance. The machine learning model framework can be readily extended to predict other properties.

quantitative structure–property relationship (QSPR↗
DOE OSTI2026

Data-based filtered dissipation rate modelling for multi-modal turbulent combustion: evaluating a priori model generalizability

Manifold-based models offer a computationally efficient alternative to directly transporting the thermochemical state in computational simulations of turbulent reacting flows, projecting the high-dimensional thermochemical state-space onto a low-dimensional manifold. Recent efforts have yielded a manifold-based model applicable to multi-modal combustion, enabling reconstruction of the thermochemical state from solutions to two-dimensional manifold equations in mixture fraction and generalized progress variable that are parameterised by three scalar dissipation rates. In coarse-grained simulations such as Large Eddy Simulation (LES), closure of the multi-modal manifold equations and subfilter variances/covariance requires closure of three filtered scalar dissipation rates. Here, the present work adopts a data-based approach, providing closure for the three filtered scalar dissipation rates via deep neural networks (DNNs). High-fidelity datasets corresponding to an autoigniting n-dodecane jet flame and a bluff body swirl-stabilized confined lifted spray flame of two aviation fuels (Jet-A and C1) with different ignition propensities are leveraged to generate training data that spans a diverse range of thermodynamic conditions and combustion modes, including low- and high-temperature ignition regimes in addition to premixed and nonpremixed behaviour. A final DNN model is trained to enforce inherent physical constraints by learning nonlinear functional transformations of the three filtered scalar dissipation rates. The generalizability of this constrained DNN model is demonstrated a priori via conditional statistics evaluated on the lifted spray flame with C1–a configuration that had not been included in the training data. Excellent DNN agreement with conditional DNS statistics is observed, and integrated gradients are computed to identify the most sensitive input variables. The similarity of the marginal PDFs of the most informative input variables and outputs across configurations are quantified via the Wasserstein metric, demonstrating that data-based models may successfully generalize to unseen parametric conditions so long as the most informative input variables share similar distributions across training and testing datasets.

Data-based modelling↗
DOE OSTI2020

Deep learning-based model for progress variable dissipation rate in turbulent premixed flames

A deep neural network (DNN) based large eddy simulation (LES) model for progress variable dissipation rate in turbulent premixed flames is presented. The DNN model is trained using filtered data from direct numerical simulations (DNS) of statistically planar turbulent premixed flames with n-heptane as fuel. Training data was comprised of flames with varying turbulence levels leading to a range of Karlovitz numbers. Through a-priori tests the DNN model is shown to predict the subfilter contribution to progress variable dissipation rate accurately over a range of filter widths and for all Karlovitz numbers examined in this study. Superior performance of the DNN model relative to an established physics-based model is also demonstrated. Additionally, transferability of the DNN model is highlighted by a-priori evaluation of the model using filtered DNS data from multiple cases with different Karlovitz numbers and fuel species than those that were used for training the model.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2024

Newton-Raphson AC Power Flow Convergence Based on Deep Learning Initialization and Homotopy Continuation

Power flow forms the basis of many power system studies. With the increased penetration of renewable energy, grid planners tend to perform multiple power flow simulations under various operating conditions and not just selected snapshots at peak or light load conditions. Getting a converged AC power flow (ACPF) case remains a significant challenge for grid planners especially in large power grid networks. This paper proposes a two-stage approach to improve Newton-Raphson ACPF convergence and was applied to a 6102 bus Electric Reliability Council of Texas (ERCOT) system. The first stage utilizes a deep learning-based initializer with data re-training. Here a deep neural network (DNN) initializer is developed to provide better initial voltage magnitude and angle guesses to aid in power flow convergence. This is because Newton-Raphson ACPF is quite sensitive to the initial conditions and bad initialization could lead to divergence. The DNN initializer includes a data re-training framework that improves the initializer's performance when faced with limited training data. The DNN initializer successfully solved 3,285 cases out of 3,899 non-converging dispatch and performed better than random forest and DC power flow initialization methods. ACPF cases not solved in this first stage are then passed through a hot-starting algorithm based on homotopy continuation with switched shunt control. The hot-starting algorithm successfully converged 416 cases out of the remaining 614 non-converging ACPF dispatch. In conclusion, the combined two-stage approach achieved a 94.9% success rate, by converging a total of 3,701 cases out of the initial 3,899 unsolved cases.

Deep learning↗
DOE OSTI2023

Iterative sampling of expensive simulations for faster deep surrogate training

Deep neural network (DNN) surrogates of expensive physics simulations are enabling a rapid change in the way that common experimental design and analysis tasks are approached. Surrogate models allow simulations to be performed in parallel and separately from downstream tasks, thereby enabling analyses that would be impossible with the simulation in-the-loop; surrogates based on DNNs can effectively emulate diverse non-scalar data of the types collected in fusion and laboratory-astrophysics experiments. The challenge is in training the surrogate model, for which large ensembles of physics simulations must be run, preferably without wasting computational effort on uninteresting simulations. Here, in this paper, we present an iterative sampling scheme that can preferentially propose simulations in interesting regions of parameter space without neglecting unexplored regions, allowing high-quality and wide-ranging surrogate models to be trained using 2–3 times fewer simulations compare to space-filling designs. Our approach uses an explicit importance function defined on the simulation output space, balanced against a measure of simulation density which serves as a proxy for surrogate accuracy. It is easy to implement and can be tuned to find interesting simulations early in the study, allowing surrogates to be trained quickly and refined as new simulations become available; this represents an important step towards the routine generation of deep surrogate models quickly enough to be truly relevant to experimental work.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Optimization of the deep neural network parameters for generating homogenized fuel assembly data for nodal codes

Homogenized fuel assembly (FA) data is a typical input data for nodal codes. Generating that data, however, could be time-consuming. One of promising ways to mitigate the computational burden of generating macroscopic cross-sections is to use trained artificial neural network (ANN) models for predicting nuclear data. However, there is a challenge to make the model support variable FA geometry. In this work, two most common types of FA were combined in one ANN model. Since there could be multiple ways of converting 2-dimensional FA data into 1-dimensional input vector for ANN, three different approaches of data flattening were evaluated. The input parameters included each fuel pin enrichment, fuel temperature, moderator temperature and boron concentration. The output parameters were 2-group macroscopic cross-sections (XS) and pin power distribution (HFF). A fully connected deep neural network (DNN) model was trained and tested using pre-generated data obtained with lattice physics code STREAM. The results of this study showed no statistically significant difference in the accuracy of XS and HFF generation for all 3 tested input vector orders. This means that fully connected DNN for XS generation demonstrated input sequence invariance. Results of comparing predicted XS data with reference solutions were found sufficiently close considering the reduction of computation time offered by ANN. Mean relative difference (MRD) for all output XS parameters was found below 0.7%, while HFF MRD was found higher compared to XS values, in some cases slightly exceeding 1%, mostly near guide tube locations. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Distilling particle knowledge for fast reconstruction at high-energy physics experiments

Knowledge distillation is a form of model compression that allows artificial neural networks of different sizes to learn from one another. Its main application is the compactification of large deep neural networks to free up computational resources, in particular on edge devices. In this article, we consider proton-proton collisions at the High-Luminosity Large Hadron Collider (HL-LHC) and demonstrate a successful knowledge transfer from an event-level graph neural network (GNN) to a particle-level small deep neural network (DNN). Our algorithm, DistillNet, is a DNN that is trained to learn about the provenance of particles, as provided by the soft labels that are the GNN outputs, to predict whether or not a particle originates from the primary interaction vertex. The results indicate that for this problem, which is one of the main challenges at the HL-LHC, there is minimal loss during the transfer of knowledge to the small student network, while improving significantly the computational resource needs compared to the teacher. This is demonstrated for the distilled student network on a CPU, as well as for a quantized and pruned student network deployed on a field programmable gate array. Our study proves that knowledge transfer between networks of different complexity can be used for fast artificial intelligence (AI) in high-energy physics that improves the expressiveness of observables over non-AI-based reconstruction algorithms. Such an approach can become essential at the HL-LHC experiments, e.g. to comply with the resource budget of their trigger stages.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Deep-learning-derived planetary boundary layer height from conventional meteorological measurements

Abstract. The planetary boundary layer (PBL) height (PBLH) is an important parameter for various meteorological and climate studies. This study presents a multi-structure deep neural network (DNN) model, which can estimate PBLH by integrating the morning temperature profiles and surface meteorological observations. The DNN model is developed by leveraging a rich dataset of PBLH derived from long-standing radiosonde records augmented with high-resolution micro-pulse lidar and Doppler lidar observations. We access the performance of the DNN with an ensemble of 10 members, each featuring distinct hidden-layer structures, which collectively yield a robust 27-year PBLH dataset over the southern Great Plains from 1994 to 2020. The influence of various meteorological factors on PBLH is rigorously analyzed through the importance test. Moreover, the DNN model's accuracy is evaluated against radiosonde observations and juxtaposed with conventional remote sensing methodologies, including Doppler lidar, ceilometer, Raman lidar, and micro-pulse lidar. The DNN model exhibits reliable performance across diverse conditions and demonstrates lower biases relative to remote sensing methods. In addition, the DNN model, originally trained over a plain region, demonstrates remarkable adaptability when applied to the heterogeneous terrains and climates encountered during the GoAmazon (Green Ocean Amazon; tropical rainforest) and CACTI (Cloud, Aerosol, and Complex Terrain Interactions; middle-latitude mountain) campaigns. These findings demonstrate the effectiveness of deep learning models in estimating PBLH, enhancing our understanding of boundary layer processes with implications for improving the representation of PBL in weather forecasting and climate modeling.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Deep-Learning-derived Boundary Layer Height from Meteorological Data over the SGP, GOAMAZON, CACTI

The planetary boundary-layer (PBL) height (PBLH) is an important parameter for various meteorological and climate studies. This study presents a multi-structure deep neural network (DNN) model, designed to estimate PBLH by integrating morning temperature profiles with surface meteorological observations. The DNN model is developed by leveraging a rich data set of PBLH derived from long-standing radiosonde records and augmented with high-resolution micropulse lidar and Doppler lidar observations. We access the performance of the DNN with an ensemble of 10 members, each featuring distinct hidden layer structures, which collectively yield a robust 27-year PBLH data set over the Southern Great Plains from 1994 to 2020. The influence of various meteorological factors on PBLH is rigorously analyzed through the importance test. Moreover, the DNN model's accuracy is evaluated against radiosonde observations and juxtaposed with conventional remote-sensing methodologies, including Doppler lidar, ceilometer, Raman lidar, and micropulse lidar. The DNN model exhibits reliable performance across diverse conditions and demonstrates lower biases relative to remote-sensing methods. In addition, the DNN model, originally trained over a plain region, demonstrates remarkable adaptability when applied to the heterogeneous terrains and climates encountered during the GoAmazon (tropical rainforest) and CACTI (middle-latitude mountain) campaigns. These findings demonstrate the effectiveness of deep learning models in estimating PBLH, enhancing our understanding of boundary-layer dynamics with implications for enhancing the representation of PBL in weather forecasting and climate modeling.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Toward Large-Scale Image Segmentation on Summit

Semantic segmentation of images is an important computer vision task that emerges in a variety of application domains such as medical imaging, robotic vision and autonomous vehicles to name a few. While these domain-specific image analysis tasks involve relatively small image sizes (~ 10 2 × 10 2 ), there are many applications that need to train machine learning models on image data with extents that are orders of magnitude larger (~10 4 × 10 4 ). Training deep neural network (DNN) models on large extent images is extremely memory-intensive and often exceeds the memory limitations of a single graphical processing unit, a hardware accelerator of choice for computer vision workloads. Here, an efficient, sample parallel approach to train U-Net models on large extent image data sets is presented. Its advantages and limitations are analyzed and near-linear strong-scaling speedup demonstrated on 256 nodes (1536 GPUs) of the Summit supercomputer. Using a single node of the Summit supercomputer, an early evaluation of a recently released model parallel framework called GPipe is demonstrated to deliver ~ 2X speedup in executing a U-Net model with an order of magnitude larger number of trainable parameters than reported before. Performance bottlenecks for pipelined training of U-Net models are identified and mitigation strategies to improve the speedups are discussed. Together, these results open up the possibility of combining both approaches into a unified scalable pipelined and data parallel algorithm to efficiently train U-Net models with very large receptive fields on data sets of ultra-large extent images.

Seal, Sudip↗