Towards Long Term Extrapolation of Phase-field Simulations with Convolution-Only Deep Neural Networks
Explore the source record for details and available documents.
Bieberdorf, Nathan [External Affiliation]↗
SEARCH · Engineering Papers
Results for “Convolutional Neural Networks”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Nano-porous Chalk Rock Construction using Deep Convolutional Generative Adversarial Neural Networks.
Abstract not provided.
Kim, Sung Eun↗
Predicting the mechanical response of oligocrystals with deep learning
In this study, we employ data-driven homogenization approaches to predict the particular mechanical evolution of polycrystalline aggregates with tens of individual crystals. In these oligocrystals the differences in stress response due to microstructural variation is pronounced. Shell-like structures produced by metal-based additive manufacturing and the like make the prediction of the behavior of oligocrystals technologically relevant. The predictions of traditional homogenization theories based on grain volumes are not sensitive to variations in local grain neighborhoods. Direct simulation of the local response with crystal plasticity finite element methods is more detailed, but the computations are expensive. To represent the stress-strain response of a polycrystalline sample given its initial grain texture and morphology we have designed a novel neural network that incorporates a convolution component to observe and reduce the information in the crystal texture field and a recursive component to represent the causal nature of the history information. This model exhibits accuracy on par with crystal plasticity simulations at minimal computational cost per prediction.
36 MATERIALS SCIENCE↗
Denoising diffusion models with geometry adaptation for high fidelity calorimeter simulation
Simulation is crucial for all aspects of collider data analysis, but the available computing budget in the High Luminosity LHC era will be severely constrained. Generative machine learning models may act as surrogates to replace physics-based full simulation of particle detectors, and diffusion models have recently emerged as the state of the art for other generative tasks. Here, we introduce CaloDiffusion, a denoising diffusion model trained on the public CaloChallenge datasets to generate calorimeter showers. Our algorithm employs 3D cylindrical convolutions, which take advantage of symmetries of the underlying data representation. To handle irregular detector geometries, we augment the diffusion model with a new geometry latent mapping (GLaM) layer to learn forward and reverse transformations to a regular geometry that is suitable for cylindrical convolutions. The showers generated by our approach are nearly indistinguishable from the full simulation, as measured by several different metrics.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
Photo-zSNthesis: Converting Type Ia Supernova Lightcurves to Redshift Estimates via Deep Learning
Abstract Upcoming photometric surveys will discover tens of thousands of Type Ia supernovae (SNe Ia), vastly outpacing the capacity of our spectroscopic resources. In order to maximize the scientific return of these observations in the absence of spectroscopic information, we must accurately extract key parameters, such as SN redshifts, with photometric information alone. We present Photo- z SNthesis, a convolutional neural network-based method for predicting full redshift probability distributions from multi-band supernova lightcurves, tested on both simulated Sloan Digital Sky Survey (SDSS) and Vera C. Rubin Legacy Survey of Space and Time data as well as observed SDSS SNe. We show major improvements over predictions from existing methods on both simulations and real observations as well as minimal redshift-dependent bias, which is a challenge due to selection effects, e.g., Malmquist bias. Specifically, we show a 61× improvement in prediction bias 〈Δ z 〉 on PLAsTiCC simulations and 5× improvement on real SDSS data compared to results from a widely used photometric redshift estimator, LCFIT+Z. The PDFs produced by this method are well constrained and will maximize the cosmological constraining power of photometric SNe Ia samples.
79 ASTRONOMY AND ASTROPHYSICS↗
Achieving robust somatic mutation detection with deep learning models derived from reference data sets of a cancer sample
Accurate detection of somatic mutations is challenging but critical in understanding cancer formation, progression, and treatment. We recently proposed NeuSomatic, the first deep convolutional neural network-based somatic mutation detection approach, and demonstrated performance advantages on in silico data. In this study, we use the first comprehensive and well-characterized somatic reference data sets from the SEQC2 consortium to investigate best practices for using a deep learning framework in cancer mutation detection. Using the high-confidence somatic mutations established for a cancer cell line by the consortium, we identify the best strategy for building robust models on multiple data sets derived from samples representing real scenarios, for example, a model trained on a combination of real and spike-in mutations had the highest average performance. The strategy identified in our study achieved high robustness across multiple sequencing technologies for fresh and FFPE DNA input, varying tumor/normal purities, and different coverages, with significant superiority over conventional detection approaches in general, as well as in challenging situations such as low coverage, low variant allele frequency, DNA damage, and difficult genomic regions.
59 BASIC BIOLOGICAL SCIENCES↗
Revealing Fundamental Physics from the Daya Bay Neutrino Experiment Using Deep Neural Networks
Experiments in particle physics produce enormous quantities of data that must be analyzed and interpreted by teams of physicists. This analysis is often exploratory, where scientists are unable to enumerate the possible types of signal prior to performing the experiment. Thus, tools for summarizing, clustering, visualizing and classifying high-dimensional data are essential. Here in this work, we show that meaningful physical content can be revealed by transforming the raw data into a learned high-level representation using deep neural networks, with measurements taken at the Daya Bay Neutrino Experiment as a case study. We further show how convolutional deep neural networks can provide an effective classification filter with greater than 97% accuracy across different classes of physics events, significantly better than other machine learning approaches.
60 APPLIED LIFE SCIENCES↗
Integrating Improved Neural Operators and Graph Convolutional Networks for Scalable Geological Carbon Storage Modeling
Explore the source record for details and available documents.
Kadeethum, Teeratorn↗
Neuromorphic Accelerator for Deep Spiking Neural Networks with NVM Crossbar Arrays
In this paper, we present a scalable digital hardware accelerator based on non-volatile memory arrays capable of realizing deep convolutional spiking neural networks (SNNs). Our design studies are conducted using a compact model for spin-transfer torque random access memory (STT-RAM) devices. Large networks are realized by tiling multiple cores which communicate by transmitting spike packets via an on-chip routing network. Compared to an equivalent SRAM based core design, we show that the STT-RAM based design achieves nearly 15X higher GSOPS (Synaptic Operations per Second) per Watt per mm 2 making it a promising platform for realizing systems with significant area and power limitations.
Kulkarni, Shruti↗
Reverse image search for scientific data within and beyond the visible spectrum
The explosion in the rate, quality and diversity of image acquisition instruments has propelled the development of expert systems to organize and query image collections more efficiently. Recommendation systems that handle scientific images are rare, particularly if records lack metadata. This paper introduces new strategies to enable fast searches and image ranking from large pictorial datasets with or without labels. The main contribution is the development of pyCBIR, a deep neural network software to search scientific images by content. This tool exploits convolutional layers with locality sensitivity hashing for querying images across domains through a user-friendly interface. Our results report image searches over databases ranging from thousands to millions of samples. We test pyCBIR search capabilities using three convNets against four scientific datasets, including samples from cell microscopy, microtomography, atomic diffraction patterns, and materials photographs to demonstrate 95% accurate recommendations in most cases. Furthermore, all scientific data collections are released.
97 MATHEMATICS AND COMPUTING↗
Scaling Deep Learning workloads: NVIDIA DGX-1/Pascal and Intel Knights Landing
Deep Learning (DL) algorithms have become ubiquitous in data analytics. As a result, major computing vendors—including NVIDIA, Intel, AMD, and IBM—have architectural road maps influenced by DL workloads. Furthermore, several vendors have recently advertised new computing products as accelerating large DL workloads. Unfortunately, it is difficult for data scientists to quantify the potential of these different products. Here, this article provides a performance and power analysis of important DL workloads on two major parallel architectures: NVIDIA DGX-1 (eight Pascal P100 GPUs interconnected with NVLink) and Intel Knights Landing (KNL) CPUs interconnected with Intel Omni-Path or Cray Aries. Our evaluation consists of a cross section of convolutional neural net workloads: CifarNet, AlexNet, GoogLeNet, and ResNet50 topologies using the Cifar10 and ImageNet datasets. The workloads are vendor-optimized for each architecture. We use sequentially equivalent implementations to maintain iso-accuracy between parallel and sequential DL models. Our analysis indicates that although GPUs provide the highest overall performance, the gap can close for some convolutional networks; and the KNL can be competitive in performance/watt. We find that NVLink facilitates scaling efficiency on GPUs. However, its importance is heavily dependent on neural network architecture. Furthermore, for weak-scaling—sometimes encouraged by restricted GPU memory—NVLink is less important.
97 MATHEMATICS AND COMPUTING↗
An Ensemble of Neural Networks for Moist Physics Processes, Its Generalizability and Stable Integration
Abstract With the recent advances in data science, machine learning has been increasingly applied to convection and cloud parameterizations in global climate models (GCMs). This study extends the work of Han et al. (2020, https://doi.org/10.1029/2020MS002076 ) and uses an ensemble of 32‐layer deep convolutional residual neural networks, referred to as ResCu‐en, to emulate convection and cloud processes simulated by a superparameterized GCM, SPCAM. ResCu‐en predicts GCM grid‐scale temperature and moisture tendencies, and cloud liquid and ice water contents from moist physics processes. The surface rainfall is derived from the column‐integrated moisture tendency. The prediction uncertainty inherent in deep learning algorithms in emulating the moist physics is reduced by ensemble averaging. Results in 1‐year independent offline validation show that ResCu‐en has high prediction accuracy for all output variables, both in the current climate and in a warmer climate with +4K sea surface temperature. The analysis of different neural net configurations shows that the success to generalize in a warmer climate is attributed to convective memory and the 1‐dimensional convolution layers incorporated into ResCu‐en. We further implement a member of ResCu‐en into CAM5 with real world geography and run the neural‐network‐enabled CAM5 (NCAM) for 5 years without encountering any numerical integration instability. The simulation generally captures the global distribution of the mean precipitation, with a better simulation of precipitation intensity and diurnal cycle. However, there are large biases in temperature and moisture in high latitudes. These results highlight the importance of convective memory and demonstrate the potential for machine learning to enhance climate modeling.
Meteorology & Atmospheric Sciences↗
Autonomous Human Activity Classification From Wearable Multi-Modal Sensors
There has been significant amount of research work on human activity classification relying either on Inertial Measurement Unit (IMU) data or data from static cameras providing a third-person view. There has been relatively less work using wearable cameras, providing first-person or egocentric view, and even fewer approaches combining egocentric video with IMU data. Utilizing only IMU data limits the variety and complexity of the activities that can be detected. For instance, the sitting activity can be detected by IMU data, but it cannot be determined whether the subject has sat on a chair or a sofa, or where the subject is. To perform fine-grained activity classification, and to distinguish between activities that cannot be differentiated by only IMU data, we present an autonomous and robust method using data from both wearable cameras and IMUs. In contrast to convolutional neural network-based approaches, we propose to employ capsule networks to obtain features from egocentric video data. Moreover, Convolutional Long Short Term Memory framework is employed both on egocentric videos and IMU data to capture the temporal aspect of actions. We also propose a genetic algorithm-based approach to autonomously and systematically set various network parameters, rather than using manual settings. Experiments have been conducted to perform 9- and 26-label activity classification, and the proposed method, using autonomously set network parameters, has provided very promising results, achieving overall accuracies of 86.6% and 77.2%, respectively. The proposed approach, combining both modalities, also provides increased accuracy compared to using only egovision data and only IMU data.
47 OTHER INSTRUMENTATION↗
Automated detection of corrosion in used nuclear fuel dry storage canisters using residual neural networks
Not Available
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
Scalable balanced training of conditional generative adversarial neural networks on image data
Here, we propose a distributed approach to train deep convolutional generative adversarial neural network (DC-CGANs) models. Our method reduces the imbalance between generator and discriminator by partitioning the training data according to data labels, and enhances scalability by performing a parallel training where multiple generators are concurrently trained, each one of them focusing on a single data label. Performance is assessed in terms of inception score, Fréchet inception distance, and image quality on MNIST, CIFAR10, CIFAR100, and ImageNet1k datasets, showing a significant improvement in comparison to state-of-the-art techniques to training DC-CGANs. Weak scaling is attained on all the four datasets using up to 1000 processes and 2000 NVIDIA V100 GPUs on the OLCF supercomputer Summit.
97 MATHEMATICS AND COMPUTING↗
Conditional Generative Adversarial Networks (cGANs) for Near Real-Time Precipitation Estimation from Multispectral GOES-16 Satellite Imageries—PERSIANN-cGAN
In this paper, we present a state-of-the-art precipitation estimation framework which leverages advances in satellite remote sensing as well as Deep Learning (DL). The framework takes advantage of the improvements in spatial, spectral and temporal resolutions of the Advanced Baseline Imager (ABI) onboard the GOES-16 platform along with elevation information to improve the precipitation estimates. The procedure begins by first deriving a Rain/No Rain (R/NR) binary mask through classification of the pixels and then applying regression to estimate the amount of rainfall for rainy pixels. A Fully Convolutional Network is used as a regressor to predict precipitation estimates. The network is trained using the non-saturating conditional Generative Adversarial Network (cGAN) and Mean Squared Error (MSE) loss terms to generate results that better learn the complex distribution of precipitation in the observed data. Common verification metrics such as Probability Of Detection (POD), False Alarm Ratio (FAR), Critical Success Index (CSI), Bias, Correlation and MSE are used to evaluate the accuracy of both R/NR classification and real-valued precipitation estimates. Statistics and visualizations of the evaluation measures show improvements in the precipitation retrieval accuracy in the proposed framework compared to the baseline models trained using conventional MSE loss terms. This framework is proposed as an augmentation for PERSIANN-CCS (Precipitation Estimation from Remotely Sensed Information using Artificial Neural Network- Cloud Classification System) algorithm for estimating global precipitation.
42 ENGINEERING↗
REMODEL: Rethinking Deep CNN Models to Detect and Count on a NeuroSynaptic System
In this work, we perform analysis of detection and counting of cars using a low-power IBM TrueNorth Neurosynaptic System. For our evaluation we looked at a publicly-available dataset that has overhead imagery of cars with context present in the image. The trained neural network for image analysis was deployed on the NS16e system using IBM's EEDN training framework. Through multiple experiments we identify the architectural bottlenecks present in TrueNorth system that does not let us deploy large neural network structures. Following these experiments we propose changes to CNN model to circumvent these architectural bottlenecks. The results of these evaluations have been compared with caffe-based implementations of standard neural networks that were deployed on a Titan-X GPU. Results showed that TrueNorth can detect cars from the dataset with 97.60% accuracy and can be used to accurately count the number of cars in the image with 69.04% accuracy. The car detection accuracy and car count (–/+ 2 error margin) accuracy are comparable to high-precision neural networks like AlexNet, GoogLeNet, and ResCeption, but show a manifold improvement in power consumption.
97 MATHEMATICS AND COMPUTING↗
Advancing spatiotemporal forecasts of CO 2 plume migration using deep learning networks with transfer learning and interpretation analysis
Accurate and timely forecasts of CO 2 plume distribution throughout the injection and post-injection phases are crucial for detecting plume migration, assessing leakage risks, and supporting operational decisions in geologic carbon storage (GCS). Current convolutional neural network-based approaches primarily focus on spatial information and overlook temporal dependencies in plume distributions, thus limiting their ability to capture dynamic movement effects and provide accurate predictions of plume migration. In this work, we propose two deep learning models, Auto-Encoder (AE)-LSTM and Encoder-Decoder (ED)-ConvLSTM, each uniquely designed to capture both spatial and temporal features. We apply the proposed methods to forecast the dynamic distribution of CO 2 plumes based on 108 reservoir simulations over a 30-year injection and a 30-year post-injection period. The results indicate that the ED-ConvLSTM model outperforms the AE-LSTM model in accurately predicting the spatiotemporal dynamics of CO 2 plume migration, achieving R 2 values above 0.99. To provide a deeper understanding of these model predictions, we employ a gradient-based explanation method on the trained models. This approach provides insights into the influence of input variables on plume migration forecasts and uncovers the underlying prediction mechanisms of the proposed models. Furthermore, we introduce a transfer learning technique, enabling fast and accurate plume migration forecasting in the post-injection phase by leveraging the trained model during the injection phase. This reduces the necessity for extensive data collection or re-training. In conclusion, the methods proposed in our work enhances the performance and interpretability of CO 2 plume migration forecasts, thereby facilitating informed decision-making throughout the entire lifecycle of GCS applications.
58 GEOSCIENCES↗