Deep learning based surrogate models for first-principles global simulations of fusion plasmas
Explore the source record for details and available documents.
Dong, G. (ORCID:0000000277343736)↗
SEARCH · Engineering Papers
Results for “deep learning”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Adiabatic Quantum Computation Applied to Deep Learning Networks
Training deep learning networks is a difficult task due to computational complexity, and this is traditionally handled by simplifying network topology to enable parallel computation on graphical processing units (GPUs). However, the emergence of quantum devices allows reconsideration of complex topologies. We illustrate a particular network topology that can be trained to classify MNIST data (an image dataset of handwritten digits) and neutrino detection data using a restricted form of adiabatic quantum computation known as quantum annealing performed by a D-Wave processor. We provide a brief description of the hardware and how it solves Ising models, how we translate our data into the corresponding Ising models, and how we use available expanded topology options to explore potential performance improvements. Although we focus on the application of quantum annealing in this article, the work discussed here is just one of three approaches we explored as part of a larger project that considers alternative means for training deep learning networks. The other approaches involve using a high performance computing (HPC) environment to automatically find network topologies with good performance and using neuromorphic computing to find a low-power solution for training deep learning networks. Our results show that our quantum approach can find good network parameters in a reasonable time despite increased network topology complexity; that HPC can find good parameters for traditional, simplified network topologies; and that neuromorphic computers can use low power memristive hardware to represent complex topologies and parameters derived from other architecture choices.
97 MATHEMATICS AND COMPUTING↗
DLIO: A DATA-CENTRIC BENCHMARK FOR DEEP LEARNING APPLICATIONS
SF-22-136 Deep learning has been shown as a successful method for various tasks, and its popularity results in numerous open-source deep learning software tools. Deep learning has been applied to a broad spectrum of scientific domains such as cosmology, particle physics, computer vision, fusion, and astrophysics. Scientists have performed a great deal of work to optimize the computational performance of deep learning frameworks. However, the same cannot be said for I/O performance. As deep learning algorithms rely on big-data volume and variety to effectively train neural networks accurately, I/O is a significant bottleneck on large-scale distributed deep learning training. DLIO, is a novel representative benchmark suite built based on the I/O profiling of the selected workloads. DLIO can be utilized to accurately emulate the I/O behavior of modern deep learning applications. Using DLIO, application developers and system software solution architects can identify potential I/O bottlenecks in their applications and guide optimizations to boost the I/O performance leading to lower training times. The storage vendor can also use DLIO as a guide for designing and optimize the storage and filesystem targeting at deep learning application.
ZHENG, HUIHUO↗
Evaluation of ASR in concrete using acoustic emission and deep learning
Not provided.
Nuclear Science & Technology↗
Low-Power Deep Learning Inference using the SpiNNaker Neuromorphic Platform.
Abstract not provided.
Vineyard, Craig↗
Applying Deep Learning for Wildfire Identification: Economical and Accessible Solutions Leveraging Small Datasets
Wildfires significantly impact human health, air quality, visibility, weather, and climate change and cause substantial economic losses. While state and county-operated air quality monitors provide critical insights during wildfires, they are not available in all regions. This highlights the need for affordable, accessible tools that allow the general public to assess air quality impacts. In this study, we apply machine learning with deep neural networks to diagnose air quality rapidly from sky images taken at the Pacific Northwest National Laboratory in Richland, WA, USA. Using a convolutional neural network (CNN) framework, we trained a deep learning model to classify air quality indices based on sky images. By leveraging transfer learning, our approach fine-tunes a pre-trained model on a small dataset of sky images, significantly reducing training time while maintaining high accuracy. Our results demonstrate the potential of deep learning to provide rapid air quality diagnostics during wildfire episodes, offering early warnings to the public and enabling timely mitigation strategies, particularly for vulnerable populations. Additionally, we show that lower respiratory infections pose the highest health risk during acute smoke exposures. Reactive oxygen species (ROS) from wildfire particles further exacerbate health risks by triggering inflammation and other adverse effects.
54 ENVIRONMENTAL SCIENCES↗
Network Scale Travel Time Prediction using Deep Learning
In recent years, deep learning models have been receiving increased attention within the artificial intelligence (AI) community because of their high prediction accuracy. In this paper, two deep learning models, long short-term memory (LSTM) and convolutional neural network (CNN), are proposed to predict travel time in a road network. One major advantage of using deep learning for travel time prediction is that it can make accurate predictions for all the segments in the transportation network with a single model structure, instead of building customized models for each segment separately. The proposed models were evaluated on a transportation network in the City of Saint Louis, Missouri. The prediction results show that deep learning can provide accurate prediction for both congested and uncongested traffic conditions, and can successfully capture the traffic dynamics of unexpected incidents or special events. The study findings show that deep learning offers a promising approach to real-time prediction of travel times on a network scale.
33 ADVANCED PROPULSION SYSTEMS↗
Deeply Uncertain: Comparing Methods of Uncertainty Quantification in Deep Learning Algorithms [Slides]
Deep learning is used in many applications in the physical sciences. In those sciences we are used to having uncertainty on every measurement or prediction. In the last years, many methods have been put forth for uncertainty quantification: Bayesian Neural Networks, Concrete Dropout, and Deep Ensembles. Uncertainty in deep learning is often divided into aleatoric or irreducible and epistemic or reducible. The problem remains: Which uncertainty quantification method should be chosen? How are those results interpreted?
97 MATHEMATICS AND COMPUTING↗
The role of chromatin state in intron retention: A case study in leveraging large scale deep learning models
Complex deep learning models trained on very large datasets have become key enabling tools for current research in natural language processing and computer vision. By providing pre-trained models that can be fine-tuned for specific applications, they enable researchers to create accurate models with minimal effort and computational resources. Large scale genomics deep learning models come in two flavors: the first are large language models of DNA sequences trained in a self-supervised fashion, similar to the corresponding natural language models; the second are supervised learning models that leverage large scale genomics datasets from ENCODE and other sources. We argue that these models are the equivalent of foundation models in natural language processing in their utility, as they encode within them chromatin state in its different aspects, providing useful representations that allow quick deployment of accurate models of gene regulation. We demonstrate this premise by leveraging the recently created Sei model to develop simple, interpretable models of intron retention, and demonstrate their advantage over models based on the DNA language model DNABERT-2. Our work also demonstrates the impact of chromatin state on the regulation of intron retention. Using representations learned by Sei, our model is able to discover the involvement of transcription factors and chromatin marks in regulating intron retention, providing better accuracy than a recently published custom model developed for this purpose.
Biochemistry & Molecular Biology↗
Deep Learning for Scientific Discovery
Deep learning has positively impacted fields were perceptive tasks are paramount, such as computer vision and natural language processing. Tremendous strides have also been made in deep reinforcement learning towards robotics and gaming. Given the success in these fields, it is apparent that deep learning may also accelerate scientific discovery. Herein, we discuss several topic areas where modern representation learning is driving innovation. This article presents a) learning non-linear models from data, b) learning scientific laws from models, c) deep learning on graphs, and d) approaches to scalable deep learning.
deep learning, scientific computing, koopman, mate↗
Graph-based deep learning frameworks for molecules and solid-state materials
Not provided.
Materials Science↗
Uncertainty quantification of a deep learning fuel property prediction model
Deep learning models are being widely used in the field of combustion. Given the black-box nature of typical neural network based models, uncertainty quantification (UQ) is critical to ensure the reliability of predictions as well as the training datasets, and for a principled quantification of noise and its various sources. Deep learning surrogate models for predicting properties of chemical compounds and mixtures have been recently shown to be promising for enabling data-driven fuel design and optimization, with the ultimate goal of improving efficiency and lowering emissions from combustion engines. In this study, UQ is performed for a multi-task deep learning model that simultaneously predicts the research octane number (RON), Motor Octane Number (MON), and Yield Sooting Index (YSI) of pure components and multicomponent blends. The deep learning model is comprised of three smaller networks: Extractor 1, Extractor 2, and Predictor, and a mixing operator. The molecular fingerprints of individual components are encoded via Extractor 1 and Extractor 2, the mixing operator generates fingerprints for mixtures/blends based on linear mixing operation, and the predictor maps the fingerprint to the target properties. Two different classes of UQ methods, Monte Carlo ensemble methods and Bayesian neural networks (BNNs), are employed for quantifying the epistemic uncertainty. Combinations of Bernoulli and Gaussian distributions with DropConnect and DropOut techniques are explored as ensemble methods. All the DropConnect, DropOut and Bayesian layers are applied to the predictor network. Aleatoric uncertainty is modeled by assuming that each data point has an independent uncertainty associated with it. The results of the UQ study are further analyzed to compare the performance of BNN and ensemble methods. Although this study is confined to UQ of fuel property prediction, the methodologies are applicable to other deep learning frameworks that are being widely used in the combustion community.
33 ADVANCED PROPULSION SYSTEMS↗
Surrogate Model Based Optimization for Finding Robust Deep Learning Model Architectures
Deep Learning (DL) models are increasingly used throughout the sciences. However, their performance and usefulness depend greatly on their architecture which is defined by hyperparameters such as the number of nodes, layers, the learning rate, etc. Tuning these hyperparameters is time-consuming because evaluating their performance requires a lengthy training step. Stochastic optimizers used in training lead to performance variability and potentially prediction reliability issues. In this talk, we will describe an automated optimization method based on surrogate models and active learning strategies for tuning DL model architectures. We take into account the prediction variability with the goal to identify architectures that make reliable and robust predictions. We demonstrate our developments on an application arising in particle physics.
deep learning↗
Scalable Deep Learning via I/O Analysis and Optimization
Scalable deep neural network training has been gaining prominence because of the increasing importance of deep learning in a multitude of scientific and commercial domains. Consequently, a number of researchers have investigated techniques to optimize deep learning systems. Much of the prior work has focused on runtime and algorithmic enhancements to optimize the computation and communication. Despite these enhancements, however, deep learning systems still suffer from scalability limitations, particularly with respect to data I/O. This situation is especially true for training models where the computation can be effectively parallelized, leaving I/O as the major bottleneck. In fact, our analysis shows that I/O can take up to 90% of the total training time. Thus, in this article, we first analyze LMDB, the most widely used I/O subsystem of deep learning frameworks, to understand the causes of this I/O inefficiency. Based on our analysis, we propose LMDBIO—an optimized I/O plugin for scalable deep learning. LMDBIO includes six novel optimizations that together address the various shortcomings in existing I/O for deep learning. Lastly, our experimental results show that LMDBIO significantly outperforms LMDB in all cases and improves overall application performance by up to 65-fold on a 9,216-core system.
97 MATHEMATICS AND COMPUTING↗