Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DEEP LEARNING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

Neurogenesis Deep Learning: Extending deep networks to accommodate new classes

Neural machine learning methods, such as deep neural networks (DNN), have achieved remarkable success in a number of complex data processing tasks. These methods have arguably had their strongest impact on tasks such as image and audio processing – data processing domains in which humans have long held clear advantages over conventional algorithms. In contrast to biological neural systems, which are capable of learning continuously, deep artificial networks have a limited ability for incorporating new information in an already trained network. As a result, methods for continuous learning are potentially highly impactful in enabling the application of deep networks to dynamic data sets. Here, inspired by the process of adult neurogenesis in the hippocampus, we explore the potential for adding new neurons to deep layers of artificial neural networks in order to facilitate their acquisition of novel information while preserving previously trained data representations. Our results on the MNIST handwritten digit dataset and the NIST SD 19 dataset, which includes lower and upper case letters and digits, demonstrate that neurogenesis is well suited for addressing the stability-plasticity dilemma that has long challenged adaptive machine learning algorithms.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Combinatorial Evaluation of Physical Feature Engineering, Classical Machine Learning, and Deep Learning Models for Synchrophasor Data at Scale

A major objective of the project was to train and evaluate the effectiveness of multiple event and anomaly detection, identification and classification deep temporal learning models for processing of real-time phasor measurement unit (PMU) data streams. A vast dataset, consisting of two years of phasor measurements from all three U.S. Interconnections, was curated and released by the Department of Energy (DOE) through Pacific Northwest National Laboratory (PNNL). The dataset also included an event log that provided event times and types (e.g. generator trips, line trips, planned service events, transformer operations, etc.). Our analysis of this dataset addressed six (6) of the eleven (11) research priorities identified in Funding Opportunity Announcement (FOA) DE-FOA-0001861 “Big Data Analysis of Synchrophasor Data” (FOA 1861). Rather than being limited to pre-determined specific algorithms, this project relied on the uniquely structured, highly performant underlying time series database capabilities of the PredictiveGrid platform to assess the vast dataset utilizing a wide variety of algorithms.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Deep Learning for Modeling Enhanced Geothermal Systems

Enhanced Geothermal Systems (EGS) offer a vast potential to expand the use of geothermal energy. Heat is extracted from this engineered system by injecting cold water into a subsurface fractures, which are in contact with the hot dry rock, and pulled through the production wells. Creating EGS requires improving the natural permeability of hot crystalline rocks. To develop economically viable EGS reservoirs, significant technical barriers (e.g., better stimulation technologies without adequate water and/or permeability) and non-technical barriers (e.g., land access and permitting) must be overcome. In this short conference paper, we present a workflow to address a part of this challenge – “How to develop economically viable EGS using existing technologies?”. Our workflow called the GeoThermalCloud (GTC) for EGS, leverages recent advances in machine learning, deep learning, and cloud computing. This GTC framework is open-source and available at https://github.com/SmartTensors/GeoThermalCloud.jl. The GTC framework provides trained deep learning (DL) models to estimate the net present value of a given EGS design scenario. The Geothermal Design Tool (https://github.com/GeoDesignTool/GeoDT.git), a fast and simplified multi-physics solver, is used to develop a database for training DL models. The database consists of EGS design parameters (inputs to DL model) and their net present value (output of DL model) in uncertain geologic systems. The EGS design parameters for constructing this training database are based on Utah FORGE but include the options of more wells and deeper depths. The DL models are trained by ingesting the EGS design parameters and estimating the corresponding net present value. Such an emulation allows us to screen various EGS designs quickly and identify good development strategies by coupling them with optimization techniques. Our preliminary results show promise in DL emulation of net present value. However, a lot more work is needed to improve the predictive capability of DL models (i.e., extensive hyperparameter tuning is necessary). This will be the primary focus of our future work.

artificial neural networks, geothermal↗
DOE OSTI2023

Deep-Learning Electron Diffractive Imaging

Here, we report the development of deep-learning coherent electron diffractive imaging at subangstrom resolution using convolutional neural networks (CNNs) trained with only simulated data. We experimentally demonstrate this method by applying the trained CNNs to recover the phase images from electron diffraction patterns of twisted hexagonal boron nitride, monolayer graphene, and a gold nanoparticle with comparable quality to those reconstructed by a conventional ptychographic algorithm. Fourier ring correlation between the CNN and ptychographic images indicates the achievement of a resolution in the range of 0.70 and 0.55 Å. We further develop CNNs to recover the probe function from the experimental data. The ability to replace iterative algorithms with CNNs and perform real-time atomic imaging from coherent diffraction patterns is expected to find applications in the physical and biological sciences.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Deeply learning deep inelastic scattering kinematics

We study the use of deep learning techniques to reconstruct the kinematics of the neutral current deep inelastic scattering (DIS) process in electron–proton collisions. In particular, we use simulated data from the ZEUS experiment at the HERA accelerator facility, and train deep neural networks to reconstruct the kinematic variables Q 2 and x. Our approach is based on the information used in the classical construction methods, the measurements of the scattered lepton, and the hadronic final state in the detector, but is enhanced through correlations and patterns revealed with the simulated data sets. We show that, with the appropriate selection of a training set, the neural networks sufficiently surpass all classical reconstruction methods on most of the kinematic range considered. Rapid access to large samples of simulated data and the ability of neural networks to effectively extract information from large data sets, both suggest that deep learning techniques to reconstruct DIS kinematics can serve as a rigorous method to combine and outperform the classical reconstruction methods.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2017

Scaling Deep Learning on GPU and Knights Landing clusters

The speed of deep neural networks training has become a big bottleneck of deep learning research and development. For example, training GoogleNet by ImageNet dataset on one Nvidia K20 GPU needs 21 days. To speed up the training process, the current deep learning systems heavily rely on the hardware accelerators. However, these accelerators have limited on-chip memory compared with CPUs. To handle large datasets, they need to fetch data from either CPU memory or remote processors. We use both self-hosted Intel Knights Landing (KNL) clusters and multi-GPU clusters as our target platforms. From an algorithm aspect, current distributed machine learning systems are mainly designed for cloud systems. These methods are asynchronous because of the slow network and high fault-tolerance requirement on cloud systems. We focus on Elastic Averaging SGD (EASGD) to design algorithms for HPC clusters. Original EASGD used round-robin method for communication and updating. The communication is ordered by the machine rank ID, which is inefficient on HPC clusters. First, we redesign four efficient algorithms for HPC systems to improve EASGD's poor scaling on clusters. Async EASGD, Async MEASGD, and Hogwild EASGD are faster \textcolor{black}{than} their existing counterparts (Async SGD, Async MSGD, and Hogwild SGD, resp.) in all the comparisons. Finally, we design Sync EASGD, which ties for the best performance among all the methods while being deterministic. In addition to the algorithmic improvements, we use some system-algorithm codesign techniques to scale up the algorithms. By reducing the percentage of communication from 87% to 14%, our Sync EASGD achieves 5.3x speedup over original EASGD on the same platform. We get 91.5% weak scaling efficiency on 4253 KNL cores, which is higher than the state-of-the-art implementation.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Noise signal identification in time projection chamber data using deep learning model

Deep learning has been employed in various scientific fields and has provided promising results. Here, in this study, a deep learning classifier was implemented to improve the quality of data obtained from a time projection chamber. Digital waveforms of the detected signals were classified into the following three categories: particles, noises, and particles piled up with noises. A simple 1-dimensional convolutional neural network was developed for the classification. The model demonstrated an excellent performance on the test dataset. Its practical performance was also examined using track images and particle identification plots by comparing the original and clean data without the noise signals. The comparison clearly showed that the deep learning model improved the quality of data. The current study presents an effective application of the deep learning model for the time projection chamber data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗