SEARCH · Engineering Papers

Results for “Convolutional Neural Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,081 records · Page 60

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Laplacian Smoothing Stochastic Gradient Markov Chain Monte Carlo

As an important Markov chain Monte Carlo (MCMC) method, the stochastic gradient Langevin dynamics (SGLD) algorithm has achieved great success in Bayesian learning and posterior sampling. Furthermore, SGLD typically suffers from a slow convergence rate due to its large variance caused by the stochastic gradient. In order to alleviate these drawbacks, we leverage the recently developed Laplacian smoothing technique and propose a Laplacian smoothing stochastic gradient Langevin dynamics (LS-SGLD) algorithm. We prove that for sampling from both log-concave and non-log-concave densities, LS-SGLD achieves strictly smaller discretization error in 2-Wasserstein distance, although its mixing rate can be slightly slower. Experiments on both synthetic and real datasets verify our theoretical results and demonstrate the superior performance of LS-SGLD on different machine learning tasks including posterior sampling, Bayesian logistic regression, and training Bayesian convolutional neural networks.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Digital holography for real-time non-invasive monitoring of larval fish at power plant intakes

Effective evaluation of technological and operational approaches to reduce entrainment of marine organisms at cooling water intake structures (CWIS) requires accurate organism-sensing systems. Current detection methods lead to large temporal data gaps, require tedious manual analysis, and are fatal to organisms. In this report we describe integrating deep learning with a non-lethal, non-intrusive imaging method—digital holography—to rapidly detect fish larvae. Laboratory experiments demonstrated that the instrument could successfully image fish larvae at flow rates exceeding ranges seen in CWIS. Holograms of two fish larvae species, in the presence of bubbles and detritus, were collected to build a large database for training a lightweight convolutional neural network. The model achieves 97% extraction accuracy in quantifying larvae, and distinguishing them from other particles, including detritus and bubbles, when applied to a dataset of manually classified images, exceeding previous metrics for non-lethal, accurate, and real-time detection. These results demonstrate the potential of in situ holographic imaging for monitoring endangered larval fish species at power plant intake structures, and for high-fidelity, real-time applications in monitoring aquatic ichthyoplankton.

endangered species↗

DOE OSTI2020

Identifying the nature of the QCD transition in relativistic collision of heavy nuclei with deep learning

Using deep convolutional neural network (CNN), the nature of the QCD transition can be identified from the final-state pion spectra from hybrid model simulations of heavy-ion collisions that combines a viscous hydrodynamic model with a hadronic cascade “after-burner”. Two different types of equations of state (EoS) of the medium are used in the hydrodynamic evolution. The resulting spectra in transverse momentum and azimuthal angle are used as the input data to train the neural network to distinguish different EoS. Different scenarios for the input data are studied and compared in a systematic way. A clear hierarchy is observed in the prediction accuracy when using the event-by-event, cascade-coarse-grained and event-fine-averaged spectra as input for the network, which are about 80%, 90% and 99%, respectively. A comparison with the prediction performance by deep neural network (DNN) with only the normalized pion transverse momentum spectra is also made. High-level features of pion spectra captured by a carefully-trained neural network were found to be able to distinguish the nature of the QCD transition even in a simulation scenario which is close to the experiments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2023

Lorentz group equivariant autoencoders

Abstract There has been significant work recently in developing machine learning (ML) models in high energy physics (HEP) for tasks such as classification, simulation, and anomaly detection. Often these models are adapted from those designed for datasets in computer vision or natural language processing, which lack inductive biases suited to HEP data, such as equivariance to its inherent symmetries. Such biases have been shown to make models more performant and interpretable, and reduce the amount of training data needed. To that end, we develop the Lorentz group autoencoder (LGAE), an autoencoder model equivariant with respect to the proper, orthochronous Lorentz group $$\textrm{SO}^+(3,1)$$ SO + ( 3 , 1 ) , with a latent space living in the representations of the group. We present our architecture and several experimental results on jets at the LHC and find it outperforms graph and convolutional neural network baseline models on several compression, reconstruction, and anomaly detection metrics. We also demonstrate the advantage of such an equivariant model in analyzing the latent space of the autoencoder, which can improve the explainability of potential anomalies discovered by such ML models.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2024

Demonstration of event position reconstruction based on diffusion in the NEXT-white detector

Noble element time projection chambers are a leading technology for rare event detection in physics, such as for dark matter and neutrinoless double beta decay searches. Time projection chambers typically assign event position in the drift direction using the relative timing of prompt scintillation and delayed charge collection signals, allowing for reconstruction of an absolute position in the drift direction. In this paper, alternate methods for assigning event drift dis tance via quantification of electron diffusion in a pure high pressure xenon gas time projection chamber are explored. Data from the NEXT-White detector demonstrate the ability to achieve good position assignment accuracy for both high and low-energy events. Using point-like energy deposits from 83m Kr calibration electron captures (E ~ 45 keV), the position of origin of low-energy events is determined to 2 cm precision with bias < 1 mm. A convolutional neural network approach is then used to quantify diffusion for longer tracks (E ≥ 1.5 MeV), from radiogenic electrons, yielding a precision of 3 cm on the event barycenter. The precision achieved with these methods indicates the feasibility energy calibrations of better than 1% FWHM at Q ββ in pure xenon, as well as the potential for event fiducialization in large future detectors using an alternate method that does not rely on primary scintillation.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2015

Optimizing deep learning hyper-parameters through an evolutionary algorithm. In: MLHPC '15 Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments, Article No. 4

There has been a recent surge of success in utilizing Deep Learning (DL) in imaging and speech applications for its relatively automatic feature generation and, in particular for convolutional neural networks (CNNs), high accuracy classification abilities. While these models learn their parameters through data-driven methods, model selection (as architecture construction) through hyper-parameter choices remains a tedious and highly intuition driven task. To address this, Multi-node Evolutionary Neural Networks for Deep Learning (MENNDL) is proposed as a method for automating network selection on computational clusters through hyper-parameter optimization performed via genetic algorithms.

Young, Steven R.↗

DOE OSTI2019

Wootz: a compiler-based framework for fast CNN pruning via composability

Convolutional Neural Networks (CNN) are widely used for Deep Learning tasks. CNN pruning is an important method to adapt a large CNN model trained on general datasets to fit a more specialized task or a smaller device. The key challenge is on deciding which filters to remove in order to maximize the quality of the pruned networks while satisfying the constraints. It is time-consuming due to the enormous configuration space and the slowness of CNN training.The problem has drawn many efforts from the machine learning field, which try to reduce the set of network configurations to explore. This work tackles the problem distinctively from a programming systems perspective, trying to speed up the evaluations of the remaining configurations through computation reuse via a compiler-based framework. We empirically uncover the existence of composability in the training of a collection of pruned CNN models, and point out the opportunities for computation reuse. We then propose composability-based CNN pruning, and design a compression-based algorithm to efficiently identify the set of CNN layers to pre-train for maximizing their reuse benefits in CNN pruning. We further develop a compiler-based framework named Wootz, which, for an arbitrary CNN, automatically generates code that builds a Teacher-Student scheme to materialize composability-based pruning. Experiments show that network pruning enabled by Wootz shortens the state-of-art pruning process by up to 186X while producing significantly better pruning results.

Guan, Hui↗

DOE OSTI2019

Automatic Differentiation for Adjoint Stencil Loops

Stencil loops are a common motif in computations including convolutional neural networks, structured-mesh solvers for partial differential equations, and image processing. Stencil loops are easy to parallelise, and their fast execution is aided by compilers, libraries, and domain-specific languages. Reverse-mode automatic differentiation, also known as algorithmic differentiation, autodiff, adjoint differentiation, or back-propagation, is sometimes used to obtain gradients of programs that contain stencil loops. Unfortunately, conventional automatic differentiation results in a memory access pattern that is not stencil-like and not easily parallelisable. In this paper we present a novel combination of automatic differentiation and loop transformations that preserves the structure and memory access pattern of stencil loops, while computing fully consistent derivatives. The generated loops can be parallelised and optimised for performance in the same way and using the same tools as the original computation. We have implemented this new technique in the Python tool PerforAD, which we release with this paper along with test cases derived from seismic imaging and computational fluid dynamics applications.

Automatic Differentiation↗

DOE OSTI2026

wa-hls4ml: A Benchmark and Surrogate Models for hls4ml Resource and Latency Estimation

As machine learning (ML) is increasingly implemented in hardware to address real-time challenges in scientific applications, the development of advanced toolchains has significantly reduced the time required to iterate on various designs. These advancements have solved major obstacles, but also exposed new challenges. For example, processes that were not previously considered bottlenecks, such as hardware synthesis, are becoming limiting factors in the rapid iteration of designs. To mitigate these emerging constraints, multiple efforts have been undertaken to develop an ML-based surrogate model that estimates resource usage of ML accelerator architectures. We introduce wa-hls4ml, a benchmark for ML accelerator resource and latency estimation, and its corresponding initial dataset of over 680,000 fully connected and convolutional neural networks, all synthesized using hls4ml and targeting Xilinx FPGAs. The benchmark evaluates the performance of resource and latency predictors against several common ML model architectures, primarily originating from scientific domains, as exemplar models, and the average performance across a subset of the dataset. Additionally, we introduce GNN- and transformer-based surrogate models that predict latency and resources for ML accelerators. We present the architecture and performance of the models and find that the models generally predict latency and resources for the 75% percentile within several percent of the synthesized resources on the synthetic test dataset.

Hawks, Benjamin [Fermilab] (ORCID:0000000157000288↗

DOE OSTI2023

Machine Learning Benchmarks for the Classification of Equivalent Circuit Models from Electrochemical Impedance Spectra

Analysis of Electrochemical Impedance Spectroscopy (EIS) data for electrochemical systems often consists of defining an Equivalent Circuit Model (ECM) using expert knowledge and then optimizing the model parameters to deconvolute various resistance, capacitive, inductive, or diffusion responses. For small data sets, this procedure can be conducted manually; however, it is not feasible to manually define a proper ECM for extensive data sets with a wide range of EIS responses. Automatic identification of an ECM would substantially accelerate the analysis of large sets of EIS data. We showcase machine learning methods to classify the ECMs of 9,300 impedance spectra provided by QuantumScape for the BatteryDEV hackathon. The best-performing approach is a gradient-boosted tree model utilizing a library to automatically generate features, followed by a random forest model using the raw spectral data. A convolutional neural network using boolean images of Nyquist representations is presented as an alternative, although it achieves a lower accuracy. We publish the data and open source the associated code. The approaches described in this article can serve as benchmarks for further studies. A key remaining challenge is the identifiability of the labels, underlined by the model performances and the comparison of misclassified spectra.

25 ENERGY STORAGE↗

DOE OSTI2018

DeepAnnotator

Given a collection of pairs ((A) written material such as reports, phrases, bulleted lists, etc., (B) structured labels called annotations) called the training data, we want to create software to take future written material (A) and correctly predict (B). In this case we have used medical reports and medical coder's structured labels as our training data. The software builds a machine learning model of type deep learning, specifically a convolutional neural network (CNN) using the training data so that when new data arrives in the future the annotations can be predicted.

Christian, JamesB.↗

DOE OSTI2020

Image Search for Scientific Data (camBIR) v1.0

camBIR is the CAMERA based image retrieval, a software that takes several different kinds of imagery, such as 2 dimensional images, both scientific as well as non scientific. camBIR is specially designed for training different kinds of Deep Convolutional Neural Network, including new Deep Learning architectures, such as ResNet, Inception-ResNet, NasNet, DenseNet…. It can be used to automate the classification, sort and rank of X-ray imagery such as microCT, GISAXS, GIWAXS, SAXS and WAXS.

Ushizima, Daniela↗

DOE OSTI2021

CNN-Encoder-Decoder Model

Code and data for training CNN-Encoder-Decoder model described in the publication 'Noise reduction in X-ray photon correlation spectroscopy with convolutional neural networks encoder-decoder models.'

Konstantinova, Tatiana [Brookhaven National Lab. (↗

DOE OSTI2021

HydraGNN

Distributed PyTorch implementation of multi-headed graph convolutional neural networks

Lupo Pasini, Massimiliano [Oak Ridge National Lab.↗

DOE OSTI2021

Attend and Decode: 4D fMRI Task State Decoding Using Attention Models

Source code for Brain Attend and Decode paper. Functional magnetic resonance imaging (fMRI) is a neuroimaging modality that captures the blood oxygen level in a subject's brain while the subject either rests or performs a variety of functional tasks under different conditions. Given fMRI data, the problem of inferring the task, known as task state decoding, is challenging due to the high dimensionality (hundreds of million sampling points per datum) and complex spatio-temporal blood flow patterns inherent in the data. In this work, we propose to tackle the fMRI task state decoding problem by casting it as a 4D spatiotemporal classification problem. We present a novel architecture called Brain Attend and Decode (BAnD), that uses residual convolutional neural networks for spatial feature extraction and self-attention mechanisms for temporal modeling. We achieve significant performance gain compared to previous works on a 7-task benchmark from the large-scale Human Connectome Project-Young Adult (HCP-YA) dataset. We also investigate the transferability of BAnD's extracted features on unseen HCP tasks, either by freezing the spatial feature extraction layers and retraining the temporal model, or finetuning the entire model. The pre-trained features from BAnD are useful on similar tasks while finetuning them yields competitive results on unseen tasks/conditions.

Ng, BrendaM.↗

DOE OSTI2022

Tomo2Mesh: Fast Reconstruction and Visualization of Tomography Data in Mesh Format

Tomo2Mesh is an open-source project targeted towards real-time reconstruction, segmentation, and visualization of computed tomography (CT) data in mesh format. The CT reconstruction scheme is based on filtered back-projection of voxel subsets. The segmentation scheme uses a 3D convolutional neural network. To allow for fast, real-time reconstruction, voxel subsets are first identified by coarse reconstruction. The detail in specific regions of interest is improved through full reconstruction of voxel subsets in that region. Data structures are implemented to store and process voxel subsets. Finally, voxel data is labeled using connected components to detect disconnected regions such as voids whose morphological attributes (e.g., Feret diameter, principal axis orientation, local number density, size, etc.) can be measured also in real-time. Finally, a fast marching cubes implementation processes labeled voxel data into a triangular face mesh (vertices and faces) in .ply format for visualization in Paraview or other mesh visualization tools. The code provides a simple programming interface for detecting, classifying, and visualizing regions of interest based on morphology. For example, detected voids can be classified as round pores or extended cracks. Highly porous neighborhoods can be identified based on local number density. The mesh texture (or color) is assigned based these morphological attributes to allow smart visualization scenarios in real-time (e.g., show only long cracks). At the time of first release (July 2022), extraction of face mesh for visualization for raw CT data from a 2 megapixel camera would take between 1-5 minutes for most scenarios.

TEKAWADE, ANIKET↗

DOE OSTI2022

InversionNet

InversionNet is a software to solve subsurface imaging problems. It leverages a convolutional neural network with an encoder-decoder structure to model the correspondence from seismic data to subsurface velocity structures.

Lin, Youzuo↗

DOE OSTI2022

Deep Learning Denoiser

SAND2022-14744 O The Deep Learning Denoiser uses a trained deep convolutional neural network (CNN) model to decompose an input waveform into a signal and noise waveform. In the approach, the CNN provides a signal mask and a noise mask for an input signal. The Short-Time Fourier Transform (STFT) of the estimated signal is obtained by multiplying the signal mask with the STFT of the input signal. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Koch, Clinton↗