Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Convolutional neural networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 793 records · Page 44
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Femtosecond pulse parameter estimation from photoelectron momenta using machine learning

Abstract Deep learning models have provided huge interpretation power for image-like data. Specifically, convolutional neural networks (CNNs) have demonstrated incredible acuity for tasks such as feature extraction or parameter estimation. Here we test CNNs on strong-field ionization photoelectron spectra, training on theoretical data sets to ‘invert’ experimental data. Pulse characterization is used as a ‘testing ground’, specifically we retrieve the laser intensity, where ‘traditional’ measurements typically lead to 20% uncertainty. We report on crucial data augmentation techniques required to successfully train on theoretical data and return consistent results from experiments, including accounting for detector saturation. The same procedure can be repeated to apply CNNs in a range of scenarios for strong-field ionization. Using a predictive uncertainty estimation, reliable laser intensity uncertainties of a few percent can be extracted, which are consistently lower than those given by traditional techniques. Using interpretability methods can reveal parts of the distribution that are most sensitive to laser intensity, which can be directly associated with holographic interferences. The CNNs employed provide an accurate and convenient ways to extract parameters, and represent a novel interpretational tool for strong-field ionization spectra.

Szołdra, Tomasz (ORCID:0000000228970506)↗
DOE OSTI2022

Reconstructing cosmic polarization rotation with ResUNet-CMB

Cosmic polarization rotation, which may result from parity-violating new physics or the presence of primordial magnetic fields, converts E-mode polarization of the cosmic microwave background (CMB) into B-mode polarization. Anisotropic cosmic polarization rotation leads to statistical anisotropy in CMB polarization and can be reconstructed with quadratic estimator techniques similar to those designed for gravitational lensing of the CMB. At the sensitivity of upcoming CMB surveys, lensing-induced B-mode polarization will act as a limiting factor in the search for anisotropic cosmic polarization rotation, meaning that an analysis which incorporates some form of delensing will be required to improve constraints on the effect with future surveys. In this study we extend the ResUNet-CMB convolutional neural network to reconstruct anisotropic cosmic polarization rotation in the presence of gravitational lensing and patchy reionization, and we show that the network simultaneously reconstructs all three effects with variance that is lower than that from the standard quadratic estimator nearly matching the performance of an iterative reconstruction method.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Extracting the gamma-ray source-count distribution below the Fermi-LAT detection limit with deep learning

We reconstruct the extra-galactic gamma-ray source-count distribution, or dN/dS, of resolved and unresolved sources by adopting machine learning techniques. Specifically, we train a convolutional neural network on synthetic 2-dimensional sky-maps, which are built by varying parameters of underlying source-counts models and incorporate the Fermi-LAT instrumental response functions. The trained neural network is then applied to the Fermi-LAT data, from which we estimate the source count distribution down to flux levels a factor of 50 below the Fermi-LAT threshold. We perform our analysis using 14 years of data collected in the (1,10) GeV energy range. The results we obtain show a source count distribution which, in the resolved regime, is in excellent agreement with the one derived from cataloged sources, and then extends as dN/dS ~ S -2 in the unresolved regime, down to fluxes of 5 · 10 -12 cm -2 s -1 . The neural network architecture and the devised methodology have the flexibility to enable future analyses to study the energy dependence of the source-count distribution.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Massive νs through the CNN lens: interpreting the field-level neutrino mass information in weak lensing

Modern cosmological surveys probe the Universe deep into the nonlinear regime, where massive neutrinos suppress cosmic structure. Traditional cosmological analyses, which use the 2-point correlation function to extract information, are no longer optimal in the nonlinear regime, and there is thus much interest in extracting beyond-2-point information to improve constraints on neutrino mass. Quantifying and interpreting the beyond-2-point information is thus a pressing task. We study the field-level information in weak lensing convergence maps using convolution neural networks. We find that the network performance increases as higher source redshifts and smaller scales are considered — investigating up to a source redshift of 2.5 and ℓ max ≃ 10 4 — verifying that massive neutrinos leave a distinct effect on weak lensing. However, the performance of the network significantly drops after scaling out the 2-point information from the maps, implying that most of the field-level information can be found in the 2-point correlation function alone. We quantify these findings in terms of the likelihood ratio and also use Integrated Gradient saliency maps to interpret which parts of the map the network is learning the most from. We find that, in the absence of noise, the network extracts a similar amount of information from the most overdense and underdense regions. However, upon adding noise, the information in underdense regions is distorted as noise disproportionately washes out void-like structures.

Golshan, Malika [University of California, Berkele↗
DOE OSTI2025

Using active learning to improve quasar identification for the DESI spectra processing pipeline

The Dark Energy Spectroscopic Instrument (DESI) survey uses an automatic spectral classification pipeline to classify spectra. QuasarNET is a convolutional neural network used as part of this pipeline originally trained using data from the Baryon Oscillation Spectroscopic Survey (BOSS). In this paper we implement an active learning algorithm to optimally select spectra to use for training a new version of the QuasarNET weights file using only DESI data, with the goal of improving classification accuracy. This active learning algorithm includes a novel outlier rejection step using a Self-Organizing Map to ensure we label spectra representative of the larger quasar sample observed in DESI. We perform two iterations of the active learning pipeline, assembling a final dataset of 5600 labeled spectra, a small subset of the approximately 1.3 million quasar targets in DESI's Data Release 1. When splitting the spectra into training and validation subsets we achieve similar performance to the previously trained weights file in completeness and purity calculated on the validation dataset but do so with less than one tenth of the amount of training data. The new weights also more consistently classify objects in the same way when used on unlabeled data compared to the old weights file. In the process of improving QuasarNET's classification accuracy we discovered a systemic error in QuasarNET's redshift estimation and used our findings to improve our understanding of QuasarNET's redshifts.

Machine learning↗
DOE OSTI2026

Pre-training Vision Models for the Classification of Alerts from Wide-field Time-domain Surveys

Modern wide-field time-domain surveys facilitate the study of transient, variable and moving phenomena by conducting image differencing and relaying alerts to their communities. Machine learning tools have been used on data from these surveys and their precursors for more than a decade, and convolutional neural networks (CNNs), which make predictions directly from input images, saw particularly broad adoption through the 2010s. Since then, continually rapid advances in computer vision have transformed the standard practices around using such models. It is now commonplace to use standardized architectures pre-trained on large corpora of everyday images (e.g., ImageNet). In contrast, time-domain astronomy studies still typically design custom CNN architectures and train them from scratch. Here, we explore the effects of adopting various pre-training regimens and standardized model architectures on the performance of alert classification. We find that the resulting models match or outperform a custom, specialized CNN like what is typically used for filtering alerts. Moreover, our results show that pre-training on galaxy images from Galaxy Zoo tends to yield better performance than pre-training on ImageNet or training from scratch. We observe that the design of standardized architectures are much better optimized than the custom CNN baseline, requiring significantly less time and memory for inference despite having more trainable parameters. On the eve of the Legacy Survey of Space and Time and other image-differencing surveys, these findings advocate for a paradigm shift in the creation of vision models for alerts, demonstrating that greater performance and efficiency, in time and in data, can be achieved by adopting the latest practices from the computer vision field.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Search for two-neutrino double-beta decay of 136 Xe to the $0$$^{+}_{1}$ excited state of 136 Ba with the complete EXO-200 dataset*

A new search for two-neutrino double-beta (2νββ) decay of 136 Xe to the $0$$^{+}_{1}$ excited state of 136 Ba is performed with the full EXO-200 dataset. A deep learning-based convolutional neural network is used to discriminate signal from background events. Signal detection efficiency is increased relative to previous searches by EXO-200 by more than a factor of two. With the addition of the Phase II dataset taken with an upgraded detector, the median 90% confidence level half-life sensitivity of 2νββ decay to the $0$$^{+}_{1}$ state of 136 Ba is 2.9 x 10 24 yr using a total 136 Xe exposure of 234.1 kg yr. Furthermore, no statistically significant evidence for 2νββ decay to the $0$$^{+}_{1}$ state is observed, leading to a lower limit of $T$$^{2v}_{1/2}$(0 + → $0$$^{+}_{1}$) > 1.4 x 10 24 yr at 90% confidence level, improved by 70% relative to the current world's best constraint.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Single-trial classification of evoked responses to auditory tones using OPM- and SQUID-MEG

Abstract Objective. Optically pumped magnetometers (OPMs) are emerging as a near-room-temperature alternative to superconducting quantum interference devices (SQUIDs) for magnetoencephalography (MEG). In contrast to SQUIDs, OPMs can be placed in a close proximity to subject’s scalp potentially increasing the signal-to-noise ratio and spatial resolution of MEG. However, experimental demonstrations of these suggested benefits are still scarce. Here, to compare a 24-channel OPM-MEG system to a commercial whole-head SQUID system in a data-driven way, we quantified their performance in classifying single-trial evoked responses. Approach. We measured evoked responses to three auditory tones in six participants using both OPM- and SQUID-MEG systems. We performed pairwise temporal classification of the single-trial responses with linear discriminant analysis as well as multiclass classification with both EEGNet convolutional neural network and xDAWN decoding. Main results. OPMs provided higher classification accuracies than SQUIDs having a similar coverage of the left hemisphere of the participant. However, the SQUID sensors covering the whole helmet had classification scores larger than those of OPMs for two of the tone pairs, demonstrating the benefits of a whole-head measurement. Significance. The results demonstrate that the current OPM-MEG system provides high-quality data about the brain with room for improvement for high bandwidth non-invasive brain–computer interfacing.

Iivanainen, Joonas (ORCID:0000000160344604)↗
DOE OSTI2021

Adaptive machine learning for time-varying systems: low dimensional latent space tuning

Machine learning (ML) tools such as encoder-decoder convolutional neural networks (CNN) can represent incredibly complex nonlinear functions which map between combinations of images and scalars. For example, CNNs can be used to map combinations of accelerator parameters and images which are 2D projections of the 6D phase space distributions of charged particle beams as they are transported between various particle accelerator locations. Despite their strengths, applying ML to time-varying systems, or systems with shifting distributions, is an open problem, especially for large systems for which collecting new data for re-training is impractical or interrupts operations. Particle accelerators are one example of large time-varying systems for which collecting detailed training data requires lengthy dedicated beam measurements which may no longer be available during regular operations. We present a novel method of adaptive ML for time-varying systems. Our approach is to map very high (N ≈ 100k) dimensional inputs (a combination of scalar parameters and images) into the low dimensional (N ≈ 2) latent space at the output of the encoder section of an encoder-decoder CNN. We then actively tune the low dimensional latent space-based representation of complex system dynamics by the addition of an adaptively tuned feedback vector directly before the decoder sections builds back up to our image-based high-dimensional phase space density representations. This method allows us to learn correlations within and to quickly tune the characteristics of incredibly large parameter space systems and to track their evolution in real time based on feedback without massive new data sets for re-training. We demonstrate that our method can accurately predict and track the phase space of charged particle beams at various locations in a particle accelerator by adaptively adjusting in real-time while the unknown input beam distribution of the accelerator is changing in shape, charge, and offset and while the RF system of the accelerator itself is also changing in an unpredictable way. For FACET-II we demonstrate that such an approach has the potential to use transverse deflecting cavity and energy spread spectrum beam measurements to accurately predict 2D projections of the 6D phase space of the electron beam at the plasma wakefield acceleration interaction point where such diagnostics are unavailable.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

A deep-learning based raw waveform region-of-interest finder for the liquid argon time projection chamber

The liquid argon time projection chamber (LArTPC) detector technology has an excellent capability to measure properties of low-energy neutrinos produced by the sun and supernovae and to look for exotic physics at very low energies. In order to achieve those physics goals, it is crucial to identify and reconstruct signals in the waveforms recorded on each TPC wire. In this work, we report on a novel algorithm based on a one-dimensional convolutional neural network (CNN) to look for the region-of-interest (ROI) in raw waveforms. We test this algorithm using data from the ArgoNeuT experiment in conjunction with an improved noise mitigation procedure and a more realistic data-driven noise model for simulated events. This deep-learning ROI finder shows promising performance in extracting small signals and gives an efficiency approximately twice that of the traditional algorithm in the low energy region of ~0.03–0.1 MeV. This method offers great potential to explore low-energy physics using LArTPCs.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Particle hit clustering and identification using point set transformers in liquid argon time projection chambers

Liquid argon time projection chambers are often used in neutrino physics and dark-matter searches because of their high spatial resolution. The images generated by these detectors are extremely sparse, as the energy values detected by most of the detector are equal to 0, meaning that despite their high resolution, most of the detector is unused in a particular interaction. Instead of representing all of the empty detections, the interaction is usually stored as a sparse matrix, a list of detection locations paired with their energy values. Traditional machine learning methods that have been applied to particle reconstruction such as convolutional neural networks (CNNs), however, cannot operate over data stored in this way and therefore must have the matrix fully instantiated as a dense matrix. Operating on dense matrices requires a lot of memory and computation time, in contrast to directly operating on the sparse matrix. We propose a machine learning model using a point set neural network that operates over a sparse matrix, greatly improving both processing speed and accuracy over methods that instantiate the dense matrix, as well as over other methods that operate over sparse matrices. Compared to competing state-of-the-art methods, our method improves classification performance by 14%, segmentation performance by more than 22%, while taking 80% less time and using 66% less memory. Compared to state-of-the-art CNN methods, our method improves classification performance by more than 86%, segmentation performance by more than 71%, while reducing runtime by 91% and reducing memory usage by 61%.

calibration and fitting methods↗
DOE OSTI2026

Predicting the single-site and multi-site event discrimination power of dual-phase time projection chambers

Dual-phase xenon time projection chambers (TPCs) are widely used in searches for rare dark matter and neutrino interactions, in part because of their excellent position reconstruction capability in 3D. Despite their millimeter-scale resolution along the charge drift axis, xenon TPCs face challenges in resolving single-site (SS) and multi-site (MS) interactions in the transverse plane. In this paper, we build a generic TPC model with an idealized signal readout, and use Fisher Information (FI) to predict its theoretical capability of differentiating SS and MS events using the electroluminescence signal. We also demonstrate via simulation that, when only statistical photon noise is present, the theoretical limits can be approached with conventional reconstruction algorithms like maximum likelihood estimation, and with a convolutional neural network classifier. The implications of this study on future TPC experiments will be discussed.

Physics↗
DOE OSTI2021

Adaptive pruning-based optimization of parameterized quantum circuits

Abstract Variational hybrid quantum–classical algorithms are powerful tools to maximize the use of noisy intermediate-scale quantum devices. While past studies have developed powerful and expressive ansatze, their near-term applications have been limited by the difficulty of optimizing in the vast parameter space. In this work, we propose a heuristic optimization strategy for such ansatze used in variational quantum algorithms, which we call ‘parameter-efficient circuit training (PECT)’. Instead of optimizing all of the ansatz parameters at once, PECT launches a sequence of variational algorithms, in which each iteration of the algorithm activates and optimizes a subset of the total parameter set. To update the parameter subset between iterations, we adapt the Dynamic Sparse Reparameterization scheme which was originally proposed for training deep convolutional neural networks. We demonstrate PECT for the Variational Quantum Eigensolver, in which we benchmark unitary coupled-cluster ansatze including UCCSD and k -UpCCGSD, as well as the Low-Depth Circuit Ansatz (LDCA), to estimate ground state energies of molecular systems. We additionally use a layerwise variant of PECT to optimize a hardware-efficient circuit for the Sycamore processor to estimate the ground state energy densities of the one-dimensional Fermi-Hubbard model. From our numerical data, we find that PECT can enable optimizations of certain ansatze that were previously difficult to converge and more generally can improve the performance of variational algorithms by reducing the optimization runtime and/or the depth of circuits that encode the solution candidate(s).

Physics↗
DOE OSTI2022

Fast decay of classification error in variational quantum circuits

Variational quantum circuits (VQCs) have shown great potential in near-term applications. However, the discriminative power of a VQC, in connection to its circuit architecture and depth, is not understood. To unleash the genuine discriminative power of a VQC, we propose a VQC system with the optimal classical post-processing—maximum-likelihood estimation on measuring all VQC output qubits. Via extensive numerical simulations, we find that the error of VQC quantum data classification typically decays exponentially with the circuit depth, when the VQC architecture is extensive—the number of gates does not shrink with the circuit depth. This fast error suppression ends at the saturation towards the ultimate Helstrom limit of quantum state discrimination. On the other hand, non-extensive VQCs such as quantum convolutional neural networks are sub-optimal and fail to achieve the Helstrom limit, demonstrating a trade-off between ansatz complexity and classification performance in general. To achieve the best performance for a given VQC, the optimal classical post-processing is crucial even for a binary classification problem. To simplify VQCs for near-term implementations, we find that utilizing the symmetry of the input properly can improve the performance, while oversimplification can lead to degradation.

Zhang, Bingzhi↗
DOE OSTI2020

Machine learning enabled discovery of application dependent design principles for two-dimensional materials

Abstract The unique electronic and mechanical properties of two-dimensional (2D) materials make them promising next-generation candidates for a variety of applications. Large-scale searches for high-performing 2D materials are limited to calculating descriptors with computationally demanding first-principles density functional theory. In this work, we alleviate this issue by extending and generalizing crystal graph convolutional neural networks to systems with planar periodicity and train an ensemble of models to predict thermodynamic, mechanical and electronic properties. We carry out a screening of nearly 45,000 structures for two separate applications: mechanical strength and photovoltaics. By collecting statistics of the screened candidates, we investigate structural and compositional design principles that impact the properties of the structures surveyed. Our approach recovers some well-accepted design rules: hybrid organic-inorganic perovskites with lead and tin tend to be good candidates for solar cell applications and titanium based MXenes usually have high stiffness coefficients. Interestingly, other members of the group 4 elements also contribute to increasing the mechanical strength of MXenes. For all-inorganic perovskites, we discover some compositions that have not been deeply studied in the field of photovoltaics and thus open up paths for further investigation. We open-source the code-base to spur further development in this space.

Venturi, Victor (ORCID:0000000156727549)↗
DOE OSTI2020

Deep learning of interface structures from simulated 4D STEM data: cation intermixing vs. roughening ∗

Abstract Interface structures in complex oxides remain an active area of condensed matter physics research, largely enabled by recent advances in scanning transmission electron microscopy (STEM). Yet the nature of the STEM contrast in which the structure is projected along the given direction precludes separation of possible structural models. Here, we utilize deep convolutional neural networks (DCNN) trained on simulated 4D STEM datasets to predict structural descriptors of interfaces. We focus on the widely studied interface between LaAlO 3 and SrTiO 3 , using dynamical diffraction theory and leveraging high performance computing to simulate thousands of possible 4D STEM datasets to train the DCNN to learn properties of the underlying structures on which the simulations are based. We test the DCNN on simulated data and show that it is possible (with >95% accuracy) to identify a physically rough from a chemically diffuse interface and create a DCNN regression model to predict step positions. We quantify the applicability of the model to different thicknesses and the transferability of the approach. The method shown here is general and can be applied for any inverse imaging problem where forward models are present.

42 ENGINEERING↗
DOE OSTI2024

Transferring predictions of formation energy across lattices of increasing size*

In this study, we show the transferability of graph convolutional neural network (GCNN) predictions of the formation energy of the nickel-platinum solid solution alloy across atomic structures of increasing sizes. The original dataset was generated with the large-scale atomic/molecular massively parallel simulator using the second nearest-neighbor modified embedded-atom method empirical interatomic potential. Geometry optimization was performed on the initially randomly generated face centered cubic crystal structures and the formation energy has been calculated at each step of the geometry optimization, with configurations spanning the whole compositional range. Using data from various steps of the geometry optimization, we first trained our open-source, scalable implementation of GCNN called HydraGNN on a lattice of 256 atoms, which accounts well for the short-range interactions. Using this data, we predicted the formation energy for lattices of 864 atoms and 2048 atoms, which resulted in lower-than-expected accuracy due to the long-range interactions present in these larger lattices. We accounted for the long-range interactions by including a small amount of training data representative for those two larger sizes, whereupon the predictions of HydraGNN scaled linearly with the size of the lattice. Therefore, our strategy ensured scalability while reducing significantly the computational cost of training on larger lattice sizes.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Physics-inspired spatiotemporal-graph AI ensemble for the detection of higher order wave mode signals of spinning binary black hole mergers

We present a new class of AI models for the detection of quasi-circular, spinning, non-precessing binary black hole mergers whose waveforms include the higher order gravitational wave modes ($\ell$, |m|) = {(2,2), (2,1), (3,3), (3,2), (4,4)}, and mode mixing effects in the $\ell$ = 3, |m| = 2 harmonics. These AI models combine hybrid dilated convolution neural networks to accurately model both short- and long-range temporal sequential information of gravitational waves; and graph neural networks to capture spatial correlations among gravitational wave observatories to consistently describe and identify the presence of a signal in a three detector network encompassing the Advanced LIGO and Virgo detectors. We first trained these spatiotemporal-graph AI models using synthetic noise, using 1.2 million modeled waveforms to densely sample this signal manifold, within 1.7 h using 256 NVIDIA A100 GPUs in the Polaris supercomputer at the Argonne Leadership Computing Facility. This distributed training approach exhibited optimal classification performance, and strong scaling up to 512 NVIDIA A100 GPUs. With these AI ensembles we processed data from a three detector network, and found that an ensemble of 4 AI models achieves state-of-the-art performance for signal detection, and reports two misclassifications for every decade of searched data. We distributed AI inference over 128 GPUs in the Polaris supercomputer and 128 nodes in the Theta supercomputer, and completed the processing of a decade of gravitational wave data from a three detector network within 3.5 h. Finally, we fine-tuned these AI ensembles to process the entire month of February 2020, which is part of the O3b LIGO/Virgo observation run, and found 6 gravitational waves, concurrently identified in Advanced LIGO and Advanced Virgo data, and zero false positives. This analysis was completed in one hour using one NVIDIA A100 GPU.

79 ASTRONOMY AND ASTROPHYSICS↗