Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Convolutional Neural Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Exploring End-to-end Deep Learning Applications for Event Classification at CMS

An essential part of new physics searches at the Large Hadron Collider (LHC) at CERN involves event classification, or distinguishing potential signal events from those coming from background processes. Current machine learning techniques accomplish this using traditional hand-engineered features like particle 4-momenta, motivated by our understanding of particle decay phenomenology. While such techniques have proven useful for simple decays, they are highly dependent on our ability to model all aspects of the phenomenology and detector response. Meanwhile, powerful deep learning algorithms are capable of not only training on high-level features, but of performing feature extraction. In computer vision, convolutional neural networks have become the state-of-the-art for many applications. Motivated by their success, we apply deep learning algorithms to low-level detector data from the 2012 CMS Simulated Open Data to directly learn useful features, in what we call, end-to-end event classification. We demonstrate the power of this approach in the context of a physics search and offer solutions to some of the inherent challenges, such as image construction, image sparsity, combining multiple sub-detectors, and de-correlating the classifier from the search observable, among others.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2019

Mapping XANES spectra on structural descriptors of copper oxide clusters using supervised machine learning

Understanding the origins of enhanced reactivity of supported, subnanometer in size, metal oxide clusters is challenging due to the scarcity of methods capable to extract atomic-level information from the experimental data. Due to both the sensitivity of X-ray absorption near edge structure (XANES) spectroscopy to the local geometry around metal ions and reliability of theoretical spectroscopy codes for modeling XANES spectra, supervised machine learning approach has become a powerful tool for extracting structural information from the experimental spectra. Here, we present the application of this method to grazing incidence XANES spectra of size-selective Cu oxide clusters on flat support, measured in operando conditions of the methanation reaction. We demonstrate that the convolution neural network can be trained on theoretical spectra and utilized to “invert” experimental XANES data to obtain structural descriptors—the Cu–Cu coordination numbers. As a result, we were able to distinguish between different structural motifs (Cu 2 O-like and CuO-like) of Cu oxide clusters, transforming in reaction conditions, and reliably evaluate average cluster sizes, with important implications for the understanding of structure, composition, and function relationships in catalysis.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

AI-enabled high-resolution scanning coherent diffraction imaging

Ptychographic imaging is a powerful means of imaging beyond the resolution limits of typical x-ray optics. Capturing recovered images from raw ptychographic data, however, requires the solution of an inverse problem, namely, phase retrieval. Phase retrieval algorithms are computationally expensive, which precludes real-time imaging. In this work, we propose PtychoNN, an approach to solve the ptychography data inversion problem based on a deep convolutional neural network. We demonstrate how the proposed method can be used to predict real-space structure and phase at each scan point solely from the corresponding far-field diffraction data. Our results demonstrate the practical application of machine learning to recover high fidelity amplitude and phase contrast images of a real sample hundreds of times faster than current ptychography reconstruction packages. Furthermore, by overcoming the constraints of iterative model-based methods, we can significantly relax sampling constraints on data acquisition while still producing an excellent image of the sample. Besides drastically accelerating acquisition and analysis, this capability has profound implications for the imaging of dose sensitive, dynamic, and extremely voluminous samples.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Phase behavior of continuous-space systems: A supervised machine learning approach

The phase behavior of complex fluids is a challenging problem for molecular simulations. Supervised machine learning (ML) methods have shown potential for identifying the phase boundaries of lattice models. In this work, we extend these ML methods to continuous-space systems. We propose a convolutional neural network model that utilizes grid-interpolated coordinates of molecules as input data of ML and optimizes the search for phase transitions with different filter sizes. We test the method for the phase diagram of two off-lattice models, namely, the Widom–Rowlinson model and a symmetric freely jointed polymer blend, for which results are available from standard molecular simulations techniques. The ML results show good agreement with results of previous simulation studies with the added advantage that there is no critical slowing down. We find that understanding intermediate structures near a phase transition and including them in the training set is important to obtain the phase boundary near the critical point. The method is quite general and easy to implement and could find wide application to study the phase behavior of complex fluids.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Rapid 3D nanoscale coherent imaging via physics-aware deep learning

Phase retrieval, the problem of recovering lost phase information from measured intensity alone, is an inverse problem that is widely faced in various imaging modalities ranging from astronomy to nanoscale imaging. The current process of phase recovery is iterative in nature. As a result, the image formation is time consuming and computationally expensive, precluding real-time imaging. Here, we use 3D nanoscale X-ray imaging as a representative example to develop a deep learning model to address this phase retrieval problem. We introduce 3D-CDI-NN, a deep convolutional neural network and differential programing framework trained to predict 3D structure and strain, solely from input 3D X-ray coherent scattering data. Our networks are designed to be “physics-aware” in multiple aspects; in that the physics of the X-ray scattering process is explicitly enforced in the training of the network, and the training data are drawn from atomistic simulations that are representative of the physics of the material. We further refine the neural network prediction through a physics-based optimization procedure to enable maximum accuracy at lowest computational cost. 3D-CDI-NN can invert a 3D coherent diffraction pattern to real-space structure and strain hundreds of times faster than traditional iterative phase retrieval methods. Our integrated machine learning and differential programing solution to the phase retrieval problem is broadly applicable across inverse problems in other application areas.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2024

Deep learning-enabled probing of irradiation-induced defects in time-series micrographs

Modeling time-series data with convolutional neural networks (CNNs) requires building a model to learn in batches as opposed to training sequentially. Coupling CNNs with in situ or operando techniques opens the possibility of accurately segmenting dynamic reactions and mass transport phenomena to understand how materials behave under the conditions in which they are used. In this article, in situ ion irradiation transmission electron microscopy (TEM) images are used as inputs into the CNN to assess the defect generation rate, defect cluster density, and saturation of defects. We then use the output segmentation maps to correlate with conventional TEM micrographs to assess the model’s ability to detail nanoscale interactions. Next, we discuss the implications of preprocessing and hyperparameters on model variability, accuracy when expanded to other datasets, and the role of regularization when controlling model variance. Ultimately, we eliminate human bias when extrapolating physical metrics, speed up analysis time, decouple reactions that happen at 100 ms intervals, and deploy models that are both accurate and transferable to similar experiments.

Burns, Kory (ORCID:0000000158019909)↗
DOE OSTI2024

Low latency optical-based mode tracking with machine learning deployed on FPGAs on a tokamak

Active feedback control in magnetic confinement fusion devices is desirable to mitigate plasma instabilities and enable robust operation. Optical high-speed cameras provide a powerful, non-invasive diagnostic and can be suitable for these applications. Here, in this study, we process high-speed camera data, at rates exceeding 100 kfps, on in situ field-programmable gate array (FPGA) hardware to track magnetohydrodynamic (MHD) mode evolution and generate control signals in real time. Our system utilizes a convolutional neural network (CNN) model, which predicts the n = 1 MHD mode amplitude and phase using camera images with better accuracy than other tested non-deep-learning-based methods. By implementing this model directly within the standard FPGA readout hardware of the high-speed camera diagnostic, our mode tracking system achieves a total trigger-to-output latency of 17.6 μs and a throughput of up to 120 kfps. This study at the High Beta Tokamak-Extended Pulse (HBT-EP) experiment demonstrates an FPGA-based high-speed camera data acquisition and processing system, enabling application in real-time machine-learning-based tokamak diagnostic and control as well as potential applications in other scientific domains.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Simulation-trained machine learning models for Lorentz transmission electron microscopy

Understanding the collective behavior of complex spin textures, such as lattices of magnetic skyrmions, is of fundamental importance for exploring and controlling the emergent ordering of these spin textures and inducing phase transitions. It is also critical to understand the skyrmion–skyrmion interactions for applications such as magnetic skyrmion-enabled reservoir or neuromorphic computing. Magnetic skyrmion lattices can be studied using in situ Lorentz transmission electron microscopy (LTEM), but quantitative and statistically robust analysis of the skyrmion lattices from LTEM images can be difficult. In this work, we show that a convolutional neural network, trained on simulated data, can be applied to perform segmentation of spin textures and to extract quantitative data, such as spin texture size and location, from experimental LTEM images, which cannot be obtained manually. This includes quantitative information about skyrmion size, position, and shape, which can, in turn, be used to calculate skyrmion–skyrmion interactions and lattice ordering. We apply this approach to segmenting images of Néel skyrmion lattices so that we can accurately identify skyrmion size and deformation in both dense and sparse lattices. The model is trained using a large set of micromagnetic simulations as well as simulated LTEM images. This entirely open-source training pipeline can be applied to a wide variety of magnetic features and materials, enabling large-scale statistical studies of spin textures using LTEM.

McCray, Arthur R. C. (ORCID:0000000160774698)↗
DOE OSTI2024

Effective optimization of atomic decoration in giant and superstructurally ordered crystals with machine learning

Crystals with complicated geometry are often observed with mixed chemical occupancy among Wyckoff sites, presenting a unique challenge for accurate atomic modeling. Similar systems possessing exact occupancy on all the sites can exhibit superstructural ordering, dramatically inflating the unit cell size. In this work, a crystal graph convolutional neural network (CGCNN) is used to predict optimal atomic decorations on fixed crystalline geometries. This is achieved with a site permutation search (SPS) optimization algorithm based on Monte Carlo moves combined with simulated annealing and basin-hopping techniques. Our approach relies on the evidence that, for a given chemical composition, a CGCNN estimates the correct energetic ordering of different atomic decorations, as predicted by electronic structure calculations. This provides a suitable energy landscape that can be optimized according to site occupation, allowing the prediction of chemical decoration in crystals exhibiting mixed or disordered occupancy, or superstructural ordering. Verification of the procedure is carried out on several known compounds, including the superstructurally ordered clathrate compound Rb8Ga27Sb16 and vacancy-ordered perovskite Cs2SnI6, neither of which was previously seen during the neural network training. In addition, the critical temperature of an order–disorder phase transition in solid solution CuZn is probed with our SPS routines by sampling site configuration trajectories in the canonical ensemble. This strategy provides an accurate method for determining favorable decoration in complex crystals and analyzing site occupation at unprecedented speed and scale.

Chemistry↗
DOE OSTI2024

Toward ultra-efficient high-fidelity predictions of wind turbine wakes: Augmenting the accuracy of engineering models with machine learning

This study proposes a novel machine learning (ML) methodology for the efficient and cost-effective prediction of high-fidelity three-dimensional velocity fields in the wake of utility-scale turbines. The model consists of an autoencoder convolutional neural network with U-Net skipped connections, fine-tuned using high-fidelity data from large-eddy simulations (LES). The trained model takes the low-fidelity velocity field cost-effectively generated from the analytical engineering wake model as input and produces the high-fidelity velocity fields. The accuracy of the proposed ML model is demonstrated in a utility-scale wind farm for which datasets of wake flow fields were previously generated using LES under various wind speeds, wind directions, and yaw angles. Comparing the ML model results with those of LES, the ML model was shown to reduce the error in the prediction from 20% obtained from the Gauss Curl hybrid (GCH) model to less than 5%. In addition, the ML model captured the non-symmetric wake deflection observed for opposing yaw angles for wake steering cases, demonstrating a greater accuracy than the GCH model. The computational cost of the ML model is on par with that of the analytical wake model while generating numerical outcomes nearly as accurate as those of the high-fidelity LES.

Mechanics↗
DOE OSTI2024

Three-dimensional reconstruction of laser-direct-drive inertial confinement fusion hot-spot plasma from x-ray diagnostics on the OMEGA laser facility (invited)

A deep-learning convolutional neural network (CNN) is used to infer, from x-ray images along multiple lines of sight, the low-mode shape of the hot-spot emission of deuterium–tritium (DT) laser-direct-drive cryogenic implosions on OMEGA. The motivation of this approach is to develop a physics-informed 3-D reconstruction technique that can be performed within minutes to facilitate the use of the results to inform changes to the initial target and laser conditions for the subsequent implosion. The CNN is trained on a 3D radiation-hydrodynamic simulation database to relate 2D x-ray images to 3D emissivity at stagnation. The CNN accounts for the lack of an absolute spatial reference and the different bands of photon energies in the x-ray images. While previous works studied the effect of mode-1 asymmetries on implosion performance using nuclear diagnostics, this work focuses on the effect of mode 2 inferred from x-ray diagnostics on implosion performance. A current analysis of 19 DT cryogenic implosions indicates there is an upper limit of ~20% reduction in the neutron yield caused by an ℓ = 2 amplitude for ℓ 2 /ℓ 0 ≤ 0.32. Here, these conclusions are supported by 2D simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

A hybrid neural architecture: Online attosecond x-ray characterization

The emergence of high-repetition-rate x-ray free-electron lasers (XFELs), such as SLAC’s LCLS-II, serves as our canonical example for autonomous controls that necessitate high-throughput diagnostics paired with streaming computational pipelines capable of single-shot analysis with extremely low latency. We present the deterministic characterization with an integrated parallelizable hybrid resolver architecture, a hybrid machine learning framework designed for fast, accurate analysis of XFEL diagnostics using angular streaking-based sinogram images. This architecture integrates convolutional neural networks and bidirectional long short-term memory models to denoise input, identify x-ray sub-spike features, and extract sub-spike relative delays with sub-30 attosecond temporal resolution. Deployed on low-latency hardware, it achieves over 10 kHz throughput with 168.3 μs inference latency, indicating scalability to 14 kHz with field-programmable gate array integration. By transforming regression tasks into classification problems and leveraging optimized error encoding, we achieve high precision with low-latency performance that is critical for real-time streaming event selection and experimental control feedback signals. This represents a key development in real-time control pipelines for next-generation autonomous science, generally, and high repetition-rate x-ray experiments in particular.

Accelerator Physics (physics.acc-ph)↗
DOE OSTI2024

Multiscale Flow for robust and optimal cosmological analysis

We propose Multiscale Flow, a generative Normalizing Flow that creates samples and models the field-level likelihood of two-dimensional cosmological data such as weak lensing. Multiscale Flow uses hierarchical decomposition of cosmological fields via a wavelet basis and then models different wavelet components separately as Normalizing Flows. The log-likelihood of the original cosmological field can be recovered by summing over the log-likelihood of each wavelet term. This decomposition allows us to separate the information from different scales and identify distribution shifts in the data such as unknown scale-dependent systematics. The resulting likelihood analysis can not only identify these types of systematics, but can also be made optimal, in the sense that the Multiscale Flow can learn the full likelihood at the field without any dimensionality reduction. We apply Multiscale Flow to weak lensing mock datasets for cosmological inference and show that it significantly outperforms traditional summary statistics such as power spectrum and peak counts, as well as machine learning–based summary statistics such as scattering transform and convolutional neural networks. We further show that Multiscale Flow is able to identify distribution shifts not in the training data such as baryonic effects. Finally, we demonstrate that Multiscale Flow can be used to generate realistic samples of weak lensing data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2015

A catalog of visual-like morphologies in the 5 CANDELS fields using deep learning

We present a catalog of visual-like H-band morphologies of ∼50.000 galaxies (H{sub f160w} < 24.5) in the 5 CANDELS fields (GOODS-N, GOODS-S, UDS, EGS, and COSMOS). Morphologies are estimated using Convolutional Neural Networks (ConvNets). The median redshift of the sample is 〈z〉∼1.25. The algorithm is trained on GOODS-S, for which visual classifications are publicly available, and then applied to the other 4 fields. Following the CANDELS main morphology classification scheme, our model retrieves for each galaxy the probabilities of having a spheroid or a disk, presenting an irregularity, being compact or a point source, and being unclassifiable. ConvNets are able to predict the fractions of votes given to a galaxy image with zero bias and ∼10% scatter. The fraction of mis-classifications is less than 1%. Our classification scheme represents a major improvement with respect to Concentration-Asymmetry-Smoothness-based methods, which hit a 20%–30% contamination limit at high z. The catalog is released with the present paper via the Rainbow database (http://rainbowx.fis.ucm.es/Rainbow-navigator-public/).

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

MHD mode tracking using high-speed cameras and deep learning

Abstract We present a new algorithm to track the amplitude and phase of rotating magnetohydrodynamic (MHD) modes in tokamak plasmas using high speed imaging cameras and deep learning. This algorithm uses a convolutional neural network (CNN) to predict the amplitudes of the n = 1 sine and cosine mode components using solely optical measurements from one or more cameras. The model was trained and tested on an experimental dataset consisting of camera frame images and magnetic-based mode measurements from the High Beta Tokamak - Extended Pulse (HBT-EP) device, and it outperformed other, more conventional, algorithms using identical image inputs. The effect of different input data streams on the accuracy of the model’s predictions is also explored, including using a temporal frame stack or images from two cameras viewing different toroidal regions.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Femtosecond pulse parameter estimation from photoelectron momenta using machine learning

Abstract Deep learning models have provided huge interpretation power for image-like data. Specifically, convolutional neural networks (CNNs) have demonstrated incredible acuity for tasks such as feature extraction or parameter estimation. Here we test CNNs on strong-field ionization photoelectron spectra, training on theoretical data sets to ‘invert’ experimental data. Pulse characterization is used as a ‘testing ground’, specifically we retrieve the laser intensity, where ‘traditional’ measurements typically lead to 20% uncertainty. We report on crucial data augmentation techniques required to successfully train on theoretical data and return consistent results from experiments, including accounting for detector saturation. The same procedure can be repeated to apply CNNs in a range of scenarios for strong-field ionization. Using a predictive uncertainty estimation, reliable laser intensity uncertainties of a few percent can be extracted, which are consistently lower than those given by traditional techniques. Using interpretability methods can reveal parts of the distribution that are most sensitive to laser intensity, which can be directly associated with holographic interferences. The CNNs employed provide an accurate and convenient ways to extract parameters, and represent a novel interpretational tool for strong-field ionization spectra.

Szołdra, Tomasz (ORCID:0000000228970506)↗
DOE OSTI2022

Reconstructing cosmic polarization rotation with ResUNet-CMB

Cosmic polarization rotation, which may result from parity-violating new physics or the presence of primordial magnetic fields, converts E-mode polarization of the cosmic microwave background (CMB) into B-mode polarization. Anisotropic cosmic polarization rotation leads to statistical anisotropy in CMB polarization and can be reconstructed with quadratic estimator techniques similar to those designed for gravitational lensing of the CMB. At the sensitivity of upcoming CMB surveys, lensing-induced B-mode polarization will act as a limiting factor in the search for anisotropic cosmic polarization rotation, meaning that an analysis which incorporates some form of delensing will be required to improve constraints on the effect with future surveys. In this study we extend the ResUNet-CMB convolutional neural network to reconstruct anisotropic cosmic polarization rotation in the presence of gravitational lensing and patchy reionization, and we show that the network simultaneously reconstructs all three effects with variance that is lower than that from the standard quadratic estimator nearly matching the performance of an iterative reconstruction method.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗