Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Convolutional Neural Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,333 records · Page 74
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Device-Centric Ransomware Detection using Machine Learning-Based Memory Forensics for Smart Inverters

Ransomware attacks are the fastest-growing form of cyberattacks worldwide. Recently, ransomware attacks have targeted industrial control systems (ICSs), including power grids. Lessons learned from recent incidents in ICSs show that ransomware groups can deliver ransomware into not only the organization’s control servers, but also the operational technology (OT) devices such as smart inverters and smart grid devices. This paper proposes a machine learning (ML)- based memory forensics method enabling the detection of ransomware binaries stored in the memory of a commercial smart inverter. Device firmware binary files are extracted from a Serial Peripheral Interface (SPI) flash memory, and samples of both benign and ransomware binaries are generated by a binary manipulation method and a real-world ransomware encryption, separately. A deep transfer learning (DTL) method is used to retrain a convolutional neural network (CNN)-based ransomware detection algorithm using the generated samples. The experimental result validates that the proposed ML-based memory forensics method can accurately detect ransomware files.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

First Search for Dark-Trident Processes Using the MicroBooNE Detector

This thesis presents a first search for dark-trident scattering in a neutrino beam using a data set corresponding to $7.2 \times 10^{20}$ protons on target taken with the MicroBooNE detector at Fermilab. Proton interactions in the neutrino target at the Main Injector produce $\pi^0$ and $\eta$ mesons, which could decay into dark matter (DM) particles via a dark photon $A^{\prime}$. A convolutional neural network is trained to identify interactions of the DM particles in the liquid-argon time projection chamber (LArTPC) exploiting its image-like reconstruction capability. In the absence of a DM signal, limits at the $90\%$ confidence level on the squared kinematic mixing parameter $\varepsilon^2$ as a function of the dark-photon mass in the range $10\le M_{A^\prime}\le 400$~MeV are provided. The limits cover previously unconstrained parameter space for the production of fermion or scalar DM particles $\chi$ for two benchmark models with mass ratios $M_{\chi}/M_{A^\prime}=0.6$ and $2$ and for dark fine-structure constants $0.1\le\alpha_D\le 1$.Date of Award1 Aug 2024Original languageEnglishAwarding InstitutionThe University of ManchesterSupervisorStefan Soldner-Rembold (Supervisor) & Justin Evans (Supervisor)

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Electromagnetic shower separation in LArTPCs: study of the calorimetric method at LArIAT

This thesis reports a study of the prospects of using calorimetry for separating photon- from electron-initiated electromagnetic (EM) showers in liquid argon time projection chambers (LArTPCs), a prevalent need for neutrino oscillation experiments based on the LArTPC technology as $\gamma$-initiated showers are a significant background to electron neutrino ($\nu_e$) events, whose signature is electron showers. From events simulated in the LArIAT (Liquid Argon In A Testbeam) experiment's detector, we quantitatively evaluated the dependence of the calorimetric separation on the initial momentum and position (along the beamline) of the primary particle. We could conclude that the separation is more efficient if the primary particle is created inside the chamber, whereas the impact of the momentum is not as significant. Furthermore, a first step was taken towards applying the calorimetric method to the showers measured in LArIAT. A convolutional neural network (CNN) trained on simulation was deployed to find EM showers among the events collected by the experiment. The charge deposition at the beginning of the showers found is centered at a plausible value, although the width of the distribution demands further inquiry.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

Machine Learning for DUNE Supernova Trigger

One of the major scientific goals of the Deep Underground Neutrino Experiment (DUNE) is to detect and measure the neutrino flux originating from galactic core-collapse supernovae. These neutrinos provide an opportunity to study the end of life evolution of massive stars, and reveal information about the structure of core-collapse that is not visible in observations of the electromagnetic spectrum. Because of the rarity of these events, it is crucial that DUNE is able to detect supernova neutrino interactions when they occur. However, this will require sifting through a large quantity of data, motivating the development of a trigger algorithm to identify significant events and discard irrelevant data. Machine learning provides a potential approach to building this trigger. This project generates ADC and ground truth images of simulated neutrino interactions in a LArTPC detector to be used for machine learning, and uses them to train a sparse Convolutional Neural Network (C NN). The performance of this model when applied to the task of pixel classification based on interaction type is examined. This project found that the sparse CNN approach has the potential to have high accuracy in pixel classification, meaning it may be highly relevant to the development of a supernova neutrino trigger for the DUNE far detector.

Damish, S.↗
DOE OSTI2024

A Method for Producing Hierarchical and Statistically Calibrated Predictions of Nuclear Material Properties from Existing Models

Computer vision-based analysis of micrographs of nuclear materials is an emerging technique for property prediction, synthetic route identification, and other material analysis tasks. These analysis tasks play a pivotal role in many material characterization applications such as signature development for treaty verification, process optimization, etc. The backbone in many of the recent computer vision-based techniques is a deep learning model, which takes a fixed-size set of pixels and provides a class prediction for that set of pixels. For example, previous work developed a deep convolutional neural network (CNN) to predict the synthetic route from a 256 px x 256 px patch taken from a larger image of uranium ore concentrates. In this work, we present several methods for first calibrating these models in a manner that they can provide accurate probabilities of their predictions’ veracity, and several methods of combining these probabilities. Overall, the combination of these two steps into a pipeline allows for full-image and even full-sample (where a sample has many images) predictions with associated confidence values. Finally, we show that one can also use the patch predictions and confidence to produce a visualization to map predicted constituents through the image. Results and examples for predicting and mapping uranium ore concentrates’ synthetic process from imagery will be presented.

artificial intelligence↗
DOE OSTI2024

Towards Precision Photometric Type ia Supernova Cosmology With Machine Learning

The revolutionary discovery of dark energy and accelerating cosmic expansion was made with just 42 type Ia supernovae (SNe Ia) in 1999. Since then, large synoptic surveys, e.g., Dark Energy Survey (DES), have observed thousands more SNe Ia and the upcoming Rubin Legacy Survey of Space and Time (LSST) and Roman Space Telescope promise to deliver millions in the next decade. This unprecedented data volume can produce the required precision to unambiguously test concordance cosmology which could represent a monumental shift in our understanding of dark energy and its role in cosmic history. However, extracting a pure SN Ia sample with accurate redshifts for such a large dataset will be a challenge. Specifically, spectroscopic classification will not be possible for the vast majority of discovered objects, and only ~25% will have spectroscopic redshifts. This thesis presents a series of observational and methodological studies designed to address the questions associated with this new era of photometric SN Ia cosmology. First, we present a machine learning method for photometric classification of SNe, Supernova Classification with a COnvolutional Neural Network (SCONE). Photometric classification enables SNe with no spectroscopic information to be confidently categorized, and is a critical component of current and future analysis pipelines. SCONE achieves >99% accuracy distinguishing simulated SNe Ia from non-Ia SNe, and has been integrated into DES, LSST, and Roman analysis pipelines. We also demonstrate the efficacy of SCONE on early-time photometric classification, which will be vital for optimal allocation of spectroscopic resources. We show that SCONE can distinguish between 6 SN types with 75% accuracy on the night of initial discovery, comparable to results in the literature for full-phase SNe. Next, we study current methods for estimating SN Ia redshifts and propose a machine learning alternative that uses SN photometry alone to extract redshift information. Most SNe Ia inherit redshift information from their host galaxy, but the process of matching SNe to the correct host galaxy can be challenging. We systematically analyze the impact of incorrect redshifts from host galaxy mismatch on 5 years of DES SN data, and conclude that improved host matching or redshift estimation methods can reduce our systematic errors by ~10%. In response to this finding, we present a SN photometry-only method for estimating redshifts independent of host galaxy information, Photo-zSNthesis. We show that Photo-zSNthesis redshift estimates are accurate to within 2% across the full redshift range of LSST, a first in the literature. Finally, we focus on the robustness of machine learning (ML) algorithms for real-world and scientific applications. ML models generalize poorly beyond their training set and often experience severe performance degradation when deployed on new data. We demonstrate a general method for improving robustness that achieves new state-of-the-art results on astronomical object classification, wildlife identification, and tumor detection.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Topological and Dynamical Representations for Radio Frequency Signal Classification

Radio Frequency (RF) signals are found throughout our world, carrying over-the-air information for both digital and analog uses with applications ranging from WiFi to the radio. One area of focus in RF signal analysis is determining the modulation schemes employed in these signals which is crucial in many RF signal processing domains from secure communication to spectrum monitoring. This work investigates the accuracy and noise robustness of novel Topological Data Analysis (TDA) and dynamic representation based approaches paired with a small convolution neural network for RF signal modulation classification with a comparison to state-of-the-art deep neural network approaches. We show that using TDA tools, like Vietoris-Rips and lower star filtrations, and the Takens' embedding in conjunction with a standard shallow neural network we can capture the intrinsic dynamical, geometric, and topological features of the underlying signal's manifold, offering informative representations of the RF signals. Our approach is effective in handling the modulation classification task and is notably noise robust, outperforming the commonly used deep neural network approaches in mode classification. Moreover, our fusion of dynamical and topological information is able to attain similar performance to deep neural network architectures with significantly smaller training datasets.

Myers, Audun D.↗
DOE OSTI2024

Uncertainty-aware Continuous Implicit Neural Representations for Remote Sensing Object Counting

Many existing object counting methods rely on density map estimation (DME) of the discrete grid representation by decoding extracted image semantic features from designed convolutional neural networks (CNNs). Relying on discrete density maps not only leads to information loss dependent on the original image resolution, but also has a scalability issue when analyzing high-resolution images with cubically increasing memory complexity. Furthermore, none of the existing methods can offer reliable uncertainty quantification (UQ) for the derived count estimates. To overcome these limitations, we design UNcertainty-aware, hypernetwork-based Implicit neural representations for Counting (UNIC) to assign probabilities and the corresponding counting confidence over continuous spatial coordinates. We derive a sampling-based Bayesian counting loss function and develop the corresponding model training algorithm. UNIC outperforms existing methods on the Remote Sensing Object Counting (RSOC) dataset with reliable UQ and improved interpretability of the derived count estimates. Our code is available at https://github.com/SiyuanXu-tamu/UNIC.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Data-Driven State of Health Estimation for Second-Life Batteries Using Interpolated Synthetic Data and Feature Selection

Accurate estimation of the State of Health (SOH) for second-life batteries (SLBs) is crucial given their increasing use in energy storage applications. Precise SOH prediction is essential for safe operation and robust battery management systems. A major challenge is the limited availability of datasets for building reliable degradation models. To address this, synthetic data generation through linear interpolation is performed to extend the available data, making it more representative of real-world battery operating conditions. By analyzing feature correlation with SOH, the most relevant features are selected for the model. The proposed approach employs a convolutional neural network (CNN) model trained on this interpolated, feature-selected dataset, using time series data of voltage, temperature, and current over a cycle. By focusing on highly correlated features, the model achieves over 95% accuracy, with mean absolute error and root mean squared error up to 2.27% and 2.64%, respectively, in SOH estimation for two battery datasets tested. These results highlight the potential of combining synthetic data generation and feature selection to enhance SOH predictions, showcasing the superior performance of the proposed CNN model for both new batteries and SLBs.

feature selection↗
DOE OSTI2025

Heterogeneous Point Set Transformers for Segmentation of Multiple View Particle Detectors

NOvA is a long-baseline neutrino oscillation experiment that detects neutrino particles from the NuMI beam at Fermilab. Before data from this experiment can be used in analyses, raw hits in the detector must be matched to their source particles, and the type of each particle must be identified. This task has commonly been done using a mix of traditional clustering approaches and convolutional neural networks (CNNs). Due to the construction of the detector, the data is presented as two sparse 2D images: an XZ and a YZ view of the detector, rather than a 3D representation. We propose a point set neural network that operates on the sparse matrices with an operation that mixes information from both views. Our model uses less than 10% of the memory required using previous methods while achieving a 96.8% AUC score, a higher score than obtained when both views are processed independently (85.4%).

Robles, Edgar E. [UC, Irvine (main)]↗
DOE OSTI2026

Measurement of the muon neutrino charged-current mesonless cross section in the NOvA near detector

NOvA is a long-baseline accelerator neutrino experiment at Fermilab. Its physics goals include precision neutrino oscillation measurements, neutrino interaction cross-section measurements and beyond Standard Model explorations. We present a measurement of muon neutrino charged-current cross section with zero mesons in the final state at the NOvA near detector. This measurement is performed as a function of the kinematics of the final state muon. Our chosen interaction channel is especially sensitive to quasielastic and meson exchange current interactions and it provides handles for constraining the cross section systematic uncertainties in oscillation analyses in present and future experiments. For particle identification, we use a convolutional neural network (CNN) trained on individual particles simulated in the NOvA Near detector. This allows us to select the desired signal while reducing the potential bias from neutrino interaction modeling. We study strategies for constraining the remaining charged-pion background via Michel electron information in a template fitting approach. The main experimental result is a two-dimensional differential cross section as a function of final-state muon kinetic energy and polar angle. The parameters of this measurement, including binning and unfolding, were optimized to reduce the expected systematic uncertainty in the total cross section. The final result shows good agreement with the main GENIE-based simulation framework that was independently fine-tuned in NOvA. We finally propose improvements and subsequent steps that build on this analysis and further dissect the final states of neutrino interactions. This work has been supported by US DOE grant DE-SC0015684.

Sánchez Falero, Sebastián Jesús [Iowa State U.]↗
DOE OSTI2025

Enhancing Cluster Identification in Atom Probe Tomography Data Using Transfer Learning

Atom Probe Tomography (APT) is a powerful technique for visualizing the atomic-scale distribution of solutes in materials, but quantitative cluster analysis of APT datasets remains a challenge due to the need for subjective parameter selection in clustering algorithms. While distance-based and density-based methods such as HDBSCAN are widely used, their performance is highly sensitive to user-defined parameters, which undermines reproducibility and accuracy. This study proposes an image-based, deep learning-aided workflow for automating parameter selection and cluster detection in APT data analysis. By projecting 3D APT point clouds onto 2D planes, we leverage pretrained convolutional neural networks (ConvNeXt-Tiny and ResNet-50) through transfer learning to predict the number of clusters present in synthetic datasets. The output is used to guide K-means clustering and estimate HDBSCAN parameters, specifically minimum cluster size and minimum sample points. This approach reduces reliance on manual parameter tuning, improving consistency and scalability. The methodology demonstrates the feasibility of using image-based deep learning for interpreting complex spatial patterns in APT data, enabling faster and more objective analysis. The complete workflow and code are made publicly available to support reproducibility and future research.

Density-based clustering↗
DOE OSTI2026

Unveiling Hidden Lyman Alpha Emitters in the DESI DR1 Data

We present an automatic method based on machine-learning convolutional neural network (CNN) architecture to detect Lyman alpha emitters (LAE) hidden in the Data Release 1 spectroscopic dataset of the Dark Energy Spectroscopic Instrument (DESI). Those LAEs mostly have incorrect redshift estimations because the current DESI pipeline is not designed to detect and measure the redshifts of galaxies at $z>2$. To uncover those sources, we first visually inspect thousands of DESI spectra and construct a sample, consisting of both LAEs and non-LAEs, for training and testing the CNN-based model to (1) detect LAEs in DESI spectra and (2) determine their Ly$α$ redshifts. The final model yields $95.2\%$ purity and $95.9\%$ completeness for detecting LAEs. We apply this model to approximately $2\times10^{6}$ spectra of sources targeted as emission-line galaxies and detect 19,685 LAEs from $z\sim2$ to $3.5$ within 12 minutes with a single GPU, illustrating the high efficiency of this model for identifying LAEs. The detected LAEs are mostly at the bright end of the luminosity function with Ly$α$ luminosity $L_{\rm Lyα} \gtrsim 10^{43}$ erg/s. The high signal-to-noise composite spectrum of the detected LAEs further shows various spectral features, including P-Cygni profiles of metal lines and MgII emission lines, possible indicators of Lyman continuum escape fraction, revealing the rich astrophysical information in this LAE sample. Finally, this sample can be used to train and validate the pipelines for redshift determination of LAEs for the preparation of the DESI-II survey.

Chan, Jui-Kuan [Taiwan, Natl. Taiwan U.] (ORCID:00↗
DOE OSTI2026

Reconstruction of Six-Dimensional Phase Space

A phase space is a mathematical representation of all possible physical states of a system. Particle beams at Fermilab exist within a six-dimensional (6D) phase space defined by three positional components, (x, y, z) and three momentum components, (px, py, pz). To reconstruct this space implies taking measurement data from detectors and mapping out particle behavior using computational methods. The beam detectors, however, are only able to detect spatial distribution among the events of the beam, therefore being limited to positional data. Also, due to the vast number of events in a particle beam, it is extremely difficult to analyze and differentiate every single one’s behavior. However, with Machine Learning (ML), which can distinguish between patterns and map out particle behavior more efficiently. We first used the particle beam software, G4beamline, to simulate a 10,000-event muon beam, adjusting parameters such as initial momentum magnitude (p¬0) and virtual detector position. Using ten virtual detectors, we analyzed p0 values such that minimum 9,990 events were analyzed by every detector. We then input the data from these beam simulations to a C++ program, that randomly selects 100 events, and creates a 2D histogram based on spatial distribution, detector position, and event intensity. This process is repeated 100 times to create 100 histograms per p0 value. These images were then input to a modified ResNet18 Convolutional Neural Network (CNN) for training, and to predict p0 from some unseen set of histograms. The model was accurate when trained on momentum increments of 5 MeV/c and provided with denser training samples around highly variable test values. These results displayed machine learning being able to accurately predict p0 from being trained on different particle behaviors.

Shirlee, Jermain [Fermilab]↗
DOE OSTI2019

Improving Monsoon Precipitation Prediction Using Combined Convolutional and Long Short Term Memory Neural Network

Precipitation downscaling is widely employed for enhancing the resolution and accuracy of precipitation products from general circulation models (GCMs). In this study, we propose a novel statistical downscaling method to foster GCMs’ precipitation prediction resolution and accuracy for the monsoon region. We develop a deep neural network composed of a convolution and Long Short Term Memory (LSTM) recurrent module to estimate precipitation based on well-resolved atmospheric dynamical fields. The proposed model is compared against the GCM precipitation product and classical downscaling methods in the Xiangjiang River Basin in South China. Results show considerable improvement compared to the European Centre for Medium-Range Weather Forecasts (ECMWF)-Interim reanalysis precipitation. Also, the model outperforms benchmark downscaling approaches, including (1) quantile mapping, (2) the support vector machine, and (3) the convolutional neural network. To test the robustness of the model and its applicability in practical forecasting, we apply the trained network for precipitation prediction forced by retrospective forecasts from the ECMWF model. Compared to the ECMWF precipitation forecast, our model makes better use of the resolved dynamical field for more accurate precipitation prediction at lead times from 1 day up to 2 weeks. This superiority decreases with the forecast lead time, as the GCM’s skill in predicting atmospheric dynamics is diminished by the chaotic effect. Finally, we build a distributed hydrological model and force it with different sources of precipitation inputs. Hydrological simulation forced with the neural network precipitation estimation shows significant advantage over simulation forced with the original ERA-Interim precipitation (with NSE value increases from 0.06 to 0.64), and the performance is only slightly worse than the observed precipitation forced simulation (NSE = 0.82). This further proves the value of the proposed downscaling method, and suggests its potential for hydrological forecasts.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2022

Revisiting the Ground Magnetic Field Perturbations Challenge: A Machine Learning Perspective

Forecasting ground magnetic field perturbations has been a long-standing goal of the space weather community. The availability of ground magnetic field data and its potential to be used in geomagnetically induced current studies, such as risk assessment, have resulted in several forecasting efforts over the past few decades. One particular community effort was the Geospace Environment Modeling (GEM) challenge of ground magnetic field perturbations that evaluated the predictive capacity of several empirical and first principles models at both mid- and high-latitudes in order to choose an operative model. In this work, we use three different deep learning models-a feed-forward neural network, a long short-term memory recurrent network and a convolutional neural network-to forecast the horizontal component of the ground magnetic field rate of change (dB H /dt) over 6 different ground magnetometer stations and to compare as directly as possible with the original GEM challenge. We find that, in general, the models are able to perform at similar levels to those obtained in the original challenge, although the performance depends heavily on the particular storm being evaluated. We then discuss the limitations of such a comparison on the basis that the original challenge was not designed with machine learning algorithms in mind.

Victor A. Pinto↗
DOE OSTI2022

Application of Convolutional and Feedforward Neural Networks for Fault Detection in Particle Accelerator Power Systems

High voltage converter modulators (HVCM) provide power to the accelerating cavities of the spallation neutron source (SNS) facility. HVCM experience catastrophic failures, which increase the downtime of the SNS and reduce beam time. The faults may occur due to different reasons including failures of the resonant capacitor, core saturation due to the magnetic flux, insulated-gate bipolar transistor (IGBT) failures, and others. We recently have setup a HVCM test stand to develop and test machine learning models for anomaly detection and fault prognostics. In this work, we propose binary classifiers and autoencoder architectures based on convolutional (CNN) and feedforward neural networks (FNN) to facilitate distinguishing normal from faulty waveforms coming from the HVCM during operation. The results indicate that the CNN binary classifier is the best model among the four showing very stable performance in the training and testing sets with impressive metrics of precision and recall reaching up to 99\% with a very small uncertainty. The FNN classifier shows the least performance with a large uncertainty in its metrics. The performances of the two autoencoders based on CNN and FNN were in between, showing very good performance nonetheless.

Radaideh, Majdi↗
DOE OSTI2024

Classical analog of quantum models in synthetic dimensions

We introduce a classical analog of quantum matter in ultracold molecule-synthetic or Rydberg atom-synthetic dimensions, by extending the Potts model to include interactions J 1 between atoms adjacent in both real and synthetic space and studying its finite-temperature properties. For intermediate values of J 1 , the resulting phases and phase diagrams are similar to those of the clock and Villain models, in which three phases emerge. There exists a sheet phase analogous to that found in quantum synthetic dimension models between the high-temperature disordered phase and the low-temperature ferromagnetic phase. Furthermore, we also employ machine learning to uncover nontrivial features of the phase diagram using the learning by confusion approach, which is able to discern several successive phase transitions.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗