Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “convolutional neural network model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Construction of the damped Ly⁢𝛼 absorber catalog for DESI DR2 Ly⁢𝛼 BAO

We present the Damped Ly⁢𝛼 Toolkit for automated detection and characterization of damped Ly⁢𝛼 absorbers (DLAs) in quasar spectra. Our method uses quasar spectral templates with and without absorption from intervening DLAs to reconstruct observed quasar forest regions. The best-fitting model determines whether a DLA is present while estimating the redshift and HI column density. With an optimized quality cut on detection significance (Δ⁢𝜒$^{2}_{𝑟}$ >0.03), the technique achieves an estimated 80% purity and 79% completeness when evaluated on simulated spectra with S/N>2 that are free of broad absorption lines (BALs). We provide a catalog containing candidate DLAs from the DLA Toolkit detected in DESI DR1 quasar spectra, of which 21 719 were found in S/N>2 spectra with predicted log 10 ⁡(𝑁 𝙷𝙸 )>20.3 and detection significance Δ⁢𝜒$^{2}_{𝑟}$ >0.03. We compare the Damped Ly⁢𝛼 Toolkit to two alternative DLA finders based on a convolutional neural network and Gaussian process models. We present a strategy for combining these three techniques to produce a high-fidelity DLA catalog from DESI DR2 for the Ly⁢𝛼 forest baryon acoustic oscillation measurement. The combined catalog contains 41 152 candidate DLAs with log 10 ⁡(𝑁 𝙷𝙸 )>20.3 from quasar spectra with S/N>2. We estimate this sample to be approximately 85% pure and 79% complete when BAL quasars are excluded.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Digital Twin Empowered PV Power Prediction

The accurate prediction of photovoltaic (PV) power generation is significant to ensure the economic and safe operation of power systems. To this end, the paper proposes a new digital twin (DT) empowered PV power prediction framework that is capable of ensuring reliable data transmission and employing the DT to achieve high accuracy of power prediction. With this framework, considering potential data contamination in the collected PV data, a generative adversarial network is employed to restore the historical data set, which offers a prerequisite to ensure accurate mapping from the physical space to the digital space. Further, a new DT empowered PV power prediction method is proposed. Therein, we model a DT that encompasses a digital physical model for reflecting the physical operation mechanism and a neural network model (i.e., a parallel network of convolution and bidirectional long-short-term memory model) for capturing the hidden spatial-temporal features. The proposed method enables the use of the DT to take advantages of the digital physical model and the neural network model, resulting in enhanced prediction accuracy. Finally, a real data set is conducted to access the effectiveness of the proposed method.

14 SOLAR ENERGY↗
DOE OSTI2022

Using machine learning for particle track identification in the CLAS12 detector

Particle track reconstruction is the most computationally intensive process in nuclear physics experiments. Traditional algorithms use a combinatorial approach that exhaustively tests track measurements ("hits") to identify those that form an actual particle trajectory. In this article, we describe the development of four machine learning (ML) models that assist the tracking algorithm by identifying valid track candidates from the measurements in drift chambers. Several types of machine learning models were tested, including: Convolutional Neural Networks (CNN), Multi-Layer Perceptrons (MLP), Extremely Randomized Trees (ERT) and Recurrent Neural Networks (RNN). As a result of this work, an MLP network classifier was implemented as part of the CLAS12 reconstruction software to provide the tracking code with recommended track candidates. The resulting software achieved accuracy of greater than 99% and resulted in an end-to-end speedup of 35% compared to existing algorithms.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Estimating cluster masses from SDSS multiband images with transfer learning

ABSTRACT The total masses of galaxy clusters characterize many aspects of astrophysics and the underlying cosmology. It is crucial to obtain reliable and accurate mass estimates for numerous galaxy clusters over a wide range of redshifts and mass scales. We present a transfer-learning approach to estimate cluster masses using the ugriz-band images in the SDSS Data Release 12. The target masses are derived from X-ray or SZ measurements that are only available for a small subset of the clusters. We designed a semisupervised deep learning model consisting of two convolutional neural networks. In the first network, a feature extractor is trained to classify the SDSS photometric bands. The second network takes the previously trained features as inputs to estimate their total masses. The training and testing processes in this work depend purely on real observational data. Our algorithm reaches a mean absolute error (MAE) of 0.232 dex on average and 0.214 dex for the best fold. The performance is comparable to that given by redMaPPer, 0.192 dex. We have further applied a joint integrated gradient and class activation mapping method to interpret such a two-step neural network. The performance of our algorithm is likely to improve as the size of training data set increases. This proof-of-concept experiment demonstrates the potential of deep learning in maximizing the scientific return of the current and future large cluster surveys.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Making Invisible Visible: Data-Driven Seismic Inversion With Spatio-Temporally Constrained Data Augmentation

Deep learning and data-driven approaches have shown great potential in scientific domains. The promise of data-driven techniques relies on the availability of a large volume of high-quality training datasets. Due to the high cost of obtaining data through expensive physical experiments, instruments, and simulations, data augmentation techniques for scientific applications have emerged as a new direction for obtaining scientific data recently. However, existing data augmentation techniques originating from computer vision yield physically unacceptable data samples that are not helpful for the domain problems that we are interested in. In this article, we develop new data augmentation techniques based on convolutional neural networks. Specifically, our generative models leverage different physics knowledge (such as governing equations, observable perception, and physics phenomena) to improve the quality of the synthetic data. To validate the effectiveness of our data augmentation techniques, we apply them to solve a subsurface seismic full-waveform inversion using simulated CO 2 leakage data. Our interest is to invert for subsurface velocity models associated with very small CO 2 leakage. We validate the performance of our methods using comprehensive numerical tests. Here via comparison and analysis, we show that data-driven seismic imaging can be significantly enhanced by using our data augmentation techniques. Particularly, the imaging quality has been improved by 15% in test scenarios of general-sized leakage and 17% in small-sized leakage when using an augmented training set obtained with our techniques.

58 GEOSCIENCES↗
DOE OSTI2022

Progress on Machine Learning for the SNS High Voltage Converter Modulators

The High-Voltage Converter Modulators (HVCM) used to power the klystrons in the Spallation Neutron Source (SNS) linac were selected as one area to explore machine learning due to reliability issues in the past and the availability of large sets of archived waveforms. Progress in the past two years has resulted in generating a significant amount of simulated and measured data for training neural network models such as recurrent neural networks, convolutional neural networks, and variational autoencoders. Applications in anomaly detection, fault classification, and prognostics of capacitor degradation were pursued in collaboration with the Jefferson Laboratory, and early promising results were achieved. This paper will discuss the progress to date and present results from these efforts.

Pappas, Chris↗
DOE OSTI2021

Deep active learning for classifying cancer pathology reports

Abstract Background Automated text classification has many important applications in the clinical setting; however, obtaining labelled data for training machine learning and deep learning models is often difficult and expensive. Active learning techniques may mitigate this challenge by reducing the amount of labelled data required to effectively train a model. In this study, we analyze the effectiveness of 11 active learning algorithms on classifying subsite and histology from cancer pathology reports using a Convolutional Neural Network as the text classification model. Results We compare the performance of each active learning strategy using two differently sized datasets and two different classification tasks. Our results show that on all tasks and dataset sizes, all active learning strategies except diversity-sampling strategies outperformed random sampling, i.e., no active learning. On our large dataset (15K initial labelled samples, adding 15K additional labelled samples each iteration of active learning), there was no clear winner between the different active learning strategies. On our small dataset (1K initial labelled samples, adding 1K additional labelled samples each iteration of active learning), marginal and ratio uncertainty sampling performed better than all other active learning techniques. We found that compared to random sampling, active learning strongly helps performance on rare classes by focusing on underrepresented classes. Conclusions Active learning can save annotation cost by helping human annotators efficiently and intelligently select which samples to label. Our results show that a dataset constructed using effective active learning techniques requires less than half the amount of labelled data to achieve the same performance as a dataset constructed using random sampling.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

ExtremeMETA: High-speed Lightweight Image Segmentation Model by Remodeling Multi-channel Metamaterial Imagers

Deep neural networks (DNNs) have heavily relied on traditional computational units, such as CPUs and GPUs. However, this conventional approach brings significant computational burden, latency issues, and high power consumption, limiting their effectiveness. This has sparked the need for lightweight networks such as ExtremeC3Net. Meanwhile, there have been notable advancements in optical computational units, particularly with metamaterials, offering the exciting prospect of energy-efficient neural networks operating at the speed of light. Yet, the digital design of metamaterial neural networks (MNNs) faces precision, noise, and bandwidth challenges, limiting their application to intuitive tasks and low-resolution images. In this study, we proposed a large kernel lightweight segmentation model, ExtremeMETA. Based on ExtremeC3Net, our proposed model, ExtremeMETA maximized the ability of the first convolution layer by exploring a larger convolution kernel and multiple processing paths. With the large kernel convolution model, we extended the optic neural network application boundary to the segmentation task. To further lighten the computation burden of the digital processing part, a set of model compression methods was applied to improve model efficiency in the inference stage. The experimental results on three publicly available datasets demonstrated that the optimized efficient design improved segmentation performance from 92.45 to 95.97 on mIoU while reducing computational FLOPs from 461.07 MMacs to 166.03 MMacs. The large kernel lightweight model ExtremeMETA showcased the hybrid design’s ability on complex tasks.

large convolution kernel↗
DOE OSTI2022

Developing a heterogeneous ensemble learning framework to evaluate Alkali-silica reaction damage in concrete using acoustic emission signals

The monitoring and evaluation of Alkali-silica reaction (ASR) damage in concrete structures are required to ensure the serviceability and integrity of concrete infrastructures such as bridges and dams. The innovation of this paper lies in the development of an automatic ASR monitoring and evaluation approach by leveraging acoustic emission (AE) and a heterogeneous ensemble learning framework. Here, in this paper, ASR was monitored by AE sensors attached to a concrete specimen, which was placed in a chamber with high humidity and temperature. The recorded AE signals were filtered and divided by four ASR phases according to signal strength, crack width and expansion strain. A heterogeneous ensemble network including convolutional neural networks (CNN) and random forest models was employed to learn different features from AE signals and classify the AE signals into their corresponding phases. The results suggest that the proposed model has a high performance and classifies the signals into the assigned phases with high accuracy.

42 ENGINEERING↗
DOE OSTI2024

Toward ultra-efficient high-fidelity predictions of wind turbine wakes: Augmenting the accuracy of engineering models with machine learning

This study proposes a novel machine learning (ML) methodology for the efficient and cost-effective prediction of high-fidelity three-dimensional velocity fields in the wake of utility-scale turbines. The model consists of an autoencoder convolutional neural network with U-Net skipped connections, fine-tuned using high-fidelity data from large-eddy simulations (LES). The trained model takes the low-fidelity velocity field cost-effectively generated from the analytical engineering wake model as input and produces the high-fidelity velocity fields. The accuracy of the proposed ML model is demonstrated in a utility-scale wind farm for which datasets of wake flow fields were previously generated using LES under various wind speeds, wind directions, and yaw angles. Comparing the ML model results with those of LES, the ML model was shown to reduce the error in the prediction from 20% obtained from the Gauss Curl hybrid (GCH) model to less than 5%. In addition, the ML model captured the non-symmetric wake deflection observed for opposing yaw angles for wake steering cases, demonstrating a greater accuracy than the GCH model. The computational cost of the ML model is on par with that of the analytical wake model while generating numerical outcomes nearly as accurate as those of the high-fidelity LES.

Mechanics↗
DOE OSTI2023

SNM Radiation Signature Classification Using Different Semi-Supervised Machine Learning Models

The timely detection of special nuclear material (SNM) transfers between nuclear facilities is an important monitoring objective in nuclear nonproliferation. Persistent monitoring enabled by successful detection and characterization of radiological material movements could greatly enhance the nuclear nonproliferation mission in a range of applications. Supervised machine learning can be used to signal detections when material is present if a model is trained on sufficient volumes of labeled measurements. However, the nuclear monitoring data needed to train robust machine learning models can be costly to label since radiation spectra may require strict scrutiny for characterization. Therefore, this work investigates the application of semi-supervised learning to utilize both labeled and unlabeled data. As a demonstration experiment, radiation measurements from sodium iodide (NaI) detectors are provided by the Multi-Informatics for Nuclear Operating Scenarios (MINOS) venture at Oak Ridge National Laboratory (ORNL) as sample data. Anomalous measurements are identified using a method of statistical hypothesis testing. After background estimation, an energy-dependent spectroscopic analysis is used to characterize an anomaly based on its radiation signatures. In the absence of ground-truth information, a labeling heuristic provides data necessary for training and testing machine learning models. Supervised logistic regression serves as a baseline to compare three semi-supervised machine learning models: co-training, label propagation, and a convolutional neural network (CNN). In each case, the semi-supervised models outperform logistic regression, suggesting that unlabeled data can be valuable when training and demonstrating value in semi-supervised nonproliferation implementations.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2023

Predicting Short-Term Deformation in the Central Valley Using Machine Learning

Land subsidence caused by excessive groundwater pumping in Central Valley, California, is a major issue that has several negative impacts such as reduced aquifer storage and damaged infrastructures which, in turn, produce an economic loss due to the high reliance on crop production. This is why it is of utmost importance to routinely monitor and assess the surface deformation occurring. Two main goals that this paper attempts to accomplish are deformation characterization and deformation prediction. The first goal is realized through the use of Principal Component Analysis (PCA) applied to a series of Interferomtric Synthetic Aperture Radar (InSAR) images that produces eigenimages displaying the key characteristics of the subsidence. Water storage changes are also directly analyzed by the use of data from the Gravity Recovery and Climate Experiment (GRACE) twin satellites and the Global Land Data Assimilation System (GLDAS). The second goal is accomplished by building a Long Short-Term Memory (LSTM) model to predict short-term deformation after developing an InSAR time series using LiCSBAS, an open-source InSAR time series package. The model is applied to the city of Madera and produces better results than a baseline averaging model and a one dimensional convolutional neural network (CNN) based on a mean squared error metric showing the effectiveness of machine learning in deformation prediction as well as the potential for incorporation in hazard mitigation models. The model results can directly aid policy makers in determining the appropriate rate of groundwater withdrawal while maintaining the safety and well-being of the population as well as the aquifers’ integrity.

58 GEOSCIENCES↗
DOE OSTI2024

DASEventNet: AI‐Based Microseismic Detection on Distributed Acoustic Sensing Data From the Utah FORGE Well 16A (78)‐32 Hydraulic Stimulation

Abstract Distributed acoustic sensing (DAS) has emerged as a promising seismic technology for monitoring microearthquakes (MEQs) with high spatial resolution. Efficient algorithms are needed for processing large DAS data volumes. This study introduces a deep learning (DL) model based on a Residual Convolutional Neural Network (ResNet) for detecting MEQs using DAS data, named as DASEventNet. The test data were collected from the Utah FORGE 16A (78)‐32 hydraulic stimulation experiments conducted in April 2022. The DASEventNet model achieves a remarkable accuracy of 100% when discriminating MEQs from noise in the raw test set of 260 examples. Surprisingly, the model identified weak MEQ signatures that have been manually categorized as noise. The decision‐making process with the model is decoded by the classic activation map, which illuminates learning features of the DASEventNet model. These features provide clear illustrations of weak MEQs and varied noise types. Finally, we apply the trained model to the entire period (∼7 days) of continuous DAS recordings and find that it discovers >5,700 new MEQs, previously unregistered in the public Silixa DAS catalog. The DASEventNet model significantly outperforms the traditional seismic method Short‐Term Average/Long‐Term Average (STA/LTA), which detected only 1,307 MEQs. The DASEventNet detection threshold is M w −1.80 compared to the minimum magnitude of M w −1.14 detected by STA/LTA. The spatiotemporal distribution of the newly identified MEQs defines an extensive stimulation zone and more accurately characterizes fracture geometry. Our results highlight the potential of DL for long‐term, real‐time microseismic monitoring that can improve enhanced geothermal systems and other activities that include subsurface hydraulic fracturing.

15 GEOTHERMAL ENERGY↗
DOE OSTI2022

Graph neural networks for efficient learning of mechanical properties of polycrystals

Herein we present graph neural networks (GNNs) as an efficient and accurate machine learning approach to predict mechanical properties of polycrystalline materials. Here, a GNN was developed based on graph representation of polycrystals incorporating only fundamental features of grains including their crystallographic orientation, size, and grain neighbor connectivity information. We tested our method on modeling stiffness and yield strength of α -Ti microstructures, varying in their crystallographic texture. We find the GNN predicts both properties with high accuracy with mean relative errors of ~1% for unseen microstructures from a given set of textures and < 2% for microstructures of unseen texture, even when presented with limited training data. This accuracy is comparable to methods that require high-resolution three-dimensional (3D) microstructure data, such as 3D convolutional neural networks (3D-CNNs) and models that depend on the computation of spatial statistics. The present results show that graph-based deep learning is a promising framework for property prediction, especially considering the high cost associated with obtaining high-resolution 3D microstructure data and the general scarcity of experimental materials datasets.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Hierarchical Convolutional Neural Networks for Event Classification on PMU Measurements

Event classification is one of the central components of automated disturbance analysis based on PMU measurements. Obtaining high-quality event labels remains a challenge for supervised learning-based classification of local and system-wide events in power grids due to its labor-intensive requirement. We present a sensitivity study considering rapidly refined, partially and fully inspected event labels that leads to evidence that hierarchical convolutional neural networks (HCNNs) outperform traditional classification models regardless of the quality of the available event labels. Furthermore, it is demonstrated that performance similar to the one obtained using entirely domain-driven labeling can be achieved as long as the involved expert does not mislabel more than ~5% of the event data captured by PMU measurements.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Federated Quantum Machine Learning

Distributed training across several quantum computers could significantly improve the training time and if we could share the learned model, not the data, it could potentially improve the data privacy as the training would happen where the data is located. One of the potential schemes to achieve this property is the federated learning (FL), which consists of several clients or local nodes learning on their own data and a central node to aggregate the models collected from those local nodes. However, to the best of our knowledge, no work has been done in quantum machine learning (QML) in federation setting yet. In this work, we present the federated training on hybrid quantum-classical machine learning models although our framework could be generalized to pure quantum machine learning model. Specifically, we consider the quantum neural network (QNN) coupled with classical pre-trained convolutional model. Our distributed federated learning scheme demonstrated almost the same level of trained model accuracies and yet significantly faster distributed training. It demonstrates a promising future research direction for scaling and privacy aspects.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Direct detection of dark matter substructure in strong lens images with convolutional neural networks

Strong gravitational lensing is a potential way of uncovering the nature of dark matter, by finding perturbations to images that cannot be accounted for well by modeling the lens galaxy without additional structure, be it subhalos (smaller halos within the smooth lens) or line-of-sight (LOS) halos. Here, we introduce results attempting to infer the presence of substructure from images without requiring an intermediate step in which a smooth model has to be subtracted, using a simple convolutional neural network (CNN). We find that the network is only able to infer the presence of subhalos with greater than 75% accuracy when they have masses of greater than or equal to 5 × 10 9 M ⊙ if they lie within the main lens galaxy. Since less massive foreground LOS halos can have the same effect as higher-mass subhalos, the CNN can probe lower masses in the halo mass function. The accuracy does not improve significantly if we add a population of less massive subhalos. With the expectation of experiments such as Hubble Space Telescope and Euclid yielding thousands of high-quality strong lensing images in the next years, having a way of analyzing images quickly to identify candidates that merit further analysis to determine individual subhalo properties while preventing extensive resources being used for images that would yield null detections could be very useful. By understanding the sensitivity as a function of substructure mass, nondetections could be combined with the information from images with substructure to constrain the cold dark matter scenario, in particular if the sensitivity can be pushed to lower masses.

79 ASTRONOMY AND ASTROPHYSICS↗