Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “machine learning, neural networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Classification of Cloud Particle Imagery from Aircraft Platforms Using Convolutional Neural Networks

Abstract A vast amount of ice crystal imagery exists from a variety of field campaign initiatives that can be utilized for cloud microphysical research. Here, nine convolutional neural networks are used to classify particles into nine regimes on over 10 million images from the Cloud Particle Imager probe, including liquid and frozen states and particles with evidence of riming. A transfer learning approach proves that the Visual Geometry Group (VGG-16) network best classifies imagery with respect to multiple performance metrics. Classification accuracies on a validation dataset reach 97% and surpass traditional automated classification. Furthermore, after initial model training and preprocessing, 10 000 images can be classified in approximately 35 s using 20 central processing unit cores and two graphics processing units, which reaches real-time classification capabilities. Statistical analysis of the classified images indicates that a large portion (57%) of the dataset is unusable, meaning the images are too blurry or represent indistinguishable small fragments. In addition, 19% of the dataset is classified as liquid drops. After removal of fragments, blurry images, and cloud drops, 38% of the remaining ice particles are largely intersecting the image border (≥10% cutoff) and therefore are considered unusable because of the inability to properly classify and dimensionalize. After this filtering, an unprecedented database of 1 560 364 images across all campaigns is available for parameter extraction and bulk statistics on specific particle types in a wide variety of storm systems, which can act to improve the current state of microphysical parameterizations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Variational quantum reinforcement learning via evolutionary optimization

Abstract Recent advances in classical reinforcement learning (RL) and quantum computation point to a promising direction for performing RL on a quantum computer. However, potential applications in quantum RL are limited by the number of qubits available in modern quantum devices. Here, we present two frameworks for deep quantum RL tasks using gradient-free evolutionary optimization. First, we apply the amplitude encoding scheme to the Cart-Pole problem, where we demonstrate the quantum advantage of parameter saving using amplitude encoding. Second, we propose a hybrid framework where the quantum RL agents are equipped with a hybrid tensor network-variational quantum circuit (TN-VQC) architecture to handle inputs of dimensions exceeding the number of qubits. This allows us to perform quantum RL in the MiniGrid environment with 147-dimensional inputs. The hybrid TN-VQC architecture provides a natural way to perform efficient compression of the input dimension, enabling further quantum RL applications on noisy intermediate-scale quantum devices.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Data supporting manuscript from L. Sheneman, G. Stephanopoulos, A.E. Vasdekis titled "Deep learning classification of lipid droplets in quantitative phase images" as currently under review at PLOS ONE. This includes: 1) raw and binary labeled Quantitative Phase Images (QPI) of Y. lipolytica cells used in the analyses described within the manuscript. 2) various derived data including classifier scores, etc.

Data supporting manuscript from L. Sheneman, G. Stephanopoulos, A.E. Vasdekis titled "Deep learning classification of lipid droplets in quantitative phase images" as currently under review at PLOS ONE. This includes: 1) raw and binary labeled Quantitative Phase Images (QPI) of Y. lipolytica cells used in the analyses described within the manuscript. 2) various derived data including classifier scores, etc.

ANN↗
DOE OSTI2025

Testing convolutional neural network based deep learning systems: a statistical metamorphic approach

Machine learning technology spans many areas and today plays a significant role in addressing a wide range of problems in critical domains,i.e., healthcare, autonomous driving, finance, manufacturing, cybersecurity,etc. Metamorphic testing (MT) is considered a simple but very powerful approach in testing such computationally complex systems for which either an oracle is not available or is available but difficult to apply. Conventional metamorphic testing techniques have certain limitations in verifying deep learning-based models (i.e., convolutional neural networks (CNNs)) that have a stochastic nature (because of randomly initializing the network weights) in their training. In this article, we attempt to address this problem by using a statistical metamorphic testing (SMT) technique that does not require software testers to worry about fixing the random seeds (to get deterministic results) to verify the metamorphic relations (MRs). We propose seven MRs combined with different statistical methods to statistically verify whether the program under test adheres to the relation(s) specified in the MR(s). We further use mutation testing techniques to show the usefulness of the proposed approach in the healthcare space and test two CNN-based deep learning models (used for pneumonia detection among patients). The empirical results show that our proposed approach uncovers 85.71% of the implementation faults in the classifiers under test (CUT). Furthermore, we also propose an MRs minimization algorithm for the CUT, thus saving computational costs and organizational testing resources.

Computer Science↗
DOE OSTI2021

Learning to identify electrons

In this report we investigate whether state-of-the-art classification features commonly used to distinguish electrons from jet backgrounds in collider experiments are overlooking valuable information. A deep convolutional neural network analysis of electromagnetic and hadronic calorimeter deposits is compared to the performance of typical features, revealing a ≈ 5% gap which indicates that these lower-level data do contain untapped classification power. To reveal the nature of this unused information, we use a recently developed technique to map the deep network into a space of physically interpretable observables. We identify two simple calorimeter observables which are not typically used for electron identification, but which mimic the decisions of the convolutional network and nearly close the performance gap.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

TorchBraid: High-Performance Layer-Parallel Training of Deep Neural Networks with MPI and GPU Acceleration

TorchBraid is a high-performance implementation of layer-parallel training for deep neural networks (DNNs) supporting MPI-based parallelism and GPU acceleration. Layer-parallel training has been developed to overcome the serialization inherent in forward and backward propagation of DNNs that limits utilization of computational resources in the strong scaling limit. To achieve this, TorchBraid integrates the PyTorch neural network framework with the state-of-the-art XBraid time-parallel library. Furthermore, this article presents the use and performance of TorchBraid, in addition to solutions for overcoming the algorithmic challenges inherent in combining automatic differentiation with layer-parallel. Results are presented with and without GPU acceleration for the Tiny ImageNet and MNIST image classification data sets, as well as recurrent neural networks. Overall, TorchBraid enables fast training of DNNs, both in a strong and weak scaling context. In addition to the TorchBraid software, several new advances in applying layer-parallel algorithms are detailed. Integration of layer-parallel with data-parallel algorithms is presented for the first time, showing the computational advantages of the combination. Standard deep learning techniques, like batch-normalization, are developed for layer-parallel training. Finally, a new approach combining layer-parallel with spatial coarsening in order to accelerate training for 3D image classification shows roughly a 10× speedup over serial execution.

Layer-parallel↗
DOE OSTI2023

A flexible proton beam imaging energy spectrometer (PROBIES) for high repetition rate or single-shot high energy density (HED) experiments (invited)

The PROBIES diagnostic is a new, highly flexible, imaging and energy spectrometer designed for laser-accelerated protons. The diagnostic can detect low-mode spatial variations in the proton beam profile while resolving multiple energies on a single detector or more. When a radiochromic film stack is employed for “single-shot mode,” the energy resolution of the stack can be greatly increased while reducing the need for large numbers of films; for example, a recently deployed version allowed for 180 unique energy measurements spanning ~3 to 75 MeV with <0.4 MeV resolution using just 20 films vs 180 for a comparable traditional film and filter stack. When utilized with a scintillator, the diagnostic can be run in high-rep-rate (>Hz rate) mode to recover nine proton energy bins. Here, we also demonstrate a deep learning-based method to analyze data from synthetic PROBIES images with greater than 95% accuracy on sub-millisecond timescales and retrained with experimental data to analyze real-world images on sub-millisecond time-scales with comparable accuracy.

3D printing↗
DOE OSTI2020

Deep learning of free boundary and Stefan problems

Free boundary problems appear naturally in numerous areas of mathematics, science and engineering. These problems present a great computational challenge because they necessitate numerical methods that can yield an accurate approximation of free boundaries and complex dynamic interfaces. In this work, we propose a multi-network model based on physics-informed neural networks to tackle a general class of forward and inverse free boundary problems called Stefan problems. Specifically, we approximate the unknown solution as well as any moving boundaries by two deep neural networks. Besides, we formulate a new type of inverse Stefan problems that aim to reconstruct the solution and free boundaries directly from sparse and noisy measurements. We demonstrate the effectiveness of our approach in a series of benchmarks spanning different types of Stefan problems, and illustrate how the proposed framework can accurately recover solutions of partial differential equations with moving boundaries and dynamic interfaces. All code and data accompanying this manuscript are publicly available at https://github.com/PredictiveIntelligenceLab/DeepStefan.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Delineating Variabilities of Groundwater Level Prediction Across the Agriculturally Intensive Transboundary Aquifers of South Asia

We report the groundwater depletion in South Asia’s Himalayan, transboundary Indus-Ganges-Brahmaputra-Meghna (IGBM) rivers basin is among the highest globally. Given the high irrigation demand and population, groundwater sustainability requires an improved understanding of groundwater systems for the accurate prediction of groundwater levels (GWLs). However, the prediction of groundwater system behaviors is a significant challenge since it is dominated by spatiotemporal and subsurface depth-dependent drivers. Earlier studies that address the challenges are mainly based on the short spatial and temporal extent and/or do not separate the renewable (i.e., shallow) vs nonrenewable (i.e., deeper) groundwater signals. Here, we first identified the variable importance of spatial and depth-dependent drivers on GWL in the IGBM basin. Our results indicate a greater influence of anthropogenic factors (i.e., widespread pumping and increased population) in most parts of the IGBM basin, except in the precipitation-dominated basin of the Brahmaputra. Our next purpose was to delineate a multifactorial approach for GWL prediction using the two most used machine learning models (i.e., support vector machine and feed-forward neural network) in the literature. In general, the machine learning model outputs show a good match in comparison to the GWL from the observation wells (n = 2303 distributed across India and Bangladesh) with some limitations in areas with increased groundwater irrigation. We separately compared the results from shallow (<35 m) and deep (>35 m) observation wells, emphasizing the significance of deep groundwater pumping. Our approach highlights the importance of spatiotemporal to multidepth factors in GWL prediction and can be adopted in other parts of the globe to predict GWLs.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Predicting material microstructure evolution via data-driven machine learning

The recent rapid rise in data-driven practices in the materials science domain has led to the development of efficient, generalizable, and accurate approaches for several applications, including: material property prediction, mining (micro)structure- property and (micro)structure-processing relationships, and characterization of material microstructures. Central to the materials science domain is linking microstructure to properties and performance, and critical to building such linkages is understanding how microstructures evolve as a function of environmental exposure or processing conditions (e.g., time, temperature, applied stress or strain, irradiation). This preview article summarizes recent work on the development of recurrent neural networks for predicting spatio-temporal microstructure evolution detailed by Yang et al in their manuscript entitled 'Self-supervised Learning and Prediction of Microstructure Evolution with Recurrent Neural Networks'.

machine learning (ML), materials characterization↗
DOE OSTI2021

COVID-19 dynamics across the US: A deep learning study of human mobility and social behavior

This paper presents a deep learning framework for epidemiology system identification from noisy and sparse observations with quantified uncertainty. The proposed approach employs an ensemble of deep neural networks to infer the time-dependent reproduction number of an infectious disease by formulating a tensor-based multi-step loss function that allows us to efficiently calibrate the model on multiple observed trajectories. The method is applied to a mobility and social behavior-based SEIR model of COVID-19 spread. The model is trained on Google and Unacast mobility data spanning a period of 66 days, and is able to yield accurate future forecasts of COVID-19 spread in 203 US counties within a time-window of 15 days. Interestingly, a sensitivity analysis that assesses the importance of different mobility and social behavior parameters reveals that attendance of close places, including workplaces, residential, and retail and recreational locations, has the largest impact on the effective reproduction number. Furthermore, the model enables us to rapidly probe and quantify the effects of government interventions, such as lock-down and re-opening strategies. Taken together, the proposed framework provides a robust workflow for data-driven epidemiology model discovery under uncertainty and produces probabilistic forecasts for the evolution of a pandemic that can judiciously provide information for policy and decision making. All codes and data accompanying this manuscript are available at https://github.com/PredictiveIntelligenceLab/DeepCOVID19.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Object detection with deep learning for rare event search in the GADGET II TPC

In the pursuit of identifying rare two-particle events within the GADGET II Time Projection Chamber (TPC), this paper presents a comprehensive approach for leveraging Convolutional Neural Networks (CNNs) and various data processing methods. To address the inherent complexities of 3D TPC track reconstructions, the data is expressed in 2D projections and 1D quantities. This approach capitalizes on the diverse data modalities of the TPC, allowing for the efficient representation of the distinct features of the 3D events, with no loss in topology uniqueness. Additionally, it leverages the computational efficiency of 2D CNNs and benefits from the extensive availability of pre-trained models. Given the scarcity of real training data for the rare events of interest, simulated events are used to train the models to detect real events. To account for potential distribution shifts when predominantly depending on simulations, significant perturbations are embedded within the simulations. This produces a broad parameter space that works to account for potential physics parameter and detector response variations and uncertainties. These parameter-varied simulations are used to train sensitive 2D CNN object detectors. When combined with 1D histogram peak detection algorithms, this multi-modal detection framework is highly adept at identifying rare, two-particle events in data taken during experiment 21072 at the Facility for Rare Isotope Beams (FRIB), demonstrating a 100% recall for events of interest. Here, we present the methods and outcomes of our investigation and discuss the potential future applications of these techniques.

Convolutional neural network↗
DOE OSTI2024

Phase Selection Rules of Multi‐Principal Element Alloys

Abstract Computational prediction of phase stability of multi‐principal element alloys (MPEAs) holds a lot of promise for rapid exploration of the enormous design space and autonomous discovery of superior structural and functional properties. Regardless of many plausible works that rely on phenomenological theory and machine learning, precise prediction is still limited by insufficient data and the lack of interpretability of some machine learning algorithms, e.g., convolutional neural network. In this work, a comprehensive approach is presented, encompassing the development of a complete dataset that contains 72 387 density functional theory calculations, as well as a predictive global phenomenological descriptor. The phase selection descriptor, based on atomic electronegativity and valence electron concentration, significantly outperforms the widely used valence electron concentration, excelling in both accuracy (with an f1 score of 63% compared to 47%) and its ability to predict the HCP phase (0.48 recall compared to 0). The comprehensive data mining on the global design space of 61 425 quaternary MPEAs made from 28 possible metals, together with the phenomenological theory and physical interpretation, will set up a solid computational science foundation for data‐driven exploration of MPEAs.

Chemistry↗
DOE OSTI2022

A data-centric weak supervised learning for highway traffic incident detection

Using the data from loop detector sensors for near-real-time detection of traffic incidents on highways is crucial to averting major traffic congestion. While recent supervised machine learning methods offer solutions to incident detection by leveraging human-labeled incident data, the false alarm rate is often too high to be used in practice. Specifically, the inconsistency in the human labeling of the incidents significantly affects the performance of supervised learning models. To that end, we focus on a data-centric approach to improve the accuracy and reduce the false alarm rate of traffic incident detection on highways. We develop a weak supervised learning workflow to generate high-quality training labels for the incident data without the ground truth labels, and we use those generated labels in the supervised learning setup for final detection. This approach comprises three stages. First, we introduce a data preprocessing and curation pipeline that processes traffic sensor data to generate high-quality training data through leveraging labeling functions, which can be domain knowledge-related or simple heuristic rules. Second, we evaluate the training data generated by weak supervision using three supervised learning models-random forest, k-nearest neighbors, and a support vector machine ensemble-and long short-term memory classifiers. The results show that the accuracy of all of the models improves significantly after using the training data generated by weak supervision. Third, we develop an online real-time incident detection approach that leverages the model ensemble and the uncertainty quantification while detecting incidents. Finally, we show that our proposed weak supervised learning workflow achieves a high incident detection rate (0.90) and low false alarm rate (0.08).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Paw-Net: Stacking ensemble deep learning for segmenting scanning electron microscopy images of fine-grained shale samples

Segmentation of scanning electron microscopy (SEM) images is critical yet time-consuming for geological analyses, as it needs to differentiate the boundaries for different mineral objects to facilitate subsequent analyses, such as porosity calculation. Recently, various machine learning methods, especially convolutional neural networks (CNNs), have been explored to segment SEM images of fine-grained shale samples. However, we found that general CNNs do not yield optimal performance due to insufficient training data and imbalanced objects in SEM images. This work has revised the U-Net architecture, a popular approach for biomedical image analyses, by incorporating a loss function that addresses the imbalance issue. Furthermore, we used the ensemble learning method to train multiple models and combined the results to improve the overall performance of segmentation. We prepared 2162 sub-images from raw SEM images in our experiments and divided them into training, validation, and testing datasets. The overall results show that our method improves the average Intersection over Union (IOU) of mineral objects from 0.49 to 0.58, compared to the original U-Net model. Our method can clearly distinguish each object from others with boundaries, even in highly imbalanced images. Training our models takes less than three minutes using a single GPU, while manual labeling can take up to three hours for each image. Furthermore, the method helps geoscientists gain insights quickly and effectively by building neural network models from a small dataset of SEM images.

58 GEOSCIENCES↗
DOE OSTI2026

Short-term electricity load forecasting: Application-driven evaluation of machine learning models across spatial and temporal scales

As we transition towards a decarbonized economy, the integration of variable renewable energy resources and new demands (e.g., electric vehicles, heat pumps) into the electricity grid places unprecedented pressure on grid operators to effectively anticipate and manage peak load. In this context, machine learning algorithms are proving to be indispensable for accurate short-term load forecasting, a crucial task to address these challenges. This study benchmarks 6 machine learning algorithms, including three neural networks and three tree-based algorithms, across various levels of spatial aggregation and time horizons (1, 4, 8, 24, and 48 h). The central contribution of this work is the comparison and analysis of load forecasting models not only based on statistical metrics, but also based on a novel error metric, which evaluates the cost implications of forecast errors for power system stakeholders. Results show that tree-based models outperform neural networks, based on statistical metrics, and yield less skewed error distributions for most spatial scales. However, through the lens of the novel error metric, neural networks are the more competitive choice, especially for forecast horizons that exceed 8 h. The study concludes with actionable recommendations to grid operators and highlights the need for the development of error metrics that link forecasting accuracy to operational costs. To promote transparency and open science, the datasets and Python code are open-sourced via a supplementary repository.

Houben, Nikolaus↗
DOE OSTI2025

Graph neural networks for mechanical property prediction of 2D fiber composites

This work investigates the ability of graph neural networks (GNNs) to homogenize 2D fiber composite microstructures. We use different inhomogeneity and anisotropy indices to motivate and show that the Volume Elements (VEs) used in ML methods should ideally be far from their Representative Volume Element (RVE) size limit and, consequently, are notably anisotropic. Hence, training only the isotropic limit properties may not be acceptable. Another aspect is the need to normalize elastic stiffness values for ML, especially when high elastic contrast ratios are encountered between composite phases or in the material set. We introduce a normalization technique based on the mean-field method (MFM) to handle such high contrast ratios and train for the entire stiffness tensor. We show that the proposed GNN approaches exhibit high accuracy and efficiency compared to traditional methods and convolutional neural networks, utilizing unstructured graphs constructed from microstructure topology. Our model successfully predicts the stiffness tensor, peak strength under bulk damage, and brittle fracture initiation strength across diverse microstructure configurations while maintaining high accuracy even for extreme material contrasts and volume fractions. We also present a method to improve prediction accuracy for small dataset sizes using Voronoi partitioning.

Brittle strength↗