Cloud Segmentation & Motion Tracking in Sky Images: A Case Study using Machine Learning for PV Applications.
Abstract not provided.
Pierce, Benjamin↗
SEARCH · Engineering Papers
Results for “Image segmentation”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Satellite Image Algorithms
Presented by Conner Hatton at SIAM UQ Conference in Trieste, Italy, on February 27th, 2024. Conference details available online at https://www.siam.org/conferences/cm/conference/uq24
97 MATHEMATICS AND COMPUTING↗
Computer vision models and advanced TEM imaging for microstructures of irradiated AM316 stainless steels
Advancements were made in automating microscopy-based material characterization, particularly in studying irradiation effects on additively manufactured (AM) materials using machine learning (ML) and computer vision (CV). These automation efforts address the challenges of analyzing complex microstructures, accelerating the detection of irradiation-induced defects. Two CV models were developed at Argonne National Laboratory (ANL) to enhance transmission electron microscopy (TEM) analysis of irradiated AM 316 stainless steel. The first model focused on the detection of irradiation-induced dislocation loops, which contribute to material hardening and embrittlement. These loops, categorized as faulted or perfect, were automatically detected and classified using a Mask R-CNN model trained on TEM images from both in-situ and ex-situ ion irradiation experiments. The model achieved high accuracy, with precision, recall, and F1 scores of 0.839, 0.734, and 0.776, respectively, demonstrating its effectiveness in analyzing dislocation loops in irradiated AM materials. The second CV model was developed to analyze the size and wall thickness of dislocation cells in laser powder bed fusion (LPBF) 316 stainless steel. Using a U-Net++ architecture with EfficientNet as the encoder, the model was trained on TEM images to segment and measure cell size and wall thickness.
36 MATERIALS SCIENCE↗
Stochastic microstructural analysis of ceramic matrix composites using a high-fidelity multiscale framework
This paper presents a high-fidelity multiscale framework to simulate the mechanical behavior of ceramic matrix composites (CMCs), accounting for the complex micromorphology captured using detailed material characterization. First, high-resolution micrographs are obtained for the specific carbon fiber silicon-carbide-nitride matrix (C/SiNC) CMC to characterize the variability of the architectural features and manufacturing-induced defects at the microscale. An image processing algorithm is then used to precisely estimate the size and distribution of all subscale features and defects from the micrographs. The information is then used to generate a three-dimensional stochastic representative volume element (SRVE) to reconstruct microscale constituents accounting for the variability. As a result, the generated SRVEs are simulated using the high-fidelity generalized method of cells (HFGMC) micromechanics theory to investigate the effects of defects on the elastic properties of C/SiNC CMCs.
Ceramic Matrix Composites↗
A framework of computer vision-enhanced microfluidic approach for automated assessment of the transient sickling kinetics in sickle red blood cells
The occurrence of vaso-occlusive crisis greatly depends on the competition between the sickling delay time and the transit time of individual sickle cells, i.e., red blood cells from sickle cell disease (SCD) patients, while they are traversing the circulatory system. Many drugs for treating SCD work by inhibiting the polymerization of sickle hemoglobin (HbS), effectively delaying the sickling process in sickle cells (SS RBCs). Most previous studies on screening anti-sickling drugs, such as voxelotor, rely on in vitro testing of sickling characteristics, often conducted under prolonged deoxygenation for up to 1 hour. However, since the microcirculation of RBCs typically takes less than 1 minute, the results of these studies may be less accurate and less relevant for in vitro-in vivo correlation. In our current study, we introduce a computer vision-enhanced microfluidic framework designed to automatically capture the transient sickling kinetics of SS RBCs within a 1-min timeframe. Our study has successfully detected differences in the transient sickling kinetics between vehicle control and voxelotor-treated SS RBCs. This approach has the potential for broader applications in screening anti-sickling therapies.
59 BASIC BIOLOGICAL SCIENCES↗
Application of a deep learning semantic segmentation model to helium bubbles and voids in nuclear materials
Imaging nanoscale radiation-induced defects using the transmission electron microscope (TEM) is a key factor in the successful implementation of materials for nuclear energy structural applications. Analyzing each defect in a TEM micrograph is currently a manual task. To identify the defects in a single image can take anywhere from 15 min to an hour and a project can require the analysis of anywhere from tens to ≥ 100 images. Here, we use artificial intelligence (AI) models to automate this task. For simplification, we evaluated images with only a single type of defect; helium bubbles. Additionally, we performed semantic segmentation of these helium bubble defects in electron microscopy images of irradiated FeCrAl alloys using a deep learning DefectSegNet model. This model, which was previously used to classify crystal defects, is inspired by the classic DenseNet and U-Net image segmentation models. It claims high spatial resolution, but has poor performance at object boundaries. Our paper improves the DefectSegNet model’s application by adding two new features. First, the DefectSegNet model is applied not only to perform calculation pixel-wise but also object (or feature) wise. Because object-wise metrics are directly relevant to our final goal of detecting bubbles, whereas pixel-wise classification is only an intermediate step, it is an important part of our study. Second, a distance map loss (DML) function has been added to increase its performance at object boundaries. It is crucial to accurately represent defects boundaries, especially bubbles, in order to track the bubble-induced swelling caused by irradiation. The boundary-focused DML function is also compared to other loss functions like Cross-entropy, Weighted Binary Cross Entropy (WBCE), Dice and Intersection over Union (IOU). Finally, by incorporating new features, we found a marked improvement on segmentation quality and better shape preservation at the boundaries and areas of the bubbles.
42 ENGINEERING↗
Instance Segmentation for Direct Measurements of Satellites in Metal Powders and Automated Microstructural Characterization from Image Data
In this work, we propose instance segmentation as a useful tool for image analysis in materials science. Instance segmentation is an advanced technique in computer vision which generates individual segmentation masks for every object of interest that is recognized in an image. Using an out-of-the-box implementation of Mask R-CNN, instance segmentation is applied to images of metal powder particles produced through gas atomization. Leveraging transfer learning allows for the analysis to be conducted with a very small training set of labeled images. As well as providing another method for measuring the particle size distribution, we demonstrate the first direct measurements of the satellite content in powder samples. After analyzing the results for the labeled data dataset, the trained model was used to generate measurements for a much larger set of unlabeled images. The resulting particle size measurements showed reasonable agreement with laser scattering measurements. The satellite measurements were self-consistent and showed good agreement with the expected trends for different samples. Finally, we present a small case study showing how instance segmentation can be used to measure spheroidite content in the UltraHigh Carbon Steel DataBase, demonstrating the flexibility of the technique.
36 MATERIALS SCIENCE↗
RU Net for Automatic Characterization of TRISO Fuel Cross Sections
TRistructural ISOtropic (TRISO) particle fuel is a type of nuclear fuel known for its high-temperature and high-burnup performance. Each sub-millimeter diameter TRISO particle consists of uranium-oxycarbide (UCO) or UO2 fuel kernel, coated with buffer, inner pyrolytic carbon (IPyC), silicon carbide (SiC), and outer pyrolytic carbon (OPyC) layers. The SiC layer acts as the main containment barrier for the TRISO particle to retain the fission products, while the IPyC and OPyC layers provide additional barriers to the release of fission products, especially fission gases. During irradiation, phenomena like kernel swelling, buffer densification, and IPyC fracture may impact fuel performance. Post-irradiation microscopy on entire compact cross sections or samples of individual particles deconsolidated from compacts is often used to identify these irradiation-induced changes in morphology. However, each fuel compact generally contains thousands of TRISO particles. To get statistical information on these phenomena, it is cumbersome work if done manually. For example, to get information about swelling/densification behaviors of different layers or kernels after irradiation, researchers previously manually measured the perimeter of each TRISO layer in hundreds of particles after four rounds of iterative grinding and polishing encompassing more than 2000 cross-section images for a total of four fuel compacts. To attempt to reduce the subjectivity inherent in that process and accelerate data analysis, we conducted a study on the automatic TRISO layer segmentation on cross-sectional microscopic images using Convolutional Neural Networks (CNNs). CNNs are a class of machine learning algorithms specifically designed for processing structured grid data that have gained popularity in recent years due to their remarkable performance in various computer vision tasks, including image classification, object detection, and image segmentation. In this research, we have generated the large irradiated TRISO layer dataset with more than 2000 cross-section TRISO microscopic images and the corresponding annotated images. Based on these annotated images, we have employed different CNNs for automatic segmentation of different TRISO layers. These include RU-Net (developed in this study), as well as three existing architectures: U-Net, Residual Network (ResNet), and Attention U-Net. The preliminary results show that the model based on RU-Net has the best performance in terms of intersection-over-union (IoU). Through the aid of these CNN models, we can expedite the analysis of TRISO particle cross-sections, significantly reducing the manual labor involved and improving the objectivity of the segmentation results.
Convolutional Neural Networks↗
Fine-tuning TrailMap: The utility of transfer learning to improve the performance of deep learning in axon segmentation of light-sheet microscopy images
Light-sheet microscopy has made possible the 3D imaging of both fixed and live biological tissue, with samples as large as the entire mouse brain. However, segmentation and quantification of that data remains a time-consuming manual undertaking. Machine learning methods promise the possibility of automating this process. This study seeks to advance the performance of prior models through optimizing transfer learning. We fine-tuned the existing TrailMap model using expert-labeled data from noradrenergic axonal structures in the mouse brain. By changing the cross-entropy weights and using augmentation, we demonstrate a generally improved adjusted F1-score over using the originally trained TrailMap model within our test datasets.
97 MATHEMATICS AND COMPUTING↗
Data-driven Mapping of the Mouse Connectome: The utility of transfer learning to improve the performance of deep learning models performing axon segmentation on light-sheet microscopy images
Light sheet microscopy has made possible the high temporal and spatial 3D imaging of both fixed and live biological tissue, with samples as large as the entire mouse brain. However, segmentation and quantification of that data remains a time-consuming manual process. Machine learning methods promise the possibility of automating this process. This study seeks to advance the performance of prior models through the application of refinements such as transfer learning.
59 BASIC BIOLOGICAL SCIENCES↗
EDS-PhaSe: Phase Segmentation and Analysis from EDS Elemental Map Images Using Markers of Elemental Segregation
Scanning electron microscopy (SEM), combined with energy-dispersive spectroscopy (EDS), is an extensively used technique for in-depth microstructural analysis. Here, we present the EDS-Phase Segmentation (EDS-PhaSe) tool that enables phase segmentation and phase analysis using the EDS elemental map images. Importantly, it converts the EDS map images into estimated composition maps for calculating markers of selective elemental redistribution in the scanned area and creates a phase-segmented micrograph while providing approximate fraction and composition of each identified phase. EDS-PhaSe offers two unique advantages. Firstly, it enables the direct processing of EDS elemental map images without requiring any raw or proprietary data/software, thereby allowing the analysis of EDS results available in the published literature as images. Secondly, it enables segmentation and analysis of phases even when the phase contrast is missing in backscattered micrographs, assisting in correlating the XRD and SEM-EDS data as shown in this work for a AlCoCrFeNi high-entropy alloy.
36 MATERIALS SCIENCE↗
A Machine Learning Approach to Quantitative Analysis of Enamel Microstructure from Scanning Electron Microscopy Images
Dental enamel, the outermost tissue of mammalian teeth, must withstand a lifetime of wear and cyclic contact. To meet this demand, enamel possesses a combination of high hardness and resistance to fracture, properties that are typically mutually exclusive. The impressive damage tolerance has been attributed largely to decussation of the enamel rods, the principal unit of its microstructure. As such, enamel is inspiring the design of next‐generation structural materials. However, quantitative descriptions of the decussated enamel rod microstructure remain limited due to challenges encountered in applying computed tomography and in acquiring quality images appropriate for traditional digital processing methods. Here, a machine learning segmentation method is applied to images of the enamel obtained using scanning electron microscopy to support quantitative analysis of the microstructure. A pretrained convolutional neural network is used to expand the input training image dataset to allow the training of a random forest classifier, which ultimately segments the image with a very small training set ( n = 3 images). A validation of this segmentation method is presented, in addition to its application to calculate relevant microstructural parameters for images of tooth enamel from selected mammalian species. The methodology applied here is equally applicable to other hard tissues.
36 MATERIALS SCIENCE↗
Measurement of hybrid rocket solid fuel regression rate for a slab burner using deep learning
This study presents an imaging-based deep learning tool to measure the fuel regression rate in a 2D slab burner experiment for hybrid rocket fuels. The slab burner experiment is designed to verify mechanistic models of reacting boundary layer combustion in hybrid rockets by the measurement of fuel regression rates. A DSLR camera with a high intensity flash is used to capture images throughout the burn and the images are then used to find the fuel boundary to calculate the regression rate. A U-net convolutional neural network architecture is explored to segment the fuel from the experimental images. Here, a Monte-Carlo Dropout process is used to quantify the regression rate uncertainty produced from the network. The U-net computed regression rates are compared with values from other techniques from literature and show error less than 10%. An oxidizer flux dependency study is performed and shows the U-net predictions of regression rates are accurate and independent of the oxidizer flux, when the images in the training set are not over-saturated. Training with monochrome images is explored and is not successful at predicting the fuel regression rate from images with high noise. The network is superior at filtering out noise introduced by soot, pitting, and wax deposition on the chamber glass as well as the flame when compared to traditional image processing techniques, such as threshold binary conversion and spatial filtering. U-net consistently provides low error image segmentations to allow accurate computation of the regression rate of the fuel.
42 ENGINEERING↗
Few-Shot Learning Enables Population-Scale Analysis of Leaf Traits in Populus trichocarpa
Plant phenotyping is typically a time-consuming and expensive endeavor, requiring large groups of researchers to meticulously measure biologically relevant plant traits, and is the main bottleneck in understanding plant adaptation and the genetic architecture underlying complex traits at population scale. In this work, we address these challenges by leveraging few-shot learning with convolutional neural networks to segment the leaf body and visible venation of 2,906 Populus trichocarpa leaf images obtained in the field. In contrast to previous methods, our approach (a) does not require experimental or image preprocessing, (b) uses the raw RGB images at full resolution, and (c) requires very few samples for training (e.g., just 8 images for vein segmentation). Traits relating to leaf morphology and vein topology are extracted from the resulting segmentations using traditional open-source image-processing tools, validated using real-world physical measurements, and used to conduct a genome-wide association study to identify genes controlling the traits. In this way, the current work is designed to provide the plant phenotyping community with (a) methods for fast and accurate image-based feature extraction that require minimal training data and (b) a new population-scale dataset, including 68 different leaf phenotypes, for domain scientists and machine learning researchers. All of the few-shot learning code, data, and results are made publicly available.
59 BASIC BIOLOGICAL SCIENCES↗
Data for Clumping Index Estimation With 30°-tilted Cameras in Row Crops: Evaluation of Methods and Segment Size Effects
The clumping index (CI) quantifies the spatial distribution of foliage elements and is essential for accurately estimating the plant area index (PAI), canopy radiative transfer, and photosynthesis. Traditionally, the finite-length averaging method (LX), the gap size distribution method (CC), and a combined approach of CC and LX (CLX) have been applied to instruments like TRAC and digital hemispherical photography to estimate CI. However, a comprehensive evaluation of these methods in row crops remains limited, especially regarding the influence of segment size on CI. Meanwhile, digital cameras offer a cost-effective and user-friendly solution for canopy measurements in row crops, yet their application in this context remains underexplored. In this study, we employed a new approach using a 30°-tilted digital camera to estimate CI in corn and soybean fields, applying the LX, CC, and CLX methods. We systematically assessed the performance of these three methods by combining field measurements in real-world fields with simulations using the LESS 3D radiative transfer model. Our results showed that CLX applied to the whole image and 45° segment offered accurate estimation of CI (bias within ±0.1, RMSE < 0.2) and PAI (bias within ±0.4, RMSE < 1) in real-world fields and LESS simulations. The accuracy of the LX method was highly sensitive to segment size, with the best performance observed at the 15° segment (PAI bias within ±0.4). In contrast, the CC method remained stable across different segment sizes, and its performance was generally comparable to that of LX, except at the 15° segment. Across view zenith angles, CI derived from CC generally showed a continuous increase, while those from LX and CLX followed a rising trend at small zenith angles but began to decline at 68°, likely due to an increasing proportion of no-gap segments. Seasonally, LX tended to show decreasing CI during early growth stages but increased as the canopy matured, whereas CC and CLX showed gradually increasing CI before plateauing at peak PAI. The 30°-tilted camera effectively captured CI variations across different angles and growth stages, making it a practical and robust instrument for row crop canopy structure analysis. Applying these CI methods to digital cameras offers a low-cost and accessible CI estimation alternative, improving canopy structure monitoring accuracy in row crops.
Modeling↗
Automated Grain Boundary (GB) Segmentation and Microstructural Analysis in 347H Stainless Steel Using Deep Learning and Multimodal Microscopy
Austenitic 347H stainless steel offers superior mechanical properties and corrosion resistance required for extreme operating conditions such as high temperature. The change in microstructure due to composition and process variations is expected to impact material properties. Identifying microstructural features such as grain boundaries thus becomes an important task in the process-microstructure-properties loop. Applying convolutional neural network (CNN)-based deep learning models is a powerful technique to detect features from material micrographs in an automated manner. In contrast to microstructural classification, supervised CNN models for segmentation tasks require pixel-wise annotation labels. However, manual labeling of the images for the segmentation task poses a major bottleneck for generating training data and labels in a reliable and reproducible way within a reasonable timeframe. Microstructural characterization especially needs to be expedited for faster material discovery by changing alloy compositions. Here, in this study, we attempt to overcome such limitations by utilizing multimodal microscopy to generate labels directly instead of manual labeling. We combine scanning electron microscopy images of 347H stainless steel as training data and electron backscatter diffraction micrographs as pixel-wise labels for grain boundary detection as a semantic segmentation task. The viability of our method is evaluated by considering a set of deep CNN architectures. We demonstrate that despite producing instrumentation drift during data collection between two modes of microscopy, this method performs comparably to similar segmentation tasks that used manual labeling. Additionally, we find that naïve pixel-wise segmentation results in small gaps and missing boundaries in the predicted grain boundary map. By incorporating topological information during model training, the connectivity of the grain boundary network and segmentation performance is improved. Finally, our approach is validated by accurate computation on downstream tasks of predicting the underlying grain morphology distributions which are the ultimate quantities of interest for microstructural characterization.
36 MATERIALS SCIENCE↗
Online thermal profile prediction for large format additive manufacturing: A hybrid CNN-LSTM based approach
Large format additive manufacturing (LFAM) is an advanced 3D printing technique that efficiently fabricates large-scale components through a layer-by-layer extrusion and deposition process. Accurate surface layer temperature monitoring is essential to prevent manufacturing failures and ensure final product quality. Traditional physics-based offline approaches for simulating thermal behavior are often inefficient and complex, posing challenges on real-time, in-situ monitoring. Here, to address this, we propose a data-driven hybrid CNN-LSTM model to predict sequential thermal images of arbitrary length using real-time infrared thermal imaging. In this approach, a Convolutional Neural Networks (CNN) is trained offline to capture spatial features, reduce dimensional complexity, and enhance time efficiency, while a stacked Long Short-Term Memory (LSTM) is applied online to capture temporal information for improved prediction of future thermal behavior in subsequent printing layers. Model performance is evaluated using MSE, SSIM, and PSNR metrics and is benchmarked against stacked LSTM and convolutional LSTM models, demonstrating superior accuracy and applicability. Additionally, to mitigate noise from moving extruders and gantry backgrounds in thermal images, a fine-tuned semantic segmentation model is implemented offline to extract printing geometry, enabling precise temperature tracking along the tool path for further thermal analysis. The frameworks developed in this study significantly advance temperature monitoring, thermal analysis, and in-situ manufacturing control for LFAM, bridging the gap between theoretical modeling and practical application.
Geometry extraction↗
A One-Stage Domain Adaptation Network With Image Alignment for Unsupervised Nighttime Semantic Segmentation
Not provided.
Computer Science↗