Cloud Segmentation & Motion Tracking in Sky Images: A Case Study using Machine Learning for PV Applications.
Abstract not provided.
Pierce, Benjamin↗
SEARCH · Engineering Papers
Results for “Image segmentation”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Satellite Image Algorithms
Presented by Conner Hatton at SIAM UQ Conference in Trieste, Italy, on February 27th, 2024. Conference details available online at https://www.siam.org/conferences/cm/conference/uq24
97 MATHEMATICS AND COMPUTING↗
Computer vision models and advanced TEM imaging for microstructures of irradiated AM316 stainless steels
Advancements were made in automating microscopy-based material characterization, particularly in studying irradiation effects on additively manufactured (AM) materials using machine learning (ML) and computer vision (CV). These automation efforts address the challenges of analyzing complex microstructures, accelerating the detection of irradiation-induced defects. Two CV models were developed at Argonne National Laboratory (ANL) to enhance transmission electron microscopy (TEM) analysis of irradiated AM 316 stainless steel. The first model focused on the detection of irradiation-induced dislocation loops, which contribute to material hardening and embrittlement. These loops, categorized as faulted or perfect, were automatically detected and classified using a Mask R-CNN model trained on TEM images from both in-situ and ex-situ ion irradiation experiments. The model achieved high accuracy, with precision, recall, and F1 scores of 0.839, 0.734, and 0.776, respectively, demonstrating its effectiveness in analyzing dislocation loops in irradiated AM materials. The second CV model was developed to analyze the size and wall thickness of dislocation cells in laser powder bed fusion (LPBF) 316 stainless steel. Using a U-Net++ architecture with EfficientNet as the encoder, the model was trained on TEM images to segment and measure cell size and wall thickness.
36 MATERIALS SCIENCE↗
Stochastic microstructural analysis of ceramic matrix composites using a high-fidelity multiscale framework
This paper presents a high-fidelity multiscale framework to simulate the mechanical behavior of ceramic matrix composites (CMCs), accounting for the complex micromorphology captured using detailed material characterization. First, high-resolution micrographs are obtained for the specific carbon fiber silicon-carbide-nitride matrix (C/SiNC) CMC to characterize the variability of the architectural features and manufacturing-induced defects at the microscale. An image processing algorithm is then used to precisely estimate the size and distribution of all subscale features and defects from the micrographs. The information is then used to generate a three-dimensional stochastic representative volume element (SRVE) to reconstruct microscale constituents accounting for the variability. As a result, the generated SRVEs are simulated using the high-fidelity generalized method of cells (HFGMC) micromechanics theory to investigate the effects of defects on the elastic properties of C/SiNC CMCs.
Ceramic Matrix Composites↗
A framework of computer vision-enhanced microfluidic approach for automated assessment of the transient sickling kinetics in sickle red blood cells
The occurrence of vaso-occlusive crisis greatly depends on the competition between the sickling delay time and the transit time of individual sickle cells, i.e., red blood cells from sickle cell disease (SCD) patients, while they are traversing the circulatory system. Many drugs for treating SCD work by inhibiting the polymerization of sickle hemoglobin (HbS), effectively delaying the sickling process in sickle cells (SS RBCs). Most previous studies on screening anti-sickling drugs, such as voxelotor, rely on in vitro testing of sickling characteristics, often conducted under prolonged deoxygenation for up to 1 hour. However, since the microcirculation of RBCs typically takes less than 1 minute, the results of these studies may be less accurate and less relevant for in vitro-in vivo correlation. In our current study, we introduce a computer vision-enhanced microfluidic framework designed to automatically capture the transient sickling kinetics of SS RBCs within a 1-min timeframe. Our study has successfully detected differences in the transient sickling kinetics between vehicle control and voxelotor-treated SS RBCs. This approach has the potential for broader applications in screening anti-sickling therapies.
59 BASIC BIOLOGICAL SCIENCES↗
Identification of Polygonal Patterns on Venus Using Mathematical Morphology
We develop a detection and recognition model to fully automatize the identification of polygonal faults present in many of the Magellan SAR images. Our segmentation model is based on a modified watershed algorithm. Additional information is contained in the original extended abstract.
Moreels, P.↗
CMOS sum and difference imager with on-chip charge-leakage compensation
This paper presents a new technique for implementing a low-power CMOS imager with simultaneous on-chip computation of the difference and summ of two successive frames.
imager object segmentation↗
Assessing Alaskan boreal forest landcover affected by climate-wildfire interactions from ground truth surveys and NASA airborne remote sensing
Alaska’s boreal forest is facing unprecedented challenges under rapid climate warming (increasingly severe fires, droughts, pest/disease outbreaks) that may destabilize its function as a global carbon sink. Forests near Fairbanks may be especially vulnerable, impacting air quality and ecosystem services. We combined GT (ground truthing) with Airborne Visible InfraRed Imaging Spectrometer (AVIRIS-NG) images collected by the NASA Arctic-Boreal Vulnerability Experiment (ABoVE) program (2017-2019) to assess landcover change at five recently burned sites (2001-2019) of different fire severities and moisture regimes within 30 miles of Fairbanks. GT included tree seedling counts, understory % cover and >50% leaf canopy color assessment. 36 circular plots (1/30 ha radius) including 6 moderate to severely burned plots were selected across sites. 31 additional sites including 12 burned sites were geotagged in photos. AVIRIS images were processed from 29 spectral bands selected to identify changes in chlorophyll and water content. Images were segmented into natural boundaries (polygons) using ENVI 5.5 software. A spectral library of 8 AVIRIS bands with high between-class/low within-class variation was used in two random forest models to predict vegetation classes (model 1: 12 classes, model 2: 14 classes) in each AVIRIS scene, using 20% of the data as training data. Model 2 classified 20% more polygons overall, but only 42% of GT/geotagged polygons were correctly classified by both models. More forest sites were correctly classified (63%) than open vegetation (32%) or post-fire sites (46%). 50% of aspen forest and post-fire polygons were misclassified as shrubland. GT revealed that post-fire plots supported 134,000 (± 48,000) tree seedlings and saplings ha-1 (0.2 - 4 m height, 64% deciduous) versus 2500 (± 2100) shrubs ha-1 (1-6 m height). > 50% canopy browning was observed in conifer forest (8 plots) with no signs of insect infestation. Canopy herbivory > 50% (leaf miner, leaf beetle) and moose herbivory of tree bark was seen across aspen sites. Our study suggests: 1) low canopy vegetation presents challenges for improved landcover classification, and 2) aspen forest should be differentiated in vegetation maps which would aid in tracking herbivory.
Alaska↗
AI4MARS: A Dataset for Terrain-Aware Autonomy on Mars
Deep learning has quickly become a necessity for selfdriving vehicles on Earth. In contrast, the self-driving vehicles on Mars, including NASA’s latest rover, Perseverance, which is planned to land on Mars in February 2021, are still driven by classical machine vision systems. Deep learning capabilities, such as semantic segmentation and object recognition, would substantially benefit the safety and productivity of ongoing and future missions to the red planet. To this end, we created the first large-scale dataset, AI4Mars, for training and validating terrain classification models for Mars, consisting of ~326K semantic segmentation full image labels on 35K images from Curiosity, Opportunity, and Spirit rovers, collected through crowdsourcing. Each image was labeled by ~10 people to ensure greater quality and agreement of the crowdsourced labels. It also includes ~1.5K validation labels annotated by the rover planners and scientists from NASA’s MSL (Mars Science Laboratory) mission, which operates the Curiosity rover, and MER (Mars Exploration Rovers) mission, which operated the Spirit and Opportunity rovers. We trained a DeepLabv3 model on the AI4Mars training dataset and achieved over 96% overall classification accuracy on the test set. The dataset is made publicly available.1
Ono, Hiro↗
TPS Certification by Analysis: Model-Driven Characterization of Properties and Failure in Woven Thermal Protection Systems
Woven, ablative thermal protection system (TPS) materials provide a robust option for aggressive (re)entries and thus have been baselined for the upcoming Mars Sample Return (MSR) mission's Earth Entry System (EES). The reliability requirements for MSR-EES necessitate understanding of material property variability, which could be significant given the complex structure and anisotropic nature of properties in TPS weaves, as well as the response to potential impact with micrometeoroids or orbital debris during the EES re-entry. The TPS Certification by Analysis effort within the Entry Systems Modeling project seeks to provide computational models and analyses that support the certification against such material-based risks. For the present talk, focus will be given to the characterization of baseline woven TPS material properties and mechanical failure limits, which entails (1) use of computational techniques (e.g., machine learning) to interpret computed tomography images of the weave to generate representative structural models and (2) application of multiscale material modeling approaches to characterize thermomechanical and failure properties.
thermal protection systems↗
Multi-Mission Terrain Classifier for Safe Rover Navigation and Automated Science
We previously presented Soil Property and Object Classification (SPOC), a machine learning-based terrain classifier for Mars rovers, for automatically segmenting rover images by its surface type such as sand and bedrock. This paper presents a number of practical improvements to pave the way for potential future onboard deployment. First, we achieved 97.0% overall pixel accuracy, evaluated against the classification generated by human experts on images from Mars Science Laboratory (MSL) missions. The substantial increase in accuracy was primarily enabled by the sheer volume of data used for training; we created a new large-scale dataset of Martian terrain labels, namely AI4Mars, which contains more than 400k labels contributed by citizen scientists for 50k images taken by the Mars Exploration Rovers (MER) and Mars Science Laboratory (MSL) rover. Second, we demonstrated that SPOC can quickly adapt to a new mission landed on a previously unseen site. Specifically, we pretrained a model with MER and MSL data from the AI4Mars dataset and then adapted to the Mars 2020 Rover (M2020) by feeding a small volume of data between Sol 0 and 157; the adapted model was tested on Sol 200-203 and resulted in 84.2% overall pixel accuracy and 93.4% reliability (recall) for detecting sand, the most concerning class for rover’s traversability. Third, we found that pretraining can substantially mitigate the decline of accuracy over time. We showed that the performance of a SPOC model pretrained with the ImageNet dataset and then trained by MSL images only up to Sol 390 remains comparable to a model trained by images up to Sol 1689 on the test data after Sol 1689. Fourth, we reimplemented SPOC with a light-weight convolutional neural network (CNN), MobileNetV2, which typically runs within tens of milliseconds (ms) on mobile processors such as Qualcomm’s Snapdragon. Finally, we released the AI4Mars dataset to the public to encourage open innovation.
Ono, Masahiro↗
Image Processing Methods for Coronal Hole Segmentation, Matching, and Map Classification
The paper presents the results from a multi-year effort to develop and validate image processing methods forselecting the best physical models based on solar image observations. The approach consists of selecting the physical models based on their agreement with coronal holes extracted from the images. Ultimately, the goal is to use physical models to predict geomagnetic storms. We decompose the problem into three subproblems: (i) coronal hole segmentation based on physical constraints, (ii) matching clusters of coronal holes between different maps, and (iii) physical map classification. For segmenting coronal holes, we develop a multi-modal method that uses segmentation maps from three different methods to initialize a level-set method that evolves the initial coronal hole segmentation to the magnetic boundary. Then, we introducea new method based on Linear Programming for matchingclusters of coronal holes. The final matching is then performedusing Random Forests. The methods were carefully validatedusing consensus maps derived from multiple readers, manualclustering, manual map classification, and method validation for50 maps. The proposed multi-modal segmentation method significantly outperformed SegNet, U-net, Henney-Harvey, and FCNby providing accurate boundary detection. Overall, the methodgave a 95.5% map classification accuracy.
Jatla, Venkatesh↗
Application of a deep learning semantic segmentation model to helium bubbles and voids in nuclear materials
Imaging nanoscale radiation-induced defects using the transmission electron microscope (TEM) is a key factor in the successful implementation of materials for nuclear energy structural applications. Analyzing each defect in a TEM micrograph is currently a manual task. To identify the defects in a single image can take anywhere from 15 min to an hour and a project can require the analysis of anywhere from tens to ≥ 100 images. Here, we use artificial intelligence (AI) models to automate this task. For simplification, we evaluated images with only a single type of defect; helium bubbles. Additionally, we performed semantic segmentation of these helium bubble defects in electron microscopy images of irradiated FeCrAl alloys using a deep learning DefectSegNet model. This model, which was previously used to classify crystal defects, is inspired by the classic DenseNet and U-Net image segmentation models. It claims high spatial resolution, but has poor performance at object boundaries. Our paper improves the DefectSegNet model’s application by adding two new features. First, the DefectSegNet model is applied not only to perform calculation pixel-wise but also object (or feature) wise. Because object-wise metrics are directly relevant to our final goal of detecting bubbles, whereas pixel-wise classification is only an intermediate step, it is an important part of our study. Second, a distance map loss (DML) function has been added to increase its performance at object boundaries. It is crucial to accurately represent defects boundaries, especially bubbles, in order to track the bubble-induced swelling caused by irradiation. The boundary-focused DML function is also compared to other loss functions like Cross-entropy, Weighted Binary Cross Entropy (WBCE), Dice and Intersection over Union (IOU). Finally, by incorporating new features, we found a marked improvement on segmentation quality and better shape preservation at the boundaries and areas of the bubbles.
42 ENGINEERING↗
Instance Segmentation for Direct Measurements of Satellites in Metal Powders and Automated Microstructural Characterization from Image Data
In this work, we propose instance segmentation as a useful tool for image analysis in materials science. Instance segmentation is an advanced technique in computer vision which generates individual segmentation masks for every object of interest that is recognized in an image. Using an out-of-the-box implementation of Mask R-CNN, instance segmentation is applied to images of metal powder particles produced through gas atomization. Leveraging transfer learning allows for the analysis to be conducted with a very small training set of labeled images. As well as providing another method for measuring the particle size distribution, we demonstrate the first direct measurements of the satellite content in powder samples. After analyzing the results for the labeled data dataset, the trained model was used to generate measurements for a much larger set of unlabeled images. The resulting particle size measurements showed reasonable agreement with laser scattering measurements. The satellite measurements were self-consistent and showed good agreement with the expected trends for different samples. Finally, we present a small case study showing how instance segmentation can be used to measure spheroidite content in the UltraHigh Carbon Steel DataBase, demonstrating the flexibility of the technique.
36 MATERIALS SCIENCE↗
RU Net for Automatic Characterization of TRISO Fuel Cross Sections
TRistructural ISOtropic (TRISO) particle fuel is a type of nuclear fuel known for its high-temperature and high-burnup performance. Each sub-millimeter diameter TRISO particle consists of uranium-oxycarbide (UCO) or UO2 fuel kernel, coated with buffer, inner pyrolytic carbon (IPyC), silicon carbide (SiC), and outer pyrolytic carbon (OPyC) layers. The SiC layer acts as the main containment barrier for the TRISO particle to retain the fission products, while the IPyC and OPyC layers provide additional barriers to the release of fission products, especially fission gases. During irradiation, phenomena like kernel swelling, buffer densification, and IPyC fracture may impact fuel performance. Post-irradiation microscopy on entire compact cross sections or samples of individual particles deconsolidated from compacts is often used to identify these irradiation-induced changes in morphology. However, each fuel compact generally contains thousands of TRISO particles. To get statistical information on these phenomena, it is cumbersome work if done manually. For example, to get information about swelling/densification behaviors of different layers or kernels after irradiation, researchers previously manually measured the perimeter of each TRISO layer in hundreds of particles after four rounds of iterative grinding and polishing encompassing more than 2000 cross-section images for a total of four fuel compacts. To attempt to reduce the subjectivity inherent in that process and accelerate data analysis, we conducted a study on the automatic TRISO layer segmentation on cross-sectional microscopic images using Convolutional Neural Networks (CNNs). CNNs are a class of machine learning algorithms specifically designed for processing structured grid data that have gained popularity in recent years due to their remarkable performance in various computer vision tasks, including image classification, object detection, and image segmentation. In this research, we have generated the large irradiated TRISO layer dataset with more than 2000 cross-section TRISO microscopic images and the corresponding annotated images. Based on these annotated images, we have employed different CNNs for automatic segmentation of different TRISO layers. These include RU-Net (developed in this study), as well as three existing architectures: U-Net, Residual Network (ResNet), and Attention U-Net. The preliminary results show that the model based on RU-Net has the best performance in terms of intersection-over-union (IoU). Through the aid of these CNN models, we can expedite the analysis of TRISO particle cross-sections, significantly reducing the manual labor involved and improving the objectivity of the segmentation results.
Convolutional Neural Networks↗
The Precision Segmented Reflector program - On-orbit thermal behavior of the Submillimeter Imager and Line Survey telescope
The Precision Segmented Reflector program and the Submillimeter Imager and Line Survey mission are overviewed. The temperature requirements, thermal design and environment, as well as the on-orbit thermal performance of a 3.65-m-diameter passively-cooled submillimeter-wavelength telescope are discussed, and focus is placed on the geometric and thermal mathematical models employed in the study. Attention is given to an inflatable space-rigidized sunshade protecting the primary mirror thermally and the highly elliptical orbit chosen to reduce heat input from the earth. Primary-reflector temperature predictions are shown to indicate that a large spatial gradient can exist and the magnitude of the gradient is a strong function of the telescope pitch angle. It is suggested that the cryostat outer-shell temperature may be further lowered with the introduction of a rectangular shield between a solar array and the cryostat.
Tsuyuki, G.↗
High-resolution imaging gamma-ray spectroscopy with externally segmented germanium detectors
Externally segmented germanium detectors promise a breakthrough in gamma-ray imaging capabilities while retaining the superb energy resolution of germanium spectrometers. An angular resolution of 0.2 deg becomes practical by combining position-sensitive germanium detectors having a segment thickness of a few millimeters with a one-dimensional coded aperture located about a meter from the detectors. Correspondingly higher angular resolutions are possible with larger separations between the detectors and the coded aperture. Two-dimensional images can be obtained by rotating the instrument. Although the basic concept is similar to optical or X-ray coded-aperture imaging techniques, several complicating effects arise because of the penetrating nature of gamma rays. The complications include partial transmission through the coded aperture elements, Compton scattering in the germanium detectors, and high background count rates. Extensive electron-photon Monte Carlo modeling of a realistic detector/coded-aperture/collimator system has been performed. Results show that these complicating effects can be characterized and accounted for with no significant loss in instrument sensitivity.
Callas, J. L.↗
Fine-tuning TrailMap: The utility of transfer learning to improve the performance of deep learning in axon segmentation of light-sheet microscopy images
Light-sheet microscopy has made possible the 3D imaging of both fixed and live biological tissue, with samples as large as the entire mouse brain. However, segmentation and quantification of that data remains a time-consuming manual undertaking. Machine learning methods promise the possibility of automating this process. This study seeks to advance the performance of prior models through optimizing transfer learning. We fine-tuned the existing TrailMap model using expert-labeled data from noradrenergic axonal structures in the mouse brain. By changing the cross-entropy weights and using augmentation, we demonstrate a generally improved adjusted F1-score over using the originally trained TrailMap model within our test datasets.
97 MATHEMATICS AND COMPUTING↗