Satellite Image Algorithms
Presented by Conner Hatton at SIAM UQ Conference in Trieste, Italy, on February 27th, 2024. Conference details available online at https://www.siam.org/conferences/cm/conference/uq24
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Presented by Conner Hatton at SIAM UQ Conference in Trieste, Italy, on February 27th, 2024. Conference details available online at https://www.siam.org/conferences/cm/conference/uq24
Advancements were made in automating microscopy-based material characterization, particularly in studying irradiation effects on additively manufactured (AM) materials using machine learning (ML) and computer vision (CV). These automation efforts address the challenges of analyzing complex microstructures, accelerating the detection of irradiation-induced defects. Two CV models were developed at Argonne National Laboratory (ANL) to enhance transmission electron microscopy (TEM) analysis of irradiated AM 316 stainless steel. The first model focused on the detection of irradiation-induced dislocation loops, which contribute to material hardening and embrittlement. These loops, categorized as faulted or perfect, were automatically detected and classified using a Mask R-CNN model trained on TEM images from both in-situ and ex-situ ion irradiation experiments. The model achieved high accuracy, with precision, recall, and F1 scores of 0.839, 0.734, and 0.776, respectively, demonstrating its effectiveness in analyzing dislocation loops in irradiated AM materials. The second CV model was developed to analyze the size and wall thickness of dislocation cells in laser powder bed fusion (LPBF) 316 stainless steel. Using a U-Net++ architecture with EfficientNet as the encoder, the model was trained on TEM images to segment and measure cell size and wall thickness.
This paper presents a high-fidelity multiscale framework to simulate the mechanical behavior of ceramic matrix composites (CMCs), accounting for the complex micromorphology captured using detailed material characterization. First, high-resolution micrographs are obtained for the specific carbon fiber silicon-carbide-nitride matrix (C/SiNC) CMC to characterize the variability of the architectural features and manufacturing-induced defects at the microscale. An image processing algorithm is then used to precisely estimate the size and distribution of all subscale features and defects from the micrographs. The information is then used to generate a three-dimensional stochastic representative volume element (SRVE) to reconstruct microscale constituents accounting for the variability. As a result, the generated SRVEs are simulated using the high-fidelity generalized method of cells (HFGMC) micromechanics theory to investigate the effects of defects on the elastic properties of C/SiNC CMCs.
The occurrence of vaso-occlusive crisis greatly depends on the competition between the sickling delay time and the transit time of individual sickle cells, i.e., red blood cells from sickle cell disease (SCD) patients, while they are traversing the circulatory system. Many drugs for treating SCD work by inhibiting the polymerization of sickle hemoglobin (HbS), effectively delaying the sickling process in sickle cells (SS RBCs). Most previous studies on screening anti-sickling drugs, such as voxelotor, rely on in vitro testing of sickling characteristics, often conducted under prolonged deoxygenation for up to 1 hour. However, since the microcirculation of RBCs typically takes less than 1 minute, the results of these studies may be less accurate and less relevant for in vitro-in vivo correlation. In our current study, we introduce a computer vision-enhanced microfluidic framework designed to automatically capture the transient sickling kinetics of SS RBCs within a 1-min timeframe. Our study has successfully detected differences in the transient sickling kinetics between vehicle control and voxelotor-treated SS RBCs. This approach has the potential for broader applications in screening anti-sickling therapies.
We develop a detection and recognition model to fully automatize the identification of polygonal faults present in many of the Magellan SAR images. Our segmentation model is based on a modified watershed algorithm. Additional information is contained in the original extended abstract.
This paper presents a new technique for implementing a low-power CMOS imager with simultaneous on-chip computation of the difference and summ of two successive frames.
Alaska’s boreal forest is facing unprecedented challenges under rapid climate warming (increasingly severe fires, droughts, pest/disease outbreaks) that may destabilize its function as a global carbon sink. Forests near Fairbanks may be especially vulnerable, impacting air quality and ecosystem services. We combined GT (ground truthing) with Airborne Visible InfraRed Imaging Spectrometer (AVIRIS-NG) images collected by the NASA Arctic-Boreal Vulnerability Experiment (ABoVE) program (2017-2019) to assess landcover change at five recently burned sites (2001-2019) of different fire severities and moisture regimes within 30 miles of Fairbanks. GT included tree seedling counts, understory % cover and >50% leaf canopy color assessment. 36 circular plots (1/30 ha radius) including 6 moderate to severely burned plots were selected across sites. 31 additional sites including 12 burned sites were geotagged in photos. AVIRIS images were processed from 29 spectral bands selected to identify changes in chlorophyll and water content. Images were segmented into natural boundaries (polygons) using ENVI 5.5 software. A spectral library of 8 AVIRIS bands with high between-class/low within-class variation was used in two random forest models to predict vegetation classes (model 1: 12 classes, model 2: 14 classes) in each AVIRIS scene, using 20% of the data as training data. Model 2 classified 20% more polygons overall, but only 42% of GT/geotagged polygons were correctly classified by both models. More forest sites were correctly classified (63%) than open vegetation (32%) or post-fire sites (46%). 50% of aspen forest and post-fire polygons were misclassified as shrubland. GT revealed that post-fire plots supported 134,000 (± 48,000) tree seedlings and saplings ha-1 (0.2 - 4 m height, 64% deciduous) versus 2500 (± 2100) shrubs ha-1 (1-6 m height). > 50% canopy browning was observed in conifer forest (8 plots) with no signs of insect infestation. Canopy herbivory > 50% (leaf miner, leaf beetle) and moose herbivory of tree bark was seen across aspen sites. Our study suggests: 1) low canopy vegetation presents challenges for improved landcover classification, and 2) aspen forest should be differentiated in vegetation maps which would aid in tracking herbivory.
Deep learning has quickly become a necessity for selfdriving vehicles on Earth. In contrast, the self-driving vehicles on Mars, including NASA’s latest rover, Perseverance, which is planned to land on Mars in February 2021, are still driven by classical machine vision systems. Deep learning capabilities, such as semantic segmentation and object recognition, would substantially benefit the safety and productivity of ongoing and future missions to the red planet. To this end, we created the first large-scale dataset, AI4Mars, for training and validating terrain classification models for Mars, consisting of ~326K semantic segmentation full image labels on 35K images from Curiosity, Opportunity, and Spirit rovers, collected through crowdsourcing. Each image was labeled by ~10 people to ensure greater quality and agreement of the crowdsourced labels. It also includes ~1.5K validation labels annotated by the rover planners and scientists from NASA’s MSL (Mars Science Laboratory) mission, which operates the Curiosity rover, and MER (Mars Exploration Rovers) mission, which operated the Spirit and Opportunity rovers. We trained a DeepLabv3 model on the AI4Mars training dataset and achieved over 96% overall classification accuracy on the test set. The dataset is made publicly available.1
Woven, ablative thermal protection system (TPS) materials provide a robust option for aggressive (re)entries and thus have been baselined for the upcoming Mars Sample Return (MSR) mission's Earth Entry System (EES). The reliability requirements for MSR-EES necessitate understanding of material property variability, which could be significant given the complex structure and anisotropic nature of properties in TPS weaves, as well as the response to potential impact with micrometeoroids or orbital debris during the EES re-entry. The TPS Certification by Analysis effort within the Entry Systems Modeling project seeks to provide computational models and analyses that support the certification against such material-based risks. For the present talk, focus will be given to the characterization of baseline woven TPS material properties and mechanical failure limits, which entails (1) use of computational techniques (e.g., machine learning) to interpret computed tomography images of the weave to generate representative structural models and (2) application of multiscale material modeling approaches to characterize thermomechanical and failure properties.
We previously presented Soil Property and Object Classification (SPOC), a machine learning-based terrain classifier for Mars rovers, for automatically segmenting rover images by its surface type such as sand and bedrock. This paper presents a number of practical improvements to pave the way for potential future onboard deployment. First, we achieved 97.0% overall pixel accuracy, evaluated against the classification generated by human experts on images from Mars Science Laboratory (MSL) missions. The substantial increase in accuracy was primarily enabled by the sheer volume of data used for training; we created a new large-scale dataset of Martian terrain labels, namely AI4Mars, which contains more than 400k labels contributed by citizen scientists for 50k images taken by the Mars Exploration Rovers (MER) and Mars Science Laboratory (MSL) rover. Second, we demonstrated that SPOC can quickly adapt to a new mission landed on a previously unseen site. Specifically, we pretrained a model with MER and MSL data from the AI4Mars dataset and then adapted to the Mars 2020 Rover (M2020) by feeding a small volume of data between Sol 0 and 157; the adapted model was tested on Sol 200-203 and resulted in 84.2% overall pixel accuracy and 93.4% reliability (recall) for detecting sand, the most concerning class for rover’s traversability. Third, we found that pretraining can substantially mitigate the decline of accuracy over time. We showed that the performance of a SPOC model pretrained with the ImageNet dataset and then trained by MSL images only up to Sol 390 remains comparable to a model trained by images up to Sol 1689 on the test data after Sol 1689. Fourth, we reimplemented SPOC with a light-weight convolutional neural network (CNN), MobileNetV2, which typically runs within tens of milliseconds (ms) on mobile processors such as Qualcomm’s Snapdragon. Finally, we released the AI4Mars dataset to the public to encourage open innovation.
Explore the source record for details and available documents.
Plan to submit this to the Journal of Statistical Analysis and Data Mining for their special issue on CoDA (Conference on Data Analysis, a meeting of analysts and scientists from the national labs)
The paper presents the results from a multi-year effort to develop and validate image processing methods forselecting the best physical models based on solar image observations. The approach consists of selecting the physical models based on their agreement with coronal holes extracted from the images. Ultimately, the goal is to use physical models to predict geomagnetic storms. We decompose the problem into three subproblems: (i) coronal hole segmentation based on physical constraints, (ii) matching clusters of coronal holes between different maps, and (iii) physical map classification. For segmenting coronal holes, we develop a multi-modal method that uses segmentation maps from three different methods to initialize a level-set method that evolves the initial coronal hole segmentation to the magnetic boundary. Then, we introducea new method based on Linear Programming for matchingclusters of coronal holes. The final matching is then performedusing Random Forests. The methods were carefully validatedusing consensus maps derived from multiple readers, manualclustering, manual map classification, and method validation for50 maps. The proposed multi-modal segmentation method significantly outperformed SegNet, U-net, Henney-Harvey, and FCNby providing accurate boundary detection. Overall, the methodgave a 95.5% map classification accuracy.
Purpose: Dictionary learning based method has attracted more and more attentions in low-dose CT due to the superior performance on suppressing noise and preserving structural details. Considering the structures and noise vary from region to region in one imaging object, we propose a region-specific dictionary learning method to improve the low-dose CT reconstruction. Methods: A set of normal-dose images was used for dictionary learning. Segmentations were performed on these images, so that the training patch sets corresponding to different regions can be extracted out. After that, region-specific dictionaries were learned from these training sets. For the low-dose CT reconstruction, a conventional reconstruction, such as filtered back-projection (FBP), was performed firstly, and then segmentation was followed to segment the image into different regions. Sparsity constraints of each region based on its dictionary were used as regularization terms. The regularization parameters were selected adaptively according to different regions. A low-dose human thorax dataset was used to evaluate the proposed method. The single dictionary based method was performed for comparison. Results: Since the lung region is very different from the other part of thorax, two dictionaries corresponding to lung region and the rest part of thorax respectively were learned to better express the structural details and avoid artifacts. With only one dictionary some artifact appeared in the body region caused by the spot atoms corresponding to the structures in the lung region. And also some structure in the lung regions cannot be recovered well by only one dictionary. The quantitative indices of the result by the proposed method were also improved a little compared to the single dictionary based method. Conclusion: Region-specific dictionary can make the dictionary more adaptive to different region characteristics, which is much desirable for enhancing the performance of dictionary learning based method.
Crystalline materials exhibit long-range ordered lattice unit, within which resides nonperiodic structural features called defects. These crystallographic defects play a vital role in determining the physical and mechanical properties of a wide range of material systems. While computer vision has demonstrated success in recognizing feature patterns in images with well defined contrast, automated identification of nanometer scale crystallographic defects in electron micrographs governed by complex contrast mechanism is still a challenging task. Here, building upon an advanced defect imaging mode that offers high feature clarity, we introduce DefectSegNet - a new convolutional neural network (CNN) architecture that performs semantic segmentation of three common crystallographic defects in structural alloys: dislocation lines, precipitates and voids. Results from supervised training of a small set of high-quality defect images of steels show high pixel-wise accuracy across all three types of defects: 91.60 ± 1.77% on dislocations, 93.39 ± 1.00% on precipitates, and 98.85 ± 0.56% on voids. We discuss the sources of uncertainties in CNN prediction, and the training data in terms of feature density, representation and homogeneity and their effects on deep learning performance. Further defect quantification using DefectSegNet prediction outperforms human expert average, presenting a promising new workflow for fast and statistically meaningful quantification of materials defects.
Imaging nanoscale radiation-induced defects using the transmission electron microscope (TEM) is a key factor in the successful implementation of materials for nuclear energy structural applications. Analyzing each defect in a TEM micrograph is currently a manual task. To identify the defects in a single image can take anywhere from 15 min to an hour and a project can require the analysis of anywhere from tens to ≥ 100 images. Here, we use artificial intelligence (AI) models to automate this task. For simplification, we evaluated images with only a single type of defect; helium bubbles. Additionally, we performed semantic segmentation of these helium bubble defects in electron microscopy images of irradiated FeCrAl alloys using a deep learning DefectSegNet model. This model, which was previously used to classify crystal defects, is inspired by the classic DenseNet and U-Net image segmentation models. It claims high spatial resolution, but has poor performance at object boundaries. Our paper improves the DefectSegNet model’s application by adding two new features. First, the DefectSegNet model is applied not only to perform calculation pixel-wise but also object (or feature) wise. Because object-wise metrics are directly relevant to our final goal of detecting bubbles, whereas pixel-wise classification is only an intermediate step, it is an important part of our study. Second, a distance map loss (DML) function has been added to increase its performance at object boundaries. It is crucial to accurately represent defects boundaries, especially bubbles, in order to track the bubble-induced swelling caused by irradiation. The boundary-focused DML function is also compared to other loss functions like Cross-entropy, Weighted Binary Cross Entropy (WBCE), Dice and Intersection over Union (IOU). Finally, by incorporating new features, we found a marked improvement on segmentation quality and better shape preservation at the boundaries and areas of the bubbles.
We present advances with a 32 element scalable, segmented dual mode imager. Scaling up the number of cells results in a 1.4 increase in efficiency over a system we deployed last year. Variable plane separation has been incorporated which further improves the efficiency of the detector. By using 20 cm diameter cells we demonstrate that we could increase sensitivity by a factor of 6. We further demonstrate gamma ray imaging in from Compton scattering. This feature allows for powerful dual mode imaging. Selected results are presented that demonstrate these new capabilities.
We present advances with a 32 element scalable, segmented dual mode imager. Scaling up the number of cells results in a 1.4 increase in efficiency over a system we deployed last year. Variable plane separation has been incorporated which further improves the efficiency of the detector. By using 20 cm diameter cells we demonstrate that we could increase sensitivity by a factor of 6. We further demonstrate gamma ray imaging in from Compton scattering. This feature allows for powerful dual mode imaging. Selected results are presented that demonstrate these new capabilities.