Topology Data Analysis Guided Prompt Optimization of Segment Anything Model for Zero-Shot Segmentation of Biological Images
Explore the source record for details and available documents.
Biological and medical sciences, Computer science↗
SEARCH · Engineering Papers
Results for “Image segmentation”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Beyond Binary: Automated PLC Memory Forensics through RGB Image Analysis and Deep Learning
The introduction of Industry 4.0 and the evolution of industrial control systems (ICS) to adopt Internet-based technologies enhanced productivity, but have inadvertently increased their vulnerability to cyber-based malicious attacks. When an ICS system is compromised, security analysts need to identify the root cause quickly to start the recovery process and develop mitigation strategies to safeguard against future instances. Memory forensics is critical in the analysis process to ascertain what occurred. To date, approaches to analyze the persistent memory in ICS devices are limited, and almost nonexistent for volatile memory. This paper proposes an automated methodology, COMA, for PLC memory dump analysis using computer vision and deep learning techniques. Specifically, COMA converts the sequences of bytes in a PLC memory dump to RGB pixels and creates a deep learning model that learns the underlying patterns and features of pre-labeled forensic artifacts in images and segments them into distinct regions. COMA then uses the trained model to automatically segment new memory images and extract forensic artifacts. We evaluate COMA on a Schneider Electric Modicon M221 PLC involving two cyber-based attack scenarios: (i) code injection and (ii) code modification. The empirical results show that COMA can successfully detect attack artifacts in memory dumps in both scenarios.
Asmar Awad, Rima↗
Deep-learning-based workflow for boundary and small target segmentation in digital rock images using UNet++ and IK-EBM
We report three-dimensional (3D) X-ray micro-computed tomography (μCT) has been widely used in petroleum engineering because it can provide detailed pore structural information for a reservoir rock, which can be imported into a pore-scale numerical model to simulate the transport and distribution of multiple fluids in the pore space. The partial volume blurring (PVB) problem is a major challenge in segmenting raw μCT images of rock samples, which impacts boundaries and small targets near the resolution limit. We developed a deep-learning (DL)-based workflow for accurate and fast partial volume segmentation. The DL model's performance depends primarily on the training data quality and model architecture. This study employed the entropy-based-masking indicator kriging (IK-EBM) to segment 3D Berea sandstone images as training datasets. The comparison between IK-EBM and manual segmentation using a 3D synthetic sphere pack, which had a known ground truth, showed that IK-EBM had higher accuracy on partial volume segmentation. We then trained and tested the UNet++ model, a state-of-the-art supervised encoder-decoder model, for binary (i.e., void and solid) and four-class segmentation. We compared the UNet++ with the commonly used U-Net and wide U-Net models and showed that the UNet++ had the best performance in terms of pixel-wise and physics-based evaluation metrics. Specifically, boundary-scaled accuracy demonstrated that the UNet++ architecture outperformed the regular U-Net architecture in the segmentation of pixels near boundaries and small targets, which were subjected to the PVB effect. Feature map visualization illustrated that the UNet++ bridged the semantic gaps between the feature maps extracted at different depths of the network, thereby enabling faster convergence and more accurate extraction of fine-scale features. The developed workflow significantly enhances the performance of supervised encoder-decoder models in partial volume segmentation, which has extensive applications in fundamental studies of subsurface energy, water, and environmental systems.
02 PETROLEUM↗
TEMImageNet training library and AtomSegNet deep-learning models for high-precision atom segmentation, localization, denoising, and deblurring of atomic-resolution images
Abstract Atom segmentation and localization, noise reduction and deblurring of atomic-resolution scanning transmission electron microscopy (STEM) images with high precision and robustness is a challenging task. Although several conventional algorithms, such has thresholding, edge detection and clustering, can achieve reasonable performance in some predefined sceneries, they tend to fail when interferences from the background are strong and unpredictable. Particularly, for atomic-resolution STEM images, so far there is no well-established algorithm that is robust enough to segment or detect all atomic columns when there is large thickness variation in a recorded image. Herein, we report the development of a training library and a deep learning method that can perform robust and precise atom segmentation, localization, denoising, and super-resolution processing of experimental images. Despite using simulated images as training datasets, the deep-learning model can self-adapt to experimental STEM images and shows outstanding performance in atom detection and localization in challenging contrast conditions and the precision consistently outperforms the state-of-the-art two-dimensional Gaussian fit method. Taking a step further, we have deployed our deep-learning models to a desktop app with a graphical user interface and the app is free and open-source. We have also built a TEM ImageNet project website for easy browsing and downloading of the training data.
25 ENERGY STORAGE↗
High-Resolution Gamma-Ray Imaging Measurements Using Externally Segmented Germanium Detectors
Fully two-dimensional gamma-ray imaging with simultaneous high-resolution spectroscopy has been demonstrated using an externally segmented germanium sensor. The system employs a single high-purity coaxial detector with its outer electrode segmented into 5 distinct charge collection regions and a lead coded aperture with a uniformly redundant array (URA) pattern. A series of one-dimensional responses was collected around 511 keV while the system was rotated in steps through 180 degrees. A non-negative, linear least-squares algorithm was then employed to reconstruct a 2-dimensional image. Corrections for multiple scattering in the detector, and the finite distance of source and detector are made in the reconstruction process.
spectroscopy gamma ray germanium sensor nucleosynt↗
Universal image representation based on a multimodal graph
A system for classifying a target image with segments having attributes is provided. The system generates a graph for the target image that includes vertices representing segments of the image and edges representing relationships between the connected vertices. For each vertex, the system generates a subgraph that includes the vertex as a home vertex and neighboring vertices representing segments of the target image within a neighborhood of the segment represented by the home vertex. The system applies an autoencoder to each subgraph to generate latent variables to represent the subgraph. The system applies a machine learning algorithm to a feature vector comprising a universal image representation of the target image that is derived from the generated latent variables of the subgraphs to generate a classification for the target image.
Bremer, Peer-Timo↗
Universal image representation based on a multimodal graph
A system for classifying a target image with segments having attributes is provided. The system generates a graph for the target image that includes vertices representing segments of the image and edges representing relationships between the connected vertices. For each vertex, the system generates a subgraph that includes the vertex as a home vertex and neighboring vertices representing segments of the target image within a neighborhood of the segment represented by the home vertex. The system applies an autoencoder to each subgraph to generate latent variables to represent the subgraph. The system applies a machine learning algorithm to a feature vector comprising a universal image representation of the target image that is derived from the generated latent variables of the subgraphs to generate a classification for the target image.
Bremer, Peer-Timo↗
Rapid macrovoid characterization in membranes prepared via nonsolvent-induced phase separation: A comparison between 2D and 3D techniques
Optimizing the performance of asymmetric membranes prepared via nonsolvent-induced phase separation (NIPS) requires a quantitative understanding of how processing variables influence membrane morphology. Presently, the most useful structural quantification techniques require 3D visualization of the membrane structure and are best suited for studies seeking detailed information on small datasets. This study proposes and validates a rapid and accurate technique for quantifying macroporosity (i.e., D m ), a simple descriptor of sublayer macrovoid content in asymmetric membranes D m . values measured from segmented cross-sectional imaging performed via X-ray computed tomography (XCT) and scanning electron microscopy (SEM) are presented and compared for three asymmetric membranes prepared from commercial polymers. Importantly, analyses of 3D XCT membrane reconstructions reveal that D m is described by a single, centralized mean, which demonstrates that macrovoid content is spatially homogenous. Thus, D m can be approximated from limited sampling of the 2D cross-sectional membrane structure via SEM. A proposed 2D SEM sampling method provides D m estimates within ±6% of corresponding 3D XCT values with 30 independent measurements for the three membranes. Further sensitivity is achieved using complementary descriptors such as macrovoid count density (i.e., C m ). This technique is thus a useful tool for characterizing macroporosity from a broad selection of membrane samples.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
A proximity‐based image‐processing algorithm for colloid assignment in segmented multiphase flow datasets
Summary Colloidal transport and deposition are of both environmental and engineering importance. Easier access to x‐ray microtomography (XMT) coupled with improved imaging resolution has made XMT a unique and viable tool for visualizing and quantifying these processes. Currently, there is scant information in the literature addressing colloid segmentation and analysis in saturated and unsaturated porous media, in particular related to spatial partitioning of colloids. To support this need, an approach to assign segmented colloidal particles and aggregates to different partitioning classes based on their proximity to different phases is presented here. The method uses different markers for each attachment site (e.g. wetting‐nonwetting phase interfaces). An example XMT dataset from a drainage experiment is used to demonstrate the efficacy of the image processing algorithms. Flow conditions, and fluid and colloid properties, can thus be compared to the behaviour of colloids within the porous medium. This algorithm can help elucidate colloidal deposition mechanisms and the importance of different attachment sites, explore the importance of fluid properties, as well as the arrangement and shape of the colloids.
BRUECK, C. L.↗
Multi-Task Learning of Scanning Electron Microscopy and Synthetic Thermal Tomography Images for Detection of Defects in Additively Manufactured Metals
One of the key challenges in laser powder bed fusion (LPBF) additive manufacturing of metals is the appearance of microscopic pores in 3D-printed metallic structures. Quality control in LPBF can be accomplished with non-destructive imaging of the actual 3D-printed structures. Thermal tomography (TT) is a promising non-contact, non-destructive imaging method, which allows for the visualization of subsurface defects in arbitrary-sized metallic structures. However, because imaging is based on heat diffusion, TT images suffer from blurring, which increases with depth. We have been investigating the enhancement of TT imaging capability using machine learning. In this work, we introduce a novel multi-task learning (MTL) approach, which simultaneously performs the classification of synthetic TT images, and segmentation of experimental scanning electron microscopy (SEM) images. Synthetic TT images are obtained from computer simulations of metallic structures with subsurface elliptical-shaped defects, while experimental SEM images are obtained from imaging of LPBF-printed stainless-steel coupons. MTL network is implemented as a shared U-net encoder between the classification and the segmentation tasks. Results of this study show that the MTL network performs better in both the classification of synthetic TT images and the segmentation of SEM images tasks, as compared to the conventional approach when the individual tasks are performed independently of each other.
36 MATERIALS SCIENCE↗
Machine learning-assisted upscaling analysis of reservoir rock core properties based on micro-computed tomography imagery
Optimum solutions for geologic modeling and reservoir simulation in industries such as oil and gas recovery and carbon capture and storage require accurate characterization of reservoir properties, which are often heterogeneous. In this study, high-quality micro-computed tomography (CT) images (1.475-μm/pixel resolution) of a sandstone core acquired from the Bell Creek oil field, USA, were used to provide nondestructive analysis of pore- and core-scale heterogeneity across measurement scales of 94–566 μm. In addition to characterizing the as-received sample, the core sample was flooded with brine to evaluate the capacity of the core sample to receive injected fluids. The micro-CT images were systematically segmented into pore spaces and grains via machine learning (ML) steps including image preprocessing, label creation using a traditional ML method based on limited manual image annotation, and finally U-Net segmentation. The segmented image stacks were reconstructed into digital cubes of various scales of voxel lengths. The 3D porosity values were calculated for all the digital cubes, and the fractal dimensions of the cubes were estimated using a box-counting method. The results showed that smaller cubes had greater heterogeneity and that the porosity values could be accurately estimated by fractal dimension and voxel lengths using ML models. For the core sample with brine flooding, the ratio of pores filled by brine to the total pore space was related to the porosity and could also be accurately estimated by porosity, fractal dimension, and voxel lengths using ML models. In conclusion, the results of this study demonstrate that the concept of fractal dimension can be a useful vector to perform upscaling analysis of sandstone rock heterogeneity from the pore to core scale and that fractal dimensions can be used to estimate porosity values and pore space-filling capacity across those scales.
58 GEOSCIENCES↗
Compressing TV-image data
Compressing technique calculates activity estimator for each segment of image line. Estimator is used in conjunction with allowable bits per line, N, to determine number of bits necessary to code each segment and which segments can tolerate truncation. Preprocessed line data are then passed to adaptive variable-length coder, which selects optimum transmission code. Method increases capacity of broadcast and cable television transmissions and helps reduce size of storage medium for video and digital audio recordings.
Hilbert, E. E.↗
A Flexible Method for Producing F.E.M. Analysis of Bone Using Open-Source Software
This project, performed in support of the NASA GRC Space Academy summer program, sought to develop an open-source workflow methodology that segmented medical image data, created a 3D model from the segmented data, and prepared the model for finite-element analysis. In an initial step, a technological survey evaluated the performance of various existing open-source software that claim to perform these tasks. However, the survey concluded that no single software exhibited the wide array of functionality required for the potential NASA application in the area of bone, muscle and bio fluidic studies. As a result, development of a series of Python scripts provided the bridging mechanism to address the shortcomings of the available open source tools. The implementation of the VTK library provided the most quick and effective means of segmenting regions of interest from the medical images; it allowed for the export of a 3D model by using the marching cubes algorithm to build a surface mesh. To facilitate the development of the model domain from this extracted information required a surface mesh to be processed in the open-source software packages Blender and Gmsh. The Preview program of the FEBio suite proved to be sufficient for volume filling the model with an unstructured mesh and preparing boundaries specifications for finite element analysis. To fully allow FEM modeling, an in house developed Python script allowed assignment of material properties on an element by element basis by performing a weighted interpolation of voxel intensity of the parent medical image correlated to published information of image intensity to material properties, such as ash density. A graphical user interface combined the Python scripts and other software into a user friendly interface. The work using Python scripts provides a potential alternative to expensive commercial software and inadequate, limited open-source freeware programs for the creation of 3D computational models. More work will be needed to validate this approach in creating finite-element models.
gravitational physiology↗
Power spectral ensity of markov texture fields
Texture is an important image characteristic. A variety of spatial domain techniques were proposed for extracting and utilizing textural features for segmenting and classifying images. for the most part, these spatial domain techniques are ad hos in nature. A markov random field model for image texture is discussed. A frequency domain description of image texture is derived in terms of the power spectral density. This model is used for designing optimum frequency domain filters for enhancing, restoring and segmenting images based on their textural properties.
Shanmugan, K. S.↗
Statistical characterization of experimental magnetized liner inertial fusion stagnation images using deep-learning-based fuel–background segmentation
Significant variety is observed in spherical crystal x-ray imager (SCXI) data for the stagnated fuel–liner system created in Magnetized Liner Inertial Fusion (MagLIF) experiments conducted at the Sandia National Laboratories Z-facility. As a result, image analysis tasks involving, e.g., region-of-interest selection (i.e. segmentation), background subtraction and image registration have generally required tedious manual treatment leading to increased risk of irreproducibility, lack of uncertainty quantification and smaller-scale studies using only a fraction of available data. We present a convolutional neural network (CNN)-based pipeline to automate much of the image processing workflow. This tool enabled batch preprocessing of an ensemble of N scans = 139 SCXI images across N exp = 67 different experiments for subsequent study. The pipeline begins by segmenting images into the stagnated fuel and background using a CNN trained on synthetic images generated from a geometric model of a physical three-dimensional plasma. The resulting segmentation allows for a rules-based registration. Our approach flexibly handles rarely occurring artifacts through minimal user input and avoids the need for extensive hand labelling and augmentation of our experimental dataset that would be needed to train an end-to-end pipeline. Here we also fit background pixels using low-degree polynomials, and perform a statistical assessment of the background and noise properties over the entire image database. Our results provide a guide for choices made in statistical inference models using stagnation image data and can be applied in the generation of synthetic datasets with realistic choices of noise statistics and background models used for machine learning tasks in MagLIF data analysis. We anticipate that the method may be readily extended to automate other MagLIF stagnation imaging applications.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Predicting Mechanical Properties from Microstructure Images in Fiber-Reinforced Polymers Using Convolutional Neural Networks
Evaluating the mechanical response of fiber-reinforced composites can be extremely time-consuming and expensive. Machine learning (ML) techniques offer a means for faster predictions via models trained on existing input–output pairs and have exhibited success in composite research. This paper explores a fully convolutional neural network modified from StressNet, which was originally used for linear elastic materials, and extended here for a non-linear finite element (FE) simulation to predict the stress field in 2D slices of segmented tomography images of a fiber-reinforced polymer specimen. The network was trained and evaluated on data generated from the FE simulations of the exact microstructure. The testing results show that the trained network accurately captures the characteristics of the stress distribution, especially on fibers, solely from the segmented microstructure images. The trained model can make predictions within seconds in a single forward pass on an ordinary laptop, given the input microstructure, compared to 92.5 h to run the full FE simulation on a high-performance computing cluster. These results show promise in using ML techniques to conduct fast structural analysis for fiber-reinforced composites and suggest a corollary that the trained model can be used to identify the location of potential damage sites in fiber-reinforced polymers.
Sun, Yixuan (ORCID:0000000311093380)↗
Identification of uncommon objects in containers
A system for identifying in an image an object that is commonly found in a collection of images and for identifying a portion of an image that represents an object based on a consensus analysis of segmentations of the image. The system collects images of containers that contain objects for generating a collection of common objects within the containers. To process the images, the system generates a segmentation of each image. The image analysis system may also generate multiple segmentations for each image by introducing variations in the selection of voxels to be merged into a segment. The system then generates clusters of the segments based on similarity among the segments. Each cluster represents a common object found in the containers. Once the clustering is complete, the system may be used to identify common objects in images of new containers based on similarity between segments of images and the clusters.
97 MATHEMATICS AND COMPUTING↗
Deep learning for lipid droplet recognition in quantitative phase images
This library of Python code is used for performing semantic segmentation of images using 6 different machine learning methods. Five of the methods are implemented entirely within the scikit-learn framework. The Convolutional Neural Network (CNN) method requires Keras with a TensorFlow backend and generally uses a different set of scripts in order to perform the complete training and evaluation.
Sheneman, Lucas↗