Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “microscopy automation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Rapid detection of rare events from in situ X-ray diffraction data using machine learning

High-energy X-ray diffraction methods can non-destructively map the 3D microstructure and associated attributes of metallic polycrystalline engineering materials in their bulk form. These methods are often combined with external stimuli such as thermo-mechanical loading to take snapshots of the evolving microstructure and attributes over time. However, the extreme data volumes and the high costs of traditional data acquisition and reduction approaches pose a barrier to quickly extracting actionable insights and improving the temporal resolution of these snapshots. This article presents a fully automated technique capable of rapidly detecting the onset of plasticity in high-energy X-ray microscopy data. The technique is computationally faster by at least 50 times than the traditional approaches and works for data sets that are up to nine times sparser than a full data set. This new technique leverages self-supervised image representation learning and clustering to transform massive data sets into compact, semantic-rich representations of visually salient characteristics ( e.g. peak shapes). These characteristics can rapidly indicate anomalous events, such as changes in diffraction peak shapes. It is anticipated that this technique will provide just-in-time actionable information to drive smarter experiments that effectively deploy multi-modal X-ray diffraction methods spanning many decades of length scales.

Zheng, Weijian↗

Analysis of biofilm assembly by large area automated AFM

Biofilms are complex microbial communities critical in medical, industrial, and environmental contexts. Understanding their assembly, structure, genetic regulation, interspecies interactions, and environmental responses is key to developing effective control and mitigation strategies. While atomic force microscopy (AFM) offers critically important high-resolution insights on structural and functional properties at the cellular and even sub-cellular level, its limited scan range and labor-intensive nature restricts the ability to link these smaller scale features to the functional macroscale organization of the films. We begin to address this limitation by introducing an automated large area AFM approach capable of capturing high-resolution images over millimeter-scale areas, aided by machine learning for seamless image stitching, cell detection, and classification. Large area AFM is shown to provide a very detailed view of spatial heterogeneity and cellular morphology during the early stages of biofilm formation which were previously obscured. Using this approach, we examined the organization of Pantoea sp. YR343 on PFOTS-treated glass surfaces. Our findings reveal a preferred cellular orientation among surface-attached cells, forming a distinctive honeycomb pattern. Detailed mapping of flagella interactions suggests that flagellar coordination plays a role in biofilm assembly beyond initial attachment. Additionally, we use large-area AFM to characterize surface modifications on silicon substrates, observing a significant reduction in bacterial density. This highlights the potential of this method for studying surface modifications to better understand and control bacterial adhesion and biofilm formation.

59 BASIC BIOLOGICAL SCIENCES↗

MultiTaskDeltaNet: change detection-based image segmentation for operando ETEM with application to carbon gasification kinetics

Transforming in situ transmission electron microscopy (TEM) imaging into a tool for spatially-resolved operando characterization of solid-state reactions requires automated, high-precision semantic segmentation of dynamically evolving features. However, traditional deep learning methods for semantic segmentation often face limitations due to the scarcity of labeled data, visually ambiguous features of interest, and scenarios involving small objects. To tackle these challenges, we introduce MultiTaskDeltaNet (MTDN), a novel deep learning architecture that creatively reconceptualizes the segmentation task as a change detection problem. By implementing a unique Siamese network with a U-Net backbone and using paired images to capture feature changes, MTDN effectively leverages minimal data to produce high-quality segmentations. Furthermore, MTDN utilizes a multi-task learning strategy to exploit correlations between physical features of interest. In an evaluation using data from in situ environmental TEM (ETEM) videos of filamentous carbon gasification, MTDN demonstrated a significant advantage over conventional segmentation models, particularly in accurately delineating fine structural features. Notably, MTDN achieved a 10.22% performance improvement over conventional segmentation models in predicting small and visually ambiguous physical features. This work bridges key gaps between deep learning and practical TEM image analysis, advancing automated characterization of nanomaterials in complex experimental settings.

08 HYDROGEN↗

Developing Open-Source Training Materials for AI/ML and Space Biological Sciences Using NASA Cloud-Based Data

Artificial Intelligence (AI) and Machine Learning (ML) has gained significant traction in the biological and biomedical research fields in the last two decades, in part thanks to an increasing culture of open data sharing and reuse. Due to its capability for identifying complex relationships and patterns, AI/ML methodology is particularly well suited to recognize and predict biological patterns from high-dimensional next-generation sequencing data (e.g. whole genome sequencing, transcriptomic sequencing), as well as from biological or medical imaging data (e.g. microscopy, computed tomography, ultrasound, magnetic resonance imaging, radiography). These methodologies hold particular promise for space biosciences research and automated space health monitoring systems. However, there are many key considerations for properly training, validating, and testing a machine learning model in biological research or clinical application. Even with the positive culture of Open Science and data sharing, inexperienced researchers working quickly without proper checks can produce models that perform poorly outside of the immediate training dataset. Lessons learned from biological AI/ML research indicate that Open Science principles such as data sharing and open-source code must go hand-in-hand with publicly available, high-quality training curricula in best practices, with modules centered on real-life scientific use cases and data so future AI/ML practitioners gain experience on real problems. Here we present the development of open-source training materials for AI/ML and space biosciences, as part of the NASA Transform to Open Science Training (TOPST) initiative. We develop 4 independent training programs, focused on the following topics: 1) Fundamentals of Machine Learning and Space Biosciences Domain, 2) Open Science, Artificial Intelligence, and Ethical Best Practices for Data Sharing and Analysis, 3) Using AI/ML Classification to Identify Gene Networks Affected By Space Exposure in Mouse Liver, and 4) Using Neural Networks to Find DNA Damage Patterns in Immune Cells after Radiation. All programs leverage cloud-based NASA biological datasets. The curriculum we present will enable worldwide access to training in AI/ML and scientific analysis.

James Andrew Casaletto↗

Developing Open-Source Training Materials for AI/ML and Space Biological Sciences Using NASA Cloud-Based Data

Artificial Intelligence (AI) and Machine Learning (ML) has gained significant traction in the biological and biomedical research fields, in part due to a culture of open data sharing and reuse. AI/ML methodology is well-suited to recognize and predict biological patterns from high-dimensional next-generation sequencing data (e.g. whole genome sequencing, transcriptomic sequencing), as well as from biological or medical imaging data (e.g. microscopy, computed tomography, ultrasound, magnetic resonance imaging, radiography). These methodologies hold particular promise for space biosciences research and automated space health monitoring systems. However, there are key considerations for properly training, validating, and testing a machine learning model in biological research or clinical application. Inexperienced researchers can produce models that perform poorly outside of the training dataset. Open Science principles such as data sharing and open-source code must go hand-in-hand with publicly available, high-quality training curricula in best practices, with modules centered on real-life scientific use cases and data so future AI/ML practitioners gain experience on real problems. Here we present the development of open-source training materials for AI/ML and space biosciences, as part of the NASA Transform to Open Science Training (TOPST) initiative. We develop 4 independent training programs, focused on the following topics: 1) Fundamentals of Machine Learning and Space Biosciences Domain, 2) Open Science, Artificial Intelligence, and Ethical Best Practices for Data Sharing and Analysis, 3) Using AI/ML Classification to Identify Gene Networks Affected By Space Exposure in Mouse Liver, and 4) Using Neural Networks to Find DNA Damage Patterns in Immune Cells after Radiation. All programs leverage cloud-based NASA biological datasets. The curriculum we present will enable worldwide access to training in AI/ML and scientific analysis.

James Casaletto↗

Automated Bacterial Identification and Morphological Feature Analysis in Low‐Dose Cryo‐EM Using YOLOv11

Bacteria rapidly adapt to environmental cues through morphological and ultrastructural changes that correlate with physiology and behavior. Cryogenic transmission electron microscopy (cryo‐TEM) can capture these phenotypic changes in near‐native, vitrified states, but manual analysis of low‐dose micrographs is labor intensive and limits throughput. Here, we present an end‐to‐end workflow that combines low‐dose cryo‐TEM imaging with a YOLOv11‐based instance‐segmentation model to automatically identify bacteria and quantify key structural features directly from the micrographs. This workflow enables (i) robust bacterial localization and counting from low‐magnification atlas/montage images, (ii) automated measurements of cell‐envelope (outer–inner membrane) thickness and anisotropy from higher‐magnification views, and (iii) detection and quantification of bacteria–flagella interactions, including overlap length and curvature metrics for interacting versus noninteracting flagella. Using Pantoea sp. YR343 grown under distinct media conditions, we show that the automated measurements agree with manual annotations while substantially reducing analysis time. Together, these tools provide a practical framework for scalable bacterial identification and quantitative phenotyping in low‐dose cryo‐TEM datasets and establish a foundation for extending cryo‐TEM image analysis toward higher‐throughput studies of microbial heterogeneity and biointerfaces.

YOLOv11↗

Automated phase segmentation and quantification of high-resolution TEM image for alloy design

In the alloy design and development process, a wealth of atomically resolved structural high-resolution transmission electron microscopy (HRTEM) images are produced. Identifying the different nano-precipitate phases and tracking their evolution under various compositions and during manufacturing or post-processing requires hundreds of HRTEM images and thousands of precipitates. The nanoscopic phase information labeling and analysis purely relies on humans are prohibitively costly and time-consuming, sometimes not reliable because of the lack of authoritative knowledge. Here, in this work, we develop a novel unsupervised machine learning approach coupled with adaptive computer vision techniques with features in the Fourier space to automatically determine the number of phases and segment/quantify the phases with nanoscale resolution, allowing for quantitative correlation between nanostructure formation, processing and functional properties. To automate the phase extraction/quantification and ascertain its applicability, we have applied the developed framework to the HRTEM images from several alloy systems, processing conditions, image magnifications, and phase types and morphologies (precipitates, nano-twins, stacking faults, crystalline matrix, and amorphous structures) for verification. This study paves the road for compression, visualization, and translation of raw image structural data into physically relevant information in real-time with minimal human supervision. It shows the promise of enabling high-throughput materials characterization for the acceleration of alloy manufacturing and design.

36 MATERIALS SCIENCE↗

Evolution of Metastable Structures at Bimetallic Surfaces from Microscopy and Machine-Learning Molecular Dynamics

The restructuring of interfaces plays a crucial role in materials science and heterogeneous catalysis. Bimetallic systems, in particular, often adopt very different compositions and morphologies at surfaces compared to the bulk. For the first time, we reveal a detailed atomistic picture of long-time scale restructuring of Pd deposited on Ag using microscopy, spectroscopy, and novel simulation methods. By developing and performing accelerated machine-learning molecular dynamics followed by an automated analysis method, we discover and characterize previously unidentified surface restructuring mechanisms in an unbiased fashion, including Pd-Ag place exchange and Ag pop-out as well as step ascent and descent. Remarkably, layer-by-layer dissolution of Pd into Ag is always preceded by an encapsulation of Pd islands by Ag, resulting in a significant migration of Ag out of the surface and a formation of extensive vacancy pits within a period of microseconds. These metastable structures are of vital catalytic importance, as Ag-encapsulated Pd remains much more accessible to reactants than bulk-dissolved Pd. We report our approach is broadly applicable to complex multimetallic systems and enables the previously intractable mechanistic investigation of restructuring dynamics at atomic resolution.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Deep Neural Network Algorithm for CMC Microstructure Characterization and Variability Quantification

Microstructure characterization and variability quantification are crucial for understanding ceramic matrix composites (CMCs) mechanical behavior and deformation mechanisms across length scales. Traditionally, analyses of the micrographs obtained from microscopy are labor-intensive. However, with the vast improvement in computer vision (CV) and deep learning (DL), an automated algorithm can be designed to extract essential microstructure variability from micrographs which can then be used to construct a statistically representative volume element (SRVE). The DL-based algorithm spans the taxonomy of microstructure analyses, including semantic segmentation of microstructure constituents, secondary phases, matrix/fiber interface, and defects, and quantifying the microstructure variability in terms of probability distributions. In this work, C/SiNC and SiC/SiNC CMCs microstructures are semantically segmented through a deep convolutional neural network, followed by variability quantification through the implementation of a fully connected regression layer, hence forming a deep regression network. The deep regression network operates in a feedforward regime, in which the neuron output signal traverses through the network in a unidirectional manner. The weight tensor associated with each layer is updated through a backpropagation stochastic gradient descent approach. The input gray-scale image obtained through in-house scanning electron microscope and confocal microscope micrographs is augmented through affine transformations to increase the training set size, which is then processed through four strided convolutional layers. This compresses the image resolution by half at each layer while increasing the image depth by applying different filters (image encoding). The class activation maps (CAMs) corresponding to the applied filters highlight the key architectural features and assist with the semantic segmentation of the microstructure.

Hamza, Mohamed H.↗

Large-area silicon sheet task

A set of computer models was used to define a growth system configuration that was then built and used to grow web with lower thermally generated stress. Aspects of research in the edge-defined film-fed growth (EFG) method of making Si ribbon are reported. A technique was developed to determine base resistivity and carrier lifetime in semicrystalline wafers. Automated growth of 150 kg of 15 cm-dia ingot material per crucible is reviewed. Scanning transmisson electron microscopy (STEM) and microprobe investigations of processed EFG ribbon are reported. The chemical composition of the large precipitates was studied. The structural arrangement and the electrical activity of distentions or close to the central twin plane in processed material were studied. The electrical and structural properties of grain boundaries in silicon are discussed. Temperature-dependence measurements of zero-bias conductance, a photoconductivity technique, and deep-level transient spectroscopy (DLTS) were developed. A grooving and staining technique, secondary ion mass spectroscopy, and EBIC measurements in scanning electron microscopy were used to study enhanced diffusion of phosphorus at grain boundaries in polycrystaline silicon. The fundamental mechanisms of abrasion and wear and the deformation of Si by a diamond in various fluid environments are described. The efficiency of solar cells made from EFG ribbon and Semix Inc. material is reported.

Morrison, A. D.↗

Learning and Predicting Photonic Responses of Plasmonic Nanoparticle Assemblies via Dual Variational Autoencoders

In this work, the application of machine learning is demonstrated for rapid and accurate extraction of plasmonic particles cluster geometries from hyperspectral image data via a dual variational autoencoder (dual-VAE). In this approach, the information is shared between the latent spaces of two VAEs acting on the particle shape data and spectral data, respectively, but enforcing a common encoding on the shape-spectra pairs. It is shown that this approach can establish the relationship between the geometric characteristics of nanoparticles and their far-field photonic responses, demonstrating that hyperspectral darkfield microscopy can be used to accurately predict the geometry (number of particles, arrangement) of a multiparticle assemblies below the diffraction limit in an automated fashion with high fidelity (for monomers (0.96), dimers (0.86), and trimers (0.58). This approach of building structure-property relationships via shared encoding is universal and should have applications to a broader range of materials science and physics problems in imaging of both molecular and nanomaterial systems.

variational autoencoder↗

Multimodal imaging and machine learning to enhance microscope images of shale

A machine learning based image processing workflow is presented to enhance shale source rock microscopic images obtained using diverse imaging platforms. Images were acquired from a 30 μm diameter cylindrical Vaca Muerta shale sample using both nondestructive Transmission X-Ray Microscopy (TXM, alternately referred to as nano computed tomography) and destructive Focused Ion Beam-Scanning Electron Microscopy (FIB-SEM). Output cross-sectional images from each modality were aligned using a combination of manual and automated registration techniques to create a registered image dataset. We then apply this dataset for two image processing tasks: prediction of image cross sections with SEM-like resolution from nondestructive TXM data and repair of charged region artifacts (localized accumulation of electrons) within SEM images. The image processing algorithms for both tasks use deep learning models, specifically image-to-image Convolutional Neural Networks (CNNs) and conditional Generative Adversarial Networks (cGANs). In the image enhancement tasks, we are able to achieve significant qualitative and quantitative improvement in TXM images. Here, the best model reaches an average Peak Signal to Noise Ratio (PSNR) of 15.8 dB. Conditioning on TXM data is also shown to reduce artifacts from SEM charging, achieving an average PSNR of 25.8 dB. Furthermore, our results suggest that properly trained and validated networks are capable of significant enhancement of images obtained using nondestructive techniques, thereby improving interpretation of two- and three-dimensional images while preserving samples for future use.

58 GEOSCIENCES↗

Preparing Lamellae from Vitreous Biological Samples using a Dual-Beam Scanning Electron Microscope for Cryo-Electron Tomography

Presented here is a protocol for preparing cryo-lamellae from plunge-frozen grids of Plasmodium falciparum-infected human erythrocytes, which could easily be adapted for other biological samples. The basic principles for preparing samples, milling, and viewing lamellae are common to all instruments and the protocol can be followed as a general guide to on-grid cryo-lamella preparation for cryo-electron microscopy (cryoEM) and cryo-electron tomography (cryoET). Electron microscopy grids supporting the cells are plunge-frozen into liquid nitrogen-cooled liquid ethane using a manual or automated plunge freezer, then screened on a light microscope equipped with a cryo-stage. Frozen grids are transferred into a cryo-scanning electron microscope equipped with a focused ion beam (cryoFIB-SEM). Grids are routinely sputter coated prior to milling, which aids dispersal of charge build-up during milling. Alternatively, an e-beam rotary coater can be used to apply a layer of carbon-platinum to the grids, the exact thickness of which can be more precisely controlled. Once inside the cryoFIB-SEM an additional coating of an organoplatinum compound is applied to the surface of the grid via a gas injection system (GIS). This layer protects the front edge of the lamella as it is milled, the integrity of which is critical for achieving uniformly thin lamellae. Regions of interest are identified via SEM and milling is carried out in a step-wise fashion, reducing the current of the ion beam as the lamella reaches electron transparency, in order to avoid excessive heat generation. A grid with multiple lamellae is then transferred to a transmission electron microscope (TEM) under cryogenic conditions for tilt-series acquisition. A robust and contamination-free workflow for lamella preparation is an essential step for downstream techniques, including cellular cryoEM, cryoET, and sub-tomogram averaging. Development of these techniques, especially for lift-out and milling of high-pressure frozen samples, is of high-priority in the field.

59 BASIC BIOLOGICAL SCIENCES↗

Reward based optimization of resonance-enhanced piezoresponse spectroscopy

Dynamic spectroscopies in scanning probe microscopy (SPM) are critical for probing material properties, such as force interactions, mechanical properties, polarization switching, electrochemical reactions, and ionic dynamics. However, the practical implementation of these measurements is constrained by the need to balance imaging time and data quality. Signal to noise requirements favor long acquisition times and high frequencies to improve signal fidelity. However, these are limited on the low end by contact resonant frequency and photodiode sensitivity and on the high end by the time needed to acquire high-resolution spectra or the propensity for sample degradation under high field excitation over long times. The interdependence of key parameters such as instrument settings, acquisition times, and sampling rates makes manual tuning labor-intensive and highly dependent on user expertise, often yielding operator-dependent results. These limitations are prominent in techniques like dual amplitude resonance tracking in piezoresponse force microscopy that utilize multiple concurrent feedback loops for topography and resonance frequency tracking. Here, a reward-driven workflow is proposed that automates the tuning process, adapting experimental conditions in real time to optimize data quality. Furthermore, this approach significantly reduces the complexity and time required for manual adjustments and can be extended to other SPM spectroscopic methods, enhancing overall efficiency and reproducibility.

47 OTHER INSTRUMENTATION↗

Classifying handedness in chiral nanomaterials using label error robust deep learning

Abstract High-throughput scanning electron microscopy (SEM) coupled with classification using neural networks is an ideal method to determine the morphological handedness of large populations of chiral nanoparticles. Automated labeling removes the time-consuming manual labeling of training data, but introduces label error, and subsequently classification error in the trained neural network. Here, we evaluate methods to minimize classification error when training from automated labels of SEM datasets of chiral Tellurium nanoparticles. Using the mirror relationship between images of opposite handed particles, we artificially create populations of varying label error. We analyze the impact of label error rate and training method on the classification error of neural networks on an ideal dataset and on a practical dataset. Of the three training methods considered, we find that a pretraining approach yields the most accurate results across label error rates on ideal datasets, where size and other morphological variables are held constant, but that a co-teaching approach performs the best in practical application.

36 MATERIALS SCIENCE↗

Physics Discovery in Nanoplasmonic Systems via Autonomous Experiments in Scanning Transmission Electron Microscopy

Abstract Physics‐driven discovery in an autonomous experiment has emerged as a dream application of machine learning in physical sciences. Here, this work develops and experimentally implements a deep kernel learning (DKL) workflow combining the correlative prediction of the target functional response and its uncertainty from the structure, and physics‐based selection of acquisition function, which autonomously guides the navigation of the image space. Compared to classical Bayesian optimization (BO) methods, this approach allows to capture the complex spatial features present in the images of realistic materials, and dynamically learn structure–property relationships. In combination with the flexible scalarizer function that allows to ascribe the degree of physical interest to predicted spectra, this enables physical discovery in automated experiment. Here, this approach is illustrated for nanoplasmonic studies of nanoparticles and experimentally implemented in a truly autonomous fashion for bulk‐ and edge plasmon discovery in MnPS 3 , a lesser‐known beam‐sensitive layered 2D material. This approach is universal, can be directly used as‐is with any specimen, and is expected to be applicable to any probe‐based microscopic techniques including other STEM modalities, scanning probe microscopies, chemical, and optical imaging.

42 ENGINEERING↗

Identifying Sample Provenance From SEM/EDS Automated Particle Analysis via Few-Shot Learning Coupled With Similarity Graph Clustering

Automated particle analysis (APA) provides a vast amount of compositional data via energy-dispersive X-ray spectroscopy along with size and shape data via scanning electron microscopy for individual particles in a sample. In many instances, APA data are leveraged to support identification of the source of a sample based on the detection of particles of a specific composition. Often, the particles that provide context make up a minuscule portion of the sample. Additionally, the interpretation of complex samples can be difficult due to the diversity of compositions both in the mixture and within a particle. In this work, we demonstrate a method to compute and cluster similarity graphs that describe inter-particle relationships within a sample using a multi-modal few-shot learning neural network. Here, as a proof-of-concept, we show that samples known to have been exposed to gunshot residue can be distinguished from samples occasionally mistaken for gunshot residue. Our workflow builds upon standard APA techniques and data processing methods to unveil additional information in a readily interpretable and quantitatively comparable format.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Deep generative learning of magnetic frustration in artificial spin ice from magnetic force microscopy images

Increasingly large datasets of microscopic images with nanoscale resolution facilitate the development of machine learning methods to identify and analyze subtle physical phenomena embedded within the images. In this work, microscopic images of honeycomb lattice spin-ice samples serve as datasets from which we automate the calculation of net magnetic moments and directional orientations of spin-ice configurations. In the first stage of our workflow, machine learning models are trained to accurately predict magnetic moments and directions within spin-ice structures. Variational Autoencoders (VAEs), an emergent unsupervised deep learning technique, are employed to generate high-quality synthetic magnetic force microscopy (MFM) images and extract latent feature representations, thereby reducing experimental and segmentation errors. The second stage of proposed methodology enables precise identification and prediction of frustrated vertices and nanomagnetic segments, effectively correlating structural and functional aspects of microscopic images. This facilitates the design of optimized spin-ice configurations with controlled frustration patterns, enabling potential on-demand synthesis.

36 MATERIALS SCIENCE↗