Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “microscopy automation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Fractal analysis on Ag 2 O thin film using a data-driven approach

The synthesis of fractal Ag oxide (Ag 2 O) on the surface of Ag thin film has been achieved at room temperature by using Synchrotron X-ray irradiation. We have performed an automated quantitative analysis of a batch of 1879 fractal Ag 2 O patterns in a scanning electron microscopy (SEM) image within a radius of 2000 mm outward from the center of the X-ray beam. The morphology is similar to that of the diffusion-limited cluster aggregation (DLCA) model. The fractal dimension (D) of Ag 2 O is between 1.7 and 1.5 from the center to the edge. The area distribution density of fractal Ag 2 O follows a quadratic function with radius R. It is found that the branches’ number of fractal Ag 2 O is a key factor affecting the fractal dimension. The more branches the fractal has, the greater the D is. This is the first time that Ag fractal has been investigated by combining automated data analysis methods with batch experimental data. This data-driven approach provides a new research perspective for rationally regulating materials’ fractal morphology and performance.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Spectro-Microscopic Analysis of Soot Particle Composition and Source Attribution

Ambient soot particles significantly impact Earth’s radiative balance, human health, and atmospheric visibility. Their microstructural properties depend on formation and aging mechanisms, which vary by emission source and atmospheric processes. Hence, accurately identifying sources of soot enhances our understanding of their physicochemical properties and atmospheric implications. This study used a multi-modal approach to characterize and attribute sources of submicron soot particles collected in Israel during new particle formation events, biomass burning episodes, and background atmospheric conditions. Synchrotron-based X-ray microscopy was used to map soot (elemental carbon), organic carbon, and inorganic species. Implemented atomic force microscopy showed highly diverse phase states, with soot consistently exhibiting a solid-like phase. Automated µ-Raman analysis was subsequently performed on ~690 particles, identifying three soot classes based on spectral features corresponding to the "Defect" (D) and "Graphite" (G) bands of soot. We applied two-peak and five-peak fitting approaches to deconvolute the “Defect” peaks (D1, D2, D3, and D4) and G band from average Raman, revealing varying degrees of graphitic order. The degree of graphitic order was determined from metrics such as the D3 peak area, often observed when soot was internally mixed with organic material. Raman spectral features, along with temporal variations in particle classes contributions, suggest that Particle Type 1 corresponds to traffic related soot and Particle Type 2 to less graphitic soot from biomass burning, while Particle Type 3 is associated with more heterogeneous particulate representative of soot-OC mixtures emitted during new particle formation and biomass burning episodes.

Rivera-Adorno, Felipe (ORCID:0000000273557999)↗
DOE OSTI2022

Exploring Physics of Ferroelectric Domain Walls in Real Time: Deep Learning Enabled Scanning Probe Microscopy

The functionality of ferroelastic domain walls in ferroelectric materials is explored in real-time via the in situ implementation of computer vision algorithms in scanning probe microscopy (SPM) experiment. The robust deep convolutional neural network (DCNN) is implemented based on a deep residual learning framework (Res) and holistically nested edge detection (Hed), and ensembled to minimize the out-of-distribution drift effects. The DCNN is implemented for real-time operations on SPM, converting the data stream into the semantically segmented image of domain walls and the corresponding uncertainty. Further the pre-defined experimental workflows perform piezoresponse spectroscopy measurement on thus discovered domain walls, and alternating high- and low-polarization dynamic (out-of-plane) ferroelastic domain walls in a PbTiO 3 (PTO) thin film and high polarization dynamic (out-of-plane) at short ferroelastic walls (compared with long ferroelastic walls) in a lead zirconate titanate (PZT) thin film is reported. This work establishes the framework for real-time DCNN analysis of data streams in scanning probe and other microscopies and highlights the role of out-of-distribution effects and strategies to ameliorate them in real time analytics.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Adaptively driven X-ray diffraction guided by machine learning for autonomous phase identification

Machine learning (ML) has become a valuable tool to assist and improve materials characterization, enabling automated interpretation of experimental results with techniques such as X-ray diffraction (XRD) and electron microscopy. Because ML models are fast once trained, there is a key opportunity to bring interpretation in-line with experiments and make on-the-fly decisions to achieve optimal measurement effectiveness, which creates broad opportunities for rapid learning and information extraction from experiments. Here, we demonstrate such a capability with the development of autonomous and adaptive XRD. By coupling an ML algorithm with a physical diffractometer, this method integrates diffraction and analysis such that early experimental information is leveraged to steer measurements toward features that improve the confidence of a model trained to identify crystalline phases. We validate the effectiveness of an adaptive approach by showing that ML-driven XRD can accurately detect trace amounts of materials in multi-phase mixtures with short measurement times. The improved speed of phase detection also enables in situ identification of short-lived intermediate phases formed during solid-state reactions using a standard in-house diffractometer. Our findings showcase the advantages of in-line ML for materials characterization and point to the possibility of more general approaches for adaptive experimentation.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Dynamic Measurements of the Structural Evolution of Material Defects at the Mesoscale

This is a technical report to be published as part of the FY2021 annual overview package. We have designed experiments and developed analytical methods using images from x-ray and electron microscopy in order to quantify the behavior of crystal dislocation defects at their native nanometer-length and micro-length scales. The results of these analyses are used to directly inform theoretical physics models and guide interpretation of dark field x-ray microscopy (DFXM) images that offer spatial resolution surpassing the resolution of beamline optics. Additionally, we developed new analytical methods to automate the alignment of optics hardware used in both synchrotron and x-ray free electron laser beamline facilities. The tools and workflows developed for this project are now being deployed as proof-of-concept in reaction history analysis (RHA) for future programmatic integration.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Rapid detection of rare events from in situ X-ray diffraction data using machine learning

High-energy X-ray diffraction methods can non-destructively map the 3D microstructure and associated attributes of metallic polycrystalline engineering materials in their bulk form. These methods are often combined with external stimuli such as thermo-mechanical loading to take snapshots of the evolving microstructure and attributes over time. However, the extreme data volumes and the high costs of traditional data acquisition and reduction approaches pose a barrier to quickly extracting actionable insights and improving the temporal resolution of these snapshots. This article presents a fully automated technique capable of rapidly detecting the onset of plasticity in high-energy X-ray microscopy data. The technique is computationally faster by at least 50 times than the traditional approaches and works for data sets that are up to nine times sparser than a full data set. This new technique leverages self-supervised image representation learning and clustering to transform massive data sets into compact, semantic-rich representations of visually salient characteristics ( e.g. peak shapes). These characteristics can rapidly indicate anomalous events, such as changes in diffraction peak shapes. It is anticipated that this technique will provide just-in-time actionable information to drive smarter experiments that effectively deploy multi-modal X-ray diffraction methods spanning many decades of length scales.

Zheng, Weijian↗
DOE OSTI2025

Analysis of biofilm assembly by large area automated AFM

Biofilms are complex microbial communities critical in medical, industrial, and environmental contexts. Understanding their assembly, structure, genetic regulation, interspecies interactions, and environmental responses is key to developing effective control and mitigation strategies. While atomic force microscopy (AFM) offers critically important high-resolution insights on structural and functional properties at the cellular and even sub-cellular level, its limited scan range and labor-intensive nature restricts the ability to link these smaller scale features to the functional macroscale organization of the films. We begin to address this limitation by introducing an automated large area AFM approach capable of capturing high-resolution images over millimeter-scale areas, aided by machine learning for seamless image stitching, cell detection, and classification. Large area AFM is shown to provide a very detailed view of spatial heterogeneity and cellular morphology during the early stages of biofilm formation which were previously obscured. Using this approach, we examined the organization of Pantoea sp. YR343 on PFOTS-treated glass surfaces. Our findings reveal a preferred cellular orientation among surface-attached cells, forming a distinctive honeycomb pattern. Detailed mapping of flagella interactions suggests that flagellar coordination plays a role in biofilm assembly beyond initial attachment. Additionally, we use large-area AFM to characterize surface modifications on silicon substrates, observing a significant reduction in bacterial density. This highlights the potential of this method for studying surface modifications to better understand and control bacterial adhesion and biofilm formation.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

MultiTaskDeltaNet: change detection-based image segmentation for operando ETEM with application to carbon gasification kinetics

Transforming in situ transmission electron microscopy (TEM) imaging into a tool for spatially-resolved operando characterization of solid-state reactions requires automated, high-precision semantic segmentation of dynamically evolving features. However, traditional deep learning methods for semantic segmentation often face limitations due to the scarcity of labeled data, visually ambiguous features of interest, and scenarios involving small objects. To tackle these challenges, we introduce MultiTaskDeltaNet (MTDN), a novel deep learning architecture that creatively reconceptualizes the segmentation task as a change detection problem. By implementing a unique Siamese network with a U-Net backbone and using paired images to capture feature changes, MTDN effectively leverages minimal data to produce high-quality segmentations. Furthermore, MTDN utilizes a multi-task learning strategy to exploit correlations between physical features of interest. In an evaluation using data from in situ environmental TEM (ETEM) videos of filamentous carbon gasification, MTDN demonstrated a significant advantage over conventional segmentation models, particularly in accurately delineating fine structural features. Notably, MTDN achieved a 10.22% performance improvement over conventional segmentation models in predicting small and visually ambiguous physical features. This work bridges key gaps between deep learning and practical TEM image analysis, advancing automated characterization of nanomaterials in complex experimental settings.

08 HYDROGEN↗
DOE OSTI2026

Automated Bacterial Identification and Morphological Feature Analysis in Low‐Dose Cryo‐EM Using YOLOv11

Bacteria rapidly adapt to environmental cues through morphological and ultrastructural changes that correlate with physiology and behavior. Cryogenic transmission electron microscopy (cryo‐TEM) can capture these phenotypic changes in near‐native, vitrified states, but manual analysis of low‐dose micrographs is labor intensive and limits throughput. Here, we present an end‐to‐end workflow that combines low‐dose cryo‐TEM imaging with a YOLOv11‐based instance‐segmentation model to automatically identify bacteria and quantify key structural features directly from the micrographs. This workflow enables (i) robust bacterial localization and counting from low‐magnification atlas/montage images, (ii) automated measurements of cell‐envelope (outer–inner membrane) thickness and anisotropy from higher‐magnification views, and (iii) detection and quantification of bacteria–flagella interactions, including overlap length and curvature metrics for interacting versus noninteracting flagella. Using Pantoea sp. YR343 grown under distinct media conditions, we show that the automated measurements agree with manual annotations while substantially reducing analysis time. Together, these tools provide a practical framework for scalable bacterial identification and quantitative phenotyping in low‐dose cryo‐TEM datasets and establish a foundation for extending cryo‐TEM image analysis toward higher‐throughput studies of microbial heterogeneity and biointerfaces.

YOLOv11↗
DOE OSTI2023

Automated phase segmentation and quantification of high-resolution TEM image for alloy design

In the alloy design and development process, a wealth of atomically resolved structural high-resolution transmission electron microscopy (HRTEM) images are produced. Identifying the different nano-precipitate phases and tracking their evolution under various compositions and during manufacturing or post-processing requires hundreds of HRTEM images and thousands of precipitates. The nanoscopic phase information labeling and analysis purely relies on humans are prohibitively costly and time-consuming, sometimes not reliable because of the lack of authoritative knowledge. Here, in this work, we develop a novel unsupervised machine learning approach coupled with adaptive computer vision techniques with features in the Fourier space to automatically determine the number of phases and segment/quantify the phases with nanoscale resolution, allowing for quantitative correlation between nanostructure formation, processing and functional properties. To automate the phase extraction/quantification and ascertain its applicability, we have applied the developed framework to the HRTEM images from several alloy systems, processing conditions, image magnifications, and phase types and morphologies (precipitates, nano-twins, stacking faults, crystalline matrix, and amorphous structures) for verification. This study paves the road for compression, visualization, and translation of raw image structural data into physically relevant information in real-time with minimal human supervision. It shows the promise of enabling high-throughput materials characterization for the acceleration of alloy manufacturing and design.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Evolution of Metastable Structures at Bimetallic Surfaces from Microscopy and Machine-Learning Molecular Dynamics

The restructuring of interfaces plays a crucial role in materials science and heterogeneous catalysis. Bimetallic systems, in particular, often adopt very different compositions and morphologies at surfaces compared to the bulk. For the first time, we reveal a detailed atomistic picture of long-time scale restructuring of Pd deposited on Ag using microscopy, spectroscopy, and novel simulation methods. By developing and performing accelerated machine-learning molecular dynamics followed by an automated analysis method, we discover and characterize previously unidentified surface restructuring mechanisms in an unbiased fashion, including Pd-Ag place exchange and Ag pop-out as well as step ascent and descent. Remarkably, layer-by-layer dissolution of Pd into Ag is always preceded by an encapsulation of Pd islands by Ag, resulting in a significant migration of Ag out of the surface and a formation of extensive vacancy pits within a period of microseconds. These metastable structures are of vital catalytic importance, as Ag-encapsulated Pd remains much more accessible to reactants than bulk-dissolved Pd. We report our approach is broadly applicable to complex multimetallic systems and enables the previously intractable mechanistic investigation of restructuring dynamics at atomic resolution.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Deep Neural Network Algorithm for CMC Microstructure Characterization and Variability Quantification

Microstructure characterization and variability quantification are crucial for understanding ceramic matrix composites (CMCs) mechanical behavior and deformation mechanisms across length scales. Traditionally, analyses of the micrographs obtained from microscopy are labor-intensive. However, with the vast improvement in computer vision (CV) and deep learning (DL), an automated algorithm can be designed to extract essential microstructure variability from micrographs which can then be used to construct a statistically representative volume element (SRVE). The DL-based algorithm spans the taxonomy of microstructure analyses, including semantic segmentation of microstructure constituents, secondary phases, matrix/fiber interface, and defects, and quantifying the microstructure variability in terms of probability distributions. In this work, C/SiNC and SiC/SiNC CMCs microstructures are semantically segmented through a deep convolutional neural network, followed by variability quantification through the implementation of a fully connected regression layer, hence forming a deep regression network. The deep regression network operates in a feedforward regime, in which the neuron output signal traverses through the network in a unidirectional manner. The weight tensor associated with each layer is updated through a backpropagation stochastic gradient descent approach. The input gray-scale image obtained through in-house scanning electron microscope and confocal microscope micrographs is augmented through affine transformations to increase the training set size, which is then processed through four strided convolutional layers. This compresses the image resolution by half at each layer while increasing the image depth by applying different filters (image encoding). The class activation maps (CAMs) corresponding to the applied filters highlight the key architectural features and assist with the semantic segmentation of the microstructure.

Hamza, Mohamed H.↗
DOE OSTI2023

Learning and Predicting Photonic Responses of Plasmonic Nanoparticle Assemblies via Dual Variational Autoencoders

In this work, the application of machine learning is demonstrated for rapid and accurate extraction of plasmonic particles cluster geometries from hyperspectral image data via a dual variational autoencoder (dual-VAE). In this approach, the information is shared between the latent spaces of two VAEs acting on the particle shape data and spectral data, respectively, but enforcing a common encoding on the shape-spectra pairs. It is shown that this approach can establish the relationship between the geometric characteristics of nanoparticles and their far-field photonic responses, demonstrating that hyperspectral darkfield microscopy can be used to accurately predict the geometry (number of particles, arrangement) of a multiparticle assemblies below the diffraction limit in an automated fashion with high fidelity (for monomers (0.96), dimers (0.86), and trimers (0.58). This approach of building structure-property relationships via shared encoding is universal and should have applications to a broader range of materials science and physics problems in imaging of both molecular and nanomaterial systems.

variational autoencoder↗
DOE OSTI2020

Multimodal imaging and machine learning to enhance microscope images of shale

A machine learning based image processing workflow is presented to enhance shale source rock microscopic images obtained using diverse imaging platforms. Images were acquired from a 30 μm diameter cylindrical Vaca Muerta shale sample using both nondestructive Transmission X-Ray Microscopy (TXM, alternately referred to as nano computed tomography) and destructive Focused Ion Beam-Scanning Electron Microscopy (FIB-SEM). Output cross-sectional images from each modality were aligned using a combination of manual and automated registration techniques to create a registered image dataset. We then apply this dataset for two image processing tasks: prediction of image cross sections with SEM-like resolution from nondestructive TXM data and repair of charged region artifacts (localized accumulation of electrons) within SEM images. The image processing algorithms for both tasks use deep learning models, specifically image-to-image Convolutional Neural Networks (CNNs) and conditional Generative Adversarial Networks (cGANs). In the image enhancement tasks, we are able to achieve significant qualitative and quantitative improvement in TXM images. Here, the best model reaches an average Peak Signal to Noise Ratio (PSNR) of 15.8 dB. Conditioning on TXM data is also shown to reduce artifacts from SEM charging, achieving an average PSNR of 25.8 dB. Furthermore, our results suggest that properly trained and validated networks are capable of significant enhancement of images obtained using nondestructive techniques, thereby improving interpretation of two- and three-dimensional images while preserving samples for future use.

58 GEOSCIENCES↗
DOE OSTI2021

Preparing Lamellae from Vitreous Biological Samples using a Dual-Beam Scanning Electron Microscope for Cryo-Electron Tomography

Presented here is a protocol for preparing cryo-lamellae from plunge-frozen grids of Plasmodium falciparum-infected human erythrocytes, which could easily be adapted for other biological samples. The basic principles for preparing samples, milling, and viewing lamellae are common to all instruments and the protocol can be followed as a general guide to on-grid cryo-lamella preparation for cryo-electron microscopy (cryoEM) and cryo-electron tomography (cryoET). Electron microscopy grids supporting the cells are plunge-frozen into liquid nitrogen-cooled liquid ethane using a manual or automated plunge freezer, then screened on a light microscope equipped with a cryo-stage. Frozen grids are transferred into a cryo-scanning electron microscope equipped with a focused ion beam (cryoFIB-SEM). Grids are routinely sputter coated prior to milling, which aids dispersal of charge build-up during milling. Alternatively, an e-beam rotary coater can be used to apply a layer of carbon-platinum to the grids, the exact thickness of which can be more precisely controlled. Once inside the cryoFIB-SEM an additional coating of an organoplatinum compound is applied to the surface of the grid via a gas injection system (GIS). This layer protects the front edge of the lamella as it is milled, the integrity of which is critical for achieving uniformly thin lamellae. Regions of interest are identified via SEM and milling is carried out in a step-wise fashion, reducing the current of the ion beam as the lamella reaches electron transparency, in order to avoid excessive heat generation. A grid with multiple lamellae is then transferred to a transmission electron microscope (TEM) under cryogenic conditions for tilt-series acquisition. A robust and contamination-free workflow for lamella preparation is an essential step for downstream techniques, including cellular cryoEM, cryoET, and sub-tomogram averaging. Development of these techniques, especially for lift-out and milling of high-pressure frozen samples, is of high-priority in the field.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Reward based optimization of resonance-enhanced piezoresponse spectroscopy

Dynamic spectroscopies in scanning probe microscopy (SPM) are critical for probing material properties, such as force interactions, mechanical properties, polarization switching, electrochemical reactions, and ionic dynamics. However, the practical implementation of these measurements is constrained by the need to balance imaging time and data quality. Signal to noise requirements favor long acquisition times and high frequencies to improve signal fidelity. However, these are limited on the low end by contact resonant frequency and photodiode sensitivity and on the high end by the time needed to acquire high-resolution spectra or the propensity for sample degradation under high field excitation over long times. The interdependence of key parameters such as instrument settings, acquisition times, and sampling rates makes manual tuning labor-intensive and highly dependent on user expertise, often yielding operator-dependent results. These limitations are prominent in techniques like dual amplitude resonance tracking in piezoresponse force microscopy that utilize multiple concurrent feedback loops for topography and resonance frequency tracking. Here, a reward-driven workflow is proposed that automates the tuning process, adapting experimental conditions in real time to optimize data quality. Furthermore, this approach significantly reduces the complexity and time required for manual adjustments and can be extended to other SPM spectroscopic methods, enhancing overall efficiency and reproducibility.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Classifying handedness in chiral nanomaterials using label error robust deep learning

Abstract High-throughput scanning electron microscopy (SEM) coupled with classification using neural networks is an ideal method to determine the morphological handedness of large populations of chiral nanoparticles. Automated labeling removes the time-consuming manual labeling of training data, but introduces label error, and subsequently classification error in the trained neural network. Here, we evaluate methods to minimize classification error when training from automated labels of SEM datasets of chiral Tellurium nanoparticles. Using the mirror relationship between images of opposite handed particles, we artificially create populations of varying label error. We analyze the impact of label error rate and training method on the classification error of neural networks on an ideal dataset and on a practical dataset. Of the three training methods considered, we find that a pretraining approach yields the most accurate results across label error rates on ideal datasets, where size and other morphological variables are held constant, but that a co-teaching approach performs the best in practical application.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Physics Discovery in Nanoplasmonic Systems via Autonomous Experiments in Scanning Transmission Electron Microscopy

Abstract Physics‐driven discovery in an autonomous experiment has emerged as a dream application of machine learning in physical sciences. Here, this work develops and experimentally implements a deep kernel learning (DKL) workflow combining the correlative prediction of the target functional response and its uncertainty from the structure, and physics‐based selection of acquisition function, which autonomously guides the navigation of the image space. Compared to classical Bayesian optimization (BO) methods, this approach allows to capture the complex spatial features present in the images of realistic materials, and dynamically learn structure–property relationships. In combination with the flexible scalarizer function that allows to ascribe the degree of physical interest to predicted spectra, this enables physical discovery in automated experiment. Here, this approach is illustrated for nanoplasmonic studies of nanoparticles and experimentally implemented in a truly autonomous fashion for bulk‐ and edge plasmon discovery in MnPS 3 , a lesser‐known beam‐sensitive layered 2D material. This approach is universal, can be directly used as‐is with any specimen, and is expected to be applicable to any probe‐based microscopic techniques including other STEM modalities, scanning probe microscopies, chemical, and optical imaging.

42 ENGINEERING↗