Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “microscopy automation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

BEACON—automated aberration correction for scanning transmission electron microscopy using Bayesian optimization

Aberration correction is an important aspect of modern high-resolution scanning transmission electron microscopy. Most methods of aligning aberration correctors require specialized sample regions and are unsuitable for fine-tuning aberrations without interrupting on-going experiments. Here, we present an automated method of correcting first- and second-order aberrations called BEACON, which uses Bayesian optimization of the normalized image variance to efficiently determine the optimal corrector settings. We demonstrate its use on gold nanoparticles and a hafnium dioxide thin film showing its versatility in nano- and atomic-scale experiments. BEACON can correct all first- and second-order aberrations simultaneously to achieve an initial alignment and first- and second-order aberrations independently for fine alignment. Ptychographic reconstructions are used to demonstrate an improvement in probe shape and a reduction in the target aberration.

techniques and instrumentation↗
DOE OSTI2023

Autonomous Molecular Structure Imaging with High-Resolution Atomic Force Microscopy for Molecular Mixture Discovery

Due to its single-molecule sensitivity, high-resolution atomic force microscopy (HR-AFM) has proved to be a valuable and uniquely advantageous tool to study complex molecular mixtures, which hold promise for developing clean energy and achieving environmental sustainability. However, significant challenges remain to achieve the full potential of the sophisticated and time-consuming experiments. Automation combined with machine learning (ML) and artificial intelligence (AI) is key to overcoming these challenges. Here we present Auto-HR-AFM, an AI tool to automatically collect HR-AFM images of petroleum-based mixtures. In this study, we trained an instance segmentation model to teach Auto-HR-AFM how to recognize features in HR-AFM images. Auto-HR-AFM then uses that information to optimize the imaging by adjusting the probe-molecule distance for each molecule in the run. Auto-HR-AFM is the initial tool that will lead to fully automated scanning probe microscopy (SPM) experiments, from start to finish. This automation will allow SPM to become a mainstream characterization technique for complex mixtures, an otherwise unattainable target.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Leveraging generative adversarial networks to create realistic scanning transmission electron microscopy images

Abstract The rise of automation and machine learning (ML) in electron microscopy has the potential to revolutionize materials research through autonomous data collection and processing. A significant challenge lies in developing ML models that rapidly generalize to large data sets under varying experimental conditions. We address this by employing a cycle generative adversarial network (CycleGAN) with a reciprocal space discriminator, which augments simulated data with realistic spatial frequency information. This allows the CycleGAN to generate images nearly indistinguishable from real data and provide labels for ML applications. We showcase our approach by training a fully convolutional network (FCN) to identify single atom defects in a 4.5 million atom data set, collected using automated acquisition in an aberration-corrected scanning transmission electron microscope (STEM). Our method produces adaptable FCNs that can adjust to dynamically changing experimental variables with minimal intervention, marking a crucial step towards fully autonomous harnessing of microscopy big data.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2021

Machine-Learning-based Algorithms for Automated Image Segmentation Techniques of Transmission X-ray Microscopy (TXM)

Four state-of-the-art Deep Learning-based Convolutional Neural Networks (DCNN) were applied to automate the semantic segmentation of a 3D Transmission x-ray Microscopy (TXM) nanotomography image data. The standard U-Net architecture as baseline along with UNet++, PSPNet, and DeepLab v3+ networks were trained to segment the microstructural features of an AA7075 micropillar. A workflow was established to evaluate and compare the DCNN prediction dataset with the manually segmented features using the Intersection of Union (IoU) scores, time of training, confusion matrix, and visual assessment. Comparing all model segmentation accuracy metrics, it was found that using pre-trained models as a backbone along with appropriate training encoder-decoder architecture of the Unet++ can robustly handle large volumes of x-ray radiographic images in a reasonable amount of time. This opens a new window for handling accurate and efficient image segmentation of in situ time-dependent 4D x-ray microscopy experimental datasets.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Towards automating structural discovery in scanning transmission electron microscopy *

Abstract Scanning transmission electron microscopy is now the primary tool for exploring functional materials on the atomic level. Often, features of interest are highly localized in specific regions in the material, such as ferroelectric domain walls, extended defects, or second phase inclusions. Selecting regions to image for structural and chemical discovery via atomically resolved imaging has traditionally proceeded via human operators making semi-informed judgements on sampling locations and parameters. Recent efforts at automation for structural and physical discovery have pointed towards the use of ‘active learning’ methods that utilize Bayesian optimization with surrogate models to quickly find relevant regions of interest. Yet despite the potential importance of this direction, there is a general lack of certainty in selecting relevant control algorithms and how to balance a priori knowledge of the material system with knowledge derived during experimentation. Here we address this gap by developing the automated experiment workflows with several combinations to both illustrate the effects of these choices and demonstrate the tradeoffs associated with each in terms of accuracy, robustness, and susceptibility to hyperparameters for structural discovery. We discuss possible methods to build descriptors using the raw image data and deep learning based semantic segmentation, as well as the implementation of variational autoencoder based representation. Furthermore, each workflow is applied to a range of feature sizes including NiO pillars within a La:SrMnO 3 matrix, ferroelectric domains in BiFeO 3 , and topological defects in graphene. The code developed in this manuscript is open sourced and will be released at github.com/nccreang/AE_Workflows .

47 OTHER INSTRUMENTATION↗
NASA NTRS2015

Light Microsopy Module, International Space Station Premier Automated Microscope

The Light Microscopy Module (LMM) was launched to the International Space Station (ISS) in 2009 and began science operations in 2010. It continues to support Physical and Biological scientific research on ISS. During 2015, if all goes as planned, five experiments will be completed: [1] Advanced Colloids Experiments with a manual sample base -3 (ACE-M-3), [2] the Advanced Colloids Experiment with a Heated Base -1 (ACE-H-1), [3] (ACE-H-2), [4] the Advanced Plant Experiment -03 (APEX-03), and [5] the Microchannel Diffusion Experiment (MDE). Preliminary results, along with an overview of present and future LMM capabilities will be presented; this includes details on the planned data imaging processing and storage system, along with the confocal upgrade to the core microscope. [1] New York University: Paul Chaikin, Andrew Hollingsworth, and Stefano Sacanna, [2] University of Pennsylvania: Arjun Yodh and Matthew Gratale, [3] a consortium of universities from the State of Kentucky working through the Experimental Program to Stimulate Competitive Research (EPSCoR): Stuart Williams, Gerold Willing, Hemali Rathnayake, et al., [4] from the University of Florida and CASIS: Anna-Lisa Paul and Rob Ferl, and [5] from the Methodist Hospital Research Institute from CASIS: Alessandro Grattoni and Giancarlo Canavese.

Microscopy↗
DOE OSTI2022

Experimental discovery of structure–property relationships in ferroelectric materials via active learning

Emergent functionalities of structural and topological defects in ferroelectric materials underpin an extremely broad spectrum of applications ranging from domain wall electronics to high dielectric and electromechanical responses. Many of these functionalities have been discovered and quantified via local scanning probe microscopy methods. However, the search has until now been based on either trial and error, or using auxiliary information such as the topography or domain wall structure to identify potential objects of interest on the basis of the intuition of operator or pre-existing hypotheses, with subsequent manual exploration. Here we report the development and implementation of a machine learning framework that actively discovers relationships between local domain structure and polarization-switching characteristics in ferroelectric materials encoded in the hysteresis loop. The hysteresis loops and their scalar descriptors such as nucleation bias, coercive bias and the hysteresis loop area (or more complex functionals of hysteresis loop shape) and corresponding uncertainties are used to guide the discovery of these relationships via automated piezoresponse force microscopy and spectroscopy experiments. As such, this approach combines the power of machine learning methods to learn the correlative relationships between high-dimensional data, as well as human-based physics insights encoded into the acquisition function. For ferroelectric materials, this automated workflow demonstrates that the discovery path and sampling points of on- and off-field hysteresis loops are largely different, indicating that on- and off-field hysteresis loops are dominated by different mechanisms. Here, the proposed approach is universal and can be applied to a broad range of modern imaging and spectroscopy methods ranging from other scanning probe microscopy modalities to electron microscopy and chemical imaging.

36 MATERIALS SCIENCE↗
DOE OSTI2022

An Automated Scanning Transmission Electron Microscope Guided by Sparse Data Analytics

Abstract Artificial intelligence (AI) promises to reshape scientific inquiry and enable breakthrough discoveries in areas such as energy storage, quantum computing, and biomedicine. Scanning transmission electron microscopy (STEM), a cornerstone of the study of chemical and materials systems, stands to benefit greatly from AI-driven automation. However, present barriers to low-level instrument control, as well as generalizable and interpretable feature detection, make truly automated microscopy impractical. Here, we discuss the design of a closed-loop instrument control platform guided by emerging sparse data analytics. We hypothesize that a centralized controller, informed by machine learning combining limited a priori knowledge and task-based discrimination, could drive on-the-fly experimental decision-making. This platform may unlock practical, automated analysis of a variety of material features, enabling new high-throughput and statistical studies.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Hyperspectral Detection of the Fluorescence Shift between Chirality-Sorted Empty and Water-Filled Single-Wall Carbon Nanotube Enantiomers

Single-wall carbon nanotubes (SWCNTs) have extraordinary electronic and optical properties that depend strongly on their exact chiral structure and their interaction with their inner and outer environment. The fluorescence (PL) of semiconducting SWCNTs, for instance, will shift depending on the molecules with which the SWCNT’s hollow core is filled. These interaction-induced shifts are challenging to resolve on the ensemble level in samples containing a mixture of different filling contents due to the relatively large inhomogeneous line width of the ensemble SWCNT PL compared to the size of these shifts. To circumvent this inhomogeneous broadening, single-tube spectroscopy and hyperspectral imaging are often applied, which until now required time-consuming statistical studies. Here, we present hyperspectral PL microscopy combined with automated SWCNT segmenting based on either principal component analysis or a convolutional neural network, capable of both spatially and spectrally resolving the PL along the length of many individual SWCNTs at the same time and automatically fitting peak positions and line widths of individual SWCNTs. The methodology is demonstrated by accurately determining the emission shifts and line widths of thousands of left- and right-handed empty and water-filled SWCNTs coated with a chiral surfactant, resulting in four statistical distributions which cannot be resolved in ensemble spectroscopy of unsorted samples. The results demonstrate a robust method to quickly probe ensemble properties with single-enantiomer spectral resolution. Moreover, it promises to be an absolute quantitative method to characterize the relative abundances of SWCNTs with different handedness or filling content in macroscopic samples, simply by counting individual species.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Automated and High-Throughput Phase Separation Control for Supramolecular Polymer Blends Enabled by Machine Learning

Supramolecular polymer blends (SPBs) offer tunable morphologies that dictate their macroscopic properties, yet their rational design is limited by the absence of predictive structure−morphology models. Here, we introduce a data-driven highthroughput workflow that integrates modular polymer synthesis, robotic formulation, automated morphology characterization, and machine learning (ML) for accelerated SPB discovery. Using a plug-and-play synthetic strategy, 33 hydrogen-bonding endfunctional homopolymers were prepared and orthogonally combined to generate 260 SPBs in 1 day. A fully automated atomic force microscopy (AFM) pipeline enabled systematic imaging, producing 2340 morphology data sets with minimal human intervention. Domain spacings were extracted through complementary imageprocessing methods and used to train ML models. A support vector regression (SVR) model accurately predicted target phase-separation sizes (50, 100, and 150 nm), which were experimentally validated. This work demonstrates the power of coupling high-throughput experimentation with ML to accelerate morphology discovery and provides one of the first large-scale experimental data sets for supramolecular polymer systems.

ML-guided polymer design↗
DOE OSTI2024

The effect of temperature and burnup on U-10Zr metallic fuel chemical interaction with HT-9: A SEM-EDS study

The fuel cladding chemical interaction (FCCI) between Uranium-Zirconium-based metallic fuel and cladding materials during in-pile service is one of the most constraining phenomena affecting the performance of this fuel system. In this study, we investigated the effect of temperature and burnup on the FCCI development in two U-10 wt.% Zr (U-10Zr) fuel samples with HT-9 cladding irradiated as part of the MFF-3 irradiation test in the Fast Flux Test Facility (FFTF). One sample achieved a burnup of 13.1 at.% and operated with an average inner cladding temperature of 530°C, while the other achieved a burnup of 8.5 at.% and was subjected to an average inner cladding temperature of 615°C. Automated scanning electron microscopy (SEM) back-scattered electron (BSE) imaging of entire fuel cross-sections and SEM energy dispersive x-ray spectroscopy (EDS) analysis on specific fuel-cladding interface regions successfully provided a comprehensive characterization of the depth and type of interaction happening under different irradiation conditions. Further, our analysis shows that FCCI development on both fuel and cladding side is strongly influenced by the inner cladding temperature and, to some extent, the formation and integrity of Zr-rich layers between the fuel and cladding, while the impact of burnup and power is negligible. Measured FCCI thicknesses were compared to BISON simulations using both an empirical model based upon legacy data from the Experimental Breeder Reactor II (EBR II) irradiations and a mechanistic model currently under development for the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program, showing satisfactory agreement. Nonetheless, this comparison supports the need for additional microstructural characterization in intermediate ranges of temperature, power, and burnups in prototypic-length pins.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2021

Multi-modal imaging of a single mouse brain over five orders of magnitude of resolution

Mammalian neurons operate at length scales spanning six orders of magnitude; they project millimeters to centimeters across brain regions, are composed of micrometer-scale-diameter myelinated axons, and ultimately form nanometer scale synapses. Capturing these anatomical features across that breadth of scale has required imaging samples with multiple independent imaging modalities. Translating between the different modalities, however, requires imaging the same brain with each. Here, we imaged the same postmortem mouse brain over five orders of spatial resolution using MRI, whole brain micrometer-scale synchrotron x-ray tomography ($\mu$CT), and large volume automated serial electron microscopy. Using this pipeline, we can track individual myelinated axons previously relegated to axon bundles in diffusion tensor MRI or arbitrarily trace neurons and their processes brain-wide and identify individual synapses on them. This pipeline provides both an unprecedented look across a single brain's multi-scaled organization as well as a vehicle for studying the brain's multi-scale pathologies.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Use of Graph Theory and Neural Networks for Microstructural Classification

Recent advances in materials data analytics have provided new avenues for determining process-structure-property (PSP) linkages in a variety of materials. Machine learning techniques including few-shot learning have increased the efficiency of classifying microscopy images for the purposes of material characterization. Modifications in segmentation also show potential in improving the accuracy of our current pyCHIP classifier. Replacing previous encoders trained on ImageNet with those trained on microscopy images like MicroNet has initially shown better performance at classifying images of irradiated samples. Additionally, different normalization approaches were tested to show no discernable effect on classification. The Louvain method for community detection is analyzed on a set of irradiated samples with different parameters to determine which proved beneficial under what circumstances. We suggest that microscopy experiments be automated in the future using a combination of these techniques to enable high-throughput analyses.

36 MATERIALS SCIENCE↗
NASA NTRS2001

Compact Microscope Imaging System Developed

The Compact Microscope Imaging System (CMIS) is a diagnostic tool with intelligent controls for use in space, industrial, medical, and security applications. The CMIS can be used in situ with a minimum amount of user intervention. This system, which was developed at the NASA Glenn Research Center, can scan, find areas of interest, focus, and acquire images automatically. Large numbers of multiple cell experiments require microscopy for in situ observations; this is only feasible with compact microscope systems. CMIS is a miniature machine vision system that combines intelligent image processing with remote control capabilities. The software also has a user-friendly interface that can be used independently of the hardware for post-experiment analysis. CMIS has potential commercial uses in the automated online inspection of precision parts, medical imaging, security industry (examination of currency in automated teller machines and fingerprint identification in secure entry locks), environmental industry (automated examination of soil/water samples), biomedical field (automated blood/cell analysis), and microscopy community. CMIS will improve research in several ways: It will expand the capabilities of MSD experiments utilizing microscope technology. It may be used in lunar and Martian experiments (Rover Robot). Because of its reduced size, it will enable experiments that were not feasible previously. It may be incorporated into existing shuttle orbiter and space station experiments, including glove-box-sized experiments as well as ground-based experiments.

McDowell, Mark↗
DOE OSTI2025

Self-Driving Microscopy for AI/ML-Enabled Physics Discovery and Materials Optimization

Materials are the bedrock of economy and foundation for all real-world technologies. The viability of space travel, grid energy storage, solar to fuels conversion, methane removal, and photovoltaic energy solutions hinge on the discovery and optimization of novel materials and rapid scaling toward manufacturing. The last 20 years have seen an exponential growth in the theoretical predictive capability for crystalline materials and small molecules. However, it is only in the last five years that we have seen the rapid expansion of high-throughput synthesis enabled by laboratory robotics and microfluidics, as well as a resurgence of combinatorial synthesis (Abolhasani and Kumacheva 2023; Epps and Abolhasani 2021; Jiang et al. 2022; Rajan 2008; Soldatov et al. 2021; Szymanski et al. 2023). Combinatorial synthesis, microfluidics, and ultimately dip-pen megalibraries have demonstrated the ability to “write” multicomponent nanomaterials at high throughput scale, generating millions of material examples in the 3D, 4D, and 5D composition spaces (Chen et al. 2016, 2019; Jibril et al. 2022).

36 MATERIALS SCIENCE↗