SEARCH · Engineering Papers

Results for “multimodal imaging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

All sources NASA NTRS DOE OSTI

DOE OSTI2020

In situ , in vivo , and in operando imaging and spectroscopy of liquids using microfluidics in vacuum

This review offers a succinct overview of the development of a vacuum-compatible microfluidic reactor system for analysis at the liquid vacuum interface (SALVI), and its diverse applications in in situ, in vivo, and in operando imaging of liquid surfaces as well as the air-liquid (a-l), liquid-liquid (l-l), and solid-liquid (s-l) interfaces in the past decade. SALVI is one of the first microfluidics-based reactors that has enabled direct analysis of volatile liquids in vacuum surface tools such as scanning electron microscopy (SEM) and time-of-flight secondary ion mass spectrometry (ToF-SIMS). Its integration into ambient and vacuum spectroscopy and microscopy is illustrated. Several applications are highlighted including (1) imaging nanoparticles in liquid using in situ SEM; (2) mapping the evolving l-l interface using in situ x-ray absorption spectroscopy and ToF-SIMS; (3) following complex a-l interfacial oxidation reaction products using in situ ToF-SIMS; (4) capturing biological interfaces of cells and microbes via in vivo multimodal and correlative imaging; and (5) monitoring the dynamic solid electrode and liquid electrolyte interface using in operando molecular imaging. Finally, outlook and recommendations are presented. Besides showing the holistic information volume obtained by real-time multiplexed imaging, this review intends to convey the importance of tool development in revolutionizing surface and interface analysis using vacuum platforms previously limited to solid surfaces. Microfluidics is manifested to be not limited to ambient conditions in many examples in this review. Moreover, fundamental interfacial phenomena underpinning mass and charge transfer can now be pursued in real time via innovated chemical imaging and spectroscopy.

42 ENGINEERING↗

DOE OSTI2022

Multimodal analysis and characterization of the boehmite layer formed on AA6061 before and after alkaline etching

Low-enriched uranium (LEU) alloyed with 10% Mo (U-10Mo) is being considered as a promising alternative to oxide-based dispersion fuel with high-enriched uranium for use in research reactors. The configuration of this proposed LEU monolithic LEU fuel plate consists of a U-10Mo plate-type fuel foil with a 25 µm zirconium interlayer barrier clad with an aluminum alloy (AA6061). In certain research reactors, the clad AA6061 is coated with a boehmite layer to prevent corrosion. The boehmite layer has a high-pH passivation range, which makes it resistant to oxidation. Boehmite is usually formed on the AA6061 surface by autoclave processing. Before the boehmite layer is added, the surface of the AA6061 is cleaned using techniques such as polishing and wet etching. In this study, we use multimodal analysis to examine how pretreatment of AA6061 using polishing followed by alkaline etching affects the chemical composition of the boehmite layer. X-ray photoelectron microscopy (XPS), transmission electron microscopy (TEM), and x-ray diffraction (XRD) were used to study the chemical changes in the boehmite layer caused by alkaline etching pretreatment. XPS provides quantitative analysis for the Al:O ratio as well as oxidation states present on the surface, which suggests slight oxidation of the boehmite surface after alkaline etching of the AA6061 surface. We further explored this suggested oxidation of the boehmite surface using high-resolution transmission electron microscopy with selected area electron diffraction (SAED) and grazing incidence x-ray diffraction (GI-XRD), which suggested only a small amount of aluminum oxide at the surface. The multimodal analysis and imaging yielded new insights for optimizing boehmite growth on AA6061 for research reactors.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Deep Multimodal Networks for M-type Star Classification with Paired Spectrum and Photometric Image

Abstract Traditional stellar classification methods include spectral and photometric classification separately. Although satisfactory results can be achieved, the accuracy could be improved. In this paper, we pioneer a novel approach to deeply fuse the spectra and photometric images of the sources in an advanced multimodal network to enhance the model’s discriminatory ability. We use Transformer as the fusion module and apply a spectrum–image contrastive loss function to enhance the consistency of the spectrum and photometric image of the same source in two different feature spaces. We perform M-type stellar subtype classification on two data sets with high and low signal-to-noise ratio (S/N) spectra and corresponding photometric images, and the F1-score achieves 95.65% and 90.84%, respectively. In our experiments, we prove that our model effectively utilizes the information from photometric images and is more accurate than advanced spectrum and photometric image classifiers. Our contributions can be summarized as follows: (1) We propose an innovative idea for stellar classification that allows the model to simultaneously consider information from spectra and photometric images. (2) We discover the challenge of fusing low-S/N spectra and photometric images in the Transformer and provide a solution. (3) The effectiveness of Transformer for spectral classification is discussed for the first time and will inspire more Transformer-based spectral classification models.

Astronomy & Astrophysics↗

DOE OSTI2025

Seeing is Believing: Autonomous Microscopy and the Data Revolution in Materials Science

This presentation explores the transformative potential of autonomous electron microscopy and artificial intelligence (AI) in accelerating materials science discovery, particularly for energy applications and materials operating in extreme environments. We discuss pioneering self-driving laboratories at NREL designed to intelligently probe material synthesis and degradation across multiple scales, aiming to rapidly bridge the gap between atomic-level understanding and the development of high-performance, reliable materials. Utilizing advanced machine learning techniques, such as few-shot learning and multimodal analysis integrating imaging and spectroscopy, we demonstrate methods to extract actionable descriptors for material behavior, quantify complex microstructural evolution, and statistically link synthesis parameters to defect populations. This AI-driven approach promises to accelerate the creation of predictive materials tailored for specific missions, enabling faster development cycles and enhanced material assurance.

36 MATERIALS SCIENCE↗

DOE OSTI2025

Seeing is Believing: Autonomous Microscopy and the Data Revolution in Materials Science

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

NDA Tech: Multimodal Data Fusion with 3D Gamma-Ray Imaging for Safeguards

The ultimate project goal is to improve quantitative results obtained by the IAEA with the H420 gamma ray imager. Distribution to the IAEA will be through incorporation of project results into the software provided with the device by our commercial partner, H3D, Inc. Toward achieving that goal, we have started discussions with H3D to identify the requisite software components and how they will mesh with their own internally developed contextual data systems and other extant software components. We have identified the coded-aperture response data cube as a key structure. Discussions on contextual imaging were informative but somewhat inconclusive as the project lead on contextual data (LBNL) has not been available.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2023

Multimodal Data Fusion with 3D Gamma-Ray Imaging for Safeguards (2023 Annual Report)

Improve the quantitative results obtained from 3D gamma-ray imagers for use in safeguards inspections. In FY23 the work focused on continuing to improve the numerical results obtained with gamma-ray imagers and explore approaches to project such data onto 3D structures mapped with Self Localization and Mapping (SLAM) hardware and software.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2021

Helium Ion Microscopy with Secondary Ion Mass Spectrometry for Nanoscale Chemical Imaging and Analysis of Polyolefins

To date, chemical imaging of polymers and polymer blends has been primarily accomplished using time-of-flight secondary ion mass spectrometry (ToF-SIMS) to directly visualize the distribution of components in a complex material with spatial resolution ranging from 100 nm to 5 μm. However, in many cases, this resolution falls far short of visualizing interfaces directly. To overcome these limitations, recent work has focused on developing a SIMS detection system based on a helium ion microscope (HIM) enabling chemical imaging with a demonstrated ~14 nm spatial resolution. Here, we utilize HIM-SIMS for differentiation between the olefin-based polymers of polyethylene (PE) and polypropylene (PP). We illustrate both analyses for separating PE and PP using specific mass fragment ratios as well as demonstrate spatially resolved imaging of phase-separated domains within PE. Altogether, we demonstrate the abilities of HIM-SIMS as a multimodal chemical technique for imaging and quantification of polyolefin interfaces, which could be more broadly applied to the analysis of more complex polymeric systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2021

Nanoindentation-Enhanced Tip-Enhanced Raman Spectroscopy

We combine nanoindentation, herein achieved using atomic force microscopy (AFM)-based pulsed-force lithography, with tip-enhanced Raman spectroscopy (TERS) and imaging. Our approach entails indentation and multimodal characterization of otherwise flat Au substrates, followed by chemical functionalization and TERS spectral imaging of the indented nanostructures. We find that the resulting structures, which vary in shape and size depending on the tip used to produce them, may sustain nano-confined and significantly enhanced local fields. We take advantage of the latter and illustrate TERS-based ultrasensitive detection/chemical fingerprinting as well as chemical reaction imaging – all using a single platform for nano-lithography, topographic imaging, hyperspectral dark field optical microscopy, and TERS.

Wang, Chih-Feng↗

DOE OSTI2025

Iterative Reconstruction for Multimodal Neutron Tomography

Here, we describe a unified framework for model-based iterative 3-D reconstruction of multimodal neutron transmission, hydrogen-scatter, and induced-fission images from low resolution data recorded using 14.1-MeV neutrons and the associated-particle imaging (API) technique. The framework, which was developed to facilitate use in challenging field-deployment scenarios, is centered around physics-based system models and a total variation (TV) constrained implementation of the simultaneous iterative reconstruction technique (SIRT). Modified to solve a statistically weighted least squares (WLS) problem, the SIRT algorithm is accelerated using ordered subsets and Nesterov’s momentum for which we derive a near-optimal value of the governing Lipschitz constant. The approach enables the reconstruction of images that are high resolution compared to the acquired data and is robust to both limited statistics and a limited number of projection angles. Moreover, the framework is fast enough to be practical. Example images are provided that demonstrate both the ability to perform fast-neutron imaging of high-atomic-number materials with low radiation dose and the benefit of multimodal neutron imaging to identify key materials.

Hydrogen scatter↗

DOE OSTI2022

Plasmonics-Enhanced Optical Imaging Systems for Bioenergy Research

This final report summarizes the key accomplishments of the project entitled “Plasmonics-Enhanced Optical Imaging Systems for Bioenergy Research”. The goal of this project is aimed at developing innovative and improved imaging instrumentation for visualization and quantitative characterization of biomarkers related to molecular processes and cellular function in living plants. This project focused on three specific aims: 1) Development of Advanced Multimodal Optical Sensing And Imaging Combinatory (MOSAIC) System for SERS biosensing and bioimaging. 2) Evaluation of MOSAIC technologies for sensing and imaging gene expression in plant systems using plasmonic nanoprobes. 3) Application of MOSAIC techniques to bioenergy research.

47 OTHER INSTRUMENTATION↗

DOE OSTI2023

Data augmentation and multimodal learning for predicting drug response in patient-derived xenografts from gene expressions and histology images

Patient-derived xenografts (PDXs) are an appealing platform for preclinical drug studies. A primary challenge in modeling drug response prediction (DRP) with PDXs and neural networks (NNs) is the limited number of drug response samples. We investigate multimodal neural network (MM-Net) and data augmentation for DRP in PDXs. The MM-Net learns to predict response using drug descriptors, gene expressions (GE), and histology whole-slide images (WSIs). We explore whether combining WSIs with GE improves predictions as compared with models that use GE alone. We propose two data augmentation methods which allow us training multimodal and unimodal NNs without changing architectures with a single larger dataset: 1) combine single-drug and drug-pair treatments by homogenizing drug representations, and 2) augment drug-pairs which doubles the sample size of all drug-pair samples. Unimodal NNs which use GE are compared to assess the contribution of data augmentation. The NN that uses the original and the augmented drug-pair treatments as well as single-drug treatments outperforms NNs that ignore either the augmented drug-pairs or the single-drug treatments. In assessing the multimodal learning based on the MCC metric, MM-Net outperforms all the baselines. Our results show that data augmentation and integration of histology images with GE can improve prediction performance of drug response in PDXs.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2023

Accelerated Stress Testing of Perovskite Photovoltaic Modules: Differentiating Degradation Modes with Electroluminescence Imaging

Herein, electroluminescence (EL) and thermal imaging are used to examine p–i–n metal halide perovskite (MHP) photovoltaic (PV) mini-modules (MA0.6FA0.4PbI3, 20 cells, 78 cm2) before and after indoor-accelerated stress testing or outdoor deployment. Distinct spatial patterns in the EL images emerge, which depend on the external stress conditions experienced by the mini-module. Imaging results highlight a distribution of dark speckle features that dominate after UV stress, attributed to widespread interfacial contact degradation. Lateral intensity gradients across cells dominate after thermal cycling (TC) stress, attributed to current crowding near scribe defects. While current–voltage analysis alone does not give full insight on the degradation process, this study shows that distinct degradation modes can be further defined by multimodal electro-optical imaging (i.e., EL combined with photoluminescence and dark lock-in thermography). Neither UV exposure nor TC-accelerated stress testing alone replicates the same degradation signatures observed after outdoor deployment, suggesting that multiple degradation modes occur under concurrent stressors outdoors. Finally, spatial characterization of degradation modes in MHP PV mini-modules before and after accelerated stress testing lays the groundwork for developing targeted accelerated stress testing procedures through comparison with outdoor aging.

14 SOLAR ENERGY↗

DOE OSTI2023

Advanced techniques in automated high-resolution scanning transmission electron microscopy

Scanning transmission electron microscopy is a common tool used to study the atomic structure of materials. It is an inherently multimodal tool allowing for the simultaneous acquisition of multiple information channels. Despite its versatility, however, experimental workflows currently rely heavily on experienced human operators and can only acquire data from small regions of a sample at a time. Here, we demonstrate a flexible pipeline-based system for high-throughput acquisition of atomic-resolution structural data using an all-piezo sample stage applied to large-scale imaging of nanoparticles and multimodal data acquisition. As a result, the system is available as part of the user program of the Molecular Foundry at Lawrence Berkeley National Laboratory.

4D-STEM↗

DOE OSTI2023

Synchrotron-based X-ray fluorescence microscopy mapping the ionome of a toxic freshwater cyanobacterium

Harmful algal blooms (HABs) pose a major environmental concern across the globe. In abundance, cyanobacteria, or so-called green-blue algae can produce extremely dangerous cyanotoxins that harm humans and animals. This study focused on the mapping and distribution of intracellular macro-and micronutrients of the widespread freshwater cyanobacteria Microcystis aeruginosa (M. aeruginosa). Towards a better understanding of trace metal uptake and homeostasis throughout the cell cycle, we quantitatively mapped the spatial distribution of the elements P, K, Fe, Ca, Zn, Mn, and Cu across the ultrastructure of frozen-hydrated single cells using state-of-theart X-ray nanofluorescence imaging at the Advanced Photon Source (APS) at Argonne National Laboratory. Further, bulk cellular nutrient and trace metal content correlated well with the total intracellular elemental content in individual cells obtained by quantitative synchrotron X-ray fluorescence measurements. Multi-dimensional mappings showed P and K atoms colocalized as discrete semicircular hotspots that were analyzed with respect to their stoichiometry. Elevated Cu and Ca concentrations were detected along division plane of cells. P and K were found to have similar spatial elemental distribution with about 65% and 69% of the total cellular P and K, respectively, located at the hotspots. The P and K colocalization were refined further using nanotomography, showing a K envelope surrounding the P core. Inorganic P and organic P compounds were specified using solution-state 31 P nuclear magnetic resonance (NMR) spectroscopy from M. aeruginosa. Of the total extracted P determined by 31 P NMR spectroscopy, 47% were found to be nucleotides while only 11% were polyphosphates. Multimodal X-ray imaging provides a better understanding of intracellular biochemical processes in cyanobacteria, helping us monitor and combat an emerging environmental threat.

31P NMR spectroscopy↗

DOE OSTI2022

Why it is Unfortunate that Linear Machine Learning “Works” so well in Electromechanical Switching of Ferroelectric Thin Films

Machine learning (ML) is relied on for materials spectroscopy. It is challenging to make ML models fail because statistical correlations can mimic the physics without causality. Here, using a benchmark band-excitation piezoresponse force microscopy polarization spectroscopy (BEPS) dataset the pitfalls of the so-called “better”, “faster”, and “less-biased” ML of electromechanical switching are demonstrated and overcome. Using a toy and real experimental dataset, it is demonstrated how linear nontemporal ML methods result in physically reasonable embedding (eigenvalues) while producing nonsensical eigenvectors and generated spectra, promoting misleading interpretations. A new method of unsupervised multimodal hyperspectral analysis of BEPS is demonstrated using long-short-term memory (LSTM) β-variational autoencoders (β-VAEs) . By including LSTM neurons, the ordinal nature of ferroelectric switching is considered. Further, to improve the interpretability of the latent space, a variational Kullback–Leibler-divergency regularization is imposed . Finally, regularization scheduling of β as a disentanglement metric is leveraged to reduce user bias. Combining these experiment-inspired modifications enables the automated detection of ferroelectric switching mechanisms, including a complex two-step, three-state one. Ultimately, this work provides a robust ML method for the rapid discovery of electromechanical switching mechanisms in ferroelectrics and is applicable to other multimodal hyperspectral materials spectroscopies.

36 MATERIALS SCIENCE↗

DOE OSTI2024

Unveiling the nanoscale architectures and dynamics of protein assembly with in situ atomic force microscopy

Proteins play a vital role in different biological processes by forming complexes through precise folding with exclusive inter- and intra-molecular interactions. Understanding the structural and regulatory mechanisms underlying protein complex formation provides insights into biophysical processes. Furthermore, the principle of protein assembly gives guidelines for new biomimetic materials with potential applications in medicine, energy, and nanotechnology. Atomic force microscopy (AFM) is a powerful tool for investigating protein assembly and interactions across spatial scales (single molecules to cells) and temporal scales (milliseconds to days). It has significantly contributed to understanding nanoscale architectures, inter- and intra-molecular interactions, and regulatory elements that determine protein structures, assemblies, and functions. This review describes recent advancements in elucidating protein assemblies with in situ AFM. We discuss the structures, diffusions, interactions, and assembly dynamics of proteins captured by conventional and high-speed AFM in near-native environments and recent AFM developments in the multimodal high-resolution imaging, bimodal imaging, live cell imaging, and machine-learning-enhanced data analysis. These approaches show the significance of broadening the horizons of AFM and enable unprecedented explorations of protein assembly for biomaterial design and biomedical research.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Aerosol Composition, Mixing State, and Phase State of Free Tropospheric Particles and Their Role in Ice Cloud Formation

The prediction of ice cloud formation in the atmosphere remains challenging. Free tropospheric aerosols can act as ice nucleating particles (INPs), affecting cloud properties and precipitation. The physicochemical properties of free tropospheric particles are modified upon long-range transport by different atmospheric processes. These modifications affect the ice formation potential of individual particles. We investigated free tropospheric particles collected at the remote Pico Mountain Observatory at 2225m a.s.l. in the North Atlantic Ocean using multimodal microspectroscopy and chemical imaging techniques. We probed their ice nucleation activity using an ice nucleation stage interfaced with an environmental scanning electron microscope. Retroplume analysis, chemical imaging, and micro-spectroscopy analysis indicated that the size-resolved chemical composition, mixing state, and phase state of the articles with similar aging times but different transport patterns were substantially different. Relative humidity-dependent glass transition temperatures estimated from meteorological conditions were consistent with the observed organic component of the particles' phase. More viscous (solid and semi-solid-like) particles are more ice active in deposition mode at temperature ranging from 205-220 K than those less viscous particles. Finally, this study provides a better understanding of the phase and mixing state of long range transported free tropospheric aerosols and their role in ice cloud formation.

54 ENVIRONMENTAL SCIENCES↗