Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “multimodal imaging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

In situ plant materials hyperspectral imaging by multimodal scattering near-field optical microscopy

Plant cells are elaborate three-dimensional polymer nano-constructs with complex chemistry. The bulk response of plants to light, in the far-field, is ultimately encoded by optical scattering from these nano-constructs. Their chemical and physical properties may be acquired through their interaction with a modulated nano-tip using scattering scanning near-field optical microscopy. Here, using this technique, we present 20 nm spatial resolution mechanical, spectral and optical mappings of plant cell walls. We first address the problem of plant polymers tracking through pretreatment and processing. Specifically, cellulose and lignin footprints are traced within a set of delignified specimen, establishing the factors hindering complete removal of lignin, an important industrial polymer. Furthermore, we determine the frequency dependent dielectric function ${\epsilon }(\omega)={(n+ik)}^{2}$ of plant material in the range 28 ≤ ω ≤ 58 THz, and show how the environmental chemical variation is imprinted in the nanoscale variability of n and k . This nanometrology is a promise for further progress in the development of plant-based (meta-)materials.

36 MATERIALS SCIENCE↗
DOE OSTI2026

SpaceNet 9—Cross-Sensor Alignment of Optical and SAR Imagery

Precise registration of high-resolution synthetic aperture radar (SAR) and optical imagery is necessary for realizing the full potential and benefits of multimodal image analysis. However, two significant challenges presently exist. First, there is a lack of annotated datasets and benchmarks available for high-resolution SAR–optical image registration. Second, an assessment of efficient and reliable image registration methods that can precisely align these modalities is lacking. Here, we present a holistic description of the SpaceNet 9 Challenge and its results. We present a description of the dataset and baseline algorithm along with the results of the challenge, including a description of the winning algorithms. We release the SpaceNet 9 dataset along with open-sourcing the winning algorithms and baseline. The objective of SpaceNet 9 was to compute a dense displacement map that indicates the shift needed to align pixels in an optical image to the pixels in a SAR image. The challenge launched in April 2025 and was active for approximately two months. The top five solutions reduced image alignment error from approximately 34 m to under 13 m for public and private test data, with the best results obtaining a registration error of only 8.5 and 6.7 m on the public testing and private testing dataset, respectively. Usage of pretrained image matching models, robust outlier rejection with RANSAC, and estimating local displacement were common among the top solutions. The results of this challenge provide insight into high-resolution SAR–optical image registration and offer opportunities for future benchmarking in this domain. The baseline algorithm, winning solutions, and datasets are available at https://spacenet.ai/sn9-challenge/.

benchmark datasets↗
DOE OSTI2022

Multimodal correlative imaging and modelling of phosphorus uptake from soil by hyphae of mycorrhizal fungi

Phosphorus (P) is essential for plant growth. Arbuscular mycorrhizal fungi (AMF) aid its uptake by acquiring P from sources distant from roots in return for carbon. Little is known about how AMF colonise soil pore-space, and models of AMF-enhanced P-uptake are poorly validated. We used synchrotron X-ray computed tomography to visualize mycorrhizas in soil and synchrotron X-ray fluorescence/X-ray absorption near edge structure (XRF/XANES) elemental mapping for P, sulphur (S) and aluminium (Al) in combination with modelling. We found that AMF inoculation had a suppressive effect on colonisation by other soil fungi and identified differences in structure and growth rate between hyphae of AMF and nonmycorrhizal fungi. Our results showed that AMF co-locate with areas of high P and low Al, and preferentially associate with organic-type P species over Al-rich inorganic P. We discovered that AMF avoid Al-rich areas as a source of P. Sulphur-rich regions were found to be correlated with higher hyphal density and an increased organic-associated P-pool, whilst oxidized S-species were found close to AMF hyphae. Increased S oxidation close to AMF suggested the observed changes were microbiome-related. Our experimentally-validated model led to an estimate of P-uptake by AMF hyphae that is an order of magnitude lower than rates previously estimated – a result with significant implications for the modelling of plant–soil–AMF interactions.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Correlative multimodal chemical imaging via machine learning

Machine learning approach can combine mass spectral imaging (MSI) techniques, one with low spatial resolution but intact molecular spectra and the other with nanometer spatial resolution but fragmented molecular signatures, to predict molecular MSI spectra with submicron spatial resolution. The machine learning approach can perform transformations on the spectral image data of the two MSI techniques to reduce dimensionality, and using a correlation technique, find relationships between the transformed spectral image data. The determined relationships can be used to generate MSI spectra of desired resolution.

Ovchinnikova, Olga S.↗
DOE OSTI2020

Nematic transitions in iron pnictide superconductors imaged with a quantum gas

The Scanning Quantum Cryogenic Atom Microscope (SQCRAMscope) uses an atomic Bose–Einstein condensate to measure magnetic fields emanating from solid-state samples. The quantum sensor does so with unprecedented d.c. sensitivity at micrometre resolution, from room to cryogenic temperatures. An additional advantage of the SQCRAMscope is the preservation of optical access to the sample so that magnetometry imaging of, for example, electron transport may be performed in concert with other imaging techniques. Here, we apply this multimodal imaging capability to the study of nematicity in iron pnictide high-temperature superconductors, where the relationship between electronic and structural symmetry breaking resulting in a nematic phase is under debate. We combine the SQCRAMscope with an in situ microscope that measures optical birefringence near the surface. This enables simultaneous and spatially resolved detection of both bulk and near-surface manifestations of nematicity via transport and structural deformation channels, respectively. By performing local measurements of emergent resistivity anisotropy in iron pnictides, we observe sharp, nearly concurrent transport and structural transitions. More broadly, these measurements demonstrate the SQCRAMscope’s ability to reveal important insights into the physics of complex quantum materials.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2021

Airy light-sheet Raman imaging

Light-sheet fluorescence microscopy has greatly improved the speed and overall photostability of optically sectioning cellular and multi-cellular specimens. Similar gains have also been conferred by light-sheet Raman imaging; these schemes, however, rely on diffraction limited Gaussian beams that hinder the uniformity and size of the imaging field-of-view, and, as such, the resulting throughput rates. Here, we demonstrate that a digitally scanned Airy beam increases the Raman imaging throughput rates by more than an order of magnitude than conventional diffraction-limited beams. Overall, this, spectrometer-less, approach enabled 3D imaging of microparticles with high contrast and 1 µm axial resolution at 300 msec integration times per plane and orders of magnitude lower irradiation density than coherent Raman imaging schemes. We detail the apparatus and its performance, as well as its compatibility with fluorescence light-sheet and quantitative-phase imaging towards rapid and low phototoxicity multimodal imaging.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Multimodal Nanoscale Tomographic Imaging for Battery Electrodes

Accurate representations of the 3D structure within a lithium-ion battery are key to understanding performance limitations. However, obtaining exact reconstructions of electrodes, where the active particles, the carbon black and polymeric binder domain, and the pore space are visualized is challenging. In this work, it is shown that multimodal imaging can be used to overcome this challenge. High-resolution ptychographic X-ray computed tomography are combined with lower resolution but higher contrast transmission X-ray tomographic microscopy to obtain 3D reconstructions of pristine and cycled graphite-silicon composite electrodes. This cross-correlation enables quantitative analysis of the surface of active particles, including the heterogeneity of carbon-black and binder domain and solid-electrolyte interphase coverage. Capturing the active particles as well as the carbon black-binder domain allows using these segmented structures for electrochemical simulations to highlight the influence of the particle embedding on local state of charge heterogeneities.

25 ENERGY STORAGE↗
DOE OSTI2026

FIRM: federated image reconstruction using multimodal tomographic data

Here, we propose a federated algorithm for reconstructing images using multimodal tomographic data sourced from dispersed locations, addressing the challenges of traditional unimodal approaches that are prone to noise and reduced image quality, as well as the limitations of centralized multimodal approaches that require extensive data transfer, leading to significant communication overhead, storage demands, and potential data privacy concerns. Our approach formulates a joint inverse optimization problem incorporating multimodality constraints and solves it in a federated framework through local gradient computations complemented by lightweight central operations, thereby ensuring data decentralization. Leveraging the connection between our federated algorithm and the quadratic penalty method, we introduce an adaptive step-size rule with guaranteed sublinear convergence. Numerical results demonstrate superior computational efficiency and improved image reconstruction quality compared to existing approaches.

federated algorithm↗
DOE OSTI2023

Multimodal nonlinear optical imaging via evanescent wave excitation

A system for multimodal nonlinear optical imaging is provided. Each mode uses a high NA objective to cause total internal reflection excitation at a sample-substrate interface. The system has a femtosecond oscillator to generate pulses used for two beams. The objective receives at least one beam, redirects the received at least one beam through a dielectric substrate to cause the TIR and produces corresponding evanescent waves in a portion of the sample adjacent to the sample-substrate interface, and collects a backward-propagating beam of pulses of responsive light. The portion of the sample illuminated by the evanescent waves emits responsive light. Different modes or combinations of the distinct modalities may be selected to access complementary chemical and structural information for various chemical species near the sample-substrate interface. Each mode may have mode-specific control such as selective beam blocking, power ratios and filtering.

Doughty, Benjamin L.↗
NASA NTRS1999

Multimode-Optical-Fiber Imaging Probe

Currently, endoscopic surgery uses single-mode fiber-bundles to obtain in vivo image information inside the orifices of the body. This limits their use to the larger natural orifices and to surgical procedures where there is plenty of room for manipulation. The knee joint, for example, can be easily viewed with a fiber optic viewer, but joints in the finger cannot. However, there are a host of smaller orifices where fiber endoscopy would play an important role if a cost effective fiber probe were developed with small enough dimensions (less than or equal to 250 microns). Examples of beneficiaries of micro-endoscopes are the treatment of the Eustatian tube of the middle ear, the breast ducts, tear ducts, coronary arteries, fallopian tubes, as well as the treatment of salivary duct parotid disease, and the neuro endoscopy of the ventricles and spinal canal. This work describes an approach for recovering images from tightly confined spaces using multimode. The concept draws upon earlier works that concentrated on image recovery after two-way transmission through a multimode fiber as well as work that demonstrated the recovery of images after one-way transmission through a multimode fiber. Both relied on generating a phase conjugated wavefront, which was predistorted with the characteristics of the fiber. The approach described here also relies on generating a phase conjugated wavefront, but utilizes two fibers to capture the image at some intermediate point (accessible by the fibers, but which is otherwise visually inaccessible).

Jackson, Deborah↗
NASA NTRS2000

Multimode-Optical-Fiber Imaging Probe

Currently, endoscopic surgery uses single-mode fiber-bundles to obtain in vivo image information inside orifices of the body. This limits their use to the larger natural bodily orifices and to surgical procedures where there is plenty of room for manipulation. The knee joint, for example can be easily viewed with a fiber optic viewer, but joints in the finger cannot. However, there are a host of smaller orifices where fiber endoscopy would play an important role if a cost effective fiber probe were developed with small enough dimensions (< 250 microns). Examples of beneficiaries of micro-endoscopes are the treatment of the Eustatian tube of the middle ear, the breast ducts, tear ducts, coronary arteries, fallopian tubes, as well as the treatment of salivary duct parotid disease, and the neuro endoscopy of the ventricles and spinal canal. To solve this problem, this work describes an approach for recovering images from. tightly confined spaces using multimode fibers and analytically demonstrates that the concept is sound. The proof of concept draws upon earlier works that concentrated on image recovery after two-way transmission through a multimode fiber as well as work that demonstrated the recovery of images after one-way transmission through a multimode fiber. Both relied on generating a phase conjugated wavefront which was predistorted with the characteristics of the fiber. The described approach also relies on generating a phase conjugated wavefront, but utilizes two fibers to capture the image at some intermediate point (accessible by the fibers, but which is otherwise visually unaccessible).

Jackson, Deborah↗
DOE OSTI2025

Ultra-high Information-content Chemical Imaging with Broadband Coherent Anti-Stokes Raman and Two-photon Fluorescence Lifetime Microscopy

Raman fingerprint spectroscopy and fluorescence lifetime imaging are emerging tools for studying metabolic profiles of biological specimens. While Raman fingerprint spectroscopy detects intrinsic molecular vibrations that reflect the molecular composition and chemical environment of a sample, fluorescence lifetime imaging measures changes in the excited-state lifetime of fluorophores that are sensitive to their microenvironments. Here, we present a multimodal imaging platform combining broadband coherent anti-Stokes Raman scattering (BCARS) and two-photon fluorescence lifetime imaging (2p-FLIM) microscopy that can acquire biologically relevant Raman fingerprint spectra and fluorescence lifetime signals in vivo and simultaneously. The tremendous chemical information obtained from spatially co-registered BCARS and 2p-FLIM images allows us to characterize the subtle differences between sub-cellular compartments and verify the potential false-positive results generated by fluorescence imaging alone. This is demonstrated by directly comparing the BCARS, 2p-FLIM, and two-photon excitation fluorescence(TPEF) signals simultaneously obtained from the same dye-stained organelle in the live, intact C. elegans expressing a green fluorescent protein (GFP) marker. In this work, we introduce the BCARS/2p-FLIM/TPEF setup scheme, the image acquisition steps, data processing, and representative results showing that the cross-modality imaging method enables rigorous characterization and in vivo detection at sub-cellular resolution. Furthermore, this protocol provides a framework for simultaneous chemical and fluorescence lifetime imaging to improve the accuracy of biological interpretation in complex living systems.

Xu, Haoyu [Georgia Institute of Technology, Atlant↗
DOE OSTI2025

Molecular Imaging of Microbially Induced Corrosion of Synthetic Archeological Glasses by a Rhizosphere Bacterium

Microbially induced corrosion (MIC) focuses on the degradation of solid materials, such as glass or metal. Soil microbes are often associated with the corrosion of foreign objects in the rhizosphere. Paenibacillus polymyxa SCE2, a facultative anaerobic bacterium in soil, is of the same genus as bacteria found near nuclear waste disposal sites. Time-of-flight secondary ion mass spectrometry (ToF-SIMS) was used for imaging surface changes induced by P. polymyxa SCE2 cultured on two synthetic glass coupons to represent natural analogs of materials that were studied in relation to the vitrification of nuclear waste. Multimodal imaging was used to verify bacterial coverage across the glass surface after long-term growth. ToF-SIMS spectral analysis showed detection of glass component ions, such as silicon oxide (m/z – 59.96 SiO 2 – ) and aluminum oxide (m/z – 101.95 Al 2 O 3 – ), and biofilm’s extracellular polymeric substance (EPS) components, such as pentadecanoic acid (m/z – 241.22 C 15 H 29 O 2 – ) and sterol lipids (m/z – 311.16 C 20 H 23 O 3 – ). ToF-SIMS spectral, imaging, and depth profiling analyses showed that the glass rich in silica and other light elements (“granite glass”) had more “corrosion related” peaks than the glass that was less silica-rich and contained more iron (“dike glass”). Furthermore, these surface and interface compositional and spatial differences observed in the mass spectra and imaging were attributed to bacterial metabolism and an electron transfer mechanism influenced by morphological and compositional differences between the two types of glasses. ToF-SIMS is effective in studying microbial effects, bringing new molecular insights into MIC in a broader context of materials degradation.

Amorphous materials↗
DOE OSTI2022

Multimodal X-ray nano-spectromicroscopy analysis of chemically heterogeneous systems

Abstract Understanding the nanoscale chemical speciation of heterogeneous systems in their native environment is critical for several disciplines such as life and environmental sciences, biogeochemistry, and materials science. Synchrotron-based X-ray spectromicroscopy tools are widely used to understand the chemistry and morphology of complex material systems owing to their high penetration depth and sensitivity. The multidimensional (4D+) structure of spectromicroscopy data poses visualization and data-reduction challenges. This paper reports the strategies for the visualization and analysis of spectromicroscopy data. We created a new graphical user interface and data analysis platform named XMIDAS (X-ray multimodal image data analysis software) to visualize spectromicroscopy data from both image and spectrum representations. The interactive data analysis toolkit combined conventional analysis methods with well-established machine learning classification algorithms (e.g. nonnegative matrix factorization) for data reduction. The data visualization and analysis methodologies were then defined and optimized using a model particle aggregate with known chemical composition. Nanoprobe-based X-ray fluorescence (nano-XRF) and X-ray absorption near edge structure (nano-XANES) spectromicroscopy techniques were used to probe elemental and chemical state information of the aggregate sample. We illustrated the complete chemical speciation methodology of the model particle by using XMIDAS. Next, we demonstrated the application of this approach in detecting and characterizing nanoparticles associated with alveolar macrophages. Our multimodal approach combining nano-XRF, nano-XANES, and differential phase-contrast imaging efficiently visualizes the chemistry of localized nanostructure with the morphology. We believe that the optimized data-reduction strategies and tool development will facilitate the analysis of complex biological and environmental samples using X-ray spectromicroscopy techniques.

36 MATERIALS SCIENCE↗
DOE OSTI2026

FATHOMS-RAG: A Framework for the Assessment of Thinking and Observation in Multimodal Systems that use Retrieval Augmented Generation

Retrieval-augmented generation (RAG) has emerged as a promising paradigm for improving factual accuracy in large language models (LLMs). We introduce a benchmark designed to evaluate RAG pipelines as a whole, evaluating a pipelines ability to ingest several modalities of information. We present (1) a curated dataset of 93 questions designed to evaluate a pipeline's ability to ingest textual data, tables, images, multimodal data, and cross-document multimodal data; (2) a phrase-level recall metric for correctness; (3) a nearest-neighbor embedding classifier in an attempt to classify pipeline hallucinations; (4) a comparative evaluation of 2 pipelines built with open-source retrieval mechanisms and 4 closed-source foundational models; and (5) a third-party human evaluation of the alignment of our correctness and hallucination metrics. We find that closed-source pipelines significantly outperform open-source pipelines in both the correctness and halucination metrics, with a wider performance gap in questions relying on multimodal and cross-document information. We also find after a human evaluation of our correctness and hallucination metric compared with our questions and pipeline responses, average agreement was 4.62 for correctness 4.53 for hallucination detection on a 1-5 Likert scale with 5 being strongly agree with our determination.

Hildebrand, Samuel [ORNL] (ORCID:0009000465963104)↗