Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “multimodal imaging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Imaging from Macro to Nanoscale: Multimodal Advances in Chemical and Biomedical Imaging

Imaging increasingly serves as a multiscale framework for linking molecular mechanisms to cellular behavior, tissue architecture, and organ phenotypes in biology and unraveling fundamental processes in chemistry, physics and materials science. This Perspective highlights recent advances in chemical and biomedical imaging across macro-, micro-, and nanoscales, using representative examples published in Chemical and Biomedical Imaging (CBMI). At the macroscale, we discuss chemically selective MRI, including endogenous and exogenous CEST strategies, together with photoacoustic imaging as a hybrid modality with functional and chemical contrast. At the microscale, we consider fluorescence, label-free optical and vibrational imaging, and selected X-ray approaches that expand sensitivity, specificity, and temporal resolution in biological and materials systems. At the nanoscale, we highlight super-resolution fluorescence microscopy, single-molecule methods, tip-enhanced Raman spectroscopy, and correlative imaging strategies that resolve local heterogeneity and molecular organization. Across scales, a common theme emerges that advances in probes, contrast mechanisms, instrumentation, and sample handling are enabling chemically informed imaging that connects molecular specificity with biological context.

multiscale imaging↗
NASA NTRS2003

Parallel Information Processing (Image Transmission Via Fiber Bundle and Multimode Fiber

Growing demand for visual, user-friendly representation of information inspires search for the new methods of image transmission. Currently used in-series (sequential) methods of information processing are inherently slow and are designed mainly for transmission of one or two dimensional arrays of data. Conventional transmission of data by fibers requires many fibers with array of laser diodes and photodetectors. In practice, fiber bundles are also used for transmission of images. Image is formed on the fiber-optic bundle entrance surface and each fiber transmits the incident image to the exit surface. Since the fibers do not preserve phase, only 2D intensity distribution can be transmitted in this way. Each single mode fiber transmit only one pixel of an image. Multimode fibers may be also used, so that each mode represent different pixel element. Direct transmission of image through multimode fiber is hindered by the mode scrambling and phase randomization. To overcome these obstacles wavelength and time-division multiplexing have been used, with each pixel transmitted on a separate wavelength or time interval. Phase-conjugate techniques also was tested in, but only in the unpractical scheme when reconstructed image return back to the fiber input end. Another method of three-dimensional imaging over single mode fibers was demonstrated in, using laser light of reduced spatial coherence. Coherence encoding, needed for a transmission of images by this methods, was realized with grating interferometer or with the help of an acousto-optic deflector. We suggest simple practical holographic method of image transmission over single multimode fiber or over fiber bundle with coherent light using filtering by holographic optical elements. Originally this method was successfully tested for the single multimode fiber. In this research we have modified holographic method for transmission of laser illuminated images over commercially available fiber bundle (fiber endoscope, or fiberscope).

Kukhtarev, Nicholai↗
DOE OSTI2021

Paramagnetic Mn 8 Fe 4 -co-Polystyrene Nanobeads as a Potential T 1 –T 2 Multimodal Magnetic Resonance Imaging Contrast Agent with In Vivo Studies

In developing a cluster-nanocarrier design, as a magnetic resonance imaging contrast agent, we have investigated the enhanced relaxivity of a manganese and iron-oxo cluster grafted within a porous polystyrene nanobead with increased relaxivity due to a higher surface area. The synthesis of the cluster-nanocarrier for the cluster Mn 8 Fe 4 O 12 (O 2 CC 6 H 4 CH=CH 2 ) 16 (H 2 O) 4 , cross-linked with polystyrene (the nano-carrier), under miniemulsion conditions is described. By including a branched hydrophobe, iso-octane, the resulting nanobeads are porous and similar to 70 nm in diameter. The increased surface area of the nanobeads compared to nonporous nanobeads leads to an enhancement in relaxivity; r 1 increases from 3.8 to 5.2 ± 0.1 mM -1 s -1 , and r 2 increases from 11.9 to 50.1 ± 4.8 mM -1 s -1 , at 9.4 teslas, strengthening the potential for T 1 and T 2 imaging. Several metrics were used to assess stability, and the porosity produced no reduction in metal stability. Synchrotron X-ray fluorescence microscopy was used to demonstrate that the nanobeads remain intact in vivo . In depth, physicochemical characteristics were determined, including extensive pharmacokinetics, in vivo imaging, and systemic biodistribution analysis.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Multimodal Mass Spectrometry Imaging (MSI) of Archean and Jurassic Geologic Samples

Current geochemical analytical techniques such as gas chromatography-mass spectrometry (GC-MS and $G\acute{C}GC-MS$) can effectively determine the composition and structure of the organic biosignatures within a sample but cannot resolve the spatial distribution of organic biosignatures within rocks or sediments. The spatial distribution can be used to determine the character of the organic biosignatures [1], which can be indigenous (deposited with host rocks), non-indigenous (incorporated after deposition via fluid migration), or contaminant. Determining the character of organic biosignatures is critical for ancient samples [2] and will be critical if organics are observed in samples returned from Mars [3].Fs-LDPI-MS has the ability to map organic compounds across the surface of samples at 2 - 10 μm lateral resolution. Furthermore, fs-LDPI-MS can be used to carry out multiple analyses in the same location for micron-scale analysis of previously buried material [4]. We used fs-LDPI-MS to examine a ~164 million year old organic-rich mudstone from SW England (14.2 wt. % total organic carbon) [5] and determined the spatial distribution of likely indigenous components buried below potentially contaminated surface layers.We then used ToF-SIMS to re-analyze the mudstone to directly compare ToF-SIMS to fs-LDPI-MS datasets, and to analyze a series of ~2.7 billion year old geologic samples from Timmins, ON, CA [6]. Ref. [6] previously analyzed the Archean samples and observed a series of archaeal biomarkers, as well as hopanes and steranes. Using ToF-SIMS analysis, we were able to detect hopanes and steranes in the mudstone samples, but not in the Archean samples indicating that the hopanes and steranes are most likely contaminants in the Archean samples. We also did not observe the archaeal biomarkers; however, this may be from the lack of molecular ion preservation during ToF-SIMS.

Pasterski, Michael↗
DOE OSTI2025

Predicting High‐Resolution Spatial and Spectral Features in Mass Spectrometry Imaging with Machine Learning and Multimodal Data Fusion

Recent advancements in molecular Mass Spectrometry Imaging have sparked interest in integrating high spatial resolution methods with molecular mass-spectrometry-based chemical imaging. Fusion-based algorithms have proven effective in generating high spatial-resolution molecular mass spectra. However, a significant challenge stems from the differing physical mechanisms underlying image generation and data upsampling techniques, potentially leading to discrepancies in integrated information channels. Integrating physical constraints into data processing workflows is essential to tackle this issue. In this study, we propose an innovative approach that merges data from Fourier transform ion cyclotron resonance (FTICR), time-of-flight matrix-assisted laser desorption/ionization, and time-of-flight secondary ion mass spectrometry imaging techniques. By leveraging FT-ICR's unparalleled spectral resolution and ToF-SIMS's exceptional spatial resolution, we achieve submicron spatial resolution, enabling the observation of intact molecular species with remarkable spectral precision. Canonical correlation analysis is employed to incorporate physical constraints. Through sophisticated image processing and machine learning techniques, the results of this fusion hold significant promise for advancing our comprehension of complex systems and unveiling concealed molecular intricacies.

canonical correlation analysis↗
DOE OSTI2023

Multimodal photon and neutron imaging devices with fast event-by-event multisampling readout and optimized particle type discrimination

A particle imaging method for distinguishing between types of incident particles, such as neutrons, photons, and alphas, and improving the position resolution of particle imaging devices with matrix readout. The method includes high frequency multisampling readout electronics that provides the sequences of multiple measurements for each detected event, resulting in recorded detailed waveform information describing the signals. Such detailed information is used to approximate each signal waveform with a parameterized function in which the extracted parameter sets determine the type of the incident particle in an optimized fashion. The detailed event-by-event multisampling information for each signal readout channel in the matrix readout of the radiation imaging devices improves and optimizes the position resolution for variable shapes of the signals. Such devices can be used in mixed radiation fields, creating a new class of multimodal photon and neutron imagers.

Degtiarenko, Pavel V.↗
DOE OSTI2025

Exploring Capability of Multimodal Foundation Model for Image-based Fault Detection of Photovoltaic Modules

Multimodal Foundation Model (MFM), like ChatGPT and Gemini, have emerged as powerful tools for their exceptional natural language processing capabilities and their emerging potential in image analysis. This paper investigates the application of MFMs for photovoltaic (PV) fault detection through image analysis, focusing on ChatGPT 4.0 and Gemini 1.5 Pro. Three types of PV images and the corresponding common PV faults are detected: bird droppings using visible images, cell cracks via electroluminescence (EL) images, and hotspots using infrared (IR) images. Among the two models, Gemini 1.5 Pro demonstrated superior performance, achieving near-perfect results with an average F1 score of 0.97, consistently outperforming ChatGPT 4.0 in accuracy and reliability. Unlike traditional machine learning (ML) models, MFMs can operate in a zero shot manner that does not require additional training by the user, and the input images are not limited by size, angle, scope, or PV technology. The strong adaptability and user-friendliness make MFM a promising tool for analyzing PV images and advancing health monitoring for PV modules.

Li, Baojie↗
DOE OSTI2021

Universal image representation based on a multimodal graph

A system for classifying a target image with segments having attributes is provided. The system generates a graph for the target image that includes vertices representing segments of the image and edges representing relationships between the connected vertices. For each vertex, the system generates a subgraph that includes the vertex as a home vertex and neighboring vertices representing segments of the target image within a neighborhood of the segment represented by the home vertex. The system applies an autoencoder to each subgraph to generate latent variables to represent the subgraph. The system applies a machine learning algorithm to a feature vector comprising a universal image representation of the target image that is derived from the generated latent variables of the subgraphs to generate a classification for the target image.

Bremer, Peer-Timo↗
DOE OSTI2023

Universal image representation based on a multimodal graph

A system for classifying a target image with segments having attributes is provided. The system generates a graph for the target image that includes vertices representing segments of the image and edges representing relationships between the connected vertices. For each vertex, the system generates a subgraph that includes the vertex as a home vertex and neighboring vertices representing segments of the target image within a neighborhood of the segment represented by the home vertex. The system applies an autoencoder to each subgraph to generate latent variables to represent the subgraph. The system applies a machine learning algorithm to a feature vector comprising a universal image representation of the target image that is derived from the generated latent variables of the subgraphs to generate a classification for the target image.

Bremer, Peer-Timo↗
DOE OSTI2023

Deep Learning on Multimodal Chemical and Whole Slide Imaging Data for Predicting Prostate Cancer Directly from Tissue Images

Prostate cancer is one of the most common cancers globally and is the second most common cancer in the male population in the US. Here we develop a study based on correlating the hematoxylin and eosin (H&E)-stained biopsy data with MALDI mass-spectrometric imaging data of the corresponding tissue to determine the cancerous regions and their unique chemical signatures and variations of the predicted regions with original pathological annotations. We obtain features from high-resolution optical micrographs of whole slide H&E stained data through deep learning and spatially register them with mass spectrometry imaging (MSI) data to correlate the chemical signature with the tissue anatomy of the data. We then use the learned correlation to predict prostate cancer from observed H&E images using trained coregistered MSI data. This multimodal approach can predict cancerous regions with ~80% accuracy, which indicates a correlation between optical H&E features and chemical information found in MSI. Further, we show that such paired multimodal data can be used for training feature extraction networks on H&E data which bypasses the need to acquire expensive MSI data and eliminates the need for manual annotation saving valuable time. Two chemical biomarkers were also found to be predicting the ground truth cancerous regions. This study shows promise in generating improved patient treatment trajectories by predicting prostate cancer directly from readily available H&E-stained biopsy images aided by coregistered MSI data.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Direct Multimodal Nanoscale Visualization of Early Phosphorus-Based Antiwear Tribofilm Formation

Understanding the mechanism of antiwear (AW) tribofilm formation and how to tune surface chemistry to control functionality is essential for the development of the next generation of oil lubricants. In particular, understanding and optimizing early AW tribofilm formation can increase the energy efficiency of mechanical systems. However, the mechanism for how these films form is not well understood. The majority of prior work has focused on analyzing only end-of-test surfaces long after the film has formed. In this work, we develop an in situ multimodal chemical imaging methodology to directly visualize the early formation of AW films on steel surfaces. We investigate an oil formulation containing a phosphorus-based additive commonly used to protect surfaces from wear and fatigue processes in machine elements, such as gears, bearings, and sliding contacts. Using nanoscale multimodal chemical imaging on combined platforms of atomic force microscopy (AFM) coupled directly with in situ nano-infrared (nano-IR) spectroscopy, and further combined ex situ with time-of-flight secondary ion mass spectrometry (ToF-SIMS), we demonstrate a direct correlation between changes in friction and local surface chemistry. In these experiments, the AFM probe acts as a single asperity contact to generate the tribofilm as well as a tool to analyze it in situ as it is forming. To verify our in situ measurements, we compare these results to the ex situ ToF-SIMS of macroscale block-on-ring tribometer-formed samples. We report the understanding gained here on how AW films form and how film properties can be modified by tuning the chemistry of the additives will facilitate developing transmission fluids to meet increasing demands for vehicle performance and efficiency.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Applying Multimodal Mass Spectrometry to Image Tumors Undergoing Ferroptosis Following In Vivo Treatment with a Ferroptosis Inducer

Epithelial ovarian cancer (EOC) is the most common form of ovarian cancer. The poor prognosis generally associated with this disease has led to the search for improved therapies such as ferroptosis-inducing agents. Ferroptosis is a form of regulated cell death that is dependent on iron and is characterized by lipid peroxidation. Precise mapping of lipids and iron within tumors exposed to ferroptosis-inducing agents may provide insight into processes of ferroptosis in vivo and ultimately assist in the optimal deployment of ferroptosis inducers in cancer therapy. In this work, we present a method for combining matrix-assisted laser desorption/ionization (MALDI) mass spectrometry imaging (MSI) with secondary ion mass spectrometry (SIMS) to analyze changes in spatial lipidomics and metal composition, respectively, in ovarian tumors following exposure to a ferroptosis inducer. Tumors were obtained by injecting human ovarian cancer tumor-initiating cells into mice, followed by treatment with the ferroptosis inducer erastin. SIMS imaging detected iron accumulation in the tumor tissue, and sequential MALDI-MS imaging of the same tissue section displayed two chemically distinct regions of lipids. Additionally, one region was associated with the iron-rich area detected with SIMS, and the other region encompassed the remainder of the tissue section. Bulk lipidomics confirmed the lipid assignments putatively assigned from the MALDI-MS data. Overall, we demonstrate the ability of multimodal MSI to identify the spatial locations of iron and lipids in the same tissue section and associate these regions with clinical pathology.

60 APPLIED LIFE SCIENCES↗