SEARCH · Engineering Papers

Results for “multimodal imaging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

All sources NASA NTRS DOE OSTI

NASA NTRS1994

Development Status of Cassini Radar For Remote Sensing of Titan

Cassini Radar is a multimode rada instrument designed to probe the optically inaccessible surface of Titan, Saturn's largest moon. The individual modes will allow surface imaging, surface emissivity measurements. Recently, the breadboard model of this instrument was built and has undergone a series of functional and perfomance tests. The results obtained from these tests indicate that the instrument design is satisfactory and that the various required performance parameters are suffieciently met.

Cassini Radar Titan↗

DOE OSTI2025

Proximity ferroelectricity in wurtzite heterostructures

Proximity ferroelectricity is an interface-associated phenomenon in electric-field-driven polarization reversal in a non-ferroelectric polar material induced by one or more adjacent ferroelectric materials. Here, in this paper, we report proximity ferroelectricity in wurtzite ferroelectric heterostructures. In the present case, the non-ferroelectric layers are AlN and ZnO, whereas the ferroelectric layers are Al 1−x B x N, Al 1−x Sc x N and Zn 1−x Mg x O. The layered structures include nitride–nitride, oxide–oxide and nitride–oxide stacks that feature two-layer (asymmetric) and three-layer (symmetric) configurations. Ferroelectric switching in both layers is validated by multimodal characterization methods, including polarization hysteresis, anisotropic chemical etching, second harmonic generation, piezo response force microscopy, electromechanical testing and atomic resolution polarization orientation imaging in real space by scanning transmission electron microscopy. We present a physical switching model in which antipolar nuclei originate in the ferroelectric layer and propagate towards the internal non-ferroelectric interface. The domain wall leading edge produces elastic and electric fields that extend beyond the interface at close proximity, reducing the switching barrier in the non-ferroelectric layer, and allowing complete domain propagation without breakdown. Density functional theory calculations of polymorph energies, reversal barriers and domain wall energies support this model. Proximity ferroelectricity enables polarization reversal in wurtzites without the chemical or structural disorder that accompanies elemental substitution, opening new questions and opportunities regarding interface-based ferroelectricity.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Recent Developments in Femtosecond Laser-Enabled TriBeam Systems

Streams of multimodal three-dimensional (3D) and four-dimensional (4D) data are revolutionizing our ability to design and predict the behavior of a broad array of advanced materials systems. Over the last 10 years, a new 3D imaging platform consisting of a femtosecond (fs) pulsed laser coupled with a focused ion beam scanning electron microscope (FIB SEM) has been developed by UC Santa Barbara in collaboration with Thermo Fisher Scientific (formerly FEI). The femtosecond-laser-enabled FIB SEM, called the TriBeam, has become one of the only 3D serial sectioning methods available that can gather millimeter-scaled multimodal datasets at sub-μm voxel resolutions; these length scales are critical for many materials problems. Multimodal chemical, crystallographic, and morphological information can be gathered rapidly on a layer-by-layer basis and reconstructed in 3D. Large (gigabyte to terabyte scale) 3D datasets have been generated for a broad array of materials systems, including metallic alloys, ceramics, biomaterials, polymer- and ceramic-matrix composites, and semiconductors. The research tasks conducted have resulted in a completely new design, operating with a dual-wavelength femtosecond-pulsed laser on a plasma focused ion beam (PFIB) platform.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Integrated transmission electron microscope

An integrated transmission electron microscope comprising multiple electron sources for tuned beams of ultrafast, scanning probe, and parallel illumination in varied beam energies can be alternated within sub-microseconds onto a sample with dynamic ‘transient state’ processes to acquire atomic-scale structural/chemical data with site specificity. The various electron sources and condenser optics enable high-resolution imaging, high-temporal resolution imaging, and chemical imaging, using fast-switching magnets to direct the different electron beams onto a single maneuverable objective pole piece where the sample resides. Such multimodal in situ characterization tools housed in a single microscope have the potential to revolutionize materials science.

Jungjohann, Katherine L.↗

DOE OSTI2024

Hiding-in-Plain-Sight (HiPS) Attack on CLIP for Targetted Object Removal from Images

Machine learning models are known to be vulnerable to adversarial attacks, but prior works have mostly focused on single-modalities. With the rise of large multi-modal models (LMMs) like CLIP, which combine vision and language capabilities, new vulnerabilities have emerged. However, these multimodal targeted attacks aim to completely change the model's output to what the adversary wants. In many realistic scenarios, an adversary might seek to make only subtle modifications to the output, so that the changes go unnoticed by downstream models or even by humans. We introduce Hiding-in-Plain-Sight (HiPS) attacks, a novel class of adversarial attacks that subtly modifies model predictions by selectively concealing target object(s), as if the target object was absent from the scene. We propose two HiPS attack variants, HiPS-cls and HiPS-cap, and demonstrate their effectiveness in transferring to downstream image captioning models, such as CLIP-Cap, for targeted object removal from image captions.

Daw, Arka [ORNL] (ORCID:0009000633191271)↗

DOE OSTI2026

Understanding the structural and morphological effects of synthesis route on NpO 2

The availability of actinide standard materials for use in nuclear safeguard applications is critical, as is thorough characterization thereof. Although accurate trace element compositions and isotopic considerations are paramount for deployment of reference standards, structural characterization is also essential towards accurately describing the chemical form and potential matrix effects in candidate materials. Here, to this end, samples of NpO 2 were synthesized via a direct denitration (DD) method and probed with powder X-ray diffraction (PXRD), Raman spectroscopy, and scanning electron microscopy (SEM) for structural and morphological characterization and comparison with NpO 2 materials produced via modified direct denitration (MDD). PXRD confirmed the bulk identity of NpO 2 , and no additional phases were identified using this method. Analysis of Raman data collected using a 532 nm excitation wavelength indicates that samples are mostly phase pure; however, some variability in spectral features is observed. Analysis of additional spectroscopic data collected with a 785 nm excitation wavelength revealed variability in the relative intensity of spectral features. Raman spectroscopy indicates that the sample is primarily NpO 2 ; however, additional signals indicate possible structural disorder, oxidized species, or potential contributions from other Np phases. To further investigate the possibility of additional phase contributions within the sample of NpO 2 , Raman spectroscopic mapping was employed to examine the homogeneity of the sample produced via DD. From this analysis, we determined that despite variability in the intensity of Raman-active vibrational modes, consistent spectra are obtained throughout the area of the sample investigated. SEM images show aggregates with variable sizes and shapes, with rounded, primary particles possessing an average diameter of approximately 100 nm. Comparison of the results of these multimodal analyses to the literature indicates that the crystal chemical, spectroscopic, and microstructural properties of NpO 2 vary based on synthesis method, even if X-ray diffraction data indicate that the bulk phase is NpO 2 .

Direct denitration↗

DOE OSTI2025

Imaging and spatially resolved mass spectrometry applications in nephrology

The application of spatially resolved mass spectrometry (MS) and MS imaging approaches for studying biomolecular processes in the kidney is rapidly growing. These powerful methods, which enable label-free and multiplexed detection of many molecular classes across omics domains (including metabolites, drugs, proteins and protein post-translational modifications), are beginning to reveal new molecular insights related to kidney health and disease. Further, the complexity of the kidney often necessitates multiple scales of analysis for interrogating biofluids, whole organs, functional tissue units, single cells and subcellular compartments. Various MS methods can generate omics data across these spatial domains and facilitate both basic science and pathological assessment of the kidney. Optimal processes related to sample preparation and handling for different MS applications are rapidly evolving. Emerging technology and methods, improvement of spatial resolution, broader molecular characterization, multimodal and multiomics approaches and the use of machine learning and artificial intelligence approaches promise to make these applications even more valuable in the field of nephology. Overall, spatially resolved MS and MS imaging methods have the potential to fill much of the omics gap in systems biology analysis of the kidney and provide functional outputs that cannot be obtained using genomics and transcriptomic methods.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2025

CAMFeND: Credibility-Aware Multimodal Fake News Detection with Rotational Attention

In the evolving digital landscape, fake news is a significant challenge, influencing public perception and decision-making. Traditional detection approaches focus on single-modal data or simple multimodal fusion, often overlooking deeper interactions and news credibility. We propose a novel model addressing these limitations by introducing rotational attention and news domain information as a feature. Unlike static attention mechanisms, our rotational attention dynamically shifts query, key, and value roles across text and image inputs, enabling richer cross-modal interaction. Incorporating news domain information further enhances the model’s reliability by associating news posts with top domains extracted from Google search results, reducing false detections. This approach assesses both the content and the broader web context in which the news is discussed. Our model outperforms existing state-of-the-art methods by providing deeper, layered multimodal integration and domain information analysis, resulting in a more robust and adaptive fake news detection system.

Gupta, Nidhi↗

DOE OSTI2022

Suppressing simulation bias in multi-modal data using transfer learning

Abstract Many problems in science and engineering require making predictions based on few observations. To build a robust predictive model, these sparse data may need to be augmented with simulated data, especially when the design space is multi-dimensional. Simulations, however, often suffer from an inherent bias. Estimation of this bias may be poorly constrained not only because of data sparsity, but also because traditional predictive models fit only one type of observed outputs, such as scalars or images, instead of all available output data modalities, which might have been acquired and simulated at great cost. To break this limitation and open up the path for multi-modal calibration, we propose to combine a novel, transfer learning technique for suppressing the bias with recent developments in deep learning, which allow building predictive models with multi-modal outputs. First, we train an initial neural network model on simulated data to learn important correlations between different output modalities and between simulation inputs and outputs. Then, the model is partially retrained, or transfer learned, to fit the experiments; a method that has never been implemented in this type of architecture. Using fewer than 10 inertial confinement fusion experiments for training, transfer learning systematically improves the simulation predictions while a simple output calibration, which we design as a baseline, makes the predictions worse. We also offer extensive cross-validation with real and carefully designed synthetic data. The method described in this paper can be applied to a wide range of problems that require transferring knowledge from simulations to the domain of experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2021

Spatially co-registered wide-field nonlinear optical imaging of living and complex biosystems in a total internal reflection geometry

Nonlinear optical microscopy that leverages an objective based total internal reflection (TIR) excitation scheme is an attractive means for rapid, wide-field imaging with enhanced surface sensitivity. Through select combinations of distinct modalities, one can, in principle, access complementary chemical and structural information for various chemical species near interfaces. Here, we report a successful implementation of such a wide-field nonlinear optical microscope system, which combines coherent anti-Stokes Raman scattering (CARS), two-photon fluorescence (TPF), second harmonic generation (SHG), and sum frequency generation (SFG) modalities on the same platform. The intense optical fields needed to drive these high order nonlinear optical processes are achieved through the use of femtosecond pulsed light in combination with the intrinsic field confinement induced by TIR over a large field of view. The performance of our multimodal microscope was first assessed through the experimental determination of its chemical fidelity, intensity and polarization dependences, and spatial resolution using a set of well-defined model systems. Subsequently, its unique capabilities were validated through imaging complex biological systems, including Hydrangea quercifolia pollen grains and Pantoea sp. YR343 bacterial cells. Specifically, the spatial distribution of different molecular groups in the former was visualized via vibrational contrast mechanisms of CARS, whereas co-registered TPF imaging allowed the identification of spatially localized intrinsic fluorophores. We further demonstrate the feasibility of our microscope for wide-field CARS imaging on live cells through independent characterization of cell viability using spatially co-registered TPF imaging. Overall, this approach to TIR enabled wide-field imaging is expected to provide new insights into bacterial strains and their interactions with other species in the rhizosphere in a time-resolved and chemically selective manner.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

Direct Imaging of Asymmetric Interfaces and Electrostatic Potentials inside a Hafnia–Zirconia Ferroelectric Nanocapacitor

In hafnia-based thin-film ferroelectric devices, chemical phenomena during growth and processing, such as oxygen vacancy formation and interfacial reactions, appear to strongly affect device performance. However, the correlation between the structure, chemistry, and electrical potentials at the nanoscale in these devices is not fully known, making it difficult to understand their influence on device properties. Here, we directly image the composition and electrostatic potential with nanometer resolution in the cross section of a nanocrystalline W/Hf 0.5 Zr 0.5 O 2−δ (HZO)/W ferroelectric capacitor using multimodal electron microscopy. This reveals a 1.4 nm wide tungsten suboxide interfacial layer formed at the bottom interface during fabrication, which introduces a potential dip and leads to asymmetric switching fields. Additionally, we compare the measured potentials to DFT calculations and find it is nearly 3 V lower than expected in the HZO, which appears to be caused by oxygen vacancies and a resulting negative built-in potential. In conclusion, these chemical and electrostatic details are important to characterize and tune to achieve high-performance ferroelectric devices.

Defects in solids↗

DOE OSTI2022

High-Throughput Microfluidics Platform for Intracellular Delivery and Sampling of Biomolecules from Live Cells

Nondestructive cell membrane permeabilization systems enable the intracellular delivery of exogenous biomolecules for cell engineering tasks as well as the temporal sampling of cytosolic contents from live cells for the analysis of dynamic processes. Here, in this paper, we report a microwell array format live-cell analysis device (LCAD) that can perform localized-electroporation induced membrane permeabilization, for cellular delivery or sampling, and directly interfaces with surface-based biosensors for analyzing the extracted contents. We demonstrate the capabilities of the LCAD via an automated high-throughput workflow for multimodal analysis of live-cell dynamics, consisting of quantitative measurements of enzyme activity using self-assembled monolayers for MALDI mass spectrometry (SAMDI) and deep-learning enhanced imaging and analysis. By combining a fabrication protocol that enables robust assembly and operation of multilayer devices with embedded gold electrodes and an automated imaging workflow, we successfully deliver functional molecules (plasmid and siRNA) into live cells at multiple time-points and track their effect on gene expression and cell morphology temporally. Furthermore, we report sampling performance enhancements, achieving saturation levels of protein tyrosine phosphatase activity measured from as few as 60 cells, and demonstrate control over the amount of sampled contents by optimization of electroporation parameters using a lumped model. Lastly, we investigate the implications of cell morphology on electroporation-induced sampling of fluorescent molecules using a deep-learning enhanced image analysis workflow.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2025

Mind the gap: Bridging the divide between AI aspirations and the reality of autonomous microscopy

What does materials science look like in the “Age of Artificial Intelligence?” Each material’s domain—synthesis, characterization, and modeling—has a different answer to this question, motivated by unique challenges and constraints. This work focuses on the tremendous potential of autonomous characterization within electron microscopy. We present our recent advancements in developing domain-aware, multimodal models for microscopy analysis capable of describing complex atomic systems. We then address the critical gap between the theoretical promise of autonomous microscopy and its current practical limitations, showcasing recent successes while highlighting the necessary developments to achieve robust, real-world autonomy.

2D materials↗

DOE OSTI2022

Multimodal (Non)Linear Optical Nanoimaging and Nanospectroscopy

This article highlights recent advances in linear and nonlinear spectral nanoimaging. The described developments are motivated by the need to characterize molecular and material systems noninvasively with nanometer spatial and femtosecond temporal resolution. Indeed, the ability to image and chemically characterize heterogeneous interfaces with joint nano–femto resolution is a prerequisite to advancing our fundamental understanding of processes as diverse as heterogeneous catalysis, microbial communication, and energy flow in pristine/defect-containing low-dimensional quantum materials, to name a few. We describe pioneering work and recent demonstrations of (non)linear optical nanoimaging and nanospectroscopy, with an emphasis on high spatial resolution measurements conducted under ambient laboratory conditions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2020

Imaging grain microstructure in a model ceramic energy material with optically generated coherent acoustic phonons

Characterization of microstructure, chemistry and function of advanced energy materials remains a challenge for instrumentation science with tremendous opportunity for new discoveries. This active area of research is starting to make considerable strides with examples ranging from methodologies that employ bright X-rays, electron microscopy and optical spectroscopy. However, further development of instruments, capable of multimodal measurements, is necessary to fully reveal complex microstructure evolution in realistic environments and dynamical systems that are far from equilibrium. In this regard, laser-based instruments have a unique advantage as multiple measurement methodologies are easily combined into a single instrument using standard optics. A new pump-probe method that uses optically generated acoustic phonons is expanding standard optical characterization by providing depth resolved information. Here we report on a noteworthy extension of this pump-probe method to image grain microstructure in ceria. Rich information regarding the orientation of individual crystallites is obtained by monitoring acoustic phonon velocities and noting how the polarization of the probe beam influences the detected signal amplitude. When paired with other optical microscopies, this methodology will provide new perspectives for optical characterization of ceramic materials.

36 MATERIALS SCIENCE↗

DOE OSTI2024

From Data to Insights: A Covariate Analysis of the IARPA BRIAR Dataset for Multimodal Biometric Recognition Algorithms at Altitude and Range

This paper examines covariate effects on fused whole body biometrics performance in the IARPA BRIAR dataset, specifically focusing on UAV platforms, elevated positions, and distances up to 1000 meters. The dataset includes outdoor videos compared with indoor images and controlled gait recordings. Normalized raw fusion scores relate directly to predicted false accept rates (FAR), offering an intuitive means for interpreting model results. A linear model is developed to predict biometric algorithm scores, analyzing their performance to identify the most influential covariates on accuracy at altitude and range. Weather factors like temperature, wind speed, solar loading, and turbulence are also investigated in this analysis. The study found that resolution and camera distance best predicted accuracy and findings can guide future research and development efforts in long-range/elevated/UAV biometrics and support the creation of more reliable and robust systems for national security and other critical domains.

Bolme, David↗

DOE OSTI2026

Neutron Grating Interferometry at the High Flux Isotope Reactor

Neutron imaging is a non-destructive probe that can spatially resolve bulk materials’ internal features/structures. Neutron imaging spatial resolution is driven by detector technology, i.e., pixel size, and is currently on the order of 10–20 µm. This limitation can be alleviated by the use of neutron grating interferometry – an advanced neutron imaging technique – that enables simultaneous access to three contrast mechanisms: attenuation, differential phase, and small-angle neutron scattering. Recently, a Talbot-Lau neutron grating interferometry has been implemented at the Multimodal Advanced Radiography Station at the High Flux Isotope Reactor. In this work, the design, setup and performance of the grating apparatus are presented.

Zhang, Yuxuan [ORNL] (ORCID:0000000200831408)↗

DOE OSTI2025

Towards Content Authenticity: Multimodal Fake News Detection and AI-Generated Text Identification

In today’s digital world, the spread of fake news and the rise of AI-generated text have become major threats to content authenticity and public trust. This thesis addresses both challenges through two complementary research directions: detecting fake news using multimodal features, and identifying AI-generated text using semantic and structural reasoning. The first part of the work focuses on fake news detection by introducing a novel model that combines text and image features through a unique rotational attention mechanism. Unlike traditional attention methods, this approach rotates the roles of query, key, and value across modalities to capture deeper interactions. Additionally, the model incorporates external domain information by linking news posts to top-ranked websites from Google search results, which helps assess the credibility of content based on its broader web context. This results in a more reliable and accurate fake news detection system that outperforms existing state-of-the-art methods. The second part presents SGG-ATD, a new framework for detecting AI-generated text. It uses masked language modeling to measure sentence coherence, followed by constructing a graph where keywords—both original and predicted—are connected based on semantic and contextual similarity. A Graph Convolutional Network (GCN) is then used to learn structural relationships within the text for final classification. Experimental results demonstrate that SGG-ATD achieves high F1-scores and consistently outperforms strong baselines. This method contributes to robust AI text detection, supporting accountability and resilience against AI-driven misinformation.

Gupta, Nidhi↗