Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “multimodal imaging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Spatio-temporal ablation dynamics and plasma chemistry of aluminum induced by temporally modulated ytterbium fiber laser

Here, in this work, we studied single-pulse ablation dynamics of a temporally modulated continuous wave laser-material interaction with Al using in situ multimodal time-resolved diagnostics that describe in detail the associated physical and chemical processes. Time-resolved scattering, emission imaging, and optical emission spectroscopy unveiled a sequence of events spread out across three distinct phases: (i) early phase ablation process, associated with particle generation and liquid Al column formation (<20 μs), (ii) secondary detonation when sufficient ejected material is accumulated over the surface (20-50 μs), and (iii) molten liquid Al pool oscillation on the surface, followed by large droplet ejection from the liquid pool (100-500 μs). Atomic Al and AlO were observed with optical emission spectroscopy at different ratios during the entire lifetime of the event, verifying the formation of oxidized Al vapor upon its interaction with air. Morphological and compositional characterization confirmed surface oxidation and material re-solidification in the form of protrusions produced during the irradiation process. This work provides insights into the complex physical and chemical mechanisms of single-pulse ablation in the sub-millisecond laser pulse regime, which are critically important for parameter optimization in a variety of laser processing, microfabrication, and deposition applications.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Automated Grain Boundary (GB) Segmentation and Microstructural Analysis in 347H Stainless Steel Using Deep Learning and Multimodal Microscopy

Austenitic 347H stainless steel offers superior mechanical properties and corrosion resistance required for extreme operating conditions such as high temperature. The change in microstructure due to composition and process variations is expected to impact material properties. Identifying microstructural features such as grain boundaries thus becomes an important task in the process-microstructure-properties loop. Applying convolutional neural network (CNN)-based deep learning models is a powerful technique to detect features from material micrographs in an automated manner. In contrast to microstructural classification, supervised CNN models for segmentation tasks require pixel-wise annotation labels. However, manual labeling of the images for the segmentation task poses a major bottleneck for generating training data and labels in a reliable and reproducible way within a reasonable timeframe. Microstructural characterization especially needs to be expedited for faster material discovery by changing alloy compositions. Here, in this study, we attempt to overcome such limitations by utilizing multimodal microscopy to generate labels directly instead of manual labeling. We combine scanning electron microscopy images of 347H stainless steel as training data and electron backscatter diffraction micrographs as pixel-wise labels for grain boundary detection as a semantic segmentation task. The viability of our method is evaluated by considering a set of deep CNN architectures. We demonstrate that despite producing instrumentation drift during data collection between two modes of microscopy, this method performs comparably to similar segmentation tasks that used manual labeling. Additionally, we find that naïve pixel-wise segmentation results in small gaps and missing boundaries in the predicted grain boundary map. By incorporating topological information during model training, the connectivity of the grain boundary network and segmentation performance is improved. Finally, our approach is validated by accurate computation on downstream tasks of predicting the underlying grain morphology distributions which are the ultimate quantities of interest for microstructural characterization.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Pursuing theranostics: a multimodal architecture approach

Theranostics is a field of nuclear medicine which uses the same targeting vector and chelating system for both a diagnostic and therapeutic radionuclide, allowing for uniformity in imaging and treatment.

Bender, Aidan A. [Nuclear Engineering Program, Uni↗
DOE OSTI2026

Integrated fluorescence light microscopy-guided cryo-focused ion beam-milling for in situ montage cryo-ET

Cryogenic-electron tomography (cryo-ET) permits the in situ visualization of biological macromolecules at the molecular level. Owing to the variable thickness of cells, tissues and organisms, frozen specimens may need to be thinned by cryo-focused ion beam (FIB) milling to produce thin (<500 nm) cryo-lamellae suitable for cryo-ET. Locating regions of interest remains a challenge because untargeted milling can lead to inadvertent ablation and removal of regions of interest. Correlative light and electron microscopy, combined with cryo-FIB milling, can guide the identification of labeled targets in the cellular milieu. Multiple transfers between cryo-imaging instruments, cumbersome correlation algorithms, limited accuracy and low throughput have hindered the routine adoption of cryo-FIB milling within a multimodal correlative workflow for in situ structural biology. Here, in this study, we present a workflow for 3D correlative cryo-fluorescence light microscopy-FIB-ET that streamlines fluorescence light microscopy-guided FIB milling, improving throughput while preserving both structural and contextual information. The complete integration of hardware and software described here minimizes sample contamination from cross-platform exchanges and greatly enhances the efficiency of 3D targeting in cryo-milling. We then describe procedures for implementing montage parallel array cryo-ET (MPACT), which can be easily adapted to any modern life-science transmission electron microscope. MPACT supports high-throughput cryo-ET acquisitions (10 tilt series in 1.5 h) for structure determination and comprehensive contextual understanding of macromolecules within their native surroundings. A complete session from sample preparation to MPACT data processing takes 5−7 d for an individual experienced in both cryo-EM and cryo-FIB milling.

Yang, Jie E. [Univ. of Wisconsin, Madison, WI (Uni↗
DOE OSTI2023

Spatially resolved multimodal vibrational spectroscopy under high pressures

In this perspective, we discuss the potential impact on in situ studies under controlled environments of a novel multimodal spectroscopic technique, optical photothermal infrared + Raman spectroscopy, which enables the simultaneous collection of infrared and Raman scattering spectra, along with hyperspectral imaging and chemical imaging with wavelength-independent sub-500 nm spatial resolution. A brief review of the current literature regarding the O-PTIR technique is presented along with recent work from our own lab on determining the crystallinity of soft and inorganic materials. The results highlight the possibility of resolving differences in the crystallinity of soft materials associated with changes in material processing. Here, we also demonstrate the first reported use of a diamond anvil cell with simultaneous infrared and Raman measurements that showcases, using a high energy material as an example, the potential use of O-PTIR spectroscopy in diamond anvil cell techniques.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Proximity ferroelectricity in wurtzite heterostructures

Proximity ferroelectricity is an interface-associated phenomenon in electric-field-driven polarization reversal in a non-ferroelectric polar material induced by one or more adjacent ferroelectric materials. Here, in this paper, we report proximity ferroelectricity in wurtzite ferroelectric heterostructures. In the present case, the non-ferroelectric layers are AlN and ZnO, whereas the ferroelectric layers are Al 1−x B x N, Al 1−x Sc x N and Zn 1−x Mg x O. The layered structures include nitride–nitride, oxide–oxide and nitride–oxide stacks that feature two-layer (asymmetric) and three-layer (symmetric) configurations. Ferroelectric switching in both layers is validated by multimodal characterization methods, including polarization hysteresis, anisotropic chemical etching, second harmonic generation, piezo response force microscopy, electromechanical testing and atomic resolution polarization orientation imaging in real space by scanning transmission electron microscopy. We present a physical switching model in which antipolar nuclei originate in the ferroelectric layer and propagate towards the internal non-ferroelectric interface. The domain wall leading edge produces elastic and electric fields that extend beyond the interface at close proximity, reducing the switching barrier in the non-ferroelectric layer, and allowing complete domain propagation without breakdown. Density functional theory calculations of polymorph energies, reversal barriers and domain wall energies support this model. Proximity ferroelectricity enables polarization reversal in wurtzites without the chemical or structural disorder that accompanies elemental substitution, opening new questions and opportunities regarding interface-based ferroelectricity.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Recent Developments in Femtosecond Laser-Enabled TriBeam Systems

Streams of multimodal three-dimensional (3D) and four-dimensional (4D) data are revolutionizing our ability to design and predict the behavior of a broad array of advanced materials systems. Over the last 10 years, a new 3D imaging platform consisting of a femtosecond (fs) pulsed laser coupled with a focused ion beam scanning electron microscope (FIB SEM) has been developed by UC Santa Barbara in collaboration with Thermo Fisher Scientific (formerly FEI). The femtosecond-laser-enabled FIB SEM, called the TriBeam, has become one of the only 3D serial sectioning methods available that can gather millimeter-scaled multimodal datasets at sub-μm voxel resolutions; these length scales are critical for many materials problems. Multimodal chemical, crystallographic, and morphological information can be gathered rapidly on a layer-by-layer basis and reconstructed in 3D. Large (gigabyte to terabyte scale) 3D datasets have been generated for a broad array of materials systems, including metallic alloys, ceramics, biomaterials, polymer- and ceramic-matrix composites, and semiconductors. The research tasks conducted have resulted in a completely new design, operating with a dual-wavelength femtosecond-pulsed laser on a plasma focused ion beam (PFIB) platform.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Integrated transmission electron microscope

An integrated transmission electron microscope comprising multiple electron sources for tuned beams of ultrafast, scanning probe, and parallel illumination in varied beam energies can be alternated within sub-microseconds onto a sample with dynamic ‘transient state’ processes to acquire atomic-scale structural/chemical data with site specificity. The various electron sources and condenser optics enable high-resolution imaging, high-temporal resolution imaging, and chemical imaging, using fast-switching magnets to direct the different electron beams onto a single maneuverable objective pole piece where the sample resides. Such multimodal in situ characterization tools housed in a single microscope have the potential to revolutionize materials science.

Jungjohann, Katherine L.↗
DOE OSTI2024

Hiding-in-Plain-Sight (HiPS) Attack on CLIP for Targetted Object Removal from Images

Machine learning models are known to be vulnerable to adversarial attacks, but prior works have mostly focused on single-modalities. With the rise of large multi-modal models (LMMs) like CLIP, which combine vision and language capabilities, new vulnerabilities have emerged. However, these multimodal targeted attacks aim to completely change the model's output to what the adversary wants. In many realistic scenarios, an adversary might seek to make only subtle modifications to the output, so that the changes go unnoticed by downstream models or even by humans. We introduce Hiding-in-Plain-Sight (HiPS) attacks, a novel class of adversarial attacks that subtly modifies model predictions by selectively concealing target object(s), as if the target object was absent from the scene. We propose two HiPS attack variants, HiPS-cls and HiPS-cap, and demonstrate their effectiveness in transferring to downstream image captioning models, such as CLIP-Cap, for targeted object removal from image captions.

Daw, Arka [ORNL] (ORCID:0009000633191271)↗
DOE OSTI2026

Understanding the structural and morphological effects of synthesis route on NpO 2

The availability of actinide standard materials for use in nuclear safeguard applications is critical, as is thorough characterization thereof. Although accurate trace element compositions and isotopic considerations are paramount for deployment of reference standards, structural characterization is also essential towards accurately describing the chemical form and potential matrix effects in candidate materials. Here, to this end, samples of NpO 2 were synthesized via a direct denitration (DD) method and probed with powder X-ray diffraction (PXRD), Raman spectroscopy, and scanning electron microscopy (SEM) for structural and morphological characterization and comparison with NpO 2 materials produced via modified direct denitration (MDD). PXRD confirmed the bulk identity of NpO 2 , and no additional phases were identified using this method. Analysis of Raman data collected using a 532 nm excitation wavelength indicates that samples are mostly phase pure; however, some variability in spectral features is observed. Analysis of additional spectroscopic data collected with a 785 nm excitation wavelength revealed variability in the relative intensity of spectral features. Raman spectroscopy indicates that the sample is primarily NpO 2 ; however, additional signals indicate possible structural disorder, oxidized species, or potential contributions from other Np phases. To further investigate the possibility of additional phase contributions within the sample of NpO 2 , Raman spectroscopic mapping was employed to examine the homogeneity of the sample produced via DD. From this analysis, we determined that despite variability in the intensity of Raman-active vibrational modes, consistent spectra are obtained throughout the area of the sample investigated. SEM images show aggregates with variable sizes and shapes, with rounded, primary particles possessing an average diameter of approximately 100 nm. Comparison of the results of these multimodal analyses to the literature indicates that the crystal chemical, spectroscopic, and microstructural properties of NpO 2 vary based on synthesis method, even if X-ray diffraction data indicate that the bulk phase is NpO 2 .

Direct denitration↗
DOE OSTI2025

Imaging and spatially resolved mass spectrometry applications in nephrology

The application of spatially resolved mass spectrometry (MS) and MS imaging approaches for studying biomolecular processes in the kidney is rapidly growing. These powerful methods, which enable label-free and multiplexed detection of many molecular classes across omics domains (including metabolites, drugs, proteins and protein post-translational modifications), are beginning to reveal new molecular insights related to kidney health and disease. Further, the complexity of the kidney often necessitates multiple scales of analysis for interrogating biofluids, whole organs, functional tissue units, single cells and subcellular compartments. Various MS methods can generate omics data across these spatial domains and facilitate both basic science and pathological assessment of the kidney. Optimal processes related to sample preparation and handling for different MS applications are rapidly evolving. Emerging technology and methods, improvement of spatial resolution, broader molecular characterization, multimodal and multiomics approaches and the use of machine learning and artificial intelligence approaches promise to make these applications even more valuable in the field of nephology. Overall, spatially resolved MS and MS imaging methods have the potential to fill much of the omics gap in systems biology analysis of the kidney and provide functional outputs that cannot be obtained using genomics and transcriptomic methods.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

CAMFeND: Credibility-Aware Multimodal Fake News Detection with Rotational Attention

In the evolving digital landscape, fake news is a significant challenge, influencing public perception and decision-making. Traditional detection approaches focus on single-modal data or simple multimodal fusion, often overlooking deeper interactions and news credibility. We propose a novel model addressing these limitations by introducing rotational attention and news domain information as a feature. Unlike static attention mechanisms, our rotational attention dynamically shifts query, key, and value roles across text and image inputs, enabling richer cross-modal interaction. Incorporating news domain information further enhances the model’s reliability by associating news posts with top domains extracted from Google search results, reducing false detections. This approach assesses both the content and the broader web context in which the news is discussed. Our model outperforms existing state-of-the-art methods by providing deeper, layered multimodal integration and domain information analysis, resulting in a more robust and adaptive fake news detection system.

Gupta, Nidhi↗
DOE OSTI2022

Suppressing simulation bias in multi-modal data using transfer learning

Abstract Many problems in science and engineering require making predictions based on few observations. To build a robust predictive model, these sparse data may need to be augmented with simulated data, especially when the design space is multi-dimensional. Simulations, however, often suffer from an inherent bias. Estimation of this bias may be poorly constrained not only because of data sparsity, but also because traditional predictive models fit only one type of observed outputs, such as scalars or images, instead of all available output data modalities, which might have been acquired and simulated at great cost. To break this limitation and open up the path for multi-modal calibration, we propose to combine a novel, transfer learning technique for suppressing the bias with recent developments in deep learning, which allow building predictive models with multi-modal outputs. First, we train an initial neural network model on simulated data to learn important correlations between different output modalities and between simulation inputs and outputs. Then, the model is partially retrained, or transfer learned, to fit the experiments; a method that has never been implemented in this type of architecture. Using fewer than 10 inertial confinement fusion experiments for training, transfer learning systematically improves the simulation predictions while a simple output calibration, which we design as a baseline, makes the predictions worse. We also offer extensive cross-validation with real and carefully designed synthetic data. The method described in this paper can be applied to a wide range of problems that require transferring knowledge from simulations to the domain of experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Spatially co-registered wide-field nonlinear optical imaging of living and complex biosystems in a total internal reflection geometry

Nonlinear optical microscopy that leverages an objective based total internal reflection (TIR) excitation scheme is an attractive means for rapid, wide-field imaging with enhanced surface sensitivity. Through select combinations of distinct modalities, one can, in principle, access complementary chemical and structural information for various chemical species near interfaces. Here, we report a successful implementation of such a wide-field nonlinear optical microscope system, which combines coherent anti-Stokes Raman scattering (CARS), two-photon fluorescence (TPF), second harmonic generation (SHG), and sum frequency generation (SFG) modalities on the same platform. The intense optical fields needed to drive these high order nonlinear optical processes are achieved through the use of femtosecond pulsed light in combination with the intrinsic field confinement induced by TIR over a large field of view. The performance of our multimodal microscope was first assessed through the experimental determination of its chemical fidelity, intensity and polarization dependences, and spatial resolution using a set of well-defined model systems. Subsequently, its unique capabilities were validated through imaging complex biological systems, including Hydrangea quercifolia pollen grains and Pantoea sp. YR343 bacterial cells. Specifically, the spatial distribution of different molecular groups in the former was visualized via vibrational contrast mechanisms of CARS, whereas co-registered TPF imaging allowed the identification of spatially localized intrinsic fluorophores. We further demonstrate the feasibility of our microscope for wide-field CARS imaging on live cells through independent characterization of cell viability using spatially co-registered TPF imaging. Overall, this approach to TIR enabled wide-field imaging is expected to provide new insights into bacterial strains and their interactions with other species in the rhizosphere in a time-resolved and chemically selective manner.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Direct Imaging of Asymmetric Interfaces and Electrostatic Potentials inside a Hafnia–Zirconia Ferroelectric Nanocapacitor

In hafnia-based thin-film ferroelectric devices, chemical phenomena during growth and processing, such as oxygen vacancy formation and interfacial reactions, appear to strongly affect device performance. However, the correlation between the structure, chemistry, and electrical potentials at the nanoscale in these devices is not fully known, making it difficult to understand their influence on device properties. Here, we directly image the composition and electrostatic potential with nanometer resolution in the cross section of a nanocrystalline W/Hf 0.5 Zr 0.5 O 2−δ (HZO)/W ferroelectric capacitor using multimodal electron microscopy. This reveals a 1.4 nm wide tungsten suboxide interfacial layer formed at the bottom interface during fabrication, which introduces a potential dip and leads to asymmetric switching fields. Additionally, we compare the measured potentials to DFT calculations and find it is nearly 3 V lower than expected in the HZO, which appears to be caused by oxygen vacancies and a resulting negative built-in potential. In conclusion, these chemical and electrostatic details are important to characterize and tune to achieve high-performance ferroelectric devices.

Defects in solids↗
DOE OSTI2022

High-Throughput Microfluidics Platform for Intracellular Delivery and Sampling of Biomolecules from Live Cells

Nondestructive cell membrane permeabilization systems enable the intracellular delivery of exogenous biomolecules for cell engineering tasks as well as the temporal sampling of cytosolic contents from live cells for the analysis of dynamic processes. Here, in this paper, we report a microwell array format live-cell analysis device (LCAD) that can perform localized-electroporation induced membrane permeabilization, for cellular delivery or sampling, and directly interfaces with surface-based biosensors for analyzing the extracted contents. We demonstrate the capabilities of the LCAD via an automated high-throughput workflow for multimodal analysis of live-cell dynamics, consisting of quantitative measurements of enzyme activity using self-assembled monolayers for MALDI mass spectrometry (SAMDI) and deep-learning enhanced imaging and analysis. By combining a fabrication protocol that enables robust assembly and operation of multilayer devices with embedded gold electrodes and an automated imaging workflow, we successfully deliver functional molecules (plasmid and siRNA) into live cells at multiple time-points and track their effect on gene expression and cell morphology temporally. Furthermore, we report sampling performance enhancements, achieving saturation levels of protein tyrosine phosphatase activity measured from as few as 60 cells, and demonstrate control over the amount of sampled contents by optimization of electroporation parameters using a lumped model. Lastly, we investigate the implications of cell morphology on electroporation-induced sampling of fluorescent molecules using a deep-learning enhanced image analysis workflow.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Mind the gap: Bridging the divide between AI aspirations and the reality of autonomous microscopy

What does materials science look like in the “Age of Artificial Intelligence?” Each material’s domain—synthesis, characterization, and modeling—has a different answer to this question, motivated by unique challenges and constraints. This work focuses on the tremendous potential of autonomous characterization within electron microscopy. We present our recent advancements in developing domain-aware, multimodal models for microscopy analysis capable of describing complex atomic systems. We then address the critical gap between the theoretical promise of autonomous microscopy and its current practical limitations, showcasing recent successes while highlighting the necessary developments to achieve robust, real-world autonomy.

2D materials↗
DOE OSTI2022

Multimodal (Non)Linear Optical Nanoimaging and Nanospectroscopy

This article highlights recent advances in linear and nonlinear spectral nanoimaging. The described developments are motivated by the need to characterize molecular and material systems noninvasively with nanometer spatial and femtosecond temporal resolution. Indeed, the ability to image and chemically characterize heterogeneous interfaces with joint nano–femto resolution is a prerequisite to advancing our fundamental understanding of processes as diverse as heterogeneous catalysis, microbial communication, and energy flow in pristine/defect-containing low-dimensional quantum materials, to name a few. We describe pioneering work and recent demonstrations of (non)linear optical nanoimaging and nanospectroscopy, with an emphasis on high spatial resolution measurements conducted under ambient laboratory conditions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗