Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “multimodal imaging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Omni‐Resonant Imaging Across the Visible

Resonant field enhancement in optical cavities is provided over only narrow linewidths and for specific spatial modes. Consequently, spectrally restrictive planar Fabry-Pérot cavities have not contributed to date to white-light imaging, which necessitates a highly multimoded broadband field to satisfy the resonance condition. It is shown that introducing judicious angular-dispersion circumvents the fundamental trade-off between cavity linewidth and finesse in a Fabry-Pérot cavity by exciting a 130-nm-bandwidth achromatic resonance across the visible spectrum, which far exceeds the finesse-limited linewidth (0.5 nm), and even exceeds the free spectral range (45 nm). This omni-resonant configuration enables broadband color-imaging over a 100-nm-bandwidth in the visible with minimal spherical and chromatic aberrations. Omni-resonant imaging is demonstrated using coherent and incoherent light, and spatially extended and localized fields comprising stationary and moving objects. This work paves the way to harnessing broadband resonant enhancements for spatially structured fields, as needed for example in solar windows.

42 ENGINEERING↗
DOE OSTI2025

Foundation Models for Zero-Shot Segmentation of Scientific Images without AI-Ready Data

Zero-shot and prompt-based models have excelled at visual reasoning tasks by leveraging large-scale natural image corpora, but they often fail on sparse and domain-specific scientific image data. We introduce Zenesis, a no-code interactive computer vision platform designed to reduce data readiness bottlenecks in scientific imaging workflows. Zenesis integrates lightweight multimodal adaptation for zero-shot inference on raw scientific data, human-in-the-loop refinement, and heuristic-based temporal enhancement. We validate our approach on Focused Ion Beam Scanning Electron Microscopy (FIB-SEM) datasets of catalyst-loaded membranes. Zenesis outperforms baselines, achieving an average accuracy of 0.947, Intersection over Union (IoU) of 0.858, and Dice score of 0.923 on amorphous catalyst samples; and 0.987 accuracy, 0.857 IoU, and 0.923 Dice on crystalline samples. These results represent a significant performance gain over conventional methods such as Otsu thresholding and standalone models like the Segment Anything Model (SAM). Zenesis enables effective image segmentation in domains where annotated datasets are limited, offering a scalable solution for scientific discovery.

Mukherjee, Shubhabrata↗
DOE OSTI2022

Tailoring Fibre Structure Enabled by X-ray Analytics for Targeted Biomedical Applications

The rising interest in designing fibres via spinning techniques combining the properties of various polymeric materials into advanced functionalised materials is directed towards targeted biomedical applications such as drug delivery, wearable sensors or tissue engineering. Understanding how these functional polymers exhibit multiscale structures ranging from the molecular level to nano-, micro-and millimetre scale is a key prerequisite for their challenging applications that can be addressed by a non-destructive X-ray based analytical approach. X-ray multimodalities combining X-ray imaging, scattering and diffraction allow the study of morphology, molecular structure, and the analysis of nano-domain size and shape, crystallinity and preferential orientation in 3D arrangements. The incorporation of X-ray analytics in the design process of polymeric fibers via their nanostructure under non-ambient conditions (i.e. temperature, mechanical load, humidity…) allows for efficient optimization of the fabrication process as well as quality control along the product lifetime under operating environmental conditions. Here, we demonstrate the successful collaboration between the laboratory of Biomimetic Textiles and Membranes and the Center of X-ray Analytics at Empa for the design, characterisation and optimisation of advanced functionalised polymeric fibrous material systems.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

Overview of MARS: The Multimodal Advanced Radiography Station at the High-Flux Isotope Reactor

Neutron imaging is a powerful tool used for inspecting internal structures and studying macroscale particle dynamics. This work is performed at the Multimodal Advanced Radiography Station (MARS), which is the cold-neutron imaging beamline at the High-Flux Isotope Reactor at Oak Ridge National Laboratory. Herein is a summary of recent upgrades to instrumentation, software, and methods, driven by needs from the user community. Planned upgrades within the next decade are included that will significantly enhance instrument capability and performance.

Torres, James [ORNL] (ORCID:0000000289407610)↗
DOE OSTI2021

Thermoresponsive polymer assemblies via variable temperature liquid-phase transmission electron microscopy and small angle X-ray scattering

Abstract Herein, phase transitions of a class of thermally-responsive polymers, namely a homopolymer, diblock, and triblock copolymer, were studied to gain mechanistic insight into nanoscale assembly dynamics via variable temperature liquid-cell transmission electron microscopy (VT-LCTEM) correlated with variable temperature small angle X-ray scattering (VT-SAXS). We study thermoresponsive poly(diethylene glycol methyl ether methacrylate) (PDEGMA)-based block copolymers and mitigate sample damage by screening electron flux and solvent conditions during LCTEM and by evaluating polymer survival via post-mortem matrix-assisted laser desorption/ionization imaging mass spectrometry (MALDI-IMS). Our multimodal approach, utilizing VT-LCTEM with MS validation and VT-SAXS, is generalizable across polymeric systems and can be used to directly image solvated nanoscale structures and thermally-induced transitions. Our strategy of correlating VT-SAXS with VT-LCTEM provided direct insight into transient nanoscale intermediates formed during the thermally-triggered morphological transformation of a PDEGMA-based triblock. Notably, we observed the temperature-triggered formation and slow relaxation of core-shell particles with complex microphase separation in the core by both VT-SAXS and VT-LCTEM.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Integrated multi-wavelength microscope combining TIRFM and IRM modalities for imaging cellulases and other processive enzymes

We describe a multimodal microscope for visualizing processive enzymes moving on immobilized substrates. The instrument combines interference reflection microscopy (IRM) with multi-wavelength total internal reflectance fluorescence microscopy (TIRFM). The microscope can localize quantum dots with a precision of 2.8 nm at 100 frames/s, and was used to image the dynamics of the cellulase, Cel7a interacting with surface-immobilized cellulose. The instrument, which was built with off-the-shelf components and is controlled by custom software, is suitable for tracking other degradative enzymes such as collagenases, as well as motor proteins moving along immobilized tracks.

09 BIOMASS FUELS↗
DOE OSTI2023

Manifold projection image segmentation for nano-XANES imaging

As spectral imaging techniques are becoming more prominent in science, advanced image segmentation algorithms are required to identify appropriate domains in these images. We present a version of image segmentation called manifold projection image segmentation (MPIS) that is generally applicable to a broad range of systems without the need for training because MPIS uses unsupervised machine learning with a few physically motivated hyperparameters. We apply MPIS to nanoscale x-ray absorption near edge structure (XANES) imaging, where XANES spectra are collected with nanometer spatial resolution. We show the superiority of manifold projection over linear transformations, such as the commonly used principal component analysis (PCA). Moreover, MPIS maintains accuracy while reducing computation time and sensitivity to noise compared to the standard nano-XANES imaging analysis procedure. Finally, we demonstrate how multimodal information, such as x-ray fluorescence data and spatial location of pixels, can be incorporated into the MPIS framework. We propose that MPIS is adaptable for any spectral imaging technique, including scanning transmission x-ray microscopy, where the length scale of domains is larger than the resolution of the experiment.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Mechanisms of graphite ablation by sub-millisecond ytterbium fiber laser pulses

Graphite is a key material in a variety of cross-cutting applications in energy conversion, energy storage, and nuclear energy. Recently, temporally modulated continuous wave lasers have been shown to produce well-defined ablation features in graphite at relatively high processing speeds. In this work, we analyze in detail the laser ablation dynamics of single-pulse ablation in the sub-millisecond time regime to elucidate the origins of the resulting well-defined ablation craters using a combination of time-resolved emission imaging, diffuse reflection/scattering imaging, and optical emission spectroscopy. These multimodal in situ diagnostics revealed three main contributors to achieve well-defined ablation features: (1) rapid ejection of particles with ~100 m/s speed, (2) ablation of the graphite in the gaseous form, and (3) absence of bulk liquid motion, which is typically observed in laser processing of metals. Plasma plume formation was sustained throughout the duration of the laser pulse (500 μs). Here, this work provides insights into the complex physical and chemical mechanisms of sub-millisecond laser–matter interactions, which are critical for parameter space optimization and tailoring of laser machining and drilling processes.

42 ENGINEERING↗
DOE OSTI2023

Mechanisms of ultrafast GHz burst fs laser ablation

Gigahertz (GHz) femtosecond (fs) lasers have opened possibilities for enhancing and controlling the laser machining quality to engineer the physicochemical properties of materials. However, fundamental understanding of laser-material interactions by GHz fs laser has remained unsolved due to the complexity of associated ablation dynamics. Here, we study the ablation dynamics of copper (Cu) by GHz fs bursts using in situ multimodal diagnostics, time-resolved scattering imaging, emission imaging, and emission spectroscopy. A combination of probing techniques reveals that GHz fs bursts rapidly remove molten Cu from the irradiated spot due to the recoil pressure exerted by following fs pulses. Material ejection essentially stops right after the burst irradiation due to the limited amount of remnant matter, combined with the suppressed heat conduction into the target material. Our work provides insights into the complex ablation mechanisms incurred by GHz fs bursts, which are critical in selecting optimal laser conditions in cross-cutting processing, micro/nano-fabrication, and spectroscopy applications.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

In situ Visible Light and Thermal Imaging Data from a Laser Powder Bed Fusion Additive Manufacturing Process Co-Registered to X-ray Computed Tomography and Fatigue Data

This dataset is comprised of in situ sensing data collected during a laser-based powder bed fusion additive manufacturing process, as well as rasterized scan path information, post-build X-ray computed tomography (XCT), and fatigue test results. A total of 64 cylinders, approximately 15 mm in diameter and 102 mm tall, were printed out of stainless steel 316H on a Colibrium Additive Concept Laser M2 Series 5 machine. Parameters known to produce dense material were used to construct 56 of these cylinders, while the remaining 8 cylinders were printed with relatively high energy density parameters prone to producing keyhole pores. In addition, two spatter generation blocks were constructed upstream of the 64 cylinders such that ejecta produced during the melting of the spatter generators were stochastically seeded onto the 64 cylinders. Based on previous experiments, these spatter particles were theorized to produce stochastic lack-of-fusion pores. During the construction of the build, high-resolution images of reflected light in the visible spectrum were captured both before and after recoating for each print layer. Additionally, temporally integrated thermal imaging in the near infrared spectrum produced integrated sum and max images on a layerwise basis. The multimodal in situ data has been co-registered to the build plate coordinate system, allowing for identification of process anomalies (e.g., spatter particles) apparent in the two sensors. Following construction of the build, the cylinders were subjected to XCT to identify internal flaws, and the resulting data have also been registered to the build plate coordinate system. Finally, 60 of the 64 cylinders were machined into fatigue coupons conforming to ASTM E466 and subsequently subjected to either high- or -low-cycle fatigue testing. The results of the fatigue tests have also been included in the dataset, and the XCT data corresponded to the approximate location of the gauge sections of the machine fatigue specimen geometry.

42 ENGINEERING↗
DOE OSTI2025

A Representation Fusion Framework for Decoupling Diagnostic Information in Multimodal Learning

Modern medicine increasingly relies on multimodal data, ranging from clinical notes to imaging and genomics, to guide diagnosis and treatment. However, integrating these heterogeneous data sources in a principled and interpretable manner remains a major challenge. We present MODES (Multi-mOdal Disentangled Embedding Space), a representation fusion framework that explicitly separates shared and modality-specific factors of variation, offering a structured latent space for multimodal information that improves both prediction and interpretability. By leveraging pre-trained unimodal foundation models, MODES mitigates the dependency on extensive paired datasets, crucial in data-scarce clinical settings. We introduce a masking strategy that optimizes representation dimensionality by eliminating low-information dimensions, to achieve compact, information-rich representations. Our framework demonstrates superior performance in predicting diagnoses and phenotypes compared to unimodal and conventional fusion models. MODES also enables robust diagnostic inference in missing data scenarios, offering an opportunity toward interpretable and efficient multimodal diagnostics in personalized healthcare.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Using a Large Language Model for Accurate Technical Language Generation in the Predictive Maintenance of Circulating Water Systems in Nuclear Power Plants

Machine learning (ML) methods for predictive maintenance (PdM) are emerging as effective proactive strategies for diagnosing equipment degradation and enabling effective decision-making. However, explainability and trustworthiness of artificial intelligence are two salient challenges that need to be addressed for wider deployment of these technologies in nuclear power plants (NPPs). Large language models (LLMs) offer a unique approach to tackle these challenges by explaining PdM, work orders, diagnosis results, and ML algorithms to users, who may not be familiar with ML and PdM in general. Moreover, by dynamically retrieving relevant information from technical documents and evaluating factuality of LLM generation, the accuracy and relevance of LLM generations can be improved. This work demonstrates using LLMs to explain the causes and consequences of circulating water system failures based on multiyear NPP work orders. This work tests the capability of multimodal LLM approaches in explaining the differences in the circulating water system from both the Salem and Hope Creek NPPs using both text and image resources. This work also demonstrates the use of multimodal LLMs in describing the diagnosis tab of a predictive maintenance software named VIsualization for PrEdictive maintenance Recommendation (VIPER) to users.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Pretraining Billion-Scale Geospatial Foundational Models on Frontier

As AI workloads increase in scope, generalization capability becomes challenging for small task-specific models and their demand for large amounts of labeled training samples increases. On the contrary, Foundation Models (FMs) are trained with internet-scale unlabeled data via self-supervised learning and have been shown to adapt to various tasks with minimal fine-tuning. Although large FMs have demonstrated significant impact in natural language processing and computer vision, efforts toward FMs for geospatial applications have been restricted to smaller size models, as pretraining larger models requires very large computing resources equipped with state-of-the-art hardware accelerators. Current satellite constellations collect 100+TBs of data a day, resulting in images that are billions of pixels and multimodal in nature. Such geospatial data poses unique challenges opening up new opportunities to develop FMs. We investigate billion scale FMs and HPC training profiles for geospatial applications by pretraining on publicly available data. We studied from end-to-end the performance and impact in the solution by scaling the model size. Our larger 3B parameter size model achieves up to 30% improvement in top1 scene classification accuracy when comparing a 100M parameter model. Moreover, we detail performance experiments on the Frontier supercomputer, America's first exascale system, where we study different model and data parallel approaches using PyTorch's Fully Sharded Data Parallel library. Specifically, we study variants of the Vision Transformer architecture (ViT), conducting performance analysis for ViT models with size up to 15B parameters. By discussing throughput and performance bottlenecks under different parallelism configurations, we offer insights on how to leverage such leadership-class HPC resources when developing large models for geospatial imagery applications.

Tsaris, Aristeidis (aris)↗
DOE OSTI2025

Toward more-robust, AI-enabled subsurface seismic imaging for geotechnical applications

Non-invasive seismic imaging has the potential to cost-effectively evaluate large volumes of subsurface material to inform geotechnical site investigation. However, seismic imaging using full waveform inversion (FWI) requires significant computational time and is dependent on an initial starting model. As a result, FWI has not yet been widely adopted into geotechnical practice. Previous efforts, on relatively simple two-layered models, indicate that data-driven artificial intelligence (AI) models may be as effective as FWI at predicting 2D images of shear wave velocity (V s ). Furthermore, the AI model predictions can be made almost instantaneously after data acquisition and do not require an initial starting model. We examine the generality of these findings by developing a new AI model for subsurface seismic imaging, whereby we make several notable contributions. First, we architect a multimodal AI model that combines time- and frequency-domain representations of the seismic wavefield to predict a 50 m by 20 m subsurface image of V s . Second, we developed a new diverse dataset of 100,000 images with their corresponding seismic wavefields to train the AI model. Third, we propose four physics-informed data augmentations for data-driven seismic imaging. Fourth, we develop two prediction consistency tests to evaluate the model’s performance when the true subsurface is unknown. Our final model, which has been made publicly available, is capable of predicting a subsurface V s image from a single seismic wavefield with an average, mean absolute percent error (MAPE) of 24 %. The predictive model is applied to a field dataset and shown to be consistent with local geology and shear-wave refraction measurements from the same location.

Artificial intelligence↗
DOE OSTI2020

Impact of krypton irradiation on a single crystal tungsten: Multi-modal X-ray imaging study

Understanding microstructural and strain evolutions induced by noble gas production in the nuclear fuel matrix or plasma-facing materials is crucial for designing next generation nuclear reactors, as they are responsible for volumetric swelling and catastrophic failure. In this paper we describe a multimodal approach combining synchrotron-based nanoscale X-ray imaging techniques with atomic-scale electron microscopy techniques for mapping chemical composition, morphology and lattice distortion in a single crystal W induced by Kr irradiation. We report that Kr-irradiated single crystal W undergoes surface deformation, forming Kr containing cavities. Furthermore, positive strain fields are observed in Kr-irradiated regions, which lead to compression of underlying W matrix.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Challenges in Lithium Metal Anodes for Solid-State Batteries

In this Perspective, we highlight recent progress and challenges related to the integration of lithium metal anodes in solid-state batteries. While prior reports have suggested that solid electrolytes may be impermeable to lithium metal, this hypothesis has been disproven under a variety of electrolyte compositions and cycling conditions. Herein, we describe the mechanistic origins and importance of lithium filament growth and interphase formation in inorganic and organic solid electrolytes. Multimodal techniques that combine real and reciprocal space imaging and modeling will be necessary to fully understand nonequilibrium dynamics at these buried interfaces. Currently, most studies on lithium electrode kinetics at solid electrolyte interfaces are completed in symmetric Li–Li configurations. To fully understand the challenges and opportunities afforded by Li-metal anodes, full-cell experiments are necessary. Lastly, the impacts of operating conditions on solid-state batteries are largely unknown with respect to pressure, geometry, and break-in protocols. Given the rapid growth of this community and the diverse portfolio of solid electrolytes, we highlight the need for detailed reporting of experimental conditions and standardization of protocols across the community.

25 ENERGY STORAGE↗
DOE OSTI2023

Identification of Blue Horizontal Branch Stars with Multimodal Fusion

Blue Horizontal Branch stars (BHBs) are ideal tracers to probe the global structure of the milky Way (MW), and the increased size of the BHB star sample could be helpful to accurately calculate the MW's enclosed mass and kinematics. Large survey telescopes have produced an increasing number of astronomical images and spectra. However, traditional methods of identifying BHBs are limited in dealing with the large scale of astronomical data. A fast and efficient way of identifying BHBs can provide a more significant sample for further analysis and research. Therefore, in order to fully use the various data observed and further improve the identification accuracy of BHBs, we have innovatively proposed and implemented a Bi-level attention mechanism-based Transformer multimodal fusion model, called Bi-level Attention in the Transformer with Multimodality (BATMM). The model consists of a spectrum encoder, an image encoder, and a Transformer multimodal fusion module. The Transformer enables the effective fusion of data from two modalities, namely image and spectrum, by using the proposed Bi-level attention mechanism, including cross-attention and self-attention. As a result, the information from the different modalities complements each other, thus improving the accuracy of the identification of BHBs. The experimental results show that the F1 score of the proposed BATMM is 94.78%, which is 21.77% and 2.76% higher than the image and spectral unimodality, respectively. It is therefore demonstrated that higher identification accuracy of BHBs can be achieved by means of using data from multiple modalities and employing an efficient data fusion strategy.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Development of Multimodal Few-Shot Analytics for Electron Micrographs

Recent advances in materials data analytics have provided new avenues for determining process-structure-property (PSP) linkages in a variety of materials. Machine learning techniques including few-shot learning have increased the efficiency of classifying microscopy images for the purposes of material characterization. Attempts at creating a multimodal approach can provide further improvements to current models and help extract more salient features from data. In this vein, raw spectrum data was taken to provide an additional modality to our current pyCHIP classifier. Modifications in segmentation also show potential in improving the accuracy of the pyCHIP classifier. Classifier output was analyzed using network graphs and unsupervised clustering algorithms such as spectral clustering to detect better segmentation methods than the current “chipping” approach. We suggest that the chip selection process can be automated in the future using a combination of these techniques to enable high-throughput analyses.

36 MATERIALS SCIENCE↗