Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “multimodal imaging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Thermoresponsive polymer assemblies via variable temperature liquid-phase transmission electron microscopy and small angle X-ray scattering

Abstract Herein, phase transitions of a class of thermally-responsive polymers, namely a homopolymer, diblock, and triblock copolymer, were studied to gain mechanistic insight into nanoscale assembly dynamics via variable temperature liquid-cell transmission electron microscopy (VT-LCTEM) correlated with variable temperature small angle X-ray scattering (VT-SAXS). We study thermoresponsive poly(diethylene glycol methyl ether methacrylate) (PDEGMA)-based block copolymers and mitigate sample damage by screening electron flux and solvent conditions during LCTEM and by evaluating polymer survival via post-mortem matrix-assisted laser desorption/ionization imaging mass spectrometry (MALDI-IMS). Our multimodal approach, utilizing VT-LCTEM with MS validation and VT-SAXS, is generalizable across polymeric systems and can be used to directly image solvated nanoscale structures and thermally-induced transitions. Our strategy of correlating VT-SAXS with VT-LCTEM provided direct insight into transient nanoscale intermediates formed during the thermally-triggered morphological transformation of a PDEGMA-based triblock. Notably, we observed the temperature-triggered formation and slow relaxation of core-shell particles with complex microphase separation in the core by both VT-SAXS and VT-LCTEM.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Integrated multi-wavelength microscope combining TIRFM and IRM modalities for imaging cellulases and other processive enzymes

We describe a multimodal microscope for visualizing processive enzymes moving on immobilized substrates. The instrument combines interference reflection microscopy (IRM) with multi-wavelength total internal reflectance fluorescence microscopy (TIRFM). The microscope can localize quantum dots with a precision of 2.8 nm at 100 frames/s, and was used to image the dynamics of the cellulase, Cel7a interacting with surface-immobilized cellulose. The instrument, which was built with off-the-shelf components and is controlled by custom software, is suitable for tracking other degradative enzymes such as collagenases, as well as motor proteins moving along immobilized tracks.

09 BIOMASS FUELS↗
DOE OSTI2023

Manifold projection image segmentation for nano-XANES imaging

As spectral imaging techniques are becoming more prominent in science, advanced image segmentation algorithms are required to identify appropriate domains in these images. We present a version of image segmentation called manifold projection image segmentation (MPIS) that is generally applicable to a broad range of systems without the need for training because MPIS uses unsupervised machine learning with a few physically motivated hyperparameters. We apply MPIS to nanoscale x-ray absorption near edge structure (XANES) imaging, where XANES spectra are collected with nanometer spatial resolution. We show the superiority of manifold projection over linear transformations, such as the commonly used principal component analysis (PCA). Moreover, MPIS maintains accuracy while reducing computation time and sensitivity to noise compared to the standard nano-XANES imaging analysis procedure. Finally, we demonstrate how multimodal information, such as x-ray fluorescence data and spatial location of pixels, can be incorporated into the MPIS framework. We propose that MPIS is adaptable for any spectral imaging technique, including scanning transmission x-ray microscopy, where the length scale of domains is larger than the resolution of the experiment.

36 MATERIALS SCIENCE↗
NASA NTRS1988

Technology transfer: Imaging tracker to robotic controller

The transformation of an imaging tracker to a robotic controller is described. A multimode tracker was developed for fire and forget missile systems. The tracker locks on to target images within an acquisition window using multiple image tracking algorithms to provide guidance commands to missile control systems. This basic tracker technology is used with the addition of a ranging algorithm based on sizing a cooperative target to perform autonomous guidance and control of a platform for an Advanced Development Project on automation and robotics. A ranging tracker is required to provide the positioning necessary for robotic control. A simple functional demonstration of the feasibility of this approach was performed and described. More realistic demonstrations are under way at NASA-JSC. In particular, this modified tracker, or robotic controller, will be used to autonomously guide the Man Maneuvering Unit (MMU) to targets such as disabled astronauts or tools as part of the EVA Retriever efforts. It will also be used to control the orbiter's Remote Manipulator Systems (RMS) in autonomous approach and positioning demonstrations. These efforts will also be discussed.

Otaguro, M. S.↗
DOE OSTI2022

Mechanisms of graphite ablation by sub-millisecond ytterbium fiber laser pulses

Graphite is a key material in a variety of cross-cutting applications in energy conversion, energy storage, and nuclear energy. Recently, temporally modulated continuous wave lasers have been shown to produce well-defined ablation features in graphite at relatively high processing speeds. In this work, we analyze in detail the laser ablation dynamics of single-pulse ablation in the sub-millisecond time regime to elucidate the origins of the resulting well-defined ablation craters using a combination of time-resolved emission imaging, diffuse reflection/scattering imaging, and optical emission spectroscopy. These multimodal in situ diagnostics revealed three main contributors to achieve well-defined ablation features: (1) rapid ejection of particles with ~100 m/s speed, (2) ablation of the graphite in the gaseous form, and (3) absence of bulk liquid motion, which is typically observed in laser processing of metals. Plasma plume formation was sustained throughout the duration of the laser pulse (500 μs). Here, this work provides insights into the complex physical and chemical mechanisms of sub-millisecond laser–matter interactions, which are critical for parameter space optimization and tailoring of laser machining and drilling processes.

42 ENGINEERING↗
DOE OSTI2023

Mechanisms of ultrafast GHz burst fs laser ablation

Gigahertz (GHz) femtosecond (fs) lasers have opened possibilities for enhancing and controlling the laser machining quality to engineer the physicochemical properties of materials. However, fundamental understanding of laser-material interactions by GHz fs laser has remained unsolved due to the complexity of associated ablation dynamics. Here, we study the ablation dynamics of copper (Cu) by GHz fs bursts using in situ multimodal diagnostics, time-resolved scattering imaging, emission imaging, and emission spectroscopy. A combination of probing techniques reveals that GHz fs bursts rapidly remove molten Cu from the irradiated spot due to the recoil pressure exerted by following fs pulses. Material ejection essentially stops right after the burst irradiation due to the limited amount of remnant matter, combined with the suppressed heat conduction into the target material. Our work provides insights into the complex ablation mechanisms incurred by GHz fs bursts, which are critical in selecting optimal laser conditions in cross-cutting processing, micro/nano-fabrication, and spectroscopy applications.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

In situ Visible Light and Thermal Imaging Data from a Laser Powder Bed Fusion Additive Manufacturing Process Co-Registered to X-ray Computed Tomography and Fatigue Data

This dataset is comprised of in situ sensing data collected during a laser-based powder bed fusion additive manufacturing process, as well as rasterized scan path information, post-build X-ray computed tomography (XCT), and fatigue test results. A total of 64 cylinders, approximately 15 mm in diameter and 102 mm tall, were printed out of stainless steel 316H on a Colibrium Additive Concept Laser M2 Series 5 machine. Parameters known to produce dense material were used to construct 56 of these cylinders, while the remaining 8 cylinders were printed with relatively high energy density parameters prone to producing keyhole pores. In addition, two spatter generation blocks were constructed upstream of the 64 cylinders such that ejecta produced during the melting of the spatter generators were stochastically seeded onto the 64 cylinders. Based on previous experiments, these spatter particles were theorized to produce stochastic lack-of-fusion pores. During the construction of the build, high-resolution images of reflected light in the visible spectrum were captured both before and after recoating for each print layer. Additionally, temporally integrated thermal imaging in the near infrared spectrum produced integrated sum and max images on a layerwise basis. The multimodal in situ data has been co-registered to the build plate coordinate system, allowing for identification of process anomalies (e.g., spatter particles) apparent in the two sensors. Following construction of the build, the cylinders were subjected to XCT to identify internal flaws, and the resulting data have also been registered to the build plate coordinate system. Finally, 60 of the 64 cylinders were machined into fatigue coupons conforming to ASTM E466 and subsequently subjected to either high- or -low-cycle fatigue testing. The results of the fatigue tests have also been included in the dataset, and the XCT data corresponded to the approximate location of the gauge sections of the machine fatigue specimen geometry.

42 ENGINEERING↗
NASA NTRS2020

Learning instrument invariant characteristics for generating high-resolution global coral reef maps

Coral reefs are one of the most biologically complex and diverse ecosystems within the shallow marine environment. Unfortunately, these underwater ecosystems are threatened by a number of anthropogenic challenges, including ocean acidification and warming, overfishing, and the continued increase of marine debris in oceans. This requires a comprehensive assessment of the world's coastal environments, including a quantitative analysis on the health and extent of coral reefs and other associated marine species, as a vital Earth Science measurement. However, limitations in observational and technological capabilities inhibit global sustained imaging of the marine environment. Harmonizing multimodal data sets acquired using different remote sensing instruments presents additional challenges, thereby limiting the availability of good quality labeled data for analysis. In this work, we develop a deep learning model for extracting domain invariant features from multimodal remote sensing imagery and creating high-resolution global maps of coral reefs by combining various sources of imagery and limited hand-labeled data available for certain regions. This framework allows us to generate, for the first time, coral reef segmentation maps at 2-meter resolution, which is a significant improvement over the kilometer-scale state-of-the-art maps. Additionally, this framework doubles accuracy and IoU metrics over baselines that do not account for domain invariance.

Domain Adaptation↗
NASA NTRSDate not supplied

Asymmetric Flow State Switching in the Space Launch System Block 2 Liftoff Environment

An in-depth investigation was conducted at the NASA Langley Research Center 14- by 22-Foot Subsonic Tunnel to evaluate flow state switching in the liftoff flow environment of the Space Launch System Block 2 Crew launch vehicle using force measurements, two-component particle image velocimetry, and tuft flow visualization. Multimodal flow states were observed in the gap flow between the centerbody and solid rocket boosters at a range of incoming flow angles and were characterized for the relative strength of state switches. When flow switches occur, the flow is predominantly bimodal, but trimodal flow states are observed with the launch tower downstream of the vehicle. Tuft visualization indicates three-dimensionality during flow state switching, which initiates at one part of the gap and quickly transitions throughout the length of the booster. Due to the long time scales between switches, statistics such as frequency of flow state switches and converged state probabilities cannot be established without significantly increased acquisition times.

Lee J. Mears↗
NASA NTRSDate not supplied

Asymmetric Flow State Switching on the Space Launch System (SLS) Block 2 Crew Vehicle

An in-depth investigation was conducted at the NASA Langley Research Center 14- by 22-Foot Subsonic Tunnel to evaluate flow state switching in the liftoff flow environment of the Space Launch System Block 2 Crew launch vehicle using side force measurements, two-component particle image velocimetry, and tufts flow visualization. Multimodal flow states were observed in the gap flow between the centerbody and solid rocket boosters at a range of incoming flow angles and were characterized for the relative strength of state switches. When flow switches occur, the flow is predominantly bimodal, but trimodal flow states are observed with the launch tower downstream of the vehicle. Tufts visualization indicates three-dimensionality during flow state switching, which initiates at one part of the gap and quickly transitions throughout the length of the booster. Due to the long time scales between switches, statistics such as frequency of flow state switches and converged state probabilities cannot be established without significantly increased acquisition times.

Lee J. Mears↗
DOE OSTI2025

A Representation Fusion Framework for Decoupling Diagnostic Information in Multimodal Learning

Modern medicine increasingly relies on multimodal data, ranging from clinical notes to imaging and genomics, to guide diagnosis and treatment. However, integrating these heterogeneous data sources in a principled and interpretable manner remains a major challenge. We present MODES (Multi-mOdal Disentangled Embedding Space), a representation fusion framework that explicitly separates shared and modality-specific factors of variation, offering a structured latent space for multimodal information that improves both prediction and interpretability. By leveraging pre-trained unimodal foundation models, MODES mitigates the dependency on extensive paired datasets, crucial in data-scarce clinical settings. We introduce a masking strategy that optimizes representation dimensionality by eliminating low-information dimensions, to achieve compact, information-rich representations. Our framework demonstrates superior performance in predicting diagnoses and phenotypes compared to unimodal and conventional fusion models. MODES also enables robust diagnostic inference in missing data scenarios, offering an opportunity toward interpretable and efficient multimodal diagnostics in personalized healthcare.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Using a Large Language Model for Accurate Technical Language Generation in the Predictive Maintenance of Circulating Water Systems in Nuclear Power Plants

Machine learning (ML) methods for predictive maintenance (PdM) are emerging as effective proactive strategies for diagnosing equipment degradation and enabling effective decision-making. However, explainability and trustworthiness of artificial intelligence are two salient challenges that need to be addressed for wider deployment of these technologies in nuclear power plants (NPPs). Large language models (LLMs) offer a unique approach to tackle these challenges by explaining PdM, work orders, diagnosis results, and ML algorithms to users, who may not be familiar with ML and PdM in general. Moreover, by dynamically retrieving relevant information from technical documents and evaluating factuality of LLM generation, the accuracy and relevance of LLM generations can be improved. This work demonstrates using LLMs to explain the causes and consequences of circulating water system failures based on multiyear NPP work orders. This work tests the capability of multimodal LLM approaches in explaining the differences in the circulating water system from both the Salem and Hope Creek NPPs using both text and image resources. This work also demonstrates the use of multimodal LLMs in describing the diagnosis tab of a predictive maintenance software named VIsualization for PrEdictive maintenance Recommendation (VIPER) to users.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Pretraining Billion-Scale Geospatial Foundational Models on Frontier

As AI workloads increase in scope, generalization capability becomes challenging for small task-specific models and their demand for large amounts of labeled training samples increases. On the contrary, Foundation Models (FMs) are trained with internet-scale unlabeled data via self-supervised learning and have been shown to adapt to various tasks with minimal fine-tuning. Although large FMs have demonstrated significant impact in natural language processing and computer vision, efforts toward FMs for geospatial applications have been restricted to smaller size models, as pretraining larger models requires very large computing resources equipped with state-of-the-art hardware accelerators. Current satellite constellations collect 100+TBs of data a day, resulting in images that are billions of pixels and multimodal in nature. Such geospatial data poses unique challenges opening up new opportunities to develop FMs. We investigate billion scale FMs and HPC training profiles for geospatial applications by pretraining on publicly available data. We studied from end-to-end the performance and impact in the solution by scaling the model size. Our larger 3B parameter size model achieves up to 30% improvement in top1 scene classification accuracy when comparing a 100M parameter model. Moreover, we detail performance experiments on the Frontier supercomputer, America's first exascale system, where we study different model and data parallel approaches using PyTorch's Fully Sharded Data Parallel library. Specifically, we study variants of the Vision Transformer architecture (ViT), conducting performance analysis for ViT models with size up to 15B parameters. By discussing throughput and performance bottlenecks under different parallelism configurations, we offer insights on how to leverage such leadership-class HPC resources when developing large models for geospatial imagery applications.

Tsaris, Aristeidis (aris)↗
DOE OSTI2025

Toward more-robust, AI-enabled subsurface seismic imaging for geotechnical applications

Non-invasive seismic imaging has the potential to cost-effectively evaluate large volumes of subsurface material to inform geotechnical site investigation. However, seismic imaging using full waveform inversion (FWI) requires significant computational time and is dependent on an initial starting model. As a result, FWI has not yet been widely adopted into geotechnical practice. Previous efforts, on relatively simple two-layered models, indicate that data-driven artificial intelligence (AI) models may be as effective as FWI at predicting 2D images of shear wave velocity (V s ). Furthermore, the AI model predictions can be made almost instantaneously after data acquisition and do not require an initial starting model. We examine the generality of these findings by developing a new AI model for subsurface seismic imaging, whereby we make several notable contributions. First, we architect a multimodal AI model that combines time- and frequency-domain representations of the seismic wavefield to predict a 50 m by 20 m subsurface image of V s . Second, we developed a new diverse dataset of 100,000 images with their corresponding seismic wavefields to train the AI model. Third, we propose four physics-informed data augmentations for data-driven seismic imaging. Fourth, we develop two prediction consistency tests to evaluate the model’s performance when the true subsurface is unknown. Our final model, which has been made publicly available, is capable of predicting a subsurface V s image from a single seismic wavefield with an average, mean absolute percent error (MAPE) of 24 %. The predictive model is applied to a field dataset and shown to be consistent with local geology and shear-wave refraction measurements from the same location.

Artificial intelligence↗
DOE OSTI2020

Impact of krypton irradiation on a single crystal tungsten: Multi-modal X-ray imaging study

Understanding microstructural and strain evolutions induced by noble gas production in the nuclear fuel matrix or plasma-facing materials is crucial for designing next generation nuclear reactors, as they are responsible for volumetric swelling and catastrophic failure. In this paper we describe a multimodal approach combining synchrotron-based nanoscale X-ray imaging techniques with atomic-scale electron microscopy techniques for mapping chemical composition, morphology and lattice distortion in a single crystal W induced by Kr irradiation. We report that Kr-irradiated single crystal W undergoes surface deformation, forming Kr containing cavities. Furthermore, positive strain fields are observed in Kr-irradiated regions, which lead to compression of underlying W matrix.

36 MATERIALS SCIENCE↗
NASA NTRS2010

DBSAR's First Multimode Flight Campaign

The Digital Beamforming SAR (DBSAR) is an airborne imaging radar system that combines phased array technology, reconfigurable on-board processing and waveform generation, and advances in signal processing to enable techniques not possible with conventional SARs. The system exploits the versatility inherently in phased-array technology with a state-of-the-art data acquisition and real-time processor in order to implement multi-mode measurement techniques in a single radar system. Operational modes include scatterometry over multiple antenna beams, Synthetic Aperture Radar (SAR) over several antenna beams, or Altimetry. The radar was flight tested in October 2008 on board of the NASA P3 aircraft over the Delmarva Peninsula, MD. The results from the DBSAR system performance is presented.

Rincon, Rafael F.↗
DOE OSTI2020

Challenges in Lithium Metal Anodes for Solid-State Batteries

In this Perspective, we highlight recent progress and challenges related to the integration of lithium metal anodes in solid-state batteries. While prior reports have suggested that solid electrolytes may be impermeable to lithium metal, this hypothesis has been disproven under a variety of electrolyte compositions and cycling conditions. Herein, we describe the mechanistic origins and importance of lithium filament growth and interphase formation in inorganic and organic solid electrolytes. Multimodal techniques that combine real and reciprocal space imaging and modeling will be necessary to fully understand nonequilibrium dynamics at these buried interfaces. Currently, most studies on lithium electrode kinetics at solid electrolyte interfaces are completed in symmetric Li–Li configurations. To fully understand the challenges and opportunities afforded by Li-metal anodes, full-cell experiments are necessary. Lastly, the impacts of operating conditions on solid-state batteries are largely unknown with respect to pressure, geometry, and break-in protocols. Given the rapid growth of this community and the diverse portfolio of solid electrolytes, we highlight the need for detailed reporting of experimental conditions and standardization of protocols across the community.

25 ENERGY STORAGE↗
NASA NTRS1968

Aerospace sensor systems.

Aerospace sensor systems, discussing adaptive multimode sensors and multispectral sensing, electronic recognition of image components, laser radar and Raman scattering technique

Mannella, G. G.↗