Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Modal”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

CrossMP: Enabling Cross-Modality Translation between Single-Cell RNA-Seq and Single-Cell ATAC-Seq through Web-Based Portal

In recent years, there has been a growing interest in profiling multiomic modalities within individual cells simultaneously. One such example is integrating combined single-cell RNA sequencing (scRNA-seq) data and single-cell transposase-accessible chromatin sequencing (scATAC-seq) data. Integrated analysis of diverse modalities has helped researchers make more accurate predictions and gain a more comprehensive understanding than with single-modality analysis. However, generating such multimodal data is technically challenging and expensive, leading to limited availability of single-cell co-assay data. Here, we propose a model for cross-modal prediction between the transcriptome and chromatin profiles in single cells. Our model is based on a deep neural network architecture that learns the latent representations from the source modality and then predicts the target modality. It demonstrates reliable performance in accurately translating between these modalities across multiple paired human scATAC-seq and scRNA-seq datasets. Additionally, we developed CrossMP, a web-based portal allowing researchers to upload their single-cell modality data through an interactive web interface and predict the other type of modality data, using high-performance computing resources plugged at the backend.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Integrating multi-modal remote sensing, deep learning, and attention mechanisms for yield prediction in plant breeding experiments

In both plant breeding and crop management, interpretability plays a crucial role in instilling trust in AI-driven approaches and enabling the provision of actionable insights. The primary objective of this research is to explore and evaluate the potential contributions of deep learning network architectures that employ stacked LSTM for end-of-season maize grain yield prediction. A secondary aim is to expand the capabilities of these networks by adapting them to better accommodate and leverage the multi-modality properties of remote sensing data. In this study, a multi-modal deep learning architecture that assimilates inputs from heterogeneous data streams, including high-resolution hyperspectral imagery, LiDAR point clouds, and environmental data, is proposed to forecast maize crop yields. The architecture includes attention mechanisms that assign varying levels of importance to different modalities and temporal features that, reflect the dynamics of plant growth and environmental interactions. The interpretability of the attention weights is investigated in multi-modal networks that seek to both improve predictions and attribute crop yield outcomes to genetic and environmental variables. This approach also contributes to increased interpretability of the model's predictions. The temporal attention weight distributions highlighted relevant factors and critical growth stages that contribute to the predictions. The results of this study affirm that the attention weights are consistent with recognized biological growth stages, thereby substantiating the network's capability to learn biologically interpretable features. Accuracies of the model's predictions of yield ranged from 0.82-0.93 R 2 ref in this genetics-focused study, further highlighting the potential of attention-based models. Further, this research facilitates understanding of how multi-modality remote sensing aligns with the physiological stages of maize. The proposed architecture shows promise in improving predictions and offering interpretable insights into the factors affecting maize crop yields, while demonstrating the impact of data collection by different modalities through the growing season. By identifying relevant factors and critical growth stages, the model's attention weights provide valuable information that can be used in both plant breeding and crop management. The consistency of attention weights with biological growth stages reinforces the potential of deep learning networks in agricultural applications, particularly in leveraging remote sensing data for yield prediction. To the best of our knowledge, this is the first study that investigates the use of hyperspectral and LiDAR UAV time series data for explaining/interpreting plant growth stages within deep learning networks and forecasting plot-level maize grain yield using late fusion modalities with attention mechanisms.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Detection of multi-modal Doppler spectra – Part 1: Establishing characteristic signals in radar moment data

Vertically pointing millimeter-wavelength radars provide a wealth of information about cloud and precipitation particle properties. Doppler spectral data can inform on how particles of varying vertical velocities contribute to the total backscattered power observed. It is more computationally cost effective to process moment data instead of spectra data, but doing so leaves valuable information on the cutting room floor. To confidently identify a multi-modal spectra event, in which two or more modes are present within a layer, Doppler spectral data are essential. This means long-term identification of layers featuring multi-modal spectra can be cost prohibitive. To address this, we explore three multi-modal spectra cases from winter precipitation events to determine characteristic signatures of these layers in the moment data averaged over short time periods (∼ 145 s) and explore how these layers differ from the rest of the vertical profiles. We find that the mean spectrum width and the standard deviation of mean Doppler velocity can be used to determine whether or not a layer is multi-modal. In particular, multi-modal layers in mixed-phase and ice clouds feature larger mean spectrum width (exceeding 0.17 m s −1 ) and smaller standard deviation of the mean Doppler velocity (below 0.1 m s −1 ). In Part 1 of this study, the identification criteria and methods are described. In Part 2 (Wugofski and Kumjian, 2025), we perform a verification of the method for three years of vertically pointing radar data, and explore the meteorological conditions associated with identified multi-modal spectral events.

Wugofski, Sarah [Pennsylvania State Univ., Univers↗
DOE OSTI2026

Multi-Modal Bayesian Neural Network Surrogates with Conjugate Last-Layer Estimation

As data collection and simulation capabilities advance, multi-modal learning, the task of learning from multiple modalities and sources of data, is becoming an increasingly important area of research. Surrogate models that learn from data of multiple auxiliary modalities to support the modeling of a highly expensive quantity of interest have the potential to aid outer loop applications such as optimization, inverse problems, or sensitivity analyses when multi-modal data are available. We develop two multi-modal Bayesian neural network surrogate models and leverage conditionally conjugate distributions in the last layer to estimate model parameters using stochastic variational inference (SVI). We provide a method to perform this conjugate SVI estimation in the presence of partially missing observations. Here, we demonstrate improved prediction accuracy and uncertainty quantification compared to unimodal surrogate models for both scalar and time series data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Cross-Modal Guidance for Fast Diffusion-Based Computed Tomography

Diffusion models have emerged as powerful priors for solving inverse problems in computed tomography (CT). In certain applications, such as neutron CT, it can be expensive to collect large amounts of measurements even for a single scan leading to sparse data sets from which it is challenging to obtain high quality reconstructions even with diffusion models. One strategy to mitigate this challenge is to leverage a complementary, easily available imaging modality; however, such approaches typically require retraining the diffusion model with large datasets. In this work, we propose incorporating an additional modality without retraining the diffusion prior, enabling accelerated imaging of costly modalities. We further examine the impact of imperfect side modalities on cross-modal guidance. Our method is evaluated on sparse-view neutron computed tomography, where reconstruction quality is substantially improved by incorporating X-ray computed tomography of the same samples.

Efimov, Timofey [ORNL] (ORCID:000900090098471X)↗
DOE OSTI2026

Multi-modal dynamic radiography using short-pulse laser-generated probe beams

Radiography is an important tool for the interrogation of dynamic experiments in the fields of dynamic properties of materials, and in condensed matter, high explosive, and high-energy-density physics. Multi-modal radiography advances the hypothesis that combining the information delivered by multiple radiographic modalities can lead to more constrained (improved) “reconstruction” of the scene than can be obtained from a single probe. We identify four modalities: multi-probe, time sequence, multi-view, and multi-messenger. Multi-probe radiography is a promising candidate for a next-generation dynamic radiographic facility. High-energy X-rays are the most frequently used probe for dynamic radiography, although recent developments show the utility of proton (pRad), electron (eRad), and neutron probe beams. Because each probing species interacts with material in the radiographic scene through quantitatively different mechanisms, each returns independent information about the scene, which can add extra constraints to the reconstruction process. How to conduct detailed, quantitative “co-analysis” of multiple data streams remains an area of active research. Multi-beam, short-pulse, laser-generated probes offer sufficient dose, an appropriate spectrum, and appropriate spatio-temporal resolution to produce high-quality dynamic radiographs. This paper reports on technology development to advance the state of the art of multi-modal/multi-probe radiography and the pursuit of both deterministic and inferential (AI/ML assisted) co-analysis methodologies to produce more constrained reconstructions from multi-modal data.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Revealing the evolution of order in materials microstructures using multi-modal computer vision

The development of high-performance materials for microelectronics, energy storage, and extreme environments depends on our ability to describe and direct property-defining microstructural order. Our present understanding is typically derived from laborious manual analysis of imaging and spectroscopy data, which is difficult to scale, challenging to reproduce, and lacks the ability to reveal latent associations needed for mechanistic models. Here, we demonstrate a multi-modal machine learning (ML) approach to describe order from electron microscopy analysis of the complex oxide La 1−x Sr x FeO 3 . We construct a hybrid pipeline based on fully and semi-supervised classification, allowing us to evaluate both the characteristics of each data modality and the value each modality adds to the ensemble. We observe distinct differences in the performance of uni- and multi-modal models, from which we draw general lessons in describing crystal order using computer vision.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Gamma-Ray and Cosmic Ray Muon Modalities for Cargo Inspection

Screening and inspection of cargo containers are two essential methods to nondestructively examine the contents of shipment. These methods enable the detection of illicit transportation of unauthorized materials such as nuclear and radioactive materials, explosives, drugs, and so on, typically at borders or secure facilities. Although high-energy X-ray transmission is a standard system and is widely used for cargo inspection, the inherent challenges of high false-positive rates and high attenuation factors necessitate the development of complementary techniques that can increase the detection efficiency and accuracy in large and dense materials. Gamma-rays, which possess higher penetration characteristics because of their high energy, offer an alternative nonintrusive modality for cargo scanning. They represent a promising inspection method when compared to X-rays for three reasons: (1) improved ability to detect nuclear and radioactive materials, (2) higher inspection throughput rates, and (3) lower false-positive rates. Currently, there are two main gamma-ray inspection techniques, active and passive interrogation. Active interrogation can be further grouped into (1) gamma-ray transmission imaging and (2) neutron-induced gamma-ray emission detection. Gamma-ray transmission imaging utilizes differences in material densities for mapping the shipment contents and detecting anomalies. It is analogous to the X-ray transmission method; however, the high-energy photons make it more difficult to shield against, which enables more efficient performance in large and dense material inspection. Neutron-induced gamma-ray emission inspection is designed for the detection of nuclear and radioactive material because those materials emit characteristic gamma-rays when they are activated by neutron absorption. On the other hand, passive interrogation techniques rely on high-efficiency detectors to detect radiation emitted from hidden special nuclear or other radioactive materials. Similar to passive interrogation, cosmic ray muon monitoring and imaging are relatively new techniques that do not require external radioactive sources. These techniques have received attention as a potential next-generation radiographic probe to identify illicit transportation of nuclear and radioactive materials in cargo containers. Cosmic ray muons have unique features, (1) much higher energies than X-rays or gamma-rays (on the order of 10−1—104 GeV), (2) enhanced penetration capability, and (3) natural occurrence, thereby eliminating the need for induced radiation sources. These features enable cosmic ray muons to be utilized for detection of special nuclear materials in high-background-noise environments. By analyzing incoming and outgoing muon trajectories, scattering angles, and energies, it has been shown that it would be possible to locate hidden and well-shielded materials in cargo containers via three-dimensional muon tomography images or signal analysis. Gamma-rays, cosmic ray muons, and other nonintrusive cargo inspection modalities are complementary to each other, allowing them to address various cargo inspection conditions (i.e., scanning time, cost, radiation exposure level, and types of target materials). This chapter presents a detailed review of the theoretical fundamentals and technical principles behind the current gamma-ray and cosmic ray muon modalities for cargo inspection. Additionally, critical assessments and suggestions for the future directions to advance the use of gamma and muon modalities are discussed.

Bae, Junghyun↗
DOE OSTI2024

Stochastic modal velocity field in rough-wall turbulence

Stochastically generated instantaneous velocity profiles are used to reproduce the outer region of rough-wall turbulent boundary layers in a range of Reynolds numbers extending from the wind tunnel to field conditions. Each profile consists in a sequence of steps, defined by the modal velocities and representing uniform momentum zones (UMZs), separated by velocity jumps representing the internal shear layers. Height-dependent UMZ is described by a minimal set of attributes: thickness, mid-height elevation, and streamwise (modal) and vertical velocities. These are informed by experimental observations and reproducing the statistical behaviour of rough-wall turbulence and attached eddy scaling, consistent with the corresponding experimental datasets. Sets of independently generated profiles are reorganized in the streamwise direction to form a spatially consistent modal velocity field, starting from any randomly selected profile. The operation allows one to stretch or compress the velocity field in space, increases the size of the domain and adjusts the size of the largest emerging structures to the Reynolds number of the simulated flow. By imposing the autocorrelation function of the modal velocity field to be anchored on the experimental measurements, we obtain a physically based spatial resolution, which is employed in the computation of the velocity spectrum, and second-order structure functions. The results reproduce the Kolmogorov inertial range extending from the UMZ and their attached-eddy vertical organization to the very-large-scale motions (VLSMs) introduced with the reordering process. The dynamic role of VLSM is confirmed in the –u'w' co-spectra and in their vertical derivative, representing a scale-dependent pressure gradient contribution.

42 ENGINEERING↗
DOE OSTI2024

AstroCLIP: a cross-modal foundation model for galaxies

ABSTRACT We present AstroCLIP, a single, versatile model that can embed both galaxy images and spectra into a shared, physically meaningful latent space. These embeddings can then be used – without any model fine-tuning – for a variety of downstream tasks including (1) accurate in-modality and cross-modality semantic similarity search, (2) photometric redshift estimation, (3) galaxy property estimation from both images and spectra, and (4) morphology classification. Our approach to implementing AstroCLIP consists of two parts. First, we embed galaxy images and spectra separately by pre-training separate transformer-based image and spectrum encoders in self-supervised settings. We then align the encoders using a contrastive loss. We apply our method to spectra from the Dark Energy Spectroscopic Instrument and images from its corresponding Legacy Imaging Survey. Overall, we find remarkable performance on all downstream tasks, even relative to supervised baselines. For example, for a task like photometric redshift prediction, we find similar performance to a specifically trained ResNet18, and for additional tasks like physical property estimation (stellar mass, age, metallicity, and specific-star-formation rate), we beat this supervised baseline by 19 per cent in terms of R2. We also compare our results with a state-of-the-art self-supervised single-modal model for galaxy images, and find that our approach outperforms this benchmark by roughly a factor of two on photometric redshift estimation and physical property prediction in terms of R2, while remaining roughly in-line in terms of morphology classification. Ultimately, our approach represents the first cross-modal self-supervised model for galaxies, and the first self-supervised transformer-based architectures for galaxy images and spectra.

Parker, Liam (ORCID:0009000749521674)↗
DOE OSTI2024

Introducing SpaceNet 9 - Cross-Modal Satellite Imagery Registration for Natural Disaster Responses

Computer vision algorithms are increasingly leveraged to accelerate geospatial analysis for disaster response and recovery. As the diversity of remote sensing imagery grows with optical, SAR, and other modalities, a perquisite for analytics is cross-modal image registration. There is a high potential to harness computer vision for this pre-processing requirement toward enabling downstream analytics such as heterogeneous change detection, automated feature extraction, and data fusion. Advancement in these areas has the potential to simplify data wrangling tasks and further accelerate disaster response timelines. The SpaceNet 9 challenge (launching in mid-2024) focuses on addressing the cross-modal image registration problem and demonstrating the utility of such modules on earthquake impacted scenarios. This paper describes the motivation for the SpaceNet 9 and provides a first overview of the dataset, the baseline algorithm, and implications for seeking cross-modal image registration in Earth observation. Code is available at https://github.com/SpaceNetChallenge/SpaceNet9.

Hansch, Ronny↗
DOE OSTI2025

Modal Analysis and Testing of a Cantilever Beam: Results and Guide

This report discusses modal analysis and testing of a cantilever beam including an introduction to modal testing, simulation, analysis, and a comparison of results. An accompanying presentation can be found from the LLNL CASIS website. Modal testing and analysis are essential for characterizing the response of a structure during vibration environments. Determining natural frequencies and mode shapes helps determine whether resonances will be reached during operations and if so, how the structure will respond and how the response can be tuned. The goal of this work is to convey the basics of modal testing and analysis through a simple project. This can be applied later to more complex, applicable structures. This includes a variety of work including data collection, finite element analysis and data processing. For this project a cantilever beam was chosen because it is a simple, well-characterized structure.

42 ENGINEERING↗
DOE OSTI2024

Beyond the Hype: Navigating the Promise and Pitfalls of Multi-Modal Models for Materials Science

Multi-modal models offer great potential for accelerating discovery in materials and chemical systems, but their adoption raises crucial questions: What materials science challenges are best addressed by multi-modal approaches? How do we weigh the benefits against the resource investment required for multi-modal data acquisition? And critically, how can we optimize experimental workflows to leverage these models effectively? In this presentation, I will delve into the development of multi-modal characterization and analytics, focusing on their application in the demanding fields of next-generation microelectronics and energy storage materials. I will share challenges encountered in designing these workflows, highlighting lessons learned and posing questions that remain unanswered.

AI↗
DOE OSTI2025

Optimization of an Impedance-Matched Test Fixture with the Modal Projection Error

Across many industries and engineering disciplines, components and systems are designed and deployed into their operational environment of intended use. It is the desire of the design agency to be able to predict whether their component or system will function in its shock and vibration environments or if it will fail due to mechanical stresses. One method to determine if the component will survive the shock and vibration environments is to expose the component to the operational environment in a laboratory. One difficulty in executing a representative laboratory test is that the component may not have the same boundary condition in the laboratory as in the operational configuration. This paper examines the use of parameterized optimization to design the test fixture in order to better match the operational configuration. Several frequency and modal-based objective functions are examined for the optimization. The study shows that the Modal Projection Error objective function performs the best of the functions studied. The efficacy of the Modal Projection Error is demonstrated with respect to dynamic test fixture design on several analytical and experimental exemplars.

dynamic↗
DOE OSTI2025

Enhancing Automotive Intrusion Detection Through Multi-Modal Fusion: A CAN FD-LiDAR Approach

As vehicles become smarter and more autonomous, they increasingly depend on advanced sensors and communication technologies to operate securely. However, such growing dependence on technology—whether it’s CAN (Controller Area Network) for internal communication or LiDAR (Light Detection and Ranging) for sensing the world around them—also expands the attack surface for the types of cyber attacks. Traditional intrusion detection systems (IDS) typically monitor these systems in isolation, limiting their ability to detect sophisticated, crosssystem attacks. To address this, we propose a multi-modal fusion approach that combines real-world CAN FD signals (from the HCRL dataset) with LiDAR features (from the nuScenes dataset) to enhance attack detection. Our method employs a twostage ensemble approach. Calibrated XGBoost and LightGBM models initially process CAN FD (Fuzzing Data) and LiDAR data independently, detecting timing anomalies and space abnormalities. They are subsequently logarithmically combined with a logistic regression meta-model along with 17 engineered features capturing cross-modal behavior, prediction conflicts, and nonlinear interactions. This approach achieves an AUC of 0.87 and an F1-score of 0.82, surpassing single-modality baselines and early fusion methods, at merely 2 ms inference latency. Compared with deep learning competitors, it is 3 times more efficient, providing a lightweight, interpretable, and real time solution to automotive cybersecurity.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Curation and Dissemination of Complex Multi-Modal Datasets for Radiation Detection, Localization, and Tracking

The PANDAWN sensor network in Chicago, IL, is a state-of-the-art testbed for networked, multi-modal sensing. It integrates AI/data science methods into its operation, from data acquisition to automated data labeling and curation workflows. The curation and dissemination of diverse multi-modal datasets will enable the development of new radiological/nuclear (R/N) detection, localization, and tracking algorithms and methods relevant across the nonproliferation mission space. This article first introduces the PANDAWN sensor network and the features that make it stand out from previous multi-modal data acquisition efforts. We then review the various data streams acquired on the PANDAWN nodes and present the implementation of an automated data curation pipeline that includes the labeling of radiation and contextual data streams. Here, we finally provide a short overview of different studies that leveraged the curated datasets.

Data curation↗
DOE OSTI2025

Modal focal adaptive optics for Bessel-focus two-photon fluorescence microscopy

Adaptive optics (AO) improves the spatial resolution of microscopy by correcting optical aberrations. While its application has been well established in microscopy modalities utilizing a circular pupil, its adaptation to systems with non-circular pupils, such as Bessel-focus two-photon fluorescence microscopy (2PFM) with an annular pupil, remains relatively uncharted. Herein, we present a modal focal AO (MFAO) method for Bessel-focus 2PFM. Measuring and correcting aberration using a spatial light modulator placed in conjugation with the focal plane of the microscope objective, MFAO employs Zernike annular polynomials — a first in AO implementation — to achieve performance on par with a previous zonal AO method, but with a notably simplified optical configuration. We validated the performance of MFAO in correcting artificial and sample-induced aberrations, as well as in in vivo imaging of zebrafish larvae and mouse brains. By expanding the application of modal AO to annular pupils as well as aberration measurement and correction to a wavefront modulator at the objective focal plane, MFAO represents a notable advancement in the implementation of AO in microscopy.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Towards Secure Autonomous Vehicles: An Integrated Edge and Multi-Modal Machine Learning Framework for Intrusion Detection

Autonomous vehicles (AVs) are vulnerable to cyberattacks targeting both internal communication networks and external perception sensors. While edge-based intrusion de- tection for Controller Area Network (CAN) buses offers real-time protection, it cannot detect cross-modal threats. Conversely, multi-modal fusion approaches improve coverage but often lack efficiency for in-vehicle deployment. This thesis integrates two complemen- tary solutions: (1) a lightweight, edge-deployable machine learning framework for CAN bus intrusion detection, and (2) a late-fusion system combining CAN FD and LiDAR data. Together, they form a hierarchical defense capable of handling single-modality and coordi- nated attacks. Simulations show that CAN-only models reach 93% accuracy on simulated DoS, spoofing, replay, and fuzzy attacks, while the fusion system achieves 0.87 AUC and 0.82 F1-score at 2 ms latency. This unified framework establishes a scalable, explainable, and field-ready strategy for AV cybersecurity.

97 MATHEMATICS AND COMPUTING↗