Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “multimodal data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

AI-supported mini-labs: combining smartphone-based experiments and multimodal AI

Multimedia learning has always been a topic in physics education—whether through specialized digital data acquisition systems, simulations, or animations. Due to increasing digitalization, more and more media from our everyday lives have also found their way into education in recent decades. This contribution introduces a conceptual framework and illustrative examples for AI-supported experimentation in physics education. It does not yet present empirical data but outlines directions for forthcoming classroom studies evaluating learning outcomes and engagement.

generative AI↗
DOE OSTI2026

Characterization of Electronic Stress-Induced Changes in Multilayer MoS 2

Transition metal dichalcogenides like molybdenum disulfide (MoS 2 ) are compelling for next-generation electronic devices. In this work, we investigate the impact of electronic stress on MoS 2 to illustrate that observational and phenomenological information on multiple devices can be useful to describe changes in the device, and caution against the rationalization of paltry results as representative or correlative to device behavior. Here, we stress MoS 2 by applying a sustained 20 V DC bias to study the material’s response. Post-stress electronic characterization revealed nonuniform shifts in current–voltage (I–V) behavior alongside microscale changes. Complementary mechanical, spectroscopic, and scanning microwave impedance measurements showed that stress-induced features locally modulate stiffness, surface potential, Raman intensity, and charge carrier density. We correlated I–V behavior with morphological features (wrinkles, tears, folds, height) and device-level geometry (MoS 2 overlap with electrodes, channel area, contact length) on 50 test structures across five chips to move beyond anecdotal conclusions. We found no universal correlations before DC stress. However, device-level geometry was correlated with I–V behavior after DC stress, suggesting that electrode contacts play a more dominant role than morphology in determining performance. Delamination and thinning induced by DC stress led to localized reductions in charge carrier density within the affected regions. Further, delamination and thinning appear to map to I–V device performance in a few samples, but the correlation is lost when a larger sample size is considered. This suggests significant sample-to-sample variability in surface electronic states of the test structures. We also discuss how environmental factors introduced during fabrication may contribute to the observed heterogeneous device response. Progress will require high-resolution, multimodal analysis across many samples constructed under controlled, clean conditions. By building data sets that capture variability, we can better identify the true drivers of performance.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Decoding diffraction and spectroscopy data with machine learning: A tutorial

This Tutorial provides a step-by-step guide on how to apply supervised machine-learning techniques to analyze diffraction and spectroscopy data. This Tutorial details four models—a reconstruction-focused model, a regression-focused model, a hybrid reconstruction/regression model, and a multimodal model—that use x-ray diffraction profiles and vibrational density of states spectra to predict various microstructural descriptors. In this Tutorial, we cover data pre-processing steps, constructions of the models via dimensionality reduction and regression, training, and analysis of these models. Comparisons of the model’s performance are provided, highlighting the strength and weakness of the various approaches utilized.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Identification of Blue Horizontal Branch Stars with Multimodal Fusion

Blue Horizontal Branch stars (BHBs) are ideal tracers to probe the global structure of the milky Way (MW), and the increased size of the BHB star sample could be helpful to accurately calculate the MW's enclosed mass and kinematics. Large survey telescopes have produced an increasing number of astronomical images and spectra. However, traditional methods of identifying BHBs are limited in dealing with the large scale of astronomical data. A fast and efficient way of identifying BHBs can provide a more significant sample for further analysis and research. Therefore, in order to fully use the various data observed and further improve the identification accuracy of BHBs, we have innovatively proposed and implemented a Bi-level attention mechanism-based Transformer multimodal fusion model, called Bi-level Attention in the Transformer with Multimodality (BATMM). The model consists of a spectrum encoder, an image encoder, and a Transformer multimodal fusion module. The Transformer enables the effective fusion of data from two modalities, namely image and spectrum, by using the proposed Bi-level attention mechanism, including cross-attention and self-attention. As a result, the information from the different modalities complements each other, thus improving the accuracy of the identification of BHBs. The experimental results show that the F1 score of the proposed BATMM is 94.78%, which is 21.77% and 2.76% higher than the image and spectral unimodality, respectively. It is therefore demonstrated that higher identification accuracy of BHBs can be achieved by means of using data from multiple modalities and employing an efficient data fusion strategy.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRSDate not supplied

PACE OCI Short-Wave Infrared Detection Assembly Optical System Design, Alignment, and Environmental Test

The Ocean Color Instrument (OCI), which will be integrated with the Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) satellite, will collect science data that will be used to monitor the health of Earth’s oceans and atmosphere. The Short-Wave Infrared (SWIR) Detection Assembly (SDA), built and characterized by Utah State University Space Dynamics Laboratory (SDL), is a subsystem of OCI consisting of 32 channels covering seven discrete optical bands of interest. A total of 16 SWIR Detection Subassemblies (SDSs) compose the SDA and house the cold optical system. The science data optical input for each SDS is supplied by a 0.22 NA multimode fiber interfacing with a fiber adapter. The diverging light from the fiber is collimated, split by a dichroic beamsplitter to two separate channels, filtered by the science filter, and then reimaged onto the single-element detectors with a final 0.76 NA. Aspheric, diamond-turned powered elements are used throughout the optical design. Fabrication and alignment tolerance analysis/budgets are balanced to ensure the optical system meets throughput requirements. All systems are aligned at ambient temperature using an InSb camera and an in-line illumination microscope system to directly image the active detector area through the science filters. Compensators used during alignment are detector focus and decenter, which are adjusted via photoetched shims in increments of 25 µm. Average focus and centering errors were less than 8 µm among all 32 flight and 10 flight spare detectors. Each SDS spectral response and conversion gain was verified at operational temperature of -65°C in vacuum.

OCI↗
DOE OSTI2023

Transition from wall modes to multimodality in liquid gallium magnetoconvection

Coupled laboratory-numerical experiments of Rayleigh-Bénard convection in liquid gallium subject to a vertical magnetic field are presented. The experiments are carried out in two cylindrical containers with diameter-to-height aspect ratio Γ = 1.0 and 2.0 at varying thermal forcing (Rayleigh numbers 10 5 ≲ Ra ≲ 10 8 ) and magnetic field strength (Chandrasekhar numbers 0 ≲ Ch ≲ 3 × 10 5 ). Laboratory measurements and numerical simulations confirm that magnetoconvection in our finite cylindrical tanks onsets via nondrifting wall-attached modes, in good agreement with asymptotic predictions for a semi-infinite domain. With increasing supercriticality, the experimental and numerical thermal measurements and the numerical velocity data reveal transitions between wall mode states with different azimuthal mode numbers and between wall-dominated convection to wall and interior multimodality. These transitions are also reflected in the heat transfer data, which combined with previous studies connect onset to supercritical turbulent behaviors in liquid metal magnetoconvection over a large parameter space. The gross heat transfer behaviors between magnetoconvection and rotating convection in liquid metals are compared and discussed.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Maven: a multimodal foundation model for supernova science

Abstract A common setting in astronomy is the availability of a small number of high-quality observations, and larger amounts of either lower-quality observations or synthetic data from simplified models. Time-domain astrophysics is a canonical example of this imbalance, with the number of supernovae observed photometrically outpacing the number observed spectroscopically by multiple orders of magnitude. At the same time, no data-driven models exist to understand these photometric and spectroscopic observables in a common context. Contrastive learning objectives, which have grown in popularity for aligning distinct data modalities in a shared embedding space, provide a potential solution to extract information from these modalities. We present Maven, the first foundation model for supernova science. To construct Maven, we first pre-train our model to align photometry and spectroscopy from 0.5 M synthetic supernovae using a contrastive objective. We then fine-tune the model on 4702 observed supernovae from the Zwicky transient facility. Maven reaches state-of-the-art performance on both classification and redshift estimation, despite the embeddings not being explicitly optimized for these tasks. Through ablation studies, we show that pre-training with synthetic data improves overall performance. In the upcoming era of the Vera C. Rubin observatory, Maven will serve as a valuable tool for leveraging large, unlabeled and multimodal time-domain datasets.

Zhang, Gemma (ORCID:0000000280198082)↗
DOE OSTI2021

Predictability as a probe of manifest and latent physics: The case of atomic scale structural, chemical, and polarization behaviors in multiferroic Sm-doped BiFeO 3

The predictability of a certain effect or phenomenon is often equated with the knowledge of relevant physical laws, typically understood as a functional or numerically derived relationship between the observations and known states of the system. Correspondingly, observations inconsistent with prior knowledge can be used to derive new knowledge on the nature of the system or indicate the presence of yet unknown mechanisms. In this work, we explore the applicability of Gaussian processes (GP) to establish predictability and uncertainty of local behaviors from multimodal observations, providing an alternative to this classical paradigm. Using atomic resolution scanning transmission electron microscopy (STEM) of multiferroic Sm-doped BiFeO 3 across a broad composition range, we directly visualize the atomic structure and structural, physical, and chemical order parameter fields for the material. GP regression is used to establish the predictability of the local polarization field from different groups of parameters, including the adjacent polarization values and several combinations of physical and chemical descriptors, including lattice parameters, column intensities, etc. We observe that certain elements of microstructure, including charged and uncharged domain walls and interfaces with the substrate, are best predicted with specific combinations of descriptors, and this predictability and associated uncertainties are consistent across the composition series. The associated generative physical mechanisms are discussed. It is also found that certain parameter combinations tend to predict the orthorhombic phase in the cases where rhombohedral phase is observed, suggesting a potential role of clamping and confinement phenomena in phase equilibrium in Sm-BiFeO 3 system close to morphotropic phase boundary. We argue that predictability and uncertainty in observational data offer a new pathway to probe the physics of condensed matter systems from multimodal local observations.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2025

A Dynamic Hierarchical Attention Framework for Multimodal Malware Detection

The increasing use of Android in the worldwide mobile ecosystem has come along with a significant increase in advanced malware, highlighting the critical necessity for efficient, scalable, and adaptable detection systems. Despite recent advancements in machine learning improving malware detection, the majority of current solutions are limited to one, two, or three data modalities, hence neglecting the comprehensive behavioral spectrum of contemporary multi-vector threats. This thesis presents the first comprehensive multimodal framework for Android malware detection, which combines textual, time-series (temporal), graph-based (structural), and visual information using an innovative hierarchical attention mechanism and Dynamic Fusion Controller (DFC). Our methodology consistently classifies and processes modalities as either sequential or structural, facilitating content-adaptive weighting and resilient cross-modal representation learning. We advance the implementation of cutting-edge time series techniques, such as MiniRocket, for malware detection, hence creating new opportunities for temporal analysis in cybersecurity. Comprehensive experimental assessment shows that our framework performs exceptionally well, with 99.46% classification accuracy and 97.15% detection accuracy, significantly outperforming existing approaches through effective multimodal integration and hierarchical attention mechanisms.

Nazmin, Tamanna↗
NASA NTRS2012

Multimodal information Management: Evaluation of Auditory and Haptic Cues for NextGen Communication Displays

Auditory communication displays within the NextGen data link system may use multiple synthetic speech messages replacing traditional ATC and company communications. The design of an interface for selecting amongst multiple incoming messages can impact both performance (time to select, audit and release a message) and preference. Two design factors were evaluated: physical pressure-sensitive switches versus flat panel "virtual switches", and the presence or absence of auditory feedback from switch contact. Performance with stimuli using physical switches was 1.2 s faster than virtual switches (2.0 s vs. 3.2 s); auditory feedback provided a 0.54 s performance advantage (2.33 s vs. 2.87 s). There was no interaction between these variables. Preference data were highly correlated with performance.

Begault, Durand R.↗
DOE OSTI2024

Utah FORGE: Well 16B(78)-32 DTS, RFS DSS Strain, and Absolute Strain Circulation Test Fiber Optic Data - August 2024

This dataset contains processed fiber optic measurements collected during the extended cross-well circulation test at the Utah FORGE site in August 2024. The data was acquired from the 16B(78)-32 well using distributed fiber optic sensing (DFOS) technology, including distributed temperature sensing (DTS) on multimode fiber and distributed strain sensing (DSS) measurements on single-mode fiber. The dataset includes Brillouin absolute strain, Rayleigh frequency shift (RFS) DSS strain change, RFS DSS strain change rate, and temperature (DTS) data, all stored in HDF5 format. Time coordinates are provided in UTC, and depth measurements are given in measured depth relative to the rotary kelly bushing (MD RKB) in feet. The spatial sampling on the RFS DSS strain data and the Brillouin Absolute Total Strain data is 20 cm and the spatial sampling on the DTS data is 1m. The dataset is accompanied by a report from Neubrex, which provides further documentation.

15 GEOTHERMAL ENERGY↗
DOE OSTI2021

Artificial generation of representative single Li-ion electrode particle architectures from microscopy data

Accurately capturing the architecture of single lithium-ion electrode particles is necessary for understanding their performance limitations and degradation mechanisms through multi-physics modeling. Information is drawn from multimodal microscopy techniques to artificially generate LiNi 0.5 Mn 0.3 Co 0.2 O 2 particles with full sub-particle grain detail. Statistical representations of particle architectures are derived from X-ray nano-computed tomography data supporting an ‘outer shell’ model, and sub-particle grain representations are derived from focused-ion beam electron backscatter diffraction data supporting a ‘grain’ model. A random field model used to characterize and generate the outer shells, and a random tessellation model used to characterize and generate grain architectures, are combined to form a multi-scale model for the generation of virtual electrode particles with full-grain detail. This work demonstrates the possibility of generating representative single electrode particle architectures for modeling and characterization that can guide synthesis approaches of particle architectures with enhanced performance.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Used Nuclear Fuel Management Using the Next Generation System Analysis Model

The U.S. Department of Energy (DOE) is leading the National effort to manage the back end of the nuclear fuel cycle, encompassing the safe transportation, storage/staging, and/or eventual disposal of used nuclear fuel (UNF) and high-level radioactive waste. The Next Generation System Analysis Model (NGSAM) is DOE’s discrete-event, agent-based simulation tool designed to model the full life cycle of UNF from reactor discharge to final disposal. NGSAM supports the DOE Office of Spent Fuel and High-Level Waste Disposition by enabling a detailed, scenario-based analysis of logistics, infrastructure, and shipping strategies. NGSAM replaces legacy models with a modern, flexible platform built on Repast Simphony and enhanced by the Process Analysis Tool. NGSAM simulates the movement and interaction of individual fuel assemblies with system components such as canisters, casks, railcars, and facilities. The model integrates with the Java Transportation Operations Model to plan and execute transportation scenarios, supporting both constrained and unconstrained resource allocation. Key features include customizable allocation and acceptance algorithms, detailed facility-level operations, and a Quick Edit tool for rapid scenario adjustments. NGSAM supports multimodal transportation modeling (e.g. rail, road, barge) and provides comprehensive cost, schedule, and infrastructure data. NGSAM utilizes data from sources such as DOE’s STANDARDS UNF database and DOE’s Stakeholder Tool for Assessing Radioactive Transportation, while also allowing user-defined inputs for scenario customization. NGSAM enables stakeholders to evaluate complex UNF management strategies, assess system performance under varying assumptions, and inform decision making for future infrastructure investments. Its modular architecture and integration with other Integrated Waste Management System tools make it a critical asset for planning the safe and efficient disposition of the Nation’s growing UNF inventory.

Craig, Brian [Argonne National Laboratory (ANL)]↗
NASA NTRS1980

Far-field radiation of AFT turbofan noise

The far-field radiation from the end of an exhaust duct is studied using both approximate and exact methods. Experimental data of narrow-band tone noise from static tests are compared to a multimodal radiation pattern. It is pointed out that possibly the exhaust noise in the far-field is inherently more difficult to attenuate than an inlet noise using duct suppressors.

Rice, E. J.↗
DOE OSTI2024

Unveiling the nanoscale architectures and dynamics of protein assembly with in situ atomic force microscopy

Proteins play a vital role in different biological processes by forming complexes through precise folding with exclusive inter- and intra-molecular interactions. Understanding the structural and regulatory mechanisms underlying protein complex formation provides insights into biophysical processes. Furthermore, the principle of protein assembly gives guidelines for new biomimetic materials with potential applications in medicine, energy, and nanotechnology. Atomic force microscopy (AFM) is a powerful tool for investigating protein assembly and interactions across spatial scales (single molecules to cells) and temporal scales (milliseconds to days). It has significantly contributed to understanding nanoscale architectures, inter- and intra-molecular interactions, and regulatory elements that determine protein structures, assemblies, and functions. This review describes recent advancements in elucidating protein assemblies with in situ AFM. We discuss the structures, diffusions, interactions, and assembly dynamics of proteins captured by conventional and high-speed AFM in near-native environments and recent AFM developments in the multimodal high-resolution imaging, bimodal imaging, live cell imaging, and machine-learning-enhanced data analysis. These approaches show the significance of broadening the horizons of AFM and enable unprecedented explorations of protein assembly for biomaterial design and biomedical research.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Computational Design of Alloys for Energy Technologies

Advanced materials that maintain their mechanical performance under elevated temperatures, corrosive environments, and a range of static and evolving stresses are needed to improve the efficiency and reduce the environmental impact of a wide spectrum of energy technologies. For instance, cost-efficient alloys that can withstand high temperatures (e.g., 700 °C) have a critical role in improving the efficiency and economics of power generation to support decarbonization of the energy sector; such is true of both the nuclear and fossil energy sectors. Considering both the threats of the energy crisis, namely soaring costs of greenhouse gas emission-producing energy and climate change, it is essential to increase the pace of material discovery and enable rapid paths for material qualification to advance clean energy technologies. Conventionally, alloy development has followed a slow Edisonian process that uses repeated cycles of making, characterizing, and modifying to arrive at optimum composition and processing conditions to achieve the desired component performance. This optimization is followed by the necessary stepwise materials qualification. Furthermore, the increasing adoption of sound data management and physics-informed machine learning represents the next step in the acceleration of materials design and development. In the integrated computational materials engineering (ICME) approach, computational modeling and simulation data from different length and time scales can be combined with complex microstructural details from multimodal experimental characterization and selective property testing to close the design loop for rapid alloy development.

Computational Design Of Materials↗
DOE OSTI2024

SCITUNE: Aligning Large Language Models with Human-Curated Scientific Multimodal Instructions

Instruction finetuning is a popular paradigm to align large language models (LLM) with human intent. Despite its popularity, this idea is less explored in improving the LLMs to align existing foundation models with scientific disciplines, concepts and goals. In this work, we present SciTune as a tuning framework to improve the ability of LLMs to follow scientific multimodal instructions. To test our methodology, we use a human-generated scientific instruction tuning dataset and train a large multimodal model LLaMA-SciTune that connects a vision encoder and LLM for science-focused visual and language understanding. LLaMA-SciTune significantly outperforms the state-of-the-art models in the generated figure types and captions in multiple scientific multimodal benchmarks. In comparison to the models that are fine-tuned with machine generated data only, LLaMA-SciTune surpasses human performance on average and in many sub-categories on the ScienceQA benchmark.

• Artificial intelligence (AI) / machine learning ↗