Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “multimodal data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

MTUQ: a framework for estimating moment tensors, point forces, and their uncertainties

SUMMARY We introduce MTUQ, an open-source Python package for seismic source estimation and uncertainty quantification, emphasizing flexibility and operational scalability. MTUQ provides MPI-parallelized grid search and global optimization capabilities, compatibility with 1-D and 3-D Green’s function database formats, customizable data processing, C-accelerated waveform and first-motion polarity misfit functions, and utilities for plotting seismic waveforms and visualizing misfit and likelihood surfaces. Applicability to a range of full- and constrained-moment tensor, point force, and centroid inversion problems is possible via a documented application programming interface, accompanied by example scripts and integration tests. We demonstrate the software using three different types of seismic events: (1) a 2009 intraslab earthquake near Anchorage, Alaska; (2) an episode of the 2021 Barry Arm landslide in Alaska; and (3) the 2017 Democratic People’s Republic of Korea underground nuclear test. With these events, we illustrate the well-known complementary character of body waves, surface waves, and polarities for constraining source parameters. We also convey the distinct misfit patterns that arise from each individual data type, the importance of uncertainty quantification for detecting multimodal or otherwise poorly constrained solutions, and the software’s flexible, modular design.

58 GEOSCIENCES↗

Multimodal Few-Shot Segmentation of Electron Micrographs

Scanning transmission electron microscopy (STEM) is one of the most used methods of analyzing the chemistry and composition of materials. By analyzing microstructures, these microscopes can help scientists better understand the molecular underpinnings of microelectronics, batteries, and more. However, STEM data can be difficult to interpret, so recent developments have been made in applications of machine learning to analyze these images. The PNNL-developed pyCHIP Classifier has achieved results in segmenting STEM these images via few-shot learning, a method which requires little data and human input, perfect for quickly analysis. In my internship I (Eli Meyers) investigated a multimodal improvement of this classifier by incorporating energy dispersive x-ray spectroscopy (EDS) data into the classification process for a more accurate segmentation. Furthermore, I encoded the spectral data by training a mass spectrometry encoder on the EDS data to extract a more meaningful representation of the data.

36 MATERIALS SCIENCE↗

AI-Based Analytics and Energy Modeling Framework for Characterizing Urban Energy Systems

Developing location-specific district energy models is essential for understanding energy patterns and supporting efficient management and planning decisions. However, accurately characterizing these models remains challenging due to gaps in building characteristics and labor-intensive traditional modeling workflows. To address these challenges, we develop an AI-based framework that integrates top-down and bottom-up building energy data to automate urban energy model characterization. The framework trains multimodal deep learning models using heterogeneous ResStockTM datasets to infer missing building characteristics from varying levels of known information and generate simulation-ready inputs for district-scale energy modeling. It also employs a conditioning-based injection approach to generate ”what-if” scenarios, enabling users to explore retrofit, efficiency, and technology-upgrade pathways. Integrated within URBANoptTM, a bottom-up district energy modeling platform for simulating co-located buildings, the framework infers detailed building-level inputs required for bottom-up simulations. Both localized and generalized AI models are developed to learn relationships across categorical, numerical, and time-series data, enabling reconstruction of missing attributes and generation of targeted upgrade scenarios. We demonstrate this methodology on a residential neighborhood in Baltimore, MD, assessing internal consistency against ResStock reference data and URBANopt simulation, and comparing selected attributes against real-world building characteristics. Results show strong overall predictive accuracy in data completion and scenario generation, with localized and generalized models offering complementary trade-offs between precision and scalability. Overall, our automated framework streamlines energy modeling and provides a reliable framework for urban building energy characterization.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Toward particle accelerator machine state embeddings as a modality for large language models

Understanding and diagnosing the state of a particle accelerator requires navigating high-dimensional control system data, often involving hundreds of interdependent parameters. We propose a novel multimodal embedding framework that jointly learns representations of machine states from both numerical control system readouts and natural language descriptions. This enables the translation of complex machine conditions into human-readable summaries while maintaining fidelity to the underlying physical system. The obtained embeddings are subsequently adapted to an open-weights large language model via cross-attention conditioning. We demonstrate a first implementation trained on European XFEL machine state data. This work covers the embedding model architecture, training methodology, and presents initial examples demonstrating the model's capabilities in action. Due to the general concept of machine state, the model can be easily adapted to other facilities and control system environments.

Accelerator Physics↗

Experimental Assessment of Mammalian Lipidome Complexity Using Multimodal 21 T FTICR Mass Spectrometry Imaging

Herein, we assess the complementarity and complexity of data that can be detected within mammalian lipidome mass spectrometry imaging (MSI) via matrix-assisted laser desorption ionization (MALDI) and nanospray desorption electrospray ionization (nano-DESI). We do so by employing 21 T Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) with absorption mode FT processing in both cases, allowing unmatched mass resolving power per unit time (≥613k at m/z 760, 1.536 s transients). While our results demonstrated that molecular coverage and dynamic range capabilities were greater in MALDI analysis, nano-DESI provided superior mass error, and all annotations for both modes had sub-ppm error. Taken together, these experiments highlight the coverage of 1676 lipids and serve as a functional guide for expected lipidome complexity within nano-DESI-MSI and MALDI-MSI. To further assess the lipidome complexity, mass splits (i.e., the difference in mass between neighboring peaks) within single pixels were collated across all pixels from each respective MSI experiment. The spatial localization of these mass splits was powerful in informing whether the observed mass splits were biological or artificial (e.g., matrix related). Mass splits down to 2.4 mDa were observed (i.e., sodium adduct ambiguity) in each experiment, and both modalities highlighted comparable degrees of lipidome complexity. Further, we highlight the persistence of certain mass splits (e.g., 8.9 mDa; double bond ambiguity) independent of ionization biases. Here, we also evaluate the need for ultrahigh mass resolving power for mass splits ≤4.6 mDa (potassium adduct ambiguity) at m/z > 1000, which may only be resolved by advanced FTICR-MS instrumentation.

21T-FTICR-MS↗

Impact of multimode range and location on urban fuel economy on a light-duty spark-ignition based powertrain using vehicle system simulations

Multimode engine operation uses two or more combustion modes to maximize engine efficiency across the operational range of a vehicle to achieve higher overall vehicle fuel economy than is possible with a single combustion mode. More specifically for this study, multimode solutions are explored that make use of boosted SI under high load operation and other advanced combustion modes such as advanced compression ignition (ACI) under part-load conditions to enable additional engine efficiency improvements across a broader range of the engine operating map. ACI combustion has well-documented potential to improve efficiency and emissions under part-load operation but poses challenges that limit full engine speed-load range. This study investigates the potential impact of ACI operational range on simulated fuel economy to help focus research on areas with the most opportunity for improving fuel economy. These simulations make use of a vehicle model, discretized engine data, and employ a systematic exploration of ACI operational range to estimate multimode fuel economy for a mid-size passenger vehicle over U.S. Environmental Protection Agency’s Urban Dynamometer Driving Schedule. The results of this study highlight operational ranges with the highest potential fuel economy and correspondingly areas of focus for multimode research and ACI engine operation1.

Curran, Scott↗

Convergent Manufacturing of Large-Scale Components for Nuclear Applications, via Additive Manufacturing and Powder Metallurgy Hot Isostatic Pressing

Powder metallurgy (PM)–hot isostatic pressing (PM-HIP) has long been recognized as a powerful route for producing fully dense, near net shape metallic components. By consolidating powders under high temperature and pressure, HIP provides isotropic properties, uniform microstructures, and scalability to complex geometries that are vital for sectors such as aerospace, energy, and nuclear power. Yet despite these advantages, the technology has remained constrained by costly trial and error canister fabrication, limitations of conventional forging, and incomplete knowledge about how the canister design influences final part properties. Additive manufacturing (AM), by contrast, thrives on design freedom and geometric flexibility but struggles with speed, scalability, and cost when applied to very large structures. The research presented in this report investigated how a convergent manufacturing approach, combining AM with PM-HIP, can merge the strengths of both technologies, leveraging AM’s flexibility for canister design and HIP’s consolidation capability to deliver reliable, large, and complex parts. The work progressed through three case studies that built on one another in scale and complexity. Small cylindrical canisters fabricated by conventional methods, laser powder bed fusion, and directed energy deposition were filled with stainless steel powders and subjected to HIP. The resulting parts demonstrated near-full density and mechanical properties on par with wrought stainless steel, showing for the first time that AM canisters can be a direct substitute for conventional ones without sacrificing quality. The next step involved a medium-scale, noncentrosymmetric T-valve, which is an enclosed, multibranch geometry that tested the limits of AM + PM-HIP integration. The T-valve achieved predictable shrinkage and uniform densification, confirming feasibility for enclosed designs. However, this study also revealed oxide inclusions and interfacial challenges at the AM + HIP boundary, underscoring the critical importance of controlling interface chemistry and employing robust, in situ strategies, such as melt pool monitoring and thermal monitoring, coupled with nondestructive evaluation techniques such as x-ray computed tomography. Finally, the effort culminated in fabricating a large-scale impeller weighing nearly 2000 lb and spanning 5 ft in diameter. Produced via multirobot wire arc AM and hot isostatic pressed to near-full density, the impeller validated industrial-scale feasibility. Predictive models closely matched experimental shrinkage, tensile properties were spatially uniform across the component, and the AM + PM-HIP interface proved mechanically sound despite the presence of oxide-decorated prior particle boundaries. This large-scale demonstration is a major milestone, showing that hybrid AM + PM‑HIP can reliably deliver components at reactor-relevant scales. Collectively, these studies charted a logical pathway: small-scale work built scientific confidence, medium-scale work highlighted opportunities and challenges, and large-scale work proved industrial impact. The overarching conclusion of this report is that AM + PM-HIP should not be seen as a replacement for forging but as a complementary pathway that provides the US with flexibility, resilience, and new options for manufacturing nuclear-grade components. Looking ahead, several directions emerge as critical to sustaining progress. Predictive modeling must become faster, more accessible, and more accurate, with digital twins and machine learning reducing reliance on trial and error. Powders and alloys must be optimized for HIP, with improved cleanliness, reduced oxides, and tailored chemistries that enhance creep, fatigue, and irradiation resistance. Interfaces between AM and HIP regions must be better engineered through coatings, machining strategies, and surface treatments to mitigate oxide formation and ensure reliable bonding to explore opportunities for HIP of targeted compositional parts, as well as multimaterial HIP cladding applications. Monitoring and nondestructive evaluation need to expand, incorporating multimodal sensors, x-ray computed tomography, and real-time data integration through platforms such as Pelican. At the same time, the pathway to industrial adoption requires techno-economic analysis, machinability studies, and qualification frameworks aligned with industry and regulatory standards. Finally, workforce and academic engagement must be strengthened. Programs that train technicians and engineers for US Navy and US Department of Energy manufacturing challenges should be paired with academic partnerships to support fundamental research, with open sharing of non-export-controlled data to accelerate innovation and build the next generation of experts. In conclusion, this report demonstrates that hybrid AM + PM-HIP is scientifically viable and strategically important. By combining the design agility of AM with the consolidation strength of HIP and embedding modeling, monitoring, and workforce development, this approach provided a transformative new capability for US manufacturing. The path forward is clear: hybrid AM + PM-HIP is not just a promising research direction but is also potentially an industrially relevant pathway that can reshape how nuclear-grade components are designed, qualified, and deployed.

36 MATERIALS SCIENCE↗

Iterative Reconstruction for Multimodal Neutron Tomography

Here, we describe a unified framework for model-based iterative 3-D reconstruction of multimodal neutron transmission, hydrogen-scatter, and induced-fission images from low resolution data recorded using 14.1-MeV neutrons and the associated-particle imaging (API) technique. The framework, which was developed to facilitate use in challenging field-deployment scenarios, is centered around physics-based system models and a total variation (TV) constrained implementation of the simultaneous iterative reconstruction technique (SIRT). Modified to solve a statistically weighted least squares (WLS) problem, the SIRT algorithm is accelerated using ordered subsets and Nesterov’s momentum for which we derive a near-optimal value of the governing Lipschitz constant. The approach enables the reconstruction of images that are high resolution compared to the acquired data and is robust to both limited statistics and a limited number of projection angles. Moreover, the framework is fast enough to be practical. Example images are provided that demonstrate both the ability to perform fast-neutron imaging of high-atomic-number materials with low radiation dose and the benefit of multimodal neutron imaging to identify key materials.

Hydrogen scatter↗

AI-supported mini-labs: combining smartphone-based experiments and multimodal AI

Multimedia learning has always been a topic in physics education—whether through specialized digital data acquisition systems, simulations, or animations. Due to increasing digitalization, more and more media from our everyday lives have also found their way into education in recent decades. This contribution introduces a conceptual framework and illustrative examples for AI-supported experimentation in physics education. It does not yet present empirical data but outlines directions for forthcoming classroom studies evaluating learning outcomes and engagement.

generative AI↗

Characterization of Electronic Stress-Induced Changes in Multilayer MoS 2

Transition metal dichalcogenides like molybdenum disulfide (MoS 2 ) are compelling for next-generation electronic devices. In this work, we investigate the impact of electronic stress on MoS 2 to illustrate that observational and phenomenological information on multiple devices can be useful to describe changes in the device, and caution against the rationalization of paltry results as representative or correlative to device behavior. Here, we stress MoS 2 by applying a sustained 20 V DC bias to study the material’s response. Post-stress electronic characterization revealed nonuniform shifts in current–voltage (I–V) behavior alongside microscale changes. Complementary mechanical, spectroscopic, and scanning microwave impedance measurements showed that stress-induced features locally modulate stiffness, surface potential, Raman intensity, and charge carrier density. We correlated I–V behavior with morphological features (wrinkles, tears, folds, height) and device-level geometry (MoS 2 overlap with electrodes, channel area, contact length) on 50 test structures across five chips to move beyond anecdotal conclusions. We found no universal correlations before DC stress. However, device-level geometry was correlated with I–V behavior after DC stress, suggesting that electrode contacts play a more dominant role than morphology in determining performance. Delamination and thinning induced by DC stress led to localized reductions in charge carrier density within the affected regions. Further, delamination and thinning appear to map to I–V device performance in a few samples, but the correlation is lost when a larger sample size is considered. This suggests significant sample-to-sample variability in surface electronic states of the test structures. We also discuss how environmental factors introduced during fabrication may contribute to the observed heterogeneous device response. Progress will require high-resolution, multimodal analysis across many samples constructed under controlled, clean conditions. By building data sets that capture variability, we can better identify the true drivers of performance.

36 MATERIALS SCIENCE↗

Decoding diffraction and spectroscopy data with machine learning: A tutorial

This Tutorial provides a step-by-step guide on how to apply supervised machine-learning techniques to analyze diffraction and spectroscopy data. This Tutorial details four models—a reconstruction-focused model, a regression-focused model, a hybrid reconstruction/regression model, and a multimodal model—that use x-ray diffraction profiles and vibrational density of states spectra to predict various microstructural descriptors. In this Tutorial, we cover data pre-processing steps, constructions of the models via dimensionality reduction and regression, training, and analysis of these models. Comparisons of the model’s performance are provided, highlighting the strength and weakness of the various approaches utilized.

36 MATERIALS SCIENCE↗

Identification of Blue Horizontal Branch Stars with Multimodal Fusion

Blue Horizontal Branch stars (BHBs) are ideal tracers to probe the global structure of the milky Way (MW), and the increased size of the BHB star sample could be helpful to accurately calculate the MW's enclosed mass and kinematics. Large survey telescopes have produced an increasing number of astronomical images and spectra. However, traditional methods of identifying BHBs are limited in dealing with the large scale of astronomical data. A fast and efficient way of identifying BHBs can provide a more significant sample for further analysis and research. Therefore, in order to fully use the various data observed and further improve the identification accuracy of BHBs, we have innovatively proposed and implemented a Bi-level attention mechanism-based Transformer multimodal fusion model, called Bi-level Attention in the Transformer with Multimodality (BATMM). The model consists of a spectrum encoder, an image encoder, and a Transformer multimodal fusion module. The Transformer enables the effective fusion of data from two modalities, namely image and spectrum, by using the proposed Bi-level attention mechanism, including cross-attention and self-attention. As a result, the information from the different modalities complements each other, thus improving the accuracy of the identification of BHBs. The experimental results show that the F1 score of the proposed BATMM is 94.78%, which is 21.77% and 2.76% higher than the image and spectral unimodality, respectively. It is therefore demonstrated that higher identification accuracy of BHBs can be achieved by means of using data from multiple modalities and employing an efficient data fusion strategy.

79 ASTRONOMY AND ASTROPHYSICS↗

Transition from wall modes to multimodality in liquid gallium magnetoconvection

Coupled laboratory-numerical experiments of Rayleigh-Bénard convection in liquid gallium subject to a vertical magnetic field are presented. The experiments are carried out in two cylindrical containers with diameter-to-height aspect ratio Γ = 1.0 and 2.0 at varying thermal forcing (Rayleigh numbers 10 5 ≲ Ra ≲ 10 8 ) and magnetic field strength (Chandrasekhar numbers 0 ≲ Ch ≲ 3 × 10 5 ). Laboratory measurements and numerical simulations confirm that magnetoconvection in our finite cylindrical tanks onsets via nondrifting wall-attached modes, in good agreement with asymptotic predictions for a semi-infinite domain. With increasing supercriticality, the experimental and numerical thermal measurements and the numerical velocity data reveal transitions between wall mode states with different azimuthal mode numbers and between wall-dominated convection to wall and interior multimodality. These transitions are also reflected in the heat transfer data, which combined with previous studies connect onset to supercritical turbulent behaviors in liquid metal magnetoconvection over a large parameter space. The gross heat transfer behaviors between magnetoconvection and rotating convection in liquid metals are compared and discussed.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Maven: a multimodal foundation model for supernova science

Abstract A common setting in astronomy is the availability of a small number of high-quality observations, and larger amounts of either lower-quality observations or synthetic data from simplified models. Time-domain astrophysics is a canonical example of this imbalance, with the number of supernovae observed photometrically outpacing the number observed spectroscopically by multiple orders of magnitude. At the same time, no data-driven models exist to understand these photometric and spectroscopic observables in a common context. Contrastive learning objectives, which have grown in popularity for aligning distinct data modalities in a shared embedding space, provide a potential solution to extract information from these modalities. We present Maven, the first foundation model for supernova science. To construct Maven, we first pre-train our model to align photometry and spectroscopy from 0.5 M synthetic supernovae using a contrastive objective. We then fine-tune the model on 4702 observed supernovae from the Zwicky transient facility. Maven reaches state-of-the-art performance on both classification and redshift estimation, despite the embeddings not being explicitly optimized for these tasks. Through ablation studies, we show that pre-training with synthetic data improves overall performance. In the upcoming era of the Vera C. Rubin observatory, Maven will serve as a valuable tool for leveraging large, unlabeled and multimodal time-domain datasets.

Zhang, Gemma (ORCID:0000000280198082)↗

Predictability as a probe of manifest and latent physics: The case of atomic scale structural, chemical, and polarization behaviors in multiferroic Sm-doped BiFeO 3

The predictability of a certain effect or phenomenon is often equated with the knowledge of relevant physical laws, typically understood as a functional or numerically derived relationship between the observations and known states of the system. Correspondingly, observations inconsistent with prior knowledge can be used to derive new knowledge on the nature of the system or indicate the presence of yet unknown mechanisms. In this work, we explore the applicability of Gaussian processes (GP) to establish predictability and uncertainty of local behaviors from multimodal observations, providing an alternative to this classical paradigm. Using atomic resolution scanning transmission electron microscopy (STEM) of multiferroic Sm-doped BiFeO 3 across a broad composition range, we directly visualize the atomic structure and structural, physical, and chemical order parameter fields for the material. GP regression is used to establish the predictability of the local polarization field from different groups of parameters, including the adjacent polarization values and several combinations of physical and chemical descriptors, including lattice parameters, column intensities, etc. We observe that certain elements of microstructure, including charged and uncharged domain walls and interfaces with the substrate, are best predicted with specific combinations of descriptors, and this predictability and associated uncertainties are consistent across the composition series. The associated generative physical mechanisms are discussed. It is also found that certain parameter combinations tend to predict the orthorhombic phase in the cases where rhombohedral phase is observed, suggesting a potential role of clamping and confinement phenomena in phase equilibrium in Sm-BiFeO 3 system close to morphotropic phase boundary. We argue that predictability and uncertainty in observational data offer a new pathway to probe the physics of condensed matter systems from multimodal local observations.

74 ATOMIC AND MOLECULAR PHYSICS↗

A Dynamic Hierarchical Attention Framework for Multimodal Malware Detection

The increasing use of Android in the worldwide mobile ecosystem has come along with a significant increase in advanced malware, highlighting the critical necessity for efficient, scalable, and adaptable detection systems. Despite recent advancements in machine learning improving malware detection, the majority of current solutions are limited to one, two, or three data modalities, hence neglecting the comprehensive behavioral spectrum of contemporary multi-vector threats. This thesis presents the first comprehensive multimodal framework for Android malware detection, which combines textual, time-series (temporal), graph-based (structural), and visual information using an innovative hierarchical attention mechanism and Dynamic Fusion Controller (DFC). Our methodology consistently classifies and processes modalities as either sequential or structural, facilitating content-adaptive weighting and resilient cross-modal representation learning. We advance the implementation of cutting-edge time series techniques, such as MiniRocket, for malware detection, hence creating new opportunities for temporal analysis in cybersecurity. Comprehensive experimental assessment shows that our framework performs exceptionally well, with 99.46% classification accuracy and 97.15% detection accuracy, significantly outperforming existing approaches through effective multimodal integration and hierarchical attention mechanisms.

Nazmin, Tamanna↗

Utah FORGE: Well 16B(78)-32 DTS, RFS DSS Strain, and Absolute Strain Circulation Test Fiber Optic Data - August 2024

This dataset contains processed fiber optic measurements collected during the extended cross-well circulation test at the Utah FORGE site in August 2024. The data was acquired from the 16B(78)-32 well using distributed fiber optic sensing (DFOS) technology, including distributed temperature sensing (DTS) on multimode fiber and distributed strain sensing (DSS) measurements on single-mode fiber. The dataset includes Brillouin absolute strain, Rayleigh frequency shift (RFS) DSS strain change, RFS DSS strain change rate, and temperature (DTS) data, all stored in HDF5 format. Time coordinates are provided in UTC, and depth measurements are given in measured depth relative to the rotary kelly bushing (MD RKB) in feet. The spatial sampling on the RFS DSS strain data and the Brillouin Absolute Total Strain data is 20 cm and the spatial sampling on the DTS data is 1m. The dataset is accompanied by a report from Neubrex, which provides further documentation.

15 GEOTHERMAL ENERGY↗

Artificial generation of representative single Li-ion electrode particle architectures from microscopy data

Accurately capturing the architecture of single lithium-ion electrode particles is necessary for understanding their performance limitations and degradation mechanisms through multi-physics modeling. Information is drawn from multimodal microscopy techniques to artificially generate LiNi 0.5 Mn 0.3 Co 0.2 O 2 particles with full sub-particle grain detail. Statistical representations of particle architectures are derived from X-ray nano-computed tomography data supporting an ‘outer shell’ model, and sub-particle grain representations are derived from focused-ion beam electron backscatter diffraction data supporting a ‘grain’ model. A random field model used to characterize and generate the outer shells, and a random tessellation model used to characterize and generate grain architectures, are combined to form a multi-scale model for the generation of virtual electrode particles with full-grain detail. This work demonstrates the possibility of generating representative single electrode particle architectures for modeling and characterization that can guide synthesis approaches of particle architectures with enhanced performance.

36 MATERIALS SCIENCE↗