Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “multimodal data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

FY2020 Energy Efficient Mobility Systems Annual Progress Report

EEMS Program activities during FY 2020 focused on analytical research and large-scale modeling and simulation to understand the impacts that new mobility technologies and services will have at the vehicle-, traveler-, and overall transportation system-level. This research included the development of a multi-fidelity, end-to-end transportation system models and tools to evaluate the complex interactions among the various actors within the mobility landscape, analysis of empirical data to characterize which solutions may provide the largest benefits, and development of new control systems and algorithms that use vehicle connectivity and automation to improve the performance and efficiency of individual vehicles as well as the overall traffic system. This document presents a brief overview of the EEMS Program and documents progress and results from projects within each of the EEMS activity areas. The Computational Modeling and Simulation key activity area summarizes work within the sub-areas of (1) the SMART (Systems and Modeling for Accelerated Research in Transportation) Mobility Lab Consortium, (2) Artificial Intelligence, High-Performance Computing, and Data Analytics, and (3) Core Simulation and Evaluation Tools. Additionally, the program’s advanced R&D projects are summarized within (4) the Connectivity and Automation Technology key activity area. Each of the individual progress reports provide a project overview and highlights of the technical results.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2026

A roadmap toward scaling, reasoning and self-evolving foundation models for nuclear and particle physics

Foundation models have revolutionized artificial intelligence, with Large Language Models demonstrating unprecedented capabilities in multimodal understanding, reasoning and tool use. Nuclear and particle physics stands at a critical juncture where similar transformative potential awaits realization. The field generates exabytes of experimental data, exascale simulations, and decades of theoretical insights — yet these remain largely disconnected from modern Artifical Intelligence (AI) capabilities, with most physics AI applications confined to narrow, task-specific models that suffer from domain shifting when applied to real experimental data. We present a roadmap for FM4NPP (Foundation Model for Nuclear and Particle Physics), systematically scaling from current proof-of-concept models to trillion-parameter architectures capable of autonomous discovery. Our approach advances three critical frontiers: unified data infrastructure integrating detector data, scientific knowledge and computational tools across global facilities; multi-facility foundation models enabling cross-experiment knowledge transfer and accelerated discovery; and agentic AI capabilities for reasoning and autonomous tool use. The resulting self-evolving FM4NPP will transform physics research by converting time-intensive data analysis, theory derivation and computational bottlenecks into rapid AI–human collaborative discovery. This paradigm shift promises to fundamentally accelerate scientific progress in nuclear and particle physics, enabling researchers to focus on high-level insights while AI handles routine analysis and explores vast parameter spaces beyond human capacity.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Heterogeneous microstructure of yttrium hydride and its relation to mechanical properties

Here, the goal of this study is to investigate the properties of yttrium hydride materials in relation to the microstructure, especially its homogeneity. High-throughput nanoindentation mapping was used to evaluate hardness distribution. Raman spectral imaging demonstrated its sensitivity to the presence of YH2 and impurities. Raman peak position maps were correlated with residual stress in the specimens. Electron backscatter diffraction mapping provided phase distributions with correlation to high-energy X-ray diffraction analysis. The experimental mapping data were combined and analyzed using unsupervised machine learning cluster procedures. The machine learning analysis revealed that yttrium hydride specimens contained a major δ-YH2 – x phase component and minor α-Y and δ-YH2 – x components with significant residual stress. The minor phase fraction decreased with increasing nominal H/Y ratio, which affected the nanoindentation and Vickers hardness. The multimodal mapping procedures described herein affect developing important microstructure–property relationships, as well as correlations in heterogeneity and mechanical properties.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Detecting Hardware Trojans in PCBs Using Side Channel Loopbacks

Malicious modifications to printed circuit boards (PCBs) are known as hardware Trojans. These may arise when malafide third parties alter PCBs premanufacturing or postmanufacturing and are a concern in safety-critical applications, such as industrial control systems. In this research, we examine how data-driven detection can be utilized to detect such Trojans at run-time. We develop a flexible and reconfigurable PCB test bed derived from the popular open-source programmable logic controller (PLC) platform “OpenPLC.” We then develop a Trojan detection framework, which utilizes and analyzes multimodal side channels (e.g., timing, magnetic signals, power, and hardware performance counters). We consider defender-configurable input/output (I/O) loopback test, comparison with design-document baselines, and magnetometer-aided monitoring of system behavior under defender-chosen excitations. Our approach can extend to golden-free environments. Golden (known-good) versions of the PCBs are assumed not available, but design information, datasheets, and component-level data are available. We demonstrate the efficacy of our approach on a range of Trojans instantiated in the test bed.

42 ENGINEERING↗
DOE OSTI2026

Accelerating Structure–Property Relationship Discovery with Multimodal Machine Learning and Self-Driving Microscopy

Microscopy combined with local spectroscopy is widely used to correlate nanoscale structure with functional properties in materials, but conventional measurements rely heavily on human-selected sampling locations and predefined targets, limiting data set diversity and the potential for discovery. Here, we present a framework that integrates autonomous microscopy with dual-novelty deep kernel learning (DN-DKL) for adaptive data acquisition and a dual variational autoencoder (VAE) for representation learning. DN-DKL actively guides the microscopy toward structurally and spectroscopically novel regions, enabling efficient collection of large spectral data sets. Dual-VAE embeds local structures and spectroscopic responses into a shared latent manifold that serves as a structure–property relationship map. We applied this framework for the investigation of halide perovskite films by using conductive atomic force microscopy. The results reveal distinct hysteresis behaviors that are linked to specific nanoscale structural motifs, including grain boundary junction points that show hysteresis under different bias conditions and asymmetric grain boundaries that suppress the charge transport. This framework establishes a general strategy that leverages the complementary strengths of self-driving microscopy, machine learning, and human expertise to accelerate scientific discovery in functional materials.

atomic force microscopy↗
DOE OSTI2022

Multimodal parameter spaces of a complex multi-channel neuron model

One of the most common types of models that helps us to understand neuron behavior is based on the Hodgkin–Huxley ion channel formulation (HH model). A major challenge with inferring parameters in HH models is non-uniqueness: many different sets of ion channel parameter values produce similar outputs for the same input stimulus. Such phenomena result in an objective function that exhibits multiple modes (i.e., multiple local minima). This non-uniqueness of local optimality poses challenges for parameter estimation with many algorithmic optimization techniques. HH models additionally have severe non-linearities resulting in further challenges for inferring parameters in an algorithmic fashion. To address these challenges with a tractable method in high-dimensional parameter spaces, we propose using a particular Markov chain Monte Carlo (MCMC) algorithm, which has the advantage of inferring parameters in a Bayesian framework. The Bayesian approach is designed to be suitable for multimodal solutions to inverse problems. We introduce and demonstrate the method using a three-channel HH model. We then focus on the inference of nine parameters in an eight-channel HH model, which we analyze in detail. We explore how the MCMC algorithm can uncover complex relationships between inferred parameters using five injected current levels. The MCMC method provides as a result a nine-dimensional posterior distribution, which we analyze visually with solution maps or landscapes of the possible parameter sets. The visualized solution maps show new complex structures of the multimodal posteriors, and they allow for selection of locally and globally optimal value sets, and they visually expose parameter sensitivities and regions of higher model robustness. We envision these solution maps as enabling experimentalists to improve the design of future experiments, increase scientific productivity and improve on model structure and ideation when the MCMC algorithm is applied to experimental data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Investigating the interconnectedness of active transportation and public transit usage as a primer for Mobility-as-a-Service adoption and deployment

With the advent of Mobility-as-a-Service packages to reduce car usage and (by extension) greenhouse gas emissions, it is crucial that researchers and practitioners consider mutual determinants and outcomes that link the adoption of multiple alternative modes. This study therefore investigates the joint usage of active travel (walking, cycling, bike-sharing) and public transit (bus, rail) modes with respect to personal and situational contexts. Online survey data (n = 826) were collected across six Midwestern U.S. states using Amazon MTurk. Respondents indicated their average weekly usage of eight travel modes across three trip purposes so that multimodality could be assessed. Several psychological constructs were extracted via (a) the stages of change framework, used to indicate willingness to adopt new behavior, and (b) confirmatory factor analysis conducted on Likert scales rooted in theories of community, identity, norms, personality, and well-being. A multiple-indicators multiple-causes structural equation model is then employed to investigate the process of adopting a modality style that incorporates active and transit modes. The model confirms that compatible physical and social contexts, as well as navigational skills and openness to learning, are key primers of multimodalism. However, a path juncture stemming from neighborhood support for mobility innovation illustrates a potential polarity in outcomes between individuals and communities. In addition, the stage of active mobility adoption is linked to identity and norm activation, offering further guidance on what influences readiness for change. The seamless integration of mobility services is critical to matching the convenience and comfort of the private vehicle; understanding potential pathways to sustainable mobility, though, requires analyses and interventions that are driven by well-being outcomes and grounded in rigorous psychological frameworks. Here, the research findings offer practical guidance for identifying intervention opportunities to be linked with MaaS enrollment while demonstrating the need to illuminate how mobility might relate to social cohesion, identity expression, and various sources of satisfaction.

42 ENGINEERING↗
DOE OSTI2023

Scaling Subspace-Driven Approaches Using Information Fusion

In this work, we seek to exploit the deep structure of multi-modal data to robustly exploit the group subspace distribution of the information using the Convolutional Neural Networks (CNNs) formalism. Upon unfolding the set of subspaces constituting each data modality, and learning their corresponding encoders, an optimized integration of the generated inherent information is carried out to yield a characterization of various classes. Referred to as deep Multimodal Robust Group Subspace Clustering (DRoGSuRe), this approach is compared against the independently developed state-of-the-art approach named Deep Multimodal Subspace Clustering (DMSC). Experiments on different multimodal datasets show that our approach is competitive and more robust in the presence of noise.

Ghanem, Sally↗
DOE OSTI2024

Agent-Based Simulation Model for Analyzing Multimodal Transportation of CO2 for CCUS

The overarching goal of this work is to determine how the United States can avoid being constrained by transportation limitations as the CCUS process evolves to a full gigaton-level system. The model is data-driven, open to CO2 supply and demand points defined as input data, and designed to allow for transportation on waterways, rail, or pipeline.

Clark, RobinJ↗
DOE OSTI2023

The effect of chemical functional groups on the octane sensitivity of fuel blends for spark-ignited and multimode engines

We report the octane sensitivity of fuel blends containing blendstocks of diverse chemical functionalities blended in concentrations up to 20% by volume into BOBs (Blendstocks for Oxygenate Blending). This study focuses on the blendstocks containing functional groups which lack reliable octane number data. The effects on octane sensitivity of blends of cyclopropanes, alkyl carbonates, oxiranes, alcohols, ketones, esters, nitrogen-containing compounds, cyclic and olefinic hydrocarbons, multifunctional materials containing combinations of these functionalities, and the co-blends of the blendstocks with ethanol in a BOB have been evaluated. Octane sensitivity of the blends containing functional groups that haven’t been evaluated before are discussed.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Mobility Energy Productivity (MEP) Metric: Partnerships, Applications, and Key Enhancements

The Mobility Energy Productivity (MEP) metric is a holistic measure of transportation systems performance that quantifies the ability of individuals to reach destinations in a cost-efficient and energy-efficient manner. This presentation highlights recent partnerships that have advanced the adoption of MEP as a decision-support tool by various agencies, stakeholders, and researchers. Applications include evaluating multimodal accessibility, comparing system-level energy impacts, and informing infrastructure investment strategies. Key enhancements to the metric - such as expanded regional applications and incorporation of emerging technologies - are also discussed. Together, these efforts highlight the potential of MEP to inform data-driven decisions that shape future transportation systems.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Imaginaries of the Pandemic in Chile: A Conceptual‐Empirical Discussion

This article aims to reconstruct the social imaginaries of Coronavirus disease 2019 (COVID‐19) in Chile. We seek to understand how families interpret their experience confronting the pandemic by identifying four main aspects: (a) the COVID‐19 pandemic, (b) working and learning, (c) health and (d) family life. Following Habermas' distinction between lifeworld and social systems, we consider these issues as constituting the social imaginary of lifeworld, different but related to the imaginaries of social systems. The qualitative empirical data was gathered through a sample of 38 families interviewed online between September 2020 and January 2021 in four Chilean cities: Iquique, Valparaíso, Santiago and Concepción. Other complementary sources of information are multimodal ethnography (digital diaries), press articles and state reports.

Vergara, Jorge Iván↗
DOE OSTI2020

Using Neural Architecture Search for Improving Software Flaw Detection in Multimodal Deep Learning Models

Software flaw detection using multimodal deep learning models has been demonstrated as a very competitive approach on benchmark problems. In this work, we demonstrate that even better performance can be achieved using neural architecture search (NAS) combined with multimodal learning models. We adapt a NAS framework aimed at investigating image classification to the problem of software flaw detection and demonstrate improved results on the Juliet Test Suite, a popular benchmarking data set for measuring performance of machine learning models in this problem domain.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Multimodal sensor fusion framework for residential building occupancy detection

For several years now, smart building energy systems have been a research area of intensive activity. In light of the increasing need for sustainable buildings and energy systems, this trend motivates an increasing need for a solution to reduce carbon dioxide emissions and improve energy efficiency. This work proposes a high-performing and transferable occupancy detection framework that combines sensor data from different data modalities, including time series environmental data (temperature, humidity, and illuminance), image data, and acoustic energy data using ensemble method. To draw out the best prediction performance in each modality, the proposed framework was developed, including various models that were designed to learn the occupancy patterns reflected in the physical data streams. To tackle the time series environmental data, we designed two variants of an occupancy detection spatiotemporal pattern network (Occ-STPN) that performs both feature level and decision level fusion, respectively. We also propose a new metric; the fading memory mean square error (FMMSE), that provides a fair evaluation and penalization of delayed occupancy predictions. Multiple open-sourced datasets, including the Electricity Consumption and Occupancy and the University of California, Irvine's (UCI) building occupancy detection dataset, along with our own real data collected from six different houses, were used to validate the algorithms' performance. The experimental results presented herein break down the performance for each sensing modality, and a detailed analysis of the performance is also discussed.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Multimodal CO2 Transportation Cost Model

This model provides a cost estimate for transporting CO2 via truck or rail in the United States using commercially available equipment and technologies. The model includes an analysis of direct and indirect CO2 emissions to determine costs per net tonne of CO2 transported. Publicly available data and methods published in the peer-reviewed literature are used to the extent possible; references are available at the bottom of the "Calculations" sheet. Upstream (i.e., liquefaction, buffer storage) and downstream (i.e., buffer storage, reconditioning) activities are included in the model of emissions and costs. All capital and operating expenditures are estimated in the "Calculations" sheet. The cost of financing the project is determined in the "FINEX" sheet. All user inputs are done via drop down menus in on the "User Interface" sheet. Summary results are also provided on the "User Interface" sheet.

Myers, CoreyA↗
DOE OSTI2020

Progress on Understanding Rayleigh–Taylor Flow and Mixing Using Synergy Between Simulation, Modeling, and Experiment

Simultaneous advances in numerical methods and computing, theoretical techniques, and experimental diagnostics have all led independently to better understanding of Rayleigh–Taylor (RT) instability, turbulence, and mixing. In particular, experiments have provided significant motivation for many simulation and modeling studies, as well as validation data. Numerical simulations have also provided data that is not currently measurable or very difficult to measure accurately in RT unstable flows. Thus, simulations have also motivated new measurements in this class of buoyancy-driven flows. This overview discusses simulation and modeling studies synergistic with experiments and examples of how experiments have motivated simulations and models of RT instability, flow, and mixing. First, a brief summary of measured experimental and calculated simulation quantities, of experimental approaches, and of issues and challenges in the simulation and modeling of RT experiments is presented. Implicit large-eddy, direct numerical, and large-eddy simulations validated using RT experimental data are then discussed. This is followed by a discussion of modeling using analytical, modal, buoyancy–drag, and turbulent transport models of RT mixing experiments. The discussion will focus on three-dimensional RT mixing arising from multimode perturbations. Finally, this focused review concludes with a perspective on future simulation, modeling, and experimental directions for further research. Research in simulation and modeling of RT unstable flows, coupled with experiments, has made significant progress over the past several decades. This overview serves as an opportunity to both discuss progress and to stimulate future research on simulation and modeling of this unique class of hydrodynamically unstable turbulent flows.

42 ENGINEERING↗
DOE OSTI2025

Mind the gap: Bridging the divide between AI aspirations and the reality of autonomous microscopy

What does materials science look like in the “Age of Artificial Intelligence?” Each material’s domain—synthesis, characterization, and modeling—has a different answer to this question, motivated by unique challenges and constraints. This work focuses on the tremendous potential of autonomous characterization within electron microscopy. We present our recent advancements in developing domain-aware, multimodal models for microscopy analysis capable of describing complex atomic systems. We then address the critical gap between the theoretical promise of autonomous microscopy and its current practical limitations, showcasing recent successes while highlighting the necessary developments to achieve robust, real-world autonomy.

2D materials↗