SEARCH · Engineering Papers

Results for “multimodal data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

All sources NASA NTRS DOE OSTI

DOE OSTI2024

ATAT: Astronomical Transformer for time series and Tabular data

Context. The advent of next-generation survey instruments, such as theVera C. RubinObservatory and its Legacy Survey of Space and Time (LSST), is opening a window for new research in time-domain astronomy. The Extended LSST Astronomical Time-Series Classification Challenge (ELAsTiCC) was created to test the capacity of brokers to deal with a simulated LSST stream. Aims. Our aim is to develop a next-generation model for the classification of variable astronomical objects. We describe ATAT, the Astronomical Transformer for time series And Tabular data, a classification model conceived by the ALeRCE alert broker to classify light curves from next-generation alert streams. ATAT was tested in production during the first round of the ELAsTiCC campaigns. Methods. ATAT consists of two transformer models that encode light curves and features using novel time modulation and quantile feature tokenizer mechanisms, respectively. ATAT was trained on different combinations of light curves, metadata, and features calculated over the light curves. We compare ATAT against the current ALeRCE classifier, a balanced hierarchical random forest (BHRF) trained on human-engineered features derived from light curves and metadata. Results. When trained on light curves and metadata, ATAT achieves a macro F1 score of 82.9 ± 0.4 in 20 classes, outperforming the BHRF model trained on 429 features, which achieves a macro F1 score of 79.4 ± 0.1. Conclusions. The use of transformer multimodal architectures, combining light curves and tabular data, opens new possibilities for classifying alerts from a new generation of large etendue telescopes, such as theVera C. RubinObservatory, in real-world brokering scenarios.

Astronomy & Astrophysics↗

DOE OSTI2023

Applying Multimodal Mass Spectrometry to Image Tumors Undergoing Ferroptosis Following In Vivo Treatment with a Ferroptosis Inducer

Epithelial ovarian cancer (EOC) is the most common form of ovarian cancer. The poor prognosis generally associated with this disease has led to the search for improved therapies such as ferroptosis-inducing agents. Ferroptosis is a form of regulated cell death that is dependent on iron and is characterized by lipid peroxidation. Precise mapping of lipids and iron within tumors exposed to ferroptosis-inducing agents may provide insight into processes of ferroptosis in vivo and ultimately assist in the optimal deployment of ferroptosis inducers in cancer therapy. In this work, we present a method for combining matrix-assisted laser desorption/ionization (MALDI) mass spectrometry imaging (MSI) with secondary ion mass spectrometry (SIMS) to analyze changes in spatial lipidomics and metal composition, respectively, in ovarian tumors following exposure to a ferroptosis inducer. Tumors were obtained by injecting human ovarian cancer tumor-initiating cells into mice, followed by treatment with the ferroptosis inducer erastin. SIMS imaging detected iron accumulation in the tumor tissue, and sequential MALDI-MS imaging of the same tissue section displayed two chemically distinct regions of lipids. Additionally, one region was associated with the iron-rich area detected with SIMS, and the other region encompassed the remainder of the tissue section. Bulk lipidomics confirmed the lipid assignments putatively assigned from the MALDI-MS data. Overall, we demonstrate the ability of multimodal MSI to identify the spatial locations of iron and lipids in the same tissue section and associate these regions with clinical pathology.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2022

Light-Duty Vehicle Trip Classification Using One-Class Novelty Detection and Exhaustive Feature Extraction

Travel mode classification within travel survey data sets, especially light-duty vehicle (LDV) trips, is foundational, though nontrivial, to emerging mobility systems, travel behavior analysis, and fuel consumption estimation. Current travel mode detection approaches require well-sampled and balanced data sets with ground truth travel mode labels. The detection approaches are rarely applied and validated on large-scale, real-world data sets, which may not satisfy the dataset requirements. This work proposes an LDV trip detection model as a supplement to current travel mode detection methods, for the case when the training set is highly (and/or completely) unbalanced, to the extent that classical machine-learning approaches become difficult or impossible to deploy. The proposed model uses a novelty detection technique - one-class support vector machines (OCSVMs) - and a novel exhaustive feature extraction (EFE) technique on continuous time series data (i.e., Global Positioning System [GPS] speed profiles) for single-mode trip trajectories. Training and validation of the model are conducted on a large-scale, real-world data set. The proposed method accurately identifies LDV trips from a broad set of multimodal trips by leveraging a wealth of preexisting in-vehicle GPS travel data. Additional sensitivity analysis sheds light on the optimal training size and feature selection, which will benefit applications limited by highly imbalanced data. The paper also discusses performance comparison with regular machine-learning approaches, the model's robustness, and the potential to extend the proposed model to multi-modal trip prediction.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2023

A guide to the BRAIN Initiative Cell Census Network data ecosystem

Characterizing cellular diversity at different levels of biological organization and across data modalities is a prerequisite to understanding the function of cell types in the brain. Classification of neurons is also essential to manipulate cell types in controlled ways and to understand their variation and vulnerability in brain disorders. The BRAIN Initiative Cell Census Network (BICCN) is an integrated network of data-generating centers, data archives, and data standards developers, with the goal of systematic multimodal brain cell type profiling and characterization. Emphasis of the BICCN is on the whole mouse brain with demonstration of prototype feasibility for human and nonhuman primate (NHP) brains. Here, we provide a guide to the cellular and spatial approaches employed by the BICCN, and to accessing and using these data and extensive resources, including the BRAIN Cell Data Center (BCDC), which serves to manage and integrate data across the ecosystem. We illustrate the power of the BICCN data ecosystem through vignettes highlighting several BICCN analysis and visualization tools. Finally, we present emerging standards that have been developed or adopted toward Findable, Accessible, Interoperable, and Reusable (FAIR) neuroscience. The combined BICCN ecosystem provides a comprehensive resource for the exploration and analysis of cell types in the brain.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2023

Probing corrosion using a simple and versatile in situ multimodal corrosion measurement system

In this work, we have developed a unique in situ multimodal corrosion system that is capable of acquiring electrochemical data, sample imaging/visualization and hydrogen collection, simultaneously. Each of these modalities yield valuable information pertaining to the ongoing corrosion process. Combining them can yield holistic information on the role of microstructure, processing history, presence of coatings, etc., on the sequence of steps occurring during the corrosion process, and how they correlate with the acquired electrochemical data. Four materials systems, namely AA6061-T6 aluminum alloy, AZ91 magnesium alloy, galvanized DP590 steel, and pure Zn, were investigated under open circuit potential and under potentiodynamic polarization. The multimodal corrosion system was utilized to observe processes such as surface passivation and dissolution, pit and filiform corrosion initiation and propagation, and was correlated with location and magnitude of hydrogen evolution. This approach is shown to yield a truly multimodal understanding of the ongoing corrosion processes.

36 MATERIALS SCIENCE↗

DOE OSTI2020

Identification of preferred multimodal ligand‐binding regions on IgG1 F C using nuclear magnetic resonance and molecular dynamics simulations

Abstract In this study, the binding of multimodal chromatographic ligands to the IgG1 F C domain were studied using nuclear magnetic resonance and molecular dynamics simulations. Nuclear magnetic resonance experiments carried out with chromatographic ligands and a perdeuterated 15 N‐labeled F C domain indicated that while single‐mode ion exchange ligands interacted very weakly throughout the F C surface, multimodal ligands containing negatively charged and aromatic moieties interacted with specific clusters of residues with relatively high affinity, forming distinct binding regions on the F C . The multimodal ligand‐binding sites on the F C were concentrated in the hinge region and near the interface of the C H 2 and C H 3 domains. Furthermore, the multimodal binding sites were primarily composed of positively charged, polar, and aliphatic residues in these regions, with histidine residues exhibiting some of the strongest binding affinities with the multimodal ligand. Interestingly, comparison of protein surface property data with ligand interaction sites indicated that the patch analysis on F C corroborated molecular‐level binding information obtained from the nuclear magnetic resonance experiments. Finally, molecular dynamics simulation results were shown to be qualitatively consistent with the nuclear magnetic resonance results and to provide further insights into the binding mechanisms. An important contribution to multimodal ligand‐F C binding in these preferred regions was shown to be electrostatic interactions and π–π stacking of surface‐exposed histidines with the ligands. This combined biophysical and simulation approach has provided a deeper molecular‐level understanding of multimodal ligand–F C interactions and sets the stage for future analyses of even more complex biotherapeutics.

Gudhka, Ronak B.↗

DOE OSTI2022

MENUS—Materials engineering by neutron scattering

Materials engineering by neutron scattering (MENUS) at the second target station will be a transformational high-flux, versatile, multiscale materials engineering diffraction beamline with unprecedented new capabilities for the study of complex materials and structures. It will support both fundamental and applied materials research in a broad range of fields. MENUS will combine unprecedented long-wavelength neutron flux and unique detector coverage to enable real-time studies of complex structural and functional materials under external stimuli. The incorporated small angle neutron scattering and transmission/imaging capabilities will extend its sensitivity to larger length scales and higher spatial resolution. Multimodal MENUS will provide crystallographic and microstructure data to the materials science and engineering community to understand lattice strain/phase transition/microstructure/texture evolution in three orthogonal directions in complex material systems under combined extreme applied conditions. Finally, the capabilities of MENUS will open new scientific opportunities and meet the research needs for science challenges to enable studies of a range of phenomena and answer the key questions in material design/exploration, advanced material processing, transformative manufacturing, and material operations of national impacts in our daily life.

47 OTHER INSTRUMENTATION↗

DOE OSTI2022

Data Fusion via Neural Network Entropy Minimization for Target Detection and Multi-Sensor Event Classification

Broadly applicable solutions to multimodal and multisensory fusion problems across domains remain a challenge because effective solutions often require substantive domain knowledge and engineering. The chief questions that arise for data fusion are in when to share information from different data sources, and how to accomplish the integration of information. The solutions explored in this work remain agnostic to input representation and terminal decision fusion approaches by sharing information through the learning objective as a compound objective function. The objective function this work uses assumes a one-to-one learning paradigm within a one-to-many domain which allows the assumption that consistency can be enforced across the one-to-many dimension. The domains and tasks we explore in this work include multi-sensor fusion for seismic event location and multimodal hyperspectral target discrimination. We find that our domain- informed consistency objectives are challenging to implement in stable and successful learning because of intersections between inherent data complexity and practical parameter optimization. While multimodal hyperspectral target discrimination was not enhanced across a range of different experiments by the fusion strategies put forward in this work, seismic event location benefited substantially, but only for label-limited scenarios.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2025

Multimodal super-resolution: discovering hidden physics and its application to fusion plasmas

Understanding complex physical systems often requires integrating data from multiple diagnostics, each with limited resolution or coverage. We present a machine learning framework that reconstructs synthetic high-temporal-resolution data for a target diagnostic using information from other diagnostics, without direct target measurements during the inference. This multimodal super-resolution technique improves diagnostic robustness and enables monitoring even in case of measurement failures or degradation. Applied to fusion plasmas, our method targets edge-localized modes (ELMs), which can damage plasma-facing materials. By reconstructing super-resolution Thomson Scattering data from complementary diagnostics, we uncover fine-scale plasma dynamics and validate the role of resonant magnetic perturbations (RMPs) in ELM suppression through magnetic island formation. The approach provides new observation supporting the plasma profile flattening due to these islands. Our results demonstrate the framework’s ability to generate high-fidelity synthetic diagnostics, offering a powerful tool for ELM control development in future reactors like ITER. The approach is broadly transferable to other domains facing sparse, incomplete, or degraded diagnostic data, opening new avenues for discovery.

Jalalvand, Azarakhsh [Princeton Univ., NJ (United ↗

DOE OSTI2023

Unsupervised classification for region of interest in X-ray ptychography

X-ray ptychography offers high-resolution imaging of large areas at a high computational cost due to the large volume of data provided. To address the cost issue, we propose a physics-informed unsupervised classification algorithm that is performed prior to reconstruction and removes data outside the region of interest (RoI) based on the multimodal features present in the diffraction patterns. The preprocessing time for the proposed method is inconsequential in contrast to the resource-intensive reconstruction process, leading to an impressive reduction in the data workload to a mere 20% of the initial dataset. This capability consequently reduces computational time dramatically while preserving reconstruction quality. Through further segmentation of the diffraction patterns, our proposed approach can also detect features that are smaller than beam size and correctly classify them as within the RoI.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Multimodal Bayesian registration of noisy functions using Hamiltonian Monte Carlo

Functional data registration is a necessary processing step for many applications. The observed data can be inherently noisy, often due to measurement error or natural process uncertainty; which most functional alignment methods cannot handle. A pair of functions can also have multiple optimal alignment solutions, which is not addressed in current literature. In this paper, a flexible Bayesian approach to functional alignment is presented, which appropriately accounts for noise in the data without any pre-smoothing required. Additionally, by running parallel MCMC chains, the method can account for multiple optimal alignments via the multi-modal posterior distribution of the warping functions. To most efficiently sample the warping functions, the approach relies on a modification of the standard Hamiltonian Monte Carlo to be well-defined on the infinite-dimensional Hilbert space. In this work, this flexible Bayesian alignment method is applied to both simulated data and real data sets to show its efficiency in handling noisy functions and successfully accounting for multiple optimal alignments in the posterior; characterizing the uncertainty surrounding the warping functions.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Mobility Energy Productivity and Equity: E-Bike Impacts for Low-Income Essential Workers in Denver

New mobility technologies such as electrified and shared mobility, combined with polices and incentive programs, are emerging to help address sustainability and equity issues in transportation planning. However, it can be difficult to understand the impacts of novel mobility trends and emerging modes on energy-efficient access. This is owing to a lack of (1) open-source tools enabling rapid data collection, and (2) open-source metrics that consider multimodal, multiactivity access and mobility within the contexts of sustainability and equity. Here, this paper addresses the topic of improving evaluation of transportation modes and incentive programs by integrating an open-source platform for tracking human travel data—the Open Platform for Agile Trip Heuristics (OpenPATH)—with a mobility metric that quantifies the efficiency of a region’s transportation system: Mobility Energy Productivity (MEP). Integration is demonstrated in the context of pilot programs in Colorado, where low-income essential workers were provided with electric bikes (e-bikes). OpenPATH-informed MEP calculations showed that several locations in downtown Denver provided comparable time-, cost-, and energy-efficient access to opportunities using e-bikes compared with driving. Additionally, providing e-bikes to low-income essential workers was found to be meaningful, as they utilized e-bikes the most to commute, despite driving still being their most utilized mode and the mode with highest MEP scores in Denver. We show how data collected from open-source tools coupled with robust metrics such as MEP can help evaluate the impacts of emerging mobility options. This could support developing policies to incentivize novel modes to achieve greater levels of sustainable, equitable, and efficient access.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2023

Manifold projection image segmentation for nano-XANES imaging

As spectral imaging techniques are becoming more prominent in science, advanced image segmentation algorithms are required to identify appropriate domains in these images. We present a version of image segmentation called manifold projection image segmentation (MPIS) that is generally applicable to a broad range of systems without the need for training because MPIS uses unsupervised machine learning with a few physically motivated hyperparameters. We apply MPIS to nanoscale x-ray absorption near edge structure (XANES) imaging, where XANES spectra are collected with nanometer spatial resolution. We show the superiority of manifold projection over linear transformations, such as the commonly used principal component analysis (PCA). Moreover, MPIS maintains accuracy while reducing computation time and sensitivity to noise compared to the standard nano-XANES imaging analysis procedure. Finally, we demonstrate how multimodal information, such as x-ray fluorescence data and spatial location of pixels, can be incorporated into the MPIS framework. We propose that MPIS is adaptable for any spectral imaging technique, including scanning transmission x-ray microscopy, where the length scale of domains is larger than the resolution of the experiment.

36 MATERIALS SCIENCE↗

DOE OSTI2026

TSDC: Transportation Secure Data Center: Real-World Data for Planning, Modeling, and Analysis

The Transportation Secure Data Center is a centralized repository for high-resolution transportation data from hundreds of travel and transit surveys and studies. It makes vital transportation data broadly available to users while preserving the privacy of survey participants. It houses surveys and studies conducted by state departments of transportation, metropolitan planning organizations, transit agencies, cities, and other public agencies. Meanwhile, the Livewire Data Platform empowers research, industry, and academic partners to easily and securely preserve, maintain, share, discover, and gain access to transportation and mobility data. Livewire accommodates a range of datasets, including behavioral, experimental, model, analytical, and raw data at the vehicle, traveler, and system levels. Datasets support mobility research and planning spanning urban science, connected and automated vehicles, fueling and charging infrastructure, mobility decision science, multimodal transportation, vehicle efficiency, and more.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2025

Informed unsupervised machine learning analysis of dislocation microstructure from high-resolution differential aperture X-ray structural microscopy data

This study leverages high-resolution differential-aperture X-ray structural microscopy (DAXM) to probe the local dislocation structure in deformed 304L-stainless steel at small strain, by measuring the lattice rotation and deviatoric elastic strain with a sub-micron resolution. For a single grain in a polycrystalline specimen, the measured lattice rotation field over the measured volume exhibited a multimodal distribution while the deviatoric elastic strain showed a single-mode distribution. An unsupervised Cauchy mixture machine learning model was developed to resolve the multimodal distribution of the lattice rotation. By mapping the lattice rotation data associated with each Cauchy peak in the model back onto the measured volume, we identify contiguous regions of the crystal rotated near the average values corresponding to the peaks of the overall rotation distribution. These regions represent the grain subdivision in the microstructure. Finally, the dislocation density tensor was also computed and its norm was laid over the rotation field to detect the subgrain boundaries. This step provided a validation of the Cauchy mixture model for the analysis of the lattice rotation distribution. The current study highlights the integration of advanced X-ray microscopy techniques with data-driven analysis methods to uncover detailed microstructure scales in deformed crystals.

Machine learning; Lattice rotation; High-energy X-↗

DOE OSTI2021

Recent Developments in Femtosecond Laser-Enabled TriBeam Systems

Streams of multimodal three-dimensional (3D) and four-dimensional (4D) data are revolutionizing our ability to design and predict the behavior of a broad array of advanced materials systems. Over the last 10 years, a new 3D imaging platform consisting of a femtosecond (fs) pulsed laser coupled with a focused ion beam scanning electron microscope (FIB SEM) has been developed by UC Santa Barbara in collaboration with Thermo Fisher Scientific (formerly FEI). The femtosecond-laser-enabled FIB SEM, called the TriBeam, has become one of the only 3D serial sectioning methods available that can gather millimeter-scaled multimodal datasets at sub-μm voxel resolutions; these length scales are critical for many materials problems. Multimodal chemical, crystallographic, and morphological information can be gathered rapidly on a layer-by-layer basis and reconstructed in 3D. Large (gigabyte to terabyte scale) 3D datasets have been generated for a broad array of materials systems, including metallic alloys, ceramics, biomaterials, polymer- and ceramic-matrix composites, and semiconductors. The research tasks conducted have resulted in a completely new design, operating with a dual-wavelength femtosecond-pulsed laser on a plasma focused ion beam (PFIB) platform.

36 MATERIALS SCIENCE↗

DOE OSTI2022

Dose Summation Strategies for External Beam Radiation Therapy and Brachytherapy in Gynecologic Malignancy: A Review from the NRG Oncology and NCTN Medical Physics Subcommittees

Definitive, nonsurgical management of gynecologic malignancies involves external beam radiation therapy (EBRT) and/or brachytherapy (BT). Summation of the cumulative dose is critical to assess the total biologic effective dose to targets and organs at risk. Cumulative dose calculation from EBRT and BT can be performed with or without image registration (IR) and biologic dose summation. Among these dose summation strategies, linear addition of dose-volume histogram (DVH) parameters without IR is the global standard for composite dose reporting. This approach stems from an era without image guidance and simple external beam and brachytherapy treatment approaches. With technological advances, EBRT and high-dose-rate BT have evolved to allow for volume-based treatment planning and delivery. Modern conformal therapeutic radiation involves volumetric or intensity modulated EBRT, capable of simultaneously treating multiple targets at different specified dose levels. Therefore, given the complexity of modern radiation treatment, the linear addition of DVH parameters from EBRT and high-dose-rate BT is challenging to represent the combined dose distribution. Deformable image registration (DIR) between EBRT and image guided brachytherapy (IGBT) data sets may provide a more nuanced calculation of multimodal dose accumulation. However, DIR is still nascent in this regard, and needs further development for accuracy and efficiency for clinical use. Biologic dose summation can combine physical dose maps from EBRT and each IGBT fraction, thereby generating a composite DVH from the biologic effective dose. However, accurate radiobiologic parameters are tissue-dependent and not well characterized. A combination of voxel-based DIR and biologic weighted dose maps may be the best approximation of dose accumulation but remains invalidated. The purpose of this report is to review dose summation strategies for EBRT and BT, including conventional equivalent dose in 2-Gy fractions dose summation without image registration, physical dose summation using 3-dimensional rigid IR and DIR, and biologic dose summation. We also provide general clinical workflows for IGBT with a focus on cervical cancer.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

DOE OSTI2022

Deep generative molecular design reshapes drug discovery

Recent advances and accomplishments of artificial intelligence (AI) and deep generative models have established their usefulness in medicinal applications, especially in drug discovery and development. To correctly apply AI, the developer and user face questions such as which protocols to consider, which factors to scrutinize, and how the deep generative models can integrate the relevant disciplines. This review summarizes classical and newly developed AI approaches, providing an updated and accessible guide to the broad computational drug discovery and development community. We introduce deep generative models from different standpoints and describe the theoretical frameworks for representing chemical and biological structures and their applications. We discuss the data and technical challenges and highlight future directions of multimodal deep generative models for accelerating drug discovery.

59 BASIC BIOLOGICAL SCIENCES↗