Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “multimodal network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Parallel measurement of transcriptomes and proteomes from same single cells using nanodroplet splitting

Single-cell multiomics provides comprehensive insights into gene regulatory networks, cellular diversity, and temporal dynamics. While tools for co-profiling single-cell genomes, transcriptomes, and epigenomes are available, accessing proteomes in parallel is more challenging. We developed nanoSPLITS (nanodroplet SPlitting for Linked-multimodal Investigations of Trace Samples), an integrated platform that enables global profiling of the transcriptome and proteome from same single cells using RNA sequencing and mass spectrometry-based proteomics, respectively. nanoSPLITS can precisely quantify over 5000 genes, 2000 proteins, and 140 phosphopeptides per single cell and identify candidate cell markers from these modalities. By exploring Cdk1-mediated cell cycle arrest, we demonstrate how nanoSPLITS single-cell multiomics can provide comprehensive cellular characterization with insights into covarying protein/gene clusters, unique phosphorylation events, and mitotic pathways.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Machine‐Learning Spectral Indicators of Topology

Abstract Topological materials discovery has emerged as an important frontier in condensed matter physics. While theoretical classification frameworks have been used to identify thousands of candidate topological materials, experimental determination of materials’ topology often poses significant technical challenges. X‐ray absorption spectroscopy (XAS) is a widely used materials characterization technique sensitive to atoms’ local symmetry and chemical bonding, which are intimately linked to band topology by the theory of topological quantum chemistry (TQC). Moreover, as a local structural probe, XAS is known to have high quantitative agreement between experiment and calculation, suggesting that insights from computational spectra can effectively inform experiments. In this work, computed X‐ray absorption near‐edge structure (XANES) spectra of more than 10 000 inorganic materials to train a neural network (NN) classifier that predicts topological class directly from XANES signatures, achieving F 1 scores of 89% and 93% for topological and trivial classes, respectively is leveraged. Given the simplicity of the XAS setup and its compatibility with multimodal sample environments, the proposed machine‐learning‐augmented XAS topological indicator has the potential to discover broader categories of topological materials, such as non‐cleavable compounds and amorphous materials, and may further inform field‐driven phenomena in situ, such as magnetic field‐driven topological phase transitions.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Solving multiphysics-based inverse problems with learned surrogates and constraints

Abstract Solving multiphysics-based inverse problems for geological carbon storage monitoring can be challenging when multimodal time-lapse data are expensive to collect and costly to simulate numerically. We overcome these challenges by combining computationally cheap learned surrogates with learned constraints. Not only does this combination lead to vastly improved inversions for the important fluid-flow property, permeability, it also provides a natural platform for inverting multimodal data including well measurements and active-source time-lapse seismic data. By adding a learned constraint, we arrive at a computationally feasible inversion approach that remains accurate. This is accomplished by including a trained deep neural network, known as a normalizing flow, which forces the model iterates to remain in-distribution, thereby safeguarding the accuracy of trained Fourier neural operators that act as surrogates for the computationally expensive multiphase flow simulations involving partial differential equation solves. By means of carefully selected experiments, centered around the problem of geological carbon storage, we demonstrate the efficacy of the proposed constrained optimization method on two different data modalities, namely time-lapse well and time-lapse seismic data. While permeability inversions from both these two modalities have their pluses and minuses, their joint inversion benefits from either, yielding valuable superior permeability inversions and CO 2 plume predictions near, and far away, from the monitoring wells.

Yin, Ziyi (ORCID:0000000250248771)↗
DOE OSTI2023

Scaling Subspace-Driven Approaches Using Information Fusion

In this work, we seek to exploit the deep structure of multi-modal data to robustly exploit the group subspace distribution of the information using the Convolutional Neural Networks (CNNs) formalism. Upon unfolding the set of subspaces constituting each data modality, and learning their corresponding encoders, an optimized integration of the generated inherent information is carried out to yield a characterization of various classes. Referred to as deep Multimodal Robust Group Subspace Clustering (DRoGSuRe), this approach is compared against the independently developed state-of-the-art approach named Deep Multimodal Subspace Clustering (DMSC). Experiments on different multimodal datasets show that our approach is competitive and more robust in the presence of noise.

Ghanem, Sally↗
DOE OSTI2024

Agent-based modeling for multimodal transportation of CO 2 for carbon capture, utilization, and storage: CCUS-agent

Here, to understand the system-level interactions between the entities in Carbon Capture, Utilization, and Storage (CCUS), an agent-based foundational modeling tool, CCUS-Agent, is developed for a large-scale study of transportation flows and infrastructure in the United States. Key features of the tool include (i) modular design, (ii) multiple transportation modes, (iii) capabilities for extension, and (iv) testing against various system components and networks of small and large sizes. Five matching algorithms for CO 2 supply agents (e.g., powerplants and industrial facilities) and demand agents (e.g., storage and utilization sites) are explored: Most Profitable First Year (MPFY), Most Profitable All Years (MPAY), Shortest Total Distance First Year (SDFY), Shortest Total Distance All Years (SDAY), and Shortest distance to long-haul transport All Years (ACAY). Before matching, the supply agent, demand agent, and route must be available, and the connection must be profitable. A profitable connection means the supply agent portion of revenue from the 45Q tax credit must cover the supply agent costs and all transportation costs, while the demand agent revenue portion must cover all demand agent costs. A case study employing over 5500 supply and demand agents and multimodal CCUS transportation infrastructure in the contiguous United States is conducted. The results suggest that it is possible to capture over 9 billion tonnes (GT) of CO 2 from 2025 to 2043, which will increase significantly to 22 GT if the capture costs are reduced by 40 %. The MPFY and SDFY algorithms capture more CO 2 earlier in the time horizon, while the MPAY and SDAY algorithms capture more later in the time horizon.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

DCTRGAN: improving the precision of generative models with reweighting

Significant advances in deep learning have led to more widely used and precise neural network-based generative models such as Generative Adversarial Networks (Gans). We introduce a post-hoc correction to deep generative models to further improve their fidelity, based on the Deep neural networks using the Classification for Tuning and Reweighting (Dctr) protocol. The correction takes the form of a reweighting function that can be applied to generated examples when making predictions from the simulation. We illustrate this approach using Gans trained on standard multimodal probability densities as well as calorimeter simulations from high energy physics. We show that the weighted Gan examples significantly improve the accuracy of the generated samples without a large loss in statistical power. This approach could be applied to any generative model and is a promising refinement method for high energy physics applications and beyond.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Multimodal sensor fusion for real-time standoff estimation in directed energy deposition

In Laser Powder-based Direct Energy Deposition (LP-DED) systems, achieving consistency, precision and quality of produced parts requires tight control over printing parameters. One of the critical parameters is the standoff distance. Maintaining an optimal standoff height is crucial for achieving correct laser power density and powder catchment efficiency, as both laser and powder streams are focused at this distance. Here, this study introduces a novel approach using multimodal sensor fusion to predict standoff height in real-time. The proposed system integrates two low-profile, cost-effective sensors: an RGB coaxial camera and a high frequency and high dynamic range microphone. By utilizing a simple fully connected neural network, trained on a limited dataset, data fusion of these sensors allowed for the real-time prediction of the standoff height. The results demonstrate high resolution and accuracy of the predictions across multiple geometries and a wide range of standoff heights. This approach offers a simple, and cost-effective solution for real-time standoff height monitoring and lays the groundwork for future integration into commercial LP-DED systems.

42 ENGINEERING↗
DOE OSTI2025

Mind the gap: Bridging the divide between AI aspirations and the reality of autonomous microscopy

What does materials science look like in the “Age of Artificial Intelligence?” Each material’s domain—synthesis, characterization, and modeling—has a different answer to this question, motivated by unique challenges and constraints. This work focuses on the tremendous potential of autonomous characterization within electron microscopy. We present our recent advancements in developing domain-aware, multimodal models for microscopy analysis capable of describing complex atomic systems. We then address the critical gap between the theoretical promise of autonomous microscopy and its current practical limitations, showcasing recent successes while highlighting the necessary developments to achieve robust, real-world autonomy.

2D materials↗
DOE OSTI2024

CrossMP: Enabling Cross-Modality Translation between Single-Cell RNA-Seq and Single-Cell ATAC-Seq through Web-Based Portal

In recent years, there has been a growing interest in profiling multiomic modalities within individual cells simultaneously. One such example is integrating combined single-cell RNA sequencing (scRNA-seq) data and single-cell transposase-accessible chromatin sequencing (scATAC-seq) data. Integrated analysis of diverse modalities has helped researchers make more accurate predictions and gain a more comprehensive understanding than with single-modality analysis. However, generating such multimodal data is technically challenging and expensive, leading to limited availability of single-cell co-assay data. Here, we propose a model for cross-modal prediction between the transcriptome and chromatin profiles in single cells. Our model is based on a deep neural network architecture that learns the latent representations from the source modality and then predicts the target modality. It demonstrates reliable performance in accurately translating between these modalities across multiple paired human scATAC-seq and scRNA-seq datasets. Additionally, we developed CrossMP, a web-based portal allowing researchers to upload their single-cell modality data through an interactive web interface and predict the other type of modality data, using high-performance computing resources plugged at the backend.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Utilizing distributed acoustic sensing and ocean bottom fiber optic cables for submarine structural characterization

The sparsity of permanent seismic instrumentation in marine environments often limits the availability of subsea information on geohazards, including active fault systems, in both time and space. One sensing resource that provides observational access to the seafloor environment are existing networks of ocean bottom fiber optic cables; these cables, coupled to modern distributed acoustic sensing (DAS) systems, can provide dense arrays of broadband seismic observations capable of recording both seismic events and the ambient noise wavefield. Here, we report a marine DAS application which demonstrates the strength and limitation of this new technique on submarine structural characterization. Based on ambient noise DAS records on a 20 km section of a fiber optic cable offshore of Moss Landing, CA, in Monterey Bay, we extract Scholte waves from DAS ambient noise records using interferometry techniques and invert the resulting multimodal dispersion curves to recover a high resolution 2D shear-wave velocity image of the near seafloor sediments. We show for the first time that the migration of coherently scattered Scholte waves observed on DAS records can provide an approach for resolving sharp lateral contrasts in subsurface properties, particularly shallow faults and depositional features near the seafloor. Our results provide improved constraints on shallow submarine features in Monterey Bay, including fault zones and paleo-channel deposits, thus highlighting one of many possible geophysical uses of the marine cable network.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Entropy-defect synergy for dual luminescence mechanism in spinel: Time-resolved anti-counterfeiting and fingerprint visualization

Multimodal luminescent materials, while promising for anti-counterfeiting, often lack dynamic time-dependent responses and controllable spatial distribution, limiting their encryption capabilities in the spatiotemporal dimension. Here, this work presents a coordinated control strategy based on entropy and defect engineering, and uses a backpropagation (BP) neural network for material screening to successfully prepare spinel Mg 0.8 (Fe 0.04 Co 0.04 Ni 0.04 Cu 0.04 Zn 0.04 )Cr 2 O 4 (MgA 5 CO) phosphors with time-dependent dynamic luminescence behavior. This phosphor simultaneously activated the d-d transition luminescence (∼618 nm) derived from Co 2+ /Cr 3+ and the defect luminescence (∼398 nm) related to zinc vacancies (V Zn ) in a single-phase solid solution. The phosphor exhibits a time-dependent color evolution from pink to purple under fixed-wavelength excitation, due to the different excited-state dynamics and decay lifetimes associated with the d-d transition and defect luminescence. Structural characterization and spectral analysis confirmed the existence of V Zn and its significant role in defect luminescence process. The fluorescent and dynamic luminescent properties of entropy-based spinel oxide enable its use in advanced anti-counterfeiting applications like fingerprint recognition and color-changing dedicated anti-counterfeiting mark, showing promise in high-end and time-dynamic anti-counterfeiting fields. This research not only developed a new type of fluorescent dynamic anti-counterfeiting material, but also provided a new idea for constructing advanced optical functional materials with multiple luminescence mechanisms.

Defect project↗
DOE OSTI2025

Knowledge Oriented Graph Unified Transformer (KOGUT) v0.1

KOGUT — Knowledge Oriented Graph Unified Transformer KOGUT implements the Relational Graph Transformer (RelGT) architecture for knowledge graph link prediction in biological domains, with a primary focus on microbial growth media prediction. While the original RelGT (arXiv:2505.10960) targets relational tables, time series, and multi-table databases, KOGUT adapts this architecture for heterogeneous biological knowledge graphs, providing first-in-class AI predictive models for microbial cultivation. Key Adaptations Beyond Original RelGT: - Knowledge Graph Focus: Applied to biological KGs with semantic node types (taxa, chemicals, media, phenotypes, environments) versus generic relational database tables, trained on the KG-Microbe knowledge graph (1.3M entities, 2.9M edges, 24 relation types). - Multimodal Node Encoding: Integrates node labels, categories, descriptions, and synonyms from KG metadata through learned embedding layers—adapting relational column features to graph node attributes with textual semantics. - Extended K-Hop Subgraph Strategy: Optimized neighborhood sampling (3-hop default, configurable up to 200 nodes) tuned for sparse biological networks, building on the original local-global attention framework with biological relation preservation. - Biolink Predicate Preservation: Type-specific transformations for 24 biological edge semantics (occurs_in, consumes, produces, has_phenotype, subclass_of) beyond standard relational foreign keys, enabling multi-relation link prediction. - Inductive Learning Support: Enables zero-shot predictions for novel taxa through feature-based embeddings (temperature, oxygen requirements, gram stain, cell shape), extending the original transductive relational benchmark scope to uncultured microorganisms. CheapSOTA Performance Optimizations (This Distribution): - VQ-EMA Centroid Attention: Vector quantization with exponential moving average for improved global context modeling (+5-10% MRR improvement). - HDF5 Precomputed Data Loading: One-time preprocessing of k-hop subgraphs to eliminate redundant graph traversals (2-5× training speedup). - Distributed Data Parallel Training: Multi-GPU support for scaling to larger knowledge graphs (tested on 4× NVIDIA A100 GPUs at NERSC Perlmutter). - Mixed Precision Training: Automatic mixed precision (AMP) for memory efficiency and faster training. Advantages Over Standard Knowledge Graph Embedding Models: Combines RelGT's proven multi-element tokenization (features, type, hop, structure) with graph-native biological representations, enabling interpretable link prediction across heterogeneous entities that standard embedding models (TransE, RotatE, ComplEx) and table-based transformers cannot directly model. Achieves near-perfect performance on microbial growth media prediction (MRR: 0.9966, Precision@1: 0.9932, Hit@10: 1.0000) while maintaining explainability through attention-based reasoning over biological pathways. Training Data: - KG-Microbe merged knowledge graph: 1,379,337 nodes, 2,960,472 edges - 24 biological relation types including taxonomic hierarchies, metabolic interactions, phenotype associations, and environmental relationships - Primary prediction task: Growth media suitability for microbial taxa (biolink:occurs_in, 50K edges) - Multi-relation capability: Predicts links for any of the 24 relation types, including chemical consumption/production, phenotype associations, and taxonomic classification Citation: Original RelGT Architecture: Dwivedi et al., "Relational Graph Transformer", arXiv:2505.10960, 2025 KOGUT Implementation: Knowledge Oriented Graph Unified Transformer for Microbial Growth Media Prediction Developed at Lawrence Berkeley National Laboratory (LBNL) Trained on NERSC Perlmutter supercomputer

Joachimiak, Marcin [Lawrence Berkeley National Lab↗
DOE OSTI2021

KS4A-Omics1.0_FspDS682

Soil fungi facilitate the translocation of inorganic nutrients from soil minerals to other microorganisms and plants. This ability is particularly advantageous in impoverished soils, because fungal mycelial networks can bridge otherwise spatially disconnected and inaccessible nutrient hotspots. However, the molecular mechanisms underlying fungal mineral weathering and transport through soil remains poorly understood. Here, we addressed this knowledge gap by directly visualizing nutrient acquisition and transport through fungal hyphae in a mineral doped soil micromodel using a multimodal imaging approach. Here, we observed how a representative of common saprotrophic soil fungi, Fusarium sp. DS 682, exhibited a mechanosensory response (thigmotropism) around obstacles and through pore spaces (~12 μm) in the presence of minerals.This study establishes the significance of fungal biology and nutrient translocation mechanisms in maintaining fungal growth under water and nutrient limitations in a soil-like microenvironment, using a high-throughput multi-omic analysis approach. Data package KS4A-Omics.1.0_FspDS682 (Publication: Fungal Mineral Weathering Mechanisms Revealed Through Direct Molecular Visualization) contents reported here are the first version (1.0) and contain pre- and post-processed data using high throughput data capture technologies for multi-omic analysis and integration, this data package contains raw and post-processed experimental data for X-Ray Absorption Near Edge Structure Spectroscopy (XANES/XRF), Optical Microscopy, Proteomics, Scanning Electron Microscope (SEM), Time-of-Flight Secondary Ion Mass Spectroscopy (ToF-SIMS), X-Ray Diffraction Spectroscopy (XRD) files, and X- Ray Photoelectron Spectroscopy (XPS) using EMSL capabilities. This data package DOI contains a comprehensive collection of high-throughput multi-omics data and process metadata catalog. Support files include additional data download contents “Read Me” with dataset descriptor information and data source method application ontologies (see data dictionary section). Reported data download content is structured for compliance with reported guidelines provided by community standard initiatives and publisher stakeholder policies supporting FAIR data principles.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Bolide Infrasound Signal Morphology and Yield Estimates: A Case Study of Two Events Detected by a Dense Acoustic Sensor Network

Two bolides (2016 June 2 and 2019 April 4) were detected at multiple regional infrasound stations, with many of the locations receiving multiple detections. Analysis of the received signals was used to estimate the yield, location, and trajectory, as well as the type of shock that produced the received signal. The results from the infrasound analysis were compared with ground-truth information that was collected through other sensing modalities. This multimodal framework offers an expanded perspective on the processes governing bolide shock generation and propagation. The majority of signal features showed reasonable agreement between the infrasound-based interpretation and the other observational modalities, though the yield estimate from the 2019 bolide was significantly lower using the infrasound detections. There was also evidence suggesting that one of the detections was from a cylindrical shock that was initially propagating upward, which is unusual though not impossible.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

3D and Multimodal X‐Ray Microscopy Reveals the Impact of Voids in CIGS Solar Cells

Small voids in the absorber layer of thin-film solar cells are generally suspected to impair photovoltaic performance. They have been studied on Cu(In,Ga)Se 2 cells with conventional laboratory techniques, albeit limited to surface characterization and often affected by sample-preparation artifacts. Here, synchrotron imaging is performed on a fully operational as-deposited solar cell containing a few tens of voids. By measuring operando current and X-ray excited optical luminescence, the local electrical and optical performance in the proximity of the voids are estimated, and via ptychographic tomography, the depth in the absorber of the voids is quantified. Besides, the complex network of material-deficit structures between the absorber and the top electrode is highlighted. Despite certain local impairments, the massive presence of voids in the absorber suggests they only have a limited detrimental impact on performance.

14 SOLAR ENERGY↗
DOE OSTI2024

Evaluating the Use of Foundational Chemical Language Models in Multimodal Graph Fusion

Rapid and accurate prediction of the physicochemical properties of molecules given their structures remains a key challenge in cheminformatics. Machine learning approaches offer high-throughput options, but the optimality of inductive biases and data representations are up for debate. For example, BERT-based masked language models (MLMs) can be trained in a self-supervised way on hundreds of millions to billions of readily available SMILES strings. Another option is graph neural networks (GNNs), which can operate directly on molecular structures. Yet, generating accurate molecular geometry is computationally expensive, leading to a relative scarcity in data compared to SMILES strings. It is attractive to combine these two paradigms by pre-training an LM on a large corpus of SMILES strings and embedding these representation into a geometric graph neural network. Despite the promise of such an approach, and contrary to previous studies, we find mixed results with the combination of the LMs and GNNs on several molecule datasets. In particular, we found evidence for improvement on the FreeSolv and QM7 benchmarks, but degraded performance on the ESOL, LIPO and QM9 datasets compared to a GNN baseline.

Francel, Collin [University of Alabama]↗
DOE OSTI2022

Development of Multimodal Few-Shot Analytics for Electron Micrographs

Recent advances in materials data analytics have provided new avenues for determining process-structure-property (PSP) linkages in a variety of materials. Machine learning techniques including few-shot learning have increased the efficiency of classifying microscopy images for the purposes of material characterization. Attempts at creating a multimodal approach can provide further improvements to current models and help extract more salient features from data. In this vein, raw spectrum data was taken to provide an additional modality to our current pyCHIP classifier. Modifications in segmentation also show potential in improving the accuracy of the pyCHIP classifier. Classifier output was analyzed using network graphs and unsupervised clustering algorithms such as spectral clustering to detect better segmentation methods than the current “chipping” approach. We suggest that the chip selection process can be automated in the future using a combination of these techniques to enable high-throughput analyses.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Seismic Monitoring near Ithaca, New York, Reveals Nonuniform Distribution of Microseismicity in an Intraplate Region

Abstract Cornell University intends to use a deep direct-use geothermal system to heat its Ithaca, New York, campus. In preparation for this project, the Cornell Seismic Network has been monitoring the background seismicity in this intraplate region since 2019. From January 2020 to June 2023, 95 events were detected within 20 km of the proposed geothermal well site, with local magnitudes ranging from −1.02 to 0.56. None of these events appear in regional or national catalogs. Events locate in a narrow geographic band, with one-fourth exhibiting multimodal hypocentral probability peaks both near the surface and at 1–4 km depth. We relocate events with a joint hypocenter and 1D velocity model inversion, in addition to a fully nonlinear method, and then compare observations with synthetic waveforms. Together, these approaches provide strong evidence for >95% of events locating at the surface or within the 3-km-thick sedimentary sequence. We explore how anthropogenic activity and regional topographic stress may contribute to frequent surficial events. This information is critical for characterizing the background microseismicity for comparison during future geothermal operations. Ithaca’s geology of Paleozoic sediments overlying Precambrian crystalline basement is typical of many continental interiors, so these results also provide insight into intraplate microseismicity patterns.

Geochemistry & Geophysics↗