Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “spatiotemporal attention”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

MSASGCN : Multi-Head Self-Attention Spatiotemporal Graph Convolutional Network for Traffic Flow Forecasting

Traffic flow forecasting is an essential task of an intelligent transportation system (ITS), closely related to intelligent transportation management and resource scheduling. Dynamic spatial-temporal dependencies in traffic data make traffic flow forecasting to be a challenging task. Most existing research cannot model dynamic spatial and temporal correlations to achieve well-forecasting performance. The multi-head self-attention mechanism is a valuable method to capture dynamic spatial-temporal correlations, and combining it with graph convolutional networks is a promising solution. Therefore, we propose a multi-head self-attention spatiotemporal graph convolutional network (MSASGCN) model. It can effectively capture local correlations and potential global correlations of spatial structures, can handle dynamic evolution of the road network, and, in the time dimension, can effectively capture dynamic temporal correlations. Experiments on two real datasets verify the stability of our proposed model, obtaining a better prediction performance than the baseline algorithms. The correlation metrics get significantly reduced compared with traditional time series prediction methods and deep learning methods without using graph neural networks, according to MAE and RMSE results. Compared with advanced traffic flow forecasting methods, our model also has a performance improvement and a more stable prediction performance. We also discuss some problems and challenges in traffic forecasting.

Cao, Yang↗
DOE OSTI2024

Transformer Neural Networks with Spatiotemporal Attention for Predictive Control and Optimization of Industrial Processes

In the context of real-time optimization and model predictive control of industrial systems, machine learning, and neural networks represent cutting-edge tools that hold promise for enhancing dynamic modeling. This work presents a novel transformer neural network architecture for real-time optimization and model predictive control. This network design includes a modified attention mechanism inspired by positional embedding attention from vision transformers and task-specific modifications to the input-output structure of the transformer’s decoder stack. Experiments were conducted using data from a 450 MW coal-fired power plant to evaluate this approach's effectiveness. The transformer neural network was compared with conventional recurrent models, including GRU and LSTM. The transformer exhibited a 6% increase in the R-squared (R2) value of predictions and an 83% reduction in mean squared error (MSE). Computation time was also reduced by 84% compared to conventional recurrent models.

Gallup, Ethan R.↗
DOE OSTI2026

MATEY: multiscale adaptive transformer models for spatiotemporal physical systems

Accurate representation of the multiscale features in spatiotemporal physical systems using vision transformer architectures requires extremely long, computationally prohibitive token sequences. To address this issue, we propose two novel adaptive tokenization schemes that dynamically adjust patch sizes based on local features: one ensures convergent behavior to uniform patch refinement, while the other offers better computational efficiency. Moreover, we present a set of spatiotemporal attention schemes, where the temporal or axial spatial dimensions are decoupled, to evaluate their baseline computational and data efficiencies and to determine whether adaptive tokenization can improve this performance. We assess the performance of the proposed multiscale adaptive model, MATEY, in a sequence of experiments. Compared to a full spatiotemporal attention scheme or a scheme that decouples only the temporal dimension, we find that fully decoupled axial attention is less efficient and expressive, requiring more training time and model parameters to achieve the same accuracy. The experiments on the adaptive tokenization schemes show that, compared to a uniformly refined model, the proposed schemes achieve comparable or improved accuracy at a much lower cost in the tested two-dimensional settings. While the asymptotic analysis suggests the potential for favorable scaling, empirical validation at substantially longer sequence lengths remains to be performed in future work. Finally, we demonstrate in two fine-tuning tasks featuring different physics that models pretrained on PDEBench data outperform the ones trained from scratch, especially in the low data regime with frozen attention.

adaptive tokenization↗
DOE OSTI2022

Ultra-Short-Term Spatiotemporal Forecasting of Renewable Resources: An Attention Temporal Convolutional Network Based Approach

The rapid increase in the penetration of renewable energy resources characterized by high variability and uncertainty is bringing new challenges to the power system operation. To ensure the efficient and reliable operation of electric grid, an accurate and general short-term forecasting algorithm with interpretability is desired. Moreover, the extensive off-site information provided by the proliferation of new renewable plants stimulates the interests in the spatiotemporal forecasting. In this paper, an attention temporal convolutional network, which is built on stacked dilated causal convolutional networks and attention mechanisms, is proposed to perform the ultra-short-term spatiotemporal forecasting of renewable resources. Compared with the existing spatiotemporal forecasting methods, the presented model needs no domain knowledge and can be applied to different forecasting tasks such as solar generation and wind speed forecasting. Here, the attention mechanism improves the interpretability. The algorithm can be used to produce both point and probabilistic forecasts. Numerical results on the data sets from National Renewable Energy Laboratory show superior performance over five baselines, in terms of skill scores. Compared with the baselines, the average improvements of accuracy introduced by the proposed method for the point and probabilistic forecasting are 15.08% and 15.85%, respectively.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Using Explainable Artificial Intelligence to Predict Perovskite Solar Cell Electrical Metastability from Operando Photoluminescence Images in Accelerated Stress Testing

Metal halide perovskite (MHP) solar cells exhibit a metastable response to bias governed by coupled ionic–electronic processes, complicating the conventional reciprocity relation between luminescence intensity and device open-circuit voltage (V oc ). This limits the use of luminescence as a diagnostic for device screening or accelerated stress testing, motivating new approaches that can interpret photoluminescence (PL) signals under nonequilibrium conditions. From the artificial intelligence perspective, we develop an explainable deep learning framework that integrates convolutional neural networks (CNN), long short-term memory (LSTM) layers, and an attention mechanism to learn spatiotemporal features from operando photoluminescence PL image sequences. The model achieves a mean absolute error of ±0.027 V in predicting open-circuit voltage transients and reduces extreme-tail errors by up to 78% compared to physics-based reciprocity calculations. Gradient-weighted Class Activation Mapping (Grad-CAM) provides interpretability by highlighting physically meaningful regions such as electrode edges and emergent defect features. From the engineering application perspective, this framework enables accurate, contactless prediction of device V oc and identification of degradation-relevant features during accelerated aging of perovskite solar cells. This approach demonstrates how explainable AI can enhance operando diagnostics and reliability analysis in photovoltaic devices under nonequilibrium conditions.

14 SOLAR ENERGY↗
DOE OSTI2024

STFM: Accurate Spatio-Temporal Fusion Model for Weather Forecasting

Meteorological prediction is crucial for various sectors, including agriculture, navigation, daily life, disaster prevention, and scientific research. However, traditional numerical weather prediction (NWP) models are constrained by their high computational resource requirements, while the accuracy of deep learning models remains suboptimal. In response to these challenges, we propose a novel deep learning-based model, the Spatiotemporal Fusion Model (STFM), designed to enhance the accuracy of meteorological predictions. Our model leverages Fifth-Generation ECMWF Reanalysis (ERA5) data and introduces two key components: a spatiotemporal encoder module and a spatiotemporal fusion module. The spatiotemporal encoder integrates the strengths of convolutional neural networks (CNNs) and recurrent neural networks (RNNs), effectively capturing both spatial and temporal dependencies. Meanwhile, the spatiotemporal fusion module employs a dual attention mechanism, decomposing spatial attention into global static attention and channel dynamic attention. This approach ensures comprehensive extraction of spatial features from meteorological data. The combination of these modules significantly improves prediction performance. Experimental results demonstrate that STFM excels in extracting spatiotemporal features from reanalysis data, yielding predictions that closely align with observed values. In comparative studies, STFM outperformed other models, achieving a 7% improvement in ground and high-altitude temperature predictions, a 5% enhancement in the prediction of the u/v components of 10 m wind speed, and an increase in the accuracy of potential height and relative humidity predictions by 3% and 1%, respectively. This enhanced performance highlights STFM’s potential to advance the accuracy and reliability of meteorological forecasting.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Causality, intermittence, and crossphase evolution during confinement transitions in the TJ-II stellarator

In this work, we study spontaneous electron to ion root transitions in TJ-II using Langmuir probes. By scanning the probe position on a shot to shot basis, we reconstruct a spatiotemporal map of the evolution of important turbulent quantities in the plasma edge region. We pay particular attention to the evolution of the cross phase between transport-relevant variables, showing the spatiotemporal evolution of this quantity for the first time, revealing the outward propagation of the changes associated with the transition. Additionally, we also compute the intermittence parameter, which allows us to conclude that the turbulence, although its amplitude increases, condenses in a reduced number of dominant modes and becomes less bursty. The causal relationship between variables is studied using the transfer entropy, clarifying the interactions between the main variables and offering a rather complete picture of the complex evolution of the plasma across the confinement transition.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Field-resolved space–time characterization of few-cycle structured light pulses

Accompanied by the rapid development of ultrafast laser platforms in recent decades, the spatiotemporal manipulation of ultrashort laser pulses has attracted much attention due to the potential for cutting-edge applications of structured light, including optical tweezers, optical communications, super-resolution imaging, time-resolved spectroscopy in molecules and quantum materials, and strong-field physics. Today, techniques capable of characterizing the full spatial, temporal, and polarization state properties of structured light are strongly desired. Here, we demonstrate a technique, termed 3D TIPTOE, for characterizing structured mid-infrared waveforms, which uses only a two-dimensional silicon-based image sensor as both the detector and the nonlinear medium. By combining the advantages of the sub-cycle time resolution afforded by nonlinear excitation and the spatial resolution inherent to the two-dimensional sensor, the 3D TIPTOE technique allows full characterization of structured electric fields, significantly reducing the complexity of detection compared to other techniques. The validity of the technique is established by measuring both few-cycle Bessel–Gaussian pulses and radially polarized femtosecond vector beams.

Liu, Yangyang (ORCID:0000000335205332)↗
NASA NTRSDate not supplied

Spatiotemporal Methane Emissions from Global Lakes and Reservoirs

Inland aquatic systems, such as lakes and reservoirs, contribute substantially to global methane (CH4) emissions; yet are among the most uncertain components of the total CH4 budget. Lakes and reservoirs have received recent attention as they may generate high CH4 fluxes. Improved quantification of these CH4 fluxes, particularly their spatiotemporal distribution, is key to realistically incorporating them in CH4 modeling and budget studies. Here we report on a new global, gridded (0.25° lat × 0.25° lon) study of lake and reservoir CH4 emissions, accounting for new knowledge regarding lake and reservoir areal extent and distribution, and spatiotemporal emission patterns influenced by diurnal variability, temperature-dependent seasonality, satellite-derived freeze-thaw dynamics, and eco-climatic and physical CH4-centric type classification. The results of this new data set comprise daily CH4 emissions from lake and reservoirs throughout the full annual cycle and are tightly anchored to field observations, in situ measurements, and remote-sensing observations. Results show that reservoirs cover 297 × 103 km2 globally and emit 10.1 Tg CH4 yr-1 from diffusive (1.2 Tg CH4 yr-1) and ebullitive (8.9 Tg CH4 yr-1) emission pathways. On a global scale, CH4 emitting areas of lakes cover 1853 × 103 km2 and emit 37.1 Tg CH4 yr-1 from diffusive (17.1 Tg CH4 yr-1) and ebullitive (23.0 Tg CH4 yr-1) emission pathways and an additional ebullition flux of 5.2 Tg CH4 yr-1 upon ice-melt due to the accumulation of bubbles during the freeze period. This analysis of lakes and reservoir CH4 emission addresses multiple gaps and uncertainties in previous studies and represents an important contribution to studies of the global CH4 budget. The new data sets and methodologies from this study provide a framework to better understand and model the current and future role of lakes and reservoirs in the global CH4 budget and to guide efforts to mitigate inland aquatic system CH4 emissions. This presentation will describe the methodologies applied and the major results of this study focusing on the spatiotemporal distribution of lake and reservoir type classification, processes driving lake and reservoir emission seasonality, and the contribution of individual lake and reservoir types to the annual cycle of inland aquatic CH4 emissions.

Spatiotemporal↗
DOE OSTI2024

Spatiotemporal Reaction Dynamics Control in Two‐Photon Polymerization for Enhancing Writing Characteristics

Since 2001, 3D microfabrication based on two‐photon polymerization (TPP) has drawn extensive attention and interest in biology, optics, photonics, material science, and high‐energy physics. The in‐volume fabrication capability due to the threshold behavior of two‐photon absorption enables TPP higher flexibility compared with other nanofabrication techniques. However, as determined by the in‐volume fabrication feature as well as various reaction dynamics, the writing characteristics of TPP, such as throughput, accuracy, surface quality, and fabrication capability, are still limited. Herein, a comprehensive study is performed on the spatiotemporal behavior of reaction dynamics during TPP fabrication, mainly focusing on spatiotemporal characteristics of radical diffusion, photothermal effect, microscale mechanics, and voxel stacking process. Based on the study, a nonsequential fabrication method is established to simultaneously improve key writing characteristics of TPP and realize sharp features, high speeds, large overhang structure, and smooth surfaces. The method established in this work can be applied to improve the performance of functional devices for various fields.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Spatiotemporal mapping of mesoscopic liquid dynamics

The study of liquid dynamics at mesoscopic scales is still strewn with difficulty due to limitations in theory and experiment. Historically, significant attention has been given to the analysis of space-time correlation functions and their frequency-Fourier transforms at a few discrete wave numbers. The massive computing power afforded by modern high performance computing clusters and the advent of a wide-angle neutron spin-echo spectrometer, however, have unlocked a more intuitive and fruitful approach to this problem. Using molecular dynamics simulations, here we demonstrate the benefits of spatiotemporally mapping intermediate scattering functions on a dense grid of correlation times and wave numbers. Four model systems are investigated: a Lennard-Jones liquid, a coarse-grained bead-spring polymer, a molten sodium chloride, and a poly(ethylene oxide) melt. We show that the spatiotemporal mapping approach is particularly useful for elucidating the mesoscopic dynamics in these liquids, where several underlying mechanisms, such as molecular relaxations, hydrodynamic modes, and nonhydrodynamic excitations, are potentially at play. Therefore, compared to the traditional method, direct visualization of density space-time correlation functions on two-dimensional color maps permits appraisals of complicated dynamical behavior at mesoscales in a global manner. For example, the scaling relations between space and time for different types of molecular motions can be straightforwardly identified on these plots, without any model-dependent analysis. Additionally, we show how theoretical ideas regarding collective mesoscopic dynamics, such as the classical hydrodynamic theory, the convolution approximation, and a recently proposed phenomenological model, can be discussed in terms of the global features of spatiotemporal maps of intermediate scattering functions. The new perspective offered by the spatiotemporal mapping method should prove useful for the study of liquid dynamics in general.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2021

Attend and Decode: 4D fMRI Task State Decoding Using Attention Models

Source code for Brain Attend and Decode paper. Functional magnetic resonance imaging (fMRI) is a neuroimaging modality that captures the blood oxygen level in a subject's brain while the subject either rests or performs a variety of functional tasks under different conditions. Given fMRI data, the problem of inferring the task, known as task state decoding, is challenging due to the high dimensionality (hundreds of million sampling points per datum) and complex spatio-temporal blood flow patterns inherent in the data. In this work, we propose to tackle the fMRI task state decoding problem by casting it as a 4D spatiotemporal classification problem. We present a novel architecture called Brain Attend and Decode (BAnD), that uses residual convolutional neural networks for spatial feature extraction and self-attention mechanisms for temporal modeling. We achieve significant performance gain compared to previous works on a 7-task benchmark from the large-scale Human Connectome Project-Young Adult (HCP-YA) dataset. We also investigate the transferability of BAnD's extracted features on unseen HCP tasks, either by freezing the spatial feature extraction layers and retraining the temporal model, or finetuning the entire model. The pre-trained features from BAnD are useful on similar tasks while finetuning them yields competitive results on unseen tasks/conditions.

Ng, BrendaM.↗
NASA NTRS2014

ldentifying Episodes of Earth Science Phenomena Using a Big-Data Technology

A significant portion of Earth Science investigations is phenomenon- (or event-) based, such as the studies of Rossby waves, volcano eruptions, tsunamis, mesoscale convective systems, and tropical cyclones. However, except for a few high-impact phenomena, e.g. tropical cyclones, comprehensive records are absent for the occurrences or events of these phenomena. Phenomenon-based studies therefore often focus on a few prominent cases while the lesser ones are overlooked. Without an automated means to gather the events, comprehensive investigation of a phenomenon is at least time-consuming if not impossible. We have constructed a prototype Automated Event Service (AES) system that is used to methodically mine custom-defined events in the reanalysis data sets of atmospheric general circulation models. Our AES will enable researchers to specify their custom, numeric event criteria using a user-friendly web interface to search the reanalysis data sets. Moreover, we have included a social component to enable dynamic formation of collaboration groups for researchers to cooperate on event definitions of common interest and for the analysis of these events. An Earth Science event (ES event) is defined here as an episode of an Earth Science phenomenon (ES phenomenon). A cumulus cloud, a thunderstorm shower, a rogue wave, a tornado, an earthquake, a tsunami, a hurricane, or an El Nino, is each an episode of a named ES phenomenon, and, from the small and insignificant to the large and potent, all are examples of ES events. An ES event has a duration (often finite) and an associated geo-location as a function of time; it's therefore an entity embedded in four-dimensional (4D) spatiotemporal space. Earth Science phenomena with the potential to cause massive economic disruption or loss of life often rivet the attention of researchers. But, broader scientific curiosity also drives the study of phenomena that pose no immediate danger, such as land/sea breezes. Due to Earth System's intricate dynamics, we are continuously discovering novel ES phenomena. We generally gain understanding of a given phenomenon by observing and studying individual events. This process usually begins by identifying the occurrences of these events. Once representative events are identified or found, we must locate associated observed or simulated data prior to commencing analysis and concerted studies of the phenomenon. Knowledge concerning the phenomenon can accumulate only after analysis has started. However, as mentioned previously, comprehensive records only exist for a very limited set of high-impact phenomena; aside from these, finding events and locating associated data currently may take a prohibitive amount of time and effort on the part of an individual investigator. The reason for the lack of comprehensive records for most of the ES phenomena is mainly due to the perception that they do not pose immediate and/or severe threat to life and property. Thus they are not consistently tracked, monitored, and catalogued. Many phenomena even lack precise and/or commonly accepted criteria for definitions. Moreover, various Earth Science observations and data have accumulated to a previously unfathomable volume; NASA Earth Observing System Data Information System (EOSDIS) alone archives several petabytes (PB) of satellite remote sensing data, which are steadily increasing. All of these factors contribute to the difficulty of methodically identifying events corresponding to a given phenomenon and significantly impede systematic investigations. We have not only envisioned AES as an environment for identifying customdefined events but also aspired for it to be an interactive environment with quick turnaround time for revisions of query criteria and results, as well as a collaborative environment where geographically distributed experts may work together on the same phenomena. A Big Data technology is thus required for the realization of such a system. In the following, we first introduce the technology selected for AES in the next section. We then demonstrate the utility of AES using a use case, Blizzard, before we conclude.

Kuo, Kwo-Sen↗
DOE OSTI2026

4D Multimodal Co-attention Fusion Network with Latent Contrastive Alignment for Alzheimer’s Diagnosis

Multimodal neuroimaging provides complementary structural and functional insights into both human brain organization and disease-related dynamics. Recent studies demonstrate enhanced diagnostic sensitivity for Alzheimer’s disease (AD) through synergistic integration of neuroimaging data (e.g., sMRI, fMRI) with tabular data (e.g., behavioral and cognitive tests). However, the intrinsic heterogeneity across modalities (e.g., 4D spatiotemporal fMRI dynamics vs. 3D anatomical sMRI structure) presents critical challenges for discriminative feature fusion, often leading to information loss or biased fusion. To bridge this gap, we propose M2M-AlignNet: a multimodal co-attention network with latent alignment for early AD diagnosis using sMRI and fMRI. At the core of our approach is a multi-patch-to-multi-patch (M2M) contrastive loss function that quantifies and reduces representational discrepancies via weighted patch correspondence, explicitly aligning fMRI components across brain regions with their sMRI structural substrates without one-to-one constraints. Additionally, we propose a latent-as-query co-attention module to autonomously discover fusion patterns, circumventing modality prioritization biases while minimizing feature redundancy. We conduct extensive experiments to confirm the effectiveness of our method and highlight the correspondence between fMRI and sMRI as AD biomarkers.

Wei, Yuxiang [Georgia Institute of Technology]↗
DOE OSTI2023

Precise Control of Dissipative Self-assembly by Light and Electricity

Nature-inspired synthetic dissipative self-assemblies have attracted much attention recently. However, it remains a major challenge to achieve precise control over dissipative supramolecular assembly structures and functions of self-contained systems. Here we combine light and electricity as two clean, and spatiotemporally addressable fuels to provide precise control over the morphology for dissipative self-assembly of a perylene bisimide glycine (PBIg) building block in a self-contained solution. In this design, electrochemical oxidation provides the positive fuel to activate PBIg self-assembly while photoreduction supplies the negative fuel to deactivate the system for disassembly. Further, through programming the two counteracting fuels, we demonstrated the control of PBIg self-assembly into a variety of assembly morphologies in a self-contained system. In addition, by exerting light and electrical dual fuels simultaneously, we could create an active homeostasis exhibiting dynamic instability, leading to morphological change to asymmetric assemblies with curvatures. Such precise control over self-assembly of self-contained systems may find future applications in programming complex active materials as well as formulating pharmaceutical reagents with desired morphologies.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Grain Growth and Coarsening Dynamics in a Compositionally Asymmetric Block Copolymer Revealed by X-ray Photon Correlation Spectroscopy

The dynamics of nanostructured soft materials crucially impact their associated macroscopic material properties, yet they are often difficult to measure due to spatiotemporal limitations of conventional instrumentation. Herein, we use X-ray photon correlation spectroscopy (XPCS) to directly observe particle-scale dynamics during grain growth and coarsening in a BCC- forming diblock polymer melt, with specific attention to the distribution of structural relaxation times associated with the interplanar (110) distance. Following sample quenching from the disordered state, these dynamical phenomena surprisingly exhibit little dependence on time and thermal quench depth. We posit that these relaxations stem from collective particle motions during grain rotation. We also observe unusual internally-referenced heterodyne correlations, which enable measurements of speed distributions within the sample. In this work, these speeds are significantly slower and appear at much longer annealing times than those previously reported during grain nucleation and growth in microphase separated block polymer melts. Drawing on analogies between polycrystalline hard and soft materials, we ascribe these speed distributions to misorientation-dependent grain boundary migration during ordered domain coarsening and anomalously fast, cooperative string-like particle motion along the grain boundaries. Thus, these coherent X-ray measurements provide new opportunities to interrogate grain boundary structure and dynamics in polycrystalline soft materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1990

Neurocomputing

The present work is intended to give technologists, research scientists, and mathematicians a graduate-level overview of the field of neurocomputing. After exploring the relationship of this field to general neuroscience, attention is given to neural network building blocks, the self-adaptation equations of learning laws, the data-transformation structures of associative networks, and the multilayer data-transformation structures of mapping networks. Also treated are the neurocomputing frontiers of spatiotemporal, stochastic, and hierarchical networks, 'neurosoftware', the creation of neural network-based computers, and neurocomputing applications in sensor processing, control, and data analysis.

Hecht-Nielsen, Robert↗