Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “temporal learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

AttentionFire_v1.0: interpretable machine learning fire model for burned-area predictions over tropics

Abstract. African and South American (ASA) wildfires account for more than 70 % of global burned areas and have strong connection to local climate for sub-seasonal to seasonal wildfire dynamics. However, representation of the wildfire–climate relationship remains challenging due to spatiotemporally heterogenous responses of wildfires to climate variability and human influences. Here, we developed an interpretable machine learning (ML) fire model (AttentionFire_v1.0) to resolve the complex controls of climate and human activities on burned areas and to better predict burned areas over ASA regions. Our ML fire model substantially improved predictability of burned areas for both spatial and temporal dynamics compared with five commonly used machine learning models. More importantly, the model revealed strong time-lagged control from climate wetness on the burned areas. The model also predicted that, under a high-emission future climate scenario, the recently observed declines in burned area will reverse in South America in the near future due to climate changes. Our study provides a reliable and interpretable fire model and highlights the importance of lagged wildfire–climate relationships in historical and future predictions.

58 GEOSCIENCES↗

Panoramic Mapping of Phonon Transport from Ultrafast Electron Diffraction and Scientific Machine Learning

Abstract One central challenge in understanding phonon thermal transport is a lack of experimental tools to investigate frequency‐resolved phonon transport. Although recent advances in computation lead to frequency‐resolved information, it is hindered by unknown defects in bulk regions and at interfaces. Here, a framework that can uncover microscopic phonon transport information in heterostructures is presented, integrating state‐of‐the‐art ultrafast electron diffraction (UED) with advanced scientific machine learning (SciML). Taking advantage of the dual temporal and reciprocal‐space resolution in UED, and the ability of SciML to solve inverse problems involving coupled Boltzmann transport equations, the frequency‐dependent interfacial transmittance and frequency‐dependent relaxation times of the heterostructure from the diffraction patterns are reliably recovered. The framework is applied to experimental Au/Si UED data, and a transport pattern beyond the diffuse mismatch model is revealed, which further enables a direct reconstruction of real‐space, real‐time, frequency‐resolved phonon dynamics across the interface. The work provides a new pathway to probe interfacial phonon transport mechanisms with unprecedented details.

36 MATERIALS SCIENCE↗

Deep learning models map rapid plant species changes from citizen science and remote sensing data

Anthropogenic habitat destruction and climate change are reshaping the geographic distribution of plants worldwide. However, we are still unable to map species shifts at high spatial, temporal, and taxonomic resolution. Here, we develop a deep learning model trained using remote sensing images from California paired with half a million citizen science observations that can map the distribution of over 2,000 plant species. Our model— Deepbiosphere— not only outperforms many common species distribution modeling approaches (AUC 0.95 vs. 0.88) but can map species at up to a few meters resolution and finely delineate plant communities with high accuracy, including the pristine and clear-cut forests of Redwood National Park. These fine-scale predictions can further be used to map the intensity of habitat fragmentation and sharp ecosystem transitions across human-altered landscapes. In addition, from frequent collections of remote sensing data, Deepbiosphere can detect the rapid effects of severe wildfire on plant community composition across a 2-y time period. These findings demonstrate that integrating public earth observations and citizen science with deep learning can pave the way toward automated systems for monitoring biodiversity change in real-time worldwide.

Gillespie, Lauren E.↗

Application of Learning Machines and Combinatorial Algorithms in Water Resources Management and Hydrologic Sciences

Contemporary and water resources engineering and management rely increasingly on pattern recognition techniques that have the ability to capitalize on the unrelenting accumulation of data that is made possible by modern information technology and remote sensing methods. In response to the growing information needs of modern water systems, advanced computational models and tools have been devised to identify and extract relevant information from the mass of data that is now available. This chapter presents innovative applications from computational learning science within the fields of hydrology, hydrogeology, hydroclimatology, and water management. The success of machine learning is evident from the growing number of studies involving the application of Artificial Neural Networks (ANN), Support Vector Machines (SVM), Relevance Vector Machines (RVM), and Locally Weighted Projection Regression (LWPR) to address various issues in hydrologic sciences. The applications that will be discussed within the chapter employ the abovementioned machine learning techniques for intelligent modeling of reservoir operations, temporal downscaling of precipitation, spatial downscaling of soil moisture and evapotranspiration, comparisons of various techniques for groundwater quality modeling, and forecasting of chaotic time series behavior. Combinatorial algorithms to capture the intrinsic complexities in the modeled phenomena and to overcome disparate scales are developed; for example, learning machines have been coupled with geostatistical techniques, non-homogenous hidden Markov models, wavelets, and evolutionary computing techniques. This chapter does not intend to be exhaustive; it reviews the progress that has been made over the past decade in the use of learning machines in applied hydrologic sciences and presents a summary of future needs and challenges for further advancement of these methods.

54 ENVIRONMENTAL SCIENCES↗

Safe Deep Reinforcement Learning for Active Distribution System Model Predictive Control with EVs and DERs

The temporal and spatial mismatch between PV generation and electric vehicle (EV) charging and discharging may cause voltage violations in active distribution networks. Despite the widespread use of deep reinforcement learning (DRL) in power system optimization and control, it lacks guarantees on constraint satisfaction during both training and deployment. This paper proposes a Lagrangian-based safe DRL approach for model predictive control (MPC) of active distribution systems with large-scale integration of PVs, EVs, and energy storage systems (ESSs). A Transformer-LSTM time-series model is proposed to forecast EV charging demand, which is then formulated as a constraint to ensure charging requirements are met. Using this prediction, a Lagrangian-based safe soft actor-critic (SAC) framework is developed for real-time control in a three-phase unbalanced distribution system, enforcing voltage safety constraints while optimizing the cumulative net reward. By integrating the forecasting model with multi-period constraints, the proposed framework jointly coordinates PV systems, EV charging and discharging, and ESS scheduling within the MPC horizon. Numerical experiments on a modified IEEE 123-bus system with real-world data show that, under a high PV penetration scenario, the proposed method increases the net reward by 30.74% and reduces average voltage violations from 0.0011 p.u. to 0.0002 p.u. compared with standard SAC. Compared with the optimal power flow (OPF) approach, it achieves similar voltage security while yielding lower line losses. It also maintains real-time control capability, reducing operation latency to 53.21 ms per 15-minute control interval. The proposed method remains effective under varying PV/EV penetrations and load conditions.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Machine learning snow depth predictions at sites in Alaska, Norway, Siberia, Colorado and New Mexico

Temporally continuous snow depth estimates are vital for understanding changing snow patterns in the Arctic and impacts on permafrost. We trained random forest machine learning models to predict snow depth from temperature data recorded at or just below the ground surface. Training data was collected at the Teller 27 Watershed and Kougarok 64 Hillslope during the 2021 - 2022 water year on the Seward Peninsula, Alaska using distributed temperature profiling (DTP) systems. We then applied this model to other sites where ground surface or shallow soil temperature data was available for at least one water year (see Related Datasets). Many of these temperature measurements were collocated with snow depth observations. Ground surface temperature (i.e. snow-ground interface temperature) is easy to measure using small, cheap and easy-to-deploy temperature sensors such as iButtons and TinyTags, and such measurements have previously been used to calculate a variety of snow metrics (e.g. snow onset date). However, this is the first study to estimate snow depth directly from ground surface temperature data. The present dataset contains one *.csv file which includes machine learning snow depth predictions at sites in Alaska, Norway, Siberia, Colorado, and New Mexico and one *.kml file including the locations of sites with snow depth predictions. No training data predictions are included in the *.csv file. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy’s Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy’s Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

Multimodal super-resolution: discovering hidden physics and its application to fusion plasmas

Understanding complex physical systems often requires integrating data from multiple diagnostics, each with limited resolution or coverage. We present a machine learning framework that reconstructs synthetic high-temporal-resolution data for a target diagnostic using information from other diagnostics, without direct target measurements during the inference. This multimodal super-resolution technique improves diagnostic robustness and enables monitoring even in case of measurement failures or degradation. Applied to fusion plasmas, our method targets edge-localized modes (ELMs), which can damage plasma-facing materials. By reconstructing super-resolution Thomson Scattering data from complementary diagnostics, we uncover fine-scale plasma dynamics and validate the role of resonant magnetic perturbations (RMPs) in ELM suppression through magnetic island formation. The approach provides new observation supporting the plasma profile flattening due to these islands. Our results demonstrate the framework’s ability to generate high-fidelity synthetic diagnostics, offering a powerful tool for ELM control development in future reactors like ITER. The approach is broadly transferable to other domains facing sparse, incomplete, or degraded diagnostic data, opening new avenues for discovery.

Jalalvand, Azarakhsh [Princeton Univ., NJ (United ↗

Use of Event-Time Embeddings via RNN to Discern Novel Event Sequences in EHRs

In highly configurable health information technology (HIT) systems, such as VistA of the Veterans Health Administration, the variations in how the system is used among different healthcare facilities and how the data are recorded can be significant. Despite the successful standardization of care efforts, some of these variations can be indicative of HIT hazards and demand further investigation. In this work, we implemented a recurrent neural network (RNN) architecture to learn clinical provider order sequences and their temporal dynamics while predicting the orders' terminal state. We demonstrate model performance and provide a use case for the model discerning novel event sequences. This model is proposed to find novel event sequences in an operational environment.

Ozmen, Ozgur↗

A Geometry-Driven Longitudal Topic Model

A simple and scalable framework for longitudinal analysis of Twitter data is developed that combines latent topic models with computational geometric methods. Dimensionality reduction tools from computational geometry are applied to learn the intrinsic manifold on which the latent, temporal topics reside. Then shortest path distances on the manifold are used to link together these topics. The proposed framework permits visualization of the low-dimensional embedding which provides clear interpretation of the complex, high-dimensional trajectories that may exist among latent topics. Practical application of the proposed framework is demonstrated through its ability to capture and effectively visualize natural progression of latent COVID-19 related topics learned from Twitter data. Interpretability of the trajectories is achieved by comparing to real-world events. In addition, the framework permits study of spatial variation in Twitter behavior for learned topics. The analysis demonstrates that the proposed framework is able to capture granular-level impact of COVID-19 on public discussions. We end by arguing that Twitter data, when analyzed within the proposed framework, can serve as a valuable supplementary data stream for COVID-related studies.

97 MATHEMATICS AND COMPUTING↗

A Graph Convolutional Network for Active Distribution System Anomaly Detection Considering Measurement Spatial-Temporal Correlations

The accuracy of distribution system state estimation may be significantly impacted by the existence of bad measure-ments and unexpected topology errors. This paper proposes a data-driven Graph Convolutional Network (GCN) for anomaly detection, including bad measurements and topology change events. Compared to many existing machine learning approaches, the proposed approach embeds both spatial-temporal measure-ment correlations, which allows us to detect and distinguish different anomalies. Numerical results carried out on the IEEE 37-node system demonstrate that the proposed-based method can obtain high accuracy in detecting bad data and topology changes as compared to other approaches, even in the presence of high PV penetrations.

active distribution system↗

Exact enforcement of temporal continuity in sequential physics-informed neural networks

The use of deep learning methods in scientific computing represents a potential paradigm shift in engineering problem solving. One of the most prominent developments is Physics-Informed Neural Networks (PINNs), in which neural networks are trained to satisfy partial differential equations (PDEs). While this method shows promise, the standard version has been shown to struggle in accurately predicting the dynamic behavior of time-dependent problems. To address this challenge, methods have been proposed that decompose the time domain into multiple segments, employing a distinct neural network in each segment and directly incorporating continuity between them in the loss function of the minimization problem. In this work we introduce a method to exactly enforce continuity between successive time segments via a solution ansatz. This hard constrained sequential PINN (HCS-PINN) method is simple to implement and eliminates the need for any loss terms associated with temporal continuity. The method is tested for a number of benchmark problems involving both linear and non-linear PDEs. Examples include various first order time dependent problems in which traditional PINNs struggle, namely advection, Allen–Cahn, and Korteweg–de Vries equations. Furthermore, second and third order time-dependent problems are demonstrated via wave and Jerky dynamics examples, respectively. Notably, the Jerky dynamics problem is chaotic, making the problem especially sensitive to temporal accuracy. Finally, the numerical experiments conducted with the proposed method demonstrated superior convergence and accuracy over both traditional PINNs and the soft-constrained counterparts.

42 ENGINEERING↗

Deep Spatiotemporal Feature Learning with Application to Image Classification

Deep machine learning is an emerging framework for dealing with complex high-dimensionality data in a hierarchical fashion which draws some inspiration from biological sources. Despite the notable progress made in the field, there remains a need for an architecture that can represent temporal information with the same ease that spatial information is discovered. In this work, we present new results using a recently introduced deep learning architecture called Deep Spatio-Temporal Inference Network (DeSTIN). DeSTIN is a discriminative deep learning architecture that combines concepts from unsupervised learning for dynamic pattern representation together with Bayesian inference. In DeSTIN the spatiotemporal dependencies that exist within the observations are modeled inherently in an unguided manner. Each node models the inputs by means of clustering and simple dynamics modeling while it constructs a belief state over the distribution of sequences using Bayesian inference. We demonstrate that information from the different layers of this hierarchical system can be extracted and utilized for the purpose of pattern classification. Earlier simulation results indicated that the framework is highly promising, consequently in this work we expand DeSTIN to a popular problem, the MNIST data set of handwritten digits. The system as a preprocessor to a neural network achieves a recognition accuracy of 97.98% on this data set. We further show related experimental results pertaining to automatic cluster adaptation and termination.

99 GENERAL AND MISCELLANEOUS↗

Full-field imaging learning machine (FILM)

A method of determining dynamic properties of a structure (linear or nonlinear) includes receiving spatio-temporal inputs, generating mode shapes and modal components corresponding to the spatio-temporal inputs using a trained deep complexity coding artificial neural network, and subsequently generating the dynamic properties by analyzing each modal component using a trained learning machine. A computing system for non-contact determination of dynamic properties of a structure includes a camera, a processor, and a memory including computer-executable instructions. When the instructions are executed, the system is caused to receive spatio-temporal image data, decompose the spatio-temporal image data into constituent manifold components using an autoencoder, and analyze the constituent manifold components using a trained learning machine to determine the dynamic properties.

Yang, Yongchao↗

A Statistician’s Overview of Physics-Informed Neural Networks for Spatio-Temporal Data

The recent success of deep neural network models with physical constraints (so-called, Physics-Informed Neural Networks, PINNs) has led to renewed interest in the incorporation of mechanistic information in predictive models. Statisticians and others have long been interested in this problem, which has led to several practical and innovative solutions dating back decades. In this overview, we focus on the problem of data-driven prediction and inference of dynamic spatio-temporal processes that include mechanistic information, such as would be available from partial differential equations, with a strong focus on the quantification of uncertainty associated with data, process, and parameters. Here, we give a brief review of several paradigms and focus our attention on Bayesian implementations given they naturally accommodate uncertainty quantification. We then show that it is straight-forward to include the Bayesian PINN (B-PINN) within the Bayesian hierarchical model (BHM) framework that has long been considered for modeling dynamic spatio-temporal processes. Such a BHM-PINN is illustrated via a simulation study in which a latent nonlinear Burgers’ equation PDE governs the dynamics of Poisson distributed spatio-temporal data. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Bayesian↗

STSR-INR: Spatiotemporal super-resolution for multivariate time-varying volumetric data via implicit neural representation

Implicit neural representation (INR) has surfaced as a promising direction for solving different scientific visualization tasks due to its continuous representation and flexible input and output settings. We present STSR-INR, an INR solution for generating simultaneous spatiotemporal super-resolution for multivariate time-varying volumetric data. Inheriting the benefits of the INR-based approach, STSR-INR supports unsupervised learning and permits data upscaling with arbitrary spatial and temporal scale factors. Unlike existing GAN- or INR-based super-resolution methods, STSR-INR focuses on tackling variables or ensembles and enabling joint training across datasets of various spatiotemporal resolutions. Here we achieve this capability via a variable embedding scheme that learns latent vectors for different variables. In conjunction with a modulated structure in the network design, we employ a variational auto-decoder to optimize the learnable latent vectors to enable latent-space interpolation. To combat the slow training of INR, we leverage a multi-head strategy to improve training and inference speed with significant speedup. We demonstrate the effectiveness of STSR-INR with multiple scalar field datasets and compare it with conventional tricubic+linear interpolation and state-of-the-art deep-learning-based solutions (STNet and CoordNet).

97 MATHEMATICS AND COMPUTING↗

Using spatio-temporal graph neural networks to estimate fleet-wide photovoltaic performance degradation patterns

Accurate estimation of photovoltaic (PV) system performance is crucial for determining its feasibility as a power generation technology and financial asset. PV-based energy solutions offer a viable alternative to traditional energy resources due to their superior Levelized Cost of Energy (LCOE). A significant challenge in assessing the LCOE of PV systems lies in understanding the Performance Loss Rate (PLR) for large fleets of PV systems. Estimating the PLR of PV systems becomes increasingly important in the rapidly growing PV industry. Precise PLR estimation benefits PV users by providing real-time monitoring of PV module performance, while explainable PLR estimation assists PV manufacturers in studying and enhancing the performance of their products. However, traditional PLR estimation methods based on statistical models have notable drawbacks. Firstly, they require user knowledge and decision-making. Secondly, they fail to leverage spatial coherence for fleet-level analysis. Additionally, these methods inherently assume the linearity of degradation, which is not representative of real world degradation. To overcome these challenges, we propose a novel graph deep learning-based decomposition method called the Spatio-Temporal Graph Neural Network for fleet-level PLR estimation (PV-stGNN-PLR). PV-stGNN-PLR decomposes the power timeseries data into aging and fluctuation components, utilizing the aging component to estimate PLR. PV-stGNN-PLR exploits spatial and temporal coherence to derive PLR estimation for all systems in a fleet and imposes flatness and smoothness regularization in loss function to ensure the successful disentanglement between aging and fluctuation. We have evaluated PV-stGNN-PLR on three simulated PV datasets consisting of 100 inverters from 5 sites. Experimental results show that PV-stGNN-PLR obtains a reduction of 33.9% and 35.1% on average in Mean Absolute Percent Error (MAPE) and Euclidean Distance (ED) in PLR degradation pattern estimation compared to the state-of-the-art PLR estimation methods.

14 SOLAR ENERGY↗

Transmission Line Fault Location Using Deep Learning Techniques

This letter presents a novel single-ended fault location approach for transmission lines that uses modern deep learning methods. A mixed convolutional neural network and long short-term memory structure are proposed to preserve the translation invariance and capture the temporal correlation of the time-series input data. Advanced deep learning techniques such as adaptive moment estimation and dropout are used to efficiently train the neural network and prevent over-fitting. Our extensive studies have demonstrated the accuracy and effectiveness of the deep-learning-based, singled-ended fault location approach.

Fan, Rui↗

Scientific machine learning for closure models in multiscale problems: A review

Here, closure problems are omnipresent when simulating multiscale systems, where some quantities and processes cannot be fully prescribed despite their effects on the simulation's accuracy. Recently, scientific machine learning approaches have been proposed as a way to tackle the closure problem, combining traditional (physics-based) modeling with data-driven (machine-learned) techniques, typically through enriching differential equations with neural networks. This paper reviews the different reduced model forms, distinguished by the degree to which they include known physics, and the different objectives of a priori and a posteriori learning. The importance of adhering to physical laws (such as symmetries and conservation laws) in choosing the reduced model form and choosing the learning method is discussed. The effect of spatial and temporal discretization and recent trends toward discretization-invariant models are reviewed. In addition, we make the connections between closure problems and several other research disciplines: inverse problems, Mori-Zwanzig theory, and multi-fidelity methods. In conclusion, much progress has been made with scientific machine learning approaches for solving closure problems, but many challenges remain. In particular, the generalizability and interpretability of learned models is a major issue that needs to be addressed further.

97 MATHEMATICS AND COMPUTING↗