Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “time series encoding”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2020

Unsupervised Anomaly Detection in High-Dimensional Flight Data Using Convolutional Variational Auto-Encoder

The modern National Airspace System (NAS) is an extremely safe system and the aviation industry has experienced a steady decrease in fatalities over the years. This can be attributed to both improved flight critical systems with redundant hardware and software protections, as well as an increased focus on active monitoring and response to real time and historically identified vulnerabilities by implementing more resilient procedures and protocols. The main approach for identifying vulnerabilities in operations leverages domain expertise using knowledge about how the system should behave within the expected tolerances to known safety margins. This approach works well when the system has a well-defined operating condition. However, the operations in the NAS can be highly complex with various nuances that render it difficult to clearly pre-define all known safety vulnerabilities. With the advancement of data science and machine learning techniques, the potential to automatically identify emerging vulnerabilities in the observed operations has become more practical in recent years. The state-of-the-art anomaly detection approaches in aerospace data usually rely on supervised or semi-supervised learning. However, in many real-world problems such as flight safety, creating labels for the data requires huge amount of effort and is largely impractical. To address this challenge, we developed a Convolutional Variational Auto-Encoder (CVAE), which is an unsupervised learning approach for anomaly detection in high-dimensional heterogeneous time-series data. We validate performance of CVAE compared to the state-of-the-art supervised learning approach as well as unsupervised clustering-based approach using KMeans++ and kernel-based approach using One-Class Support Vector Machine (OC-SVM) on Yahoo!'s benchmark time series anomaly detection data. Finally, we showcase performance of CVAE on a case study of identifying anomalies in the first 60 seconds of commercial flights' take-offs using Flight Operational Quality Assurance (FOQA) data.

Memarzadeh, Milad↗
NASA NTRS2007

A Proposal for a Thesaurus for Web Services in Solar Radiation

Metadata are necessary to discover, describe and exchange any type of information, resource and service at a large scale. A significant amount of effort has been made in the field of geography and environment to establish standards. Efforts still remain to address more specific domains such as renewable energies. This communication focuses on solar energy and more specifically on aspects in solar radiation that relate to geography and meteorology. A thesaurus in solar radiation is proposed for the keys elements in solar radiation namely time, space and radiation types. The importance of time-series in solar radiation is outlined and attributes of the key elements are discussed. An XML schema for encoding metadata is proposed. The exploitation of such a schema in web services is discussed. This proposal is a first attempt at establishing a thesaurus for describing data and applications in solar radiation.

Gschwind, Benoit↗
DOE OSTI2022

Latent-Space Dynamics for Prediction and Fault Detection in Geothermal Power Plant Operations

This paper presents a latent-space dynamic neural network (LSDNN) model for the multi-step-ahead prediction and fault detection of a geothermal power plant’s operation. The model was trained to learn the dynamics of the power generation process from multivariate time-series data and the effects of exogenous variables, such as control adjustment and ambient temperature. In the LSDNN model, an encoder–decoder architecture was designed to capture cross-correlation among different measured variables. In addition, a latent space dynamic structure was proposed to propagate the dynamics in the latent space to enable prediction. The prediction power of the LSDNN was utilized for monitoring a geothermal power plant and detecting abnormal events. The model was integrated with principal component analysis (PCA)-based process monitoring techniques to develop a fault-detection procedure. The performance of the proposed LSDNN model and fault detection approach was demonstrated using field data collected from a geothermal power plant.

15 GEOTHERMAL ENERGY↗
NASA NTRS1995

Inductive Approaches to Improving Diagnosis and Design for Diagnosability

The first research area under this grant addresses the problem of classifying time series according to their morphological features in the time domain. A supervised learning system called CALCHAS, which induces a classification procedure for signatures from preclassified examples, was developed. For each of several signature classes, the system infers a model that captures the class's morphological features using Bayesian model induction and the minimum message length approach to assign priors. After induction, a time series (signature) is classified in one of the classes when there is enough evidence to support that decision. Time series with sufficiently novel features, belonging to classes not present in the training set, are recognized as such. A second area of research assumes two sources of information about a system: a model or domain theory that encodes aspects of the system under study and data from actual system operations over time. A model, when it exists, represents strong prior expectations about how a system will perform. Our work with a diagnostic model of the RCS (Reaction Control System) of the Space Shuttle motivated the development of SIG, a system which combines information from a model (or domain theory) and data. As it tracks RCS behavior, the model computes quantitative and qualitative values. Induction is then performed over the data represented by both the 'raw' features and the model-computed high-level features. Finally, work on clustering for operating mode discovery motivated some important extensions to the clustering strategy we had used. One modification appends an iterative optimization technique onto the clustering system; this optimization strategy appears to be novel in the clustering literature. A second modification improves the noise tolerance of the clustering system. In particular, we adapt resampling-based pruning strategies used by supervised learning systems to the task of simplifying hierarchical clusterings, thus making post-clustering analysis easier.

Fisher, Douglas H.↗
DOE OSTI2026

Hierarchical-embedding autoencoder with a predictor as efficient architecture for learning time-evolution in multi-scale turbulent flows

We introduce a scale-aware, data-driven deep learning modeling framework for accurately predicting the time evolution of multi-scale turbulent plasma and liquid flows. The approach is motivated by the idea of scale separation. Structures of vastly different length scales emerge in these systems, and interactions between these structures occur only locally. To exploit this structure, the flow state is transformed by a hierarchical, fully convolutional autoencoder, not into a single embedding layer as in conventional convolutional surrogate models, but into a series of embedding layers. A stepwise training strategy ensures that fine-scale features are encoded on a high-resolution grid, while larger structures are represented on progressively coarser layers. The time evolution predictor advances all embedding layers in sync, capturing local interactions between features at the same scale as well as between all scales. This approach enables efficient modeling of multi-scale systems since negligible interactions between distant, small-scale structures do not need to be directly modeled. Our hierarchical-embedding autoencoder with a predictor framework is evaluated on canonical examples of multi-scale turbulence: two-dimensional Kolmogorov flow and Hasegawa–Wakatani plasma turbulence. In both cases, the proposed framework significantly improves predictive accuracy relative to conventional convolutional network architectures. A significant improvement in prediction accuracy was observed for crucial statistical characteristics of the Hasegawa–Wakatani plasma as well as for individual trajectories of the Kolmogorov flow turbulence. Importantly, the model's rollout for the Hasegawa–Wakatani problem demonstrates a four-order-of-magnitude speedup compared to traditional numerical solvers.

Khrabry, Alexander I. [Princeton Univ., NJ (United↗
DOE OSTI2022

Dimensionally Reduced Model for Rapid and Accurate Prediction of Gas Saturation, Pressure, and Brine Production in a CO 2 Storage Application: Case Study Using the SACROC Field as Part of SMART Task 5

This technical report presents work conducted by the sub surface analysis team of the Strategic Systems Analysis & Engineering group at NETL for Task 5 of SMART Phase 1. This study involved the development of deep learning models for CO 2 geologic storage that are capable of accurate prediction of spatio-temporal outputs of CO 2 saturation, pressure, and brine production in three dimensional space over a storage operation's injection and post-injection timeframes. The model framework involves ensembling multi-layer encoder networks that provide dimesionality reduction of geologic inputs with fully connected long short-term memory (LSTM) neural networks that generate time-series prediction This approach offers a means to maximize training time efficiency, reduce computational memory burden, and minimize prediction turnaround.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

ATAT: Astronomical Transformer for time series and Tabular data

Context. The advent of next-generation survey instruments, such as theVera C. RubinObservatory and its Legacy Survey of Space and Time (LSST), is opening a window for new research in time-domain astronomy. The Extended LSST Astronomical Time-Series Classification Challenge (ELAsTiCC) was created to test the capacity of brokers to deal with a simulated LSST stream. Aims. Our aim is to develop a next-generation model for the classification of variable astronomical objects. We describe ATAT, the Astronomical Transformer for time series And Tabular data, a classification model conceived by the ALeRCE alert broker to classify light curves from next-generation alert streams. ATAT was tested in production during the first round of the ELAsTiCC campaigns. Methods. ATAT consists of two transformer models that encode light curves and features using novel time modulation and quantile feature tokenizer mechanisms, respectively. ATAT was trained on different combinations of light curves, metadata, and features calculated over the light curves. We compare ATAT against the current ALeRCE classifier, a balanced hierarchical random forest (BHRF) trained on human-engineered features derived from light curves and metadata. Results. When trained on light curves and metadata, ATAT achieves a macro F1 score of 82.9 ± 0.4 in 20 classes, outperforming the BHRF model trained on 429 features, which achieves a macro F1 score of 79.4 ± 0.1. Conclusions. The use of transformer multimodal architectures, combining light curves and tabular data, opens new possibilities for classifying alerts from a new generation of large etendue telescopes, such as theVera C. RubinObservatory, in real-world brokering scenarios.

Astronomy & Astrophysics↗
DOE OSTI2021

Quantum Algorithm for Simulating Hamiltonian Dynamics with an Off-diagonal Series Expansion

We propose an efficient quantum algorithm for simulating the dynamics of general Hamiltonian systems. Our technique is based on a power series expansion of the time-evolution operator in its off-diagonal terms. The expansion decouples the dynamics due to the diagonal component of the Hamiltonian from the dynamics generated by its off-diagonal part, which we encode using the linear combination of unitaries technique. Our method has an optimal dependence on the desired precision and, as we illustrate, generally requires considerably fewer resources than the current state-of-the-art. We provide an analysis of resource costs for several sample models.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Long–short-term memory encoder–decoder with regularized hidden dynamics for fault detection in industrial processes

The ability of recurrent neural networks (RNN) to model nonlinear dynamics of high dimensional process data has enabled data-driven RNN-based fault detection algorithms. Previous studies have focused on detecting faults by identifying the discrepancies in data distribution between the faulty and normal data, as reflected in prediction errors generated by RNN models. However, in industrial processes, variations in data distribution can also result from changes in normal control setpoints and compensatory control adjustments in response to disturbances, making it hard to differentiate between normal and faulty conditions. This paper proposes a fault detection method utilizing a long short-term memory (LSTM) encoder–decoder structure with regularized hidden dynamics and reversible instance normalization (RevIN) to compactly represent high-dimensional measurements for effective monitoring. During training, the hidden states of the model are regularized to form a low-dimensional latent space representation of the original multivariate time series data. As a result, the prediction errors of the latent states can be used to monitor the abnormal dynamic variations, while the reconstruction errors of the measured variables are used to monitor the abnormal static variations. Furthermore, the proposed indices can reflect operating conditions, even when the distribution of test data changes, which helps distinguish faults from normal adjustments and disturbances that controllers can settle. Here, data from numerical simulation and the Tennessee Eastman process are used to illustrate the effectiveness of the proposed fault detection method.

42 ENGINEERING↗
DOE OSTI2020

CONFIDANTE Demonstration Prototype Report

This report summarizes design optimization and performance evaluation studies for a new prototype CONFIDANTE (CONfirmation using a Fast-neutron Imaging Detector with Anti-image Null-positive Time Encoding) warhead confirmation system. It was found that a spherical mask geometry and a 2” diameter cylindrical central detector is expected to best discriminate between the parametrically varied source distributions that were evaluated. The optimized design as fabricated and its performance was evaluated in a series of laboratory measurements. The performance was in good agreement with the design studies, with demonstrated discrimination between objects with differences in scale on the order of 5 centimeters or better at 1 meter stand-off

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Improving Schrödinger Equation Implementations with Gray Code for Adiabatic Quantum Computers

We reformulate the continuous-space Schrödinger equation in terms of spin Hamiltonians. For the kinetic energy operator, the critical concept facilitating the reduction in model complexity is the idea of position encoding. A binary encoding of position produces a spin-1/2 Heisenberg-like model and yields exponential improvement in space complexity when compared to classical computing. Encoding with a binary reflected Gray code (BRGC), and a Hamming-distance-2 Gray code (H2GC) reduces the model complexity down to the 𝑋⁢𝑍 and transverse Ising model, respectively. For 𝐴 qubits BRGC yields 2 𝐴 positions and is reduced to its 2-local form with O⁡(𝐴) ancillary qubits. H2GC yields 2 𝐴/2+1 positions with O⁡(𝐴 2 ) three-local penalty terms. We also identify the bijective mapping between diagonal unitaries and the Walsh series, producing the mapping of any real potential to a series of 𝑘 -local Ising models through the fast Walsh transform. Finally, in a finite volume, we provide some numerical evidence to support the claim that the total time needed for adiabatic evolution is protected by the infrared cutoff of the system. As a result, initial state preparation from a free-field wave function to an interacting system is expected to exhibit polynomial time complexity with volume and constant scaling with respect to lattice discretization for all encodings. For H2GC, if the evolution starts with the transverse Hamiltonian due to hardware restrictions, then penalties are dynamically introduced such that the low-lying spectrum reproduces the energy levels of the Laplacian. The adiabatic evolution of the penalty Hamiltonian is therefore sensitive to the ultraviolet scale. It is expected to exhibit polynomial time complexity with lattice discretization, or exponential time complexity with respect to the number of qubits given a fixed volume.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Enhancing Short-Range Weather Forecasts through Temporal Variation Encoding: A Multiperiod Embedding Approach

Machine learning (ML) techniques have emerged as promising approaches to improve regional weather forecast accuracy and reliability through data-driven methods. We propose a novel ML-based weather forecasting model, the Multiperiod Embed Net (MPENet). A key distinguishing feature of MPENet is its explicit utilization of the inherent cyclic nature in weather dynamics, unlike the autoregressive strategies commonly used in other ML weather forecasting approaches. Critical cyclic structures are identified via Fourier analyses of dynamic time series. Cyclicity in the convolutional representation is achieved by transforming one-dimensional time series of meteorological variables into two-dimensional tensors based on identified periods. This approach enables the model to leverage intrinsic weather patterns, enhancing regional forecast performance. To demonstrate the effectiveness of MPENet, we conduct a comparative analysis with Nvidia’s FourCastNet. Both models are trained on High-Resolution Rapid Refresh (HRRR) data from 2015 to 2022, over a 192 km × 192 km region in Tennessee. The comparisons are performed locally at two specific locations known to have different weather dynamics due to orographic effects: Crossville, on the relatively flat Cumberland Plateau with fewer topographic airflow disruptions, and Oak Ridge, in the ridge-and-valley region, where airflow is heavily influenced by surrounding valleys and mountains. Our results indicate that FourCastNet achieves strong accuracy at very short lead times, while MPENet maintains competitive skill and shows advantages in capturing temporal evolution over longer periods. Cross-correlation analyses of MPENet and FourCastNet predictions with the HRRR data suggest that encoding critical cyclicity into the network architecture leads to improvements in the forecasting skill.

Artificial intelligence↗
DOE OSTI2024

AIRES-NODE

This software package performs time series forecasting for engineering systems using neural controlled differential equations. Neural controlled differential equations are continuous time analogues to recurrent neural networks. AIRES-NODE is particularly well suited for dynamic systems which are subject to external forcing terms and time-dependent boundary conditions. AIRES-NODE provides an encoder-decoder architecture which allows for variable-length forecasts to be made and is not limited to a fixed time step size when forecasting. Additionally, by utilizing a novel combination of a variational autoencoder and neural controlled differential equations, the AIRES-NODE package is capable of producing probabilistic future forecasts.

Gurecky, William [Oak Ridge National Laboratory (O↗
DOE OSTI2025

Knowledge Oriented Graph Unified Transformer (KOGUT) v0.1

KOGUT — Knowledge Oriented Graph Unified Transformer KOGUT implements the Relational Graph Transformer (RelGT) architecture for knowledge graph link prediction in biological domains, with a primary focus on microbial growth media prediction. While the original RelGT (arXiv:2505.10960) targets relational tables, time series, and multi-table databases, KOGUT adapts this architecture for heterogeneous biological knowledge graphs, providing first-in-class AI predictive models for microbial cultivation. Key Adaptations Beyond Original RelGT: - Knowledge Graph Focus: Applied to biological KGs with semantic node types (taxa, chemicals, media, phenotypes, environments) versus generic relational database tables, trained on the KG-Microbe knowledge graph (1.3M entities, 2.9M edges, 24 relation types). - Multimodal Node Encoding: Integrates node labels, categories, descriptions, and synonyms from KG metadata through learned embedding layers—adapting relational column features to graph node attributes with textual semantics. - Extended K-Hop Subgraph Strategy: Optimized neighborhood sampling (3-hop default, configurable up to 200 nodes) tuned for sparse biological networks, building on the original local-global attention framework with biological relation preservation. - Biolink Predicate Preservation: Type-specific transformations for 24 biological edge semantics (occurs_in, consumes, produces, has_phenotype, subclass_of) beyond standard relational foreign keys, enabling multi-relation link prediction. - Inductive Learning Support: Enables zero-shot predictions for novel taxa through feature-based embeddings (temperature, oxygen requirements, gram stain, cell shape), extending the original transductive relational benchmark scope to uncultured microorganisms. CheapSOTA Performance Optimizations (This Distribution): - VQ-EMA Centroid Attention: Vector quantization with exponential moving average for improved global context modeling (+5-10% MRR improvement). - HDF5 Precomputed Data Loading: One-time preprocessing of k-hop subgraphs to eliminate redundant graph traversals (2-5× training speedup). - Distributed Data Parallel Training: Multi-GPU support for scaling to larger knowledge graphs (tested on 4× NVIDIA A100 GPUs at NERSC Perlmutter). - Mixed Precision Training: Automatic mixed precision (AMP) for memory efficiency and faster training. Advantages Over Standard Knowledge Graph Embedding Models: Combines RelGT's proven multi-element tokenization (features, type, hop, structure) with graph-native biological representations, enabling interpretable link prediction across heterogeneous entities that standard embedding models (TransE, RotatE, ComplEx) and table-based transformers cannot directly model. Achieves near-perfect performance on microbial growth media prediction (MRR: 0.9966, Precision@1: 0.9932, Hit@10: 1.0000) while maintaining explainability through attention-based reasoning over biological pathways. Training Data: - KG-Microbe merged knowledge graph: 1,379,337 nodes, 2,960,472 edges - 24 biological relation types including taxonomic hierarchies, metabolic interactions, phenotype associations, and environmental relationships - Primary prediction task: Growth media suitability for microbial taxa (biolink:occurs_in, 50K edges) - Multi-relation capability: Predicts links for any of the 24 relation types, including chemical consumption/production, phenotype associations, and taxonomic classification Citation: Original RelGT Architecture: Dwivedi et al., "Relational Graph Transformer", arXiv:2505.10960, 2025 KOGUT Implementation: Knowledge Oriented Graph Unified Transformer for Microbial Growth Media Prediction Developed at Lawrence Berkeley National Laboratory (LBNL) Trained on NERSC Perlmutter supercomputer

Joachimiak, Marcin [Lawrence Berkeley National Lab↗
DOE OSTI2022

AGNet: weighing black holes with deep learning

Supermassive black holes (SMBHs) are commonly found at the centres of most massive galaxies. Measuring SMBH mass is crucial for understanding the origin and evolution of SMBHs. Traditional approaches, on the other hand, necessitate the collection of spectroscopic data, which is costly. We present an algorithm that weighs SMBHs using quasar light time series information, including colours, multiband magnitudes, and the variability of the light curves, circumventing the need for expensive spectra. We train, validate, and test neural networks that directly learn from the Sloan Digital Sky Survey (SDSS) Stripe 82 light curves for a sample of 38 939 spectroscopically confirmed quasars to map out the non-linear encoding between SMBH mass and multiband optical light curves. We find a 1σ scatter of 0.37 dex between the predicted SMBH mass and the fiducial virial mass estimate based on SDSS single-epoch spectra, which is comparable to the systematic uncertainty in the virial mass estimate. Our results have direct implications for more efficient applications with future observations from the Vera C. Rubin Observatory. Our code, AGNet, is publicly available at https://github.com/snehjp2/AGNet.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Efficient hybrid explicit-implicit learning for multiscale problems

Splitting method is a powerful method to handle application problems by splitting physics, scales, domain, and so on. Many splitting algorithms have been designed for efficient temporal discretization. Here, in this paper, our goal is to use temporal splitting concepts in designing machine learning algorithms and, at the same time, help splitting algorithms by incorporating data and speeding them up. We propose a machine learning assisted splitting scheme which improves the efficiency of the scheme meanwhile preserves the accuracy. We consider a recently introduced multiscale splitting algorithms, where the multiscale problem is solved on a coarse grid. To approximate the dynamics, only a few degrees of freedom are solved implicitly, while others explicitly. This splitting concept allows identifying degrees of freedom that need implicit treatment. In this paper, we use this splitting concept in machine learning and propose several strategies. First, the implicit part of the solution can be learned as it is more difficult to solve, while the explicit part can be computed. This provides a speed-up and data incorporation for splitting approaches. Secondly, one can design a hybrid neural network architecture because handling explicit parts requires much fewer communications among neurons and can be done efficiently. Thirdly, one can solve the coarse grid component via PDEs or other approximation methods and construct simpler neural networks for the explicit part of the solutions. We discuss these options and implement one of them by interpreting it as a machine translation task. This interpretation of the splitting scheme successfully enables us using the Transformer since it can perform model reduction for multiple time series and learn the connection between them. We also find that the splitting scheme is a great platform to predict the coarse solution with insufficient information of the target model: the target problem is partially given and we need to solve it through a known problem which approximates the target. Our machine learning model can incorporate and encode the given information from two different problems and then solve the target problems. We conduct four numerical examples and the results show that our method is stable and accurate.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Trajectory prediction dimensionality reduction for low-cost connected automated vehicle systems

Here, to facilitate low-cost connected automated vehicle (CAV) system development, this study proposes two interpretable dimensionality reduction techniques in vehicle trajectory prediction, i.e., the piecewise Taylor series approximation (PTA) and the piecewise Fourier series approximation (PFA), to lower computation complexity, reduce device investment, and decrease computation energy consumption. Two benchmarks are developed, the long short-term memory (LSTM)-based model without dimensionality reduction and the LSTM-based model with encoder-decoder (a widely used dimensionality reduction technique). Results show that the four predictions have similar accuracy, and the training time (proportional to computation energy consumption) of models with dimensionality reduction techniques is greatly reduced. The reduction is even more significant when PTA/PFA is used. Sensitivity analysis advises PFA/PTA parameter selections to reduce computation complexity without significant loss of prediction accuracy. Further, the robustness of the LSTM PTA/PFA is proven by the investigation of data noises.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS1999

Multifractality in human heartbeat dynamics

There is evidence that physiological signals under healthy conditions may have a fractal temporal structure. Here we investigate the possibility that time series generated by certain physiological control systems may be members of a special class of complex processes, termed multifractal, which require a large number of exponents to characterize their scaling properties. We report on evidence for multifractality in a biological dynamical system, the healthy human heartbeat, and show that the multifractal character and nonlinear properties of the healthy heart rate are encoded in the Fourier phases. We uncover a loss of multifractality for a life-threatening condition, congestive heart failure.

Non-NASA Center↗