Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “autoregressive processes”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Studies in astronomical time series analysis. I - Modeling random processes in the time domain

Several random process models in the time domain are defined and discussed. Attention is given to the moving average model, the autoregressive model, and relationships between and combinations of these models. Consideration is then given to methods for investigating pulse structure, procedures of model construction, computational methods, and numerical experiments. A FORTRAN algorithm of time series analysis has been developed which is relatively stable numerically. Results of test cases are given to study the effect of adding noise and of different distributions for the pulse amplitudes. A preliminary analysis of the light curve of the quasar 3C 272 is considered as an example.

Scargle, J. D.↗

Wildfire Segmentation From Remotely Sensed Data Using Quantum-Compatible Conditional Vector Quantized-Variational Autoencoders

Wildfires represent a critical environmental hazard with multifaceted implications for ecosystems, communities, and public health [1]. The escalating frequency and intensity of wildfires globally have intensified the urgency for robust segmentation methodologies to facilitate effective mitigation, response, and recovery strategies [2]. Accurate wildfire segmentation is pivotal for delineating fire boundaries, assessing progression patterns, and prioritizing resource allocation during emergency scenarios. Furthermore, precise segmentation enables stakeholders, including policymakers, environmental scientists, and emergency responders, to formulate evidence-based strategies, thereby minimizing socio-economic disruptions and ecological degradation. Consequently, advancing wildfire segmentation techniques through innovative technological interventions remains a paramount research imperative. Although foundational in wildfire segmentation, traditional deterministic models exhibit inherent limitations that compromise their efficacy in dynamic and uncertain environments. These models often operate on rigid algorithms prioritizing deterministic classifications, thereby overlooking the inherent complexities and uncertainties associated with wildfire behavior and satellite data variability. Such deterministic frameworks tend to produce oversimplified representations that fail to capture the intricate nuances of evolving fire dynamics, spatial heterogeneity, and environmental interactions [1]. Consequently, the deterministic approach’s propensity for uncertainty collapsing [1, 3] hampers the accuracy, reliability, and applicability of segmentation outcomes in real-world scenarios. Contrastingly, stochastic models offer a more nuanced and adaptable framework for wildfire segmentation. By integrating probabilistic elements into the modeling paradigm, stochastic approaches, particularly probabilistic approaches such as variational auto encoders (VAEs) [4], facilitate comprehensive uncertainty assessment, enabling researchers to quantify and incorporate uncertainties into segmentation outcomes effectively. This probabilistic nature empowers stochastic models to encapsulate variability, account for data inconsistencies, and adapt to evolving environmental conditions, enhancing segmentation accuracy, reliability, and robustness. Embracing stochastic methodologies thus catalyzes advancements in wildfire science by fostering a more holistic, adaptive, and resilient segmentation framework. Despite VAEs demonstrating significant promise in various applications, they come with inherent limitations that have garnered attention within the machine learning community. One of the primary drawbacks lies in their reliance on static priors, which essentially assume a fixed distribution for latent variables, thereby limiting the model’s flexibility to capture complex data structures effectively [5]. This static nature leads to suboptimal representations, especially when dealing with complex and high-dimensional data. Additionally, VAEs often struggle with generating sharp and realistic samples, a phenomenon commonly referred to as mode collapse [5, 7, 6]. Furthermore, the optimization process in VAEs, which involves balancing the reconstruction loss and the regularization term, can sometimes be challenging to fine-tune [7]. In recent efforts to address these shortcomings, alternative approaches like Vector Quantized Variational Auto encoders(VQ-VAEs) [7], address the challenges by incorporating discrete latent variables and leveraging techniques that enhance the quality and diversity of generated samples while maintaining efficient training dynamics. VQ-VAEs propose a dynamic prior distribution generation mechanism that diverges from the static priors commonly associated with traditional VAEs. This dynamic approach allows for more adaptive and context-aware latent variable representations, thereby potentially capturing complex data structures more effectively. Unlike autoregressive prior models such as PixelCNN, which, despite their ability to model dependencies across data dimensions, suffer from significant computational inefficiencies and lack flexibility in handling diverse datasets. In our work, we propose to use a generative quantum-compatible approach to help alleviate the shortcomings of autoregressive prior model in VQ-VAEs. Restricted Boltzmann Machines (RBMs) are a viable alternative prior model that can learn prior distributions in a faster and more flexible manner. In this research endeavor, we meticulously curate a state-of-the-art dataset leveraging satellite MODIS data in conjunction with VIIRS fire masks, derived from Fire Radiative Power (FRP), thereby encapsulating diverse wildfire scenarios and environmental contexts. We developed a conditional VQ-VAE architecture with the RBM prior model that is trained in a supervised manner for segmenting wildfire masks. This innovative approach synergistically harnesses deep learning capabilities, enabling the generation of segmentation maps characterized by heightened precision, granularity, and contextual relevance. Furthermore, replacing the autoregressive prior learning method proposed by the original VQ-VAE with a prior density approximation via quantum-compatible RBM facilitates expedited inference processes, augments flexibility in prior sampling, optimizes computational efficiency and establishes a groundbreaking benchmark in wildfire segmentation methodologies.

quantum machine learning↗

Rocket engine failure detection using system identification techiques

The theoretical foundation and application of two univariate failure detection algorithms to Space Shuttle Main Engine (SSME) test firing data is presented. Both algorithms were applied to data collected during steady state operation of the engine. One algorithm, the time series algorithm, is based on time series techniques and involves the computation of autoregressive models. Times series techniques have been previously applied to SSME data. The second algorithm is based on standard signal processing techniques. It consists of tracking the variations in the average signal power with time. The average signal power algorithm is a newly proposed SSME failure detection algorithm. Seven nominal test firings were used to develop failure indication thresholds for each algorithm. These thresholds were tested using four anomalous firings and one additional nominal firing. Both algorithms provided significantly earlier failure indication times than did the current redline limit system. Neither algorithm gave false failure indications for the nominal firing. The strengths and weaknesses of the two algorithms are discussed and compared. The average signal algorithm was found to have several advantages over the time series algorithm.

Meyer, Claudia M.↗

Rocket engine failure detection using system identification techniques

The theoretical foundation and application of two univariate failure detection algorithms to Space Shuttle Main Engine (SSME) test firing data is presented. Both algorithms were applied to data collected during steady state operation of the engine. One algorithm, the time series algorithm, is based on time series techniques and involves the computation of autoregressive models. Time series techniques have been previously applied to SSME data. The second algorithm is based on standard signal processing techniques. It consists of tracking the variations in the average signal power with time. The average signal power algorithm is a newly proposed SSME failure detection algorithm. Seven nominal test firings were used to develop failure indication thresholds for each algorithm. These thresholds were tested using four anomalous firings and one additional nominal firing. Both algorithms provided significantly earlier failure indication times than did the current redline limit system. Neither algorithm gave false failure indications for the nominal firing. The strengths and weaknesses of the two algorithms are discussed and compared. The average signal algorithm was found to have several advantages over the time series algorithm.

Meyer, Claudia M.↗

In-process monitoring and prediction of droplet quality in droplet-on-demand liquid metal jetting additive manufacturing using machine learning

Abstract In droplet-on-demand liquid metal jetting (DoD-LMJ) additive manufacturing, complex physical interactions govern the droplet characteristics, such as size, velocity, and shape. These droplet characteristics, in turn, determine the functional quality of the printed parts. Hence, to ensure repeatable and reliable part quality it is necessary to monitor and control the droplet characteristics. Existing approaches for in-situ monitoring of droplet behavior in DoD-LMJ rely on high-speed imaging sensors. The resulting high volume of droplet images acquired is computationally demanding to analyze and hinders real-time control of the process. To overcome this challenge, the objective of this work is to use time series data acquired from an in-process millimeter-wave sensor for predicting the size, velocity, and shape characteristics of droplets in DoD-LMJ process. As opposed to high-speed imaging, this sensor produces data-efficient time series signatures that allows rapid, real-time process monitoring. We devise machine learning models that use the millimeter-wave sensor data to predict the droplet characteristics. Specifically, we developed multilayer perceptron-based non-linear autoregressive models to predict the size and velocity of droplets. Likewise, a supervised machine learning model was trained to classify the droplet shape using the frequency spectrum information contained in the millimeter-wave sensor signatures. High-speed imaging data served as ground truth for model training and validation. These models captured the droplet characteristics with a statistical fidelity exceeding 90%, and vastly outperformed conventional statistical modeling approaches. Thus, this work achieves a practically viable sensing approach for real-time quality monitoring of the DoD-LMJ process, in lieu of the existing data-intensive image-based techniques.

Gaikwad, Aniruddha (ORCID:0000000285642621)↗

On the Wiener-Masani algorithm for finding the generating function of multivariate stochastic processes

The algorithms developed by Wiener and Masani (1957 and 1958) and Masani (1960) for the characterization of a class of multivariate stationary stochastic processes are investigated analytically. The algorithms permit the determination of (1) the generating function, (2) the prediction-error matrix, and (3) an autoregressive representation of the linear least-squares predictor. A number of theorems and lemmas are proved, and it is shown that the range of validity of the algorithms can be extended significantly beyond that given by Wiener and Masani.

Miamee, A. G.↗

Predicting initial trans-membrane pressure across cycles in the ultrafiltration process using random forest

With growing freshwater scarcity, direct potable reuse (DPR) systems that reclaim wastewater for drinking are becoming increasingly important for sustainable water supply. Reliable operation requires minimizing downtime in ultrafiltration (UF) units, where membrane fouling leads to elevated trans-membrane pressure (TMP). This study develops data-driven regression models based on random forest (RF) and autoregressive (AR) approaches to forecast the initial TMP at the start of each UF filtration cycle in a pilot-scale DPR system. The RF model consistently outperforms baseline methods, including historical mean, last observation carried forward, and AR models, across multiple forecast horizons, achieving the lowest root mean square error. To evaluate how different classes of process variables contribute to TMP dynamics over time, we examine the feature importance of independent input variables across multiple forecast horizons. This analysis provides insight into the temporal relevance of operational and sensor-derived features, guiding control and monitoring strategies. Additionally, the impact of hyperparameter tuning on TMP prediction performance is assessed for both direct and recursive RF modelling approaches. The proposed RF framework establishes a robust foundation for predictive monitoring and real-time optimization of UF operations, supporting sustainable and reliable water reuse.

direct potable reuse↗

Diffusion Probabilistic Modeling for Video Generation

Denoising diffusion probabilistic models are a promising new class of generative models that mark a milestone in high-quality image generation. This paper showcases their ability to sequentially generate video, surpassing prior methods in perceptual and probabilistic forecasting metrics. We propose an autoregressive, end-to-end optimized video diffusion model inspired by recent advances in neural video compression. The model successively generates future frames by correcting a deterministic next-frame prediction using a stochastic residual generated by an inverse diffusion process. We compare this approach against six baselines on four datasets involving natural and simulation-based videos. We find significant improvements in terms of perceptual quality and probabilistic frame forecasting ability for all datasets.

97 MATHEMATICS AND COMPUTING↗

A data-driven linear formulation of the optimal demand response scheduling problem for an industrial air separation unit

Demand response (DR) has become a key element in balancing the power grid as the contribution of time-varying renewable power generation increases. Chemical plants are appealing candidates for DR programs as they offer large, concentrated and flexible loads. DR participation calls for frequent production rate changes over time scales that overlap with the dominant dynamics of the plant. Production scheduling should therefore consider the process dynamics explicitly. Here we present a data-driven approach for modelling the scheduling-relevant dynamics based on historical closed-loop operating data using autoregressive with extra inputs (ARX) models. We introduce a new, linear scheduling problem formulation based on the ARX representation, and demonstrate its implementation on an industrial air separation unit.

42 ENGINEERING↗

Signal processing considerations for low signal to noise ratio laser Doppler and phase Doppler signals

The relative performance of current methods used for estimating the phase and the frequency in LDV and phase Doppler applications in low signal to noise ratio conditions is analyzed. These methods include the Fourier analysis and the correlation techniques. Three methods that use the correlation function for frequency and phase estimations are evaluated in terms of accuracy and speed of processing. These methods include: (1) the frequency estimation using zero crossings counting of the auto-correlation function, (2) the Blackman-Tukey method, and (3) the AutoRegressive method (AR). The relative performance of these methods is evaluated and compared with the Fourier analysis method which provides the optimum performance in terms of the Maximum Likelihood (ML) criteria.

Ibrahim, K. M.↗

Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence

The immense computational cost of traditional numerical weather and climate models has sparked the development of machine learning (ML) based emulators. Because ML methods benefit from long records of training data, it is common to use data sets that are temporally subsampled relative to the time steps required for the numerical integration of differential equations. Here, we investigate how this often overlooked processing step affects the quality of an emulator's predictions. We implement two ML architectures from a class of methods called reservoir computing: (a) a form of Nonlinear Vector Autoregression (NVAR), and (b) an Echo State Network (ESN). Despite their simplicity, it is well documented that these architectures excel at predicting low dimensional chaotic dynamics. We are therefore motivated to test these architectures in an idealized setting of predicting high dimensional geophysical turbulence as represented by Surface Quasi-Geostrophic dynamics. In all cases, subsampling the training data consistently leads to an increased bias at small spatial scales that resembles numerical diffusion. Interestingly, the NVAR architecture becomes unstable when the temporal resolution is increased, indicating that the polynomial based interactions are insufficient at capturing the detailed nonlinearities of the turbulent flow. The ESN architecture is found to be more robust, suggesting a benefit to the more expensive but more general structure. Spectral errors are reduced by including a penalty on the kinetic energy density spectrum during training, although the subsampling related errors persist. Future work is warranted to understand how the temporal resolution of training data affects other ML architectures.

58 GEOSCIENCES↗

A stochastic-dynamical approach to the study of the natural variability of the climate

A method, suggested by Leith (1975), which employed stochastic-dynamic forecasts obtained from a general circulation model in such a way as to satisfy the definition of climatic noise, was used to validate assumptions accounting for the effects of external influences in estimating the climatic noise. Two assumptions were investigated: (1) that the weather fluctuations can be represented as a Markov process, and (2) that changing external conditions do not influence the atmosphere's statistical properties on short time scales. The general circulation model's simulation of the daily weather fluctuations was generated by performing integrations with prescribed climatological boundary conditions for random initial atmospheric states, with resulting dynamical forecasts providing an ensemble of simulated data for the autoregressive modeling of weather fluctuations. To estimate the climatic noise from the observational data (consisting of hourly values of sea level pressure and surface temperature at 54 U.S. stations for the month of January for the years 1949-1975) use of the short time-scale assumption is made. The simulated and observed data were found not to be consistent with either white noise or a Markov process of weather fluctuations. Good agreement was found between the results of the hypothetical testing of the simulated and the observed surface temperatures; and only partial support was found for the short time-scale assumption, i.e., for sea level pressure.

Straus, D. M.↗

A two dimensional power spectral estimate for some nonstationary processes

A two dimensional estimate for the power spectral density of a nonstationary process is being developed. The estimate will be applied to helicopter noise data which is clearly nonstationary. The acoustic pressure from the isolated main rotor and isolated tail rotor is known to be periodically correlated (PC) and the combined noise from the main and tail rotors is assumed to be correlation autoregressive (CAR). The results of this nonstationary analysis will be compared with the current method of assuming that the data is stationary and analyzing it as such. Another method of analysis is to introduce a random phase shift into the data as shown by Papoulis to produce a time history which can then be accurately modeled as stationary. This method will also be investigated for the helicopter data. A method used to determine the period of a PC process when the period is not know is discussed. The period of a PC process must be known in order to produce an accurate spectral representation for the process. The spectral estimate is developed. The bias and variability of the estimate are also discussed. Finally, the current method for analyzing nonstationary data is compared to that of using a two dimensional spectral representation. In addition, the method of phase shifting the data is examined.

Smith, Gregory L.↗

Laboratory for Engineering Man/Machine Systems (LEMS): System identification, model reduction and deconvolution filtering using Fourier based modulating signals and high order statistics

Several important problems in the fields of signal processing and model identification, such as system structure identification, frequency response determination, high order model reduction, high resolution frequency analysis, deconvolution filtering, and etc. Each of these topics involves a wide range of applications and has received considerable attention. Using the Fourier based sinusoidal modulating signals, it is shown that a discrete autoregressive model can be constructed for the least squares identification of continuous systems. Some identification algorithms are presented for both SISO and MIMO systems frequency response determination using only transient data. Also, several new schemes for model reduction were developed. Based upon the complex sinusoidal modulating signals, a parametric least squares algorithm for high resolution frequency estimation is proposed. Numerical examples show that the proposed algorithm gives better performance than the usual. Also, the problem was studied of deconvolution and parameter identification of a general noncausal nonminimum phase ARMA system driven by non-Gaussian stationary random processes. Algorithms are introduced for inverse cumulant estimation, both in the frequency domain via the FFT algorithms and in the domain via the least squares algorithm.

Pan, Jianqiang↗

Multi-Scale Temporal Patterns in Stream Biogeochemistry Indicate Linked Permafrost and Ecological Dynamics of Boreal Catchments

Temporal patterns in stream chemistry provide integrated signals describing the hydrological and ecological state of whole catchments. However, stream chemistry integrates multi-scale signals of processes occurring in both the catchment and stream. Deconvoluting these signals could identify mechanisms of solute transport and transformation and provide a basis for monitoring ecosystem change. Here, we applied trend analysis, wavelet decomposition, multivariate autoregressive state-space modeling, and analysis of concentration-discharge relationships to assess temporal patterns in high-frequency (15 min) stream chemistry from permafrost-influenced boreal catchments in Interior Alaska at diel, storm, and seasonal time scales. We compared catchments that varied in spatial extent of permafrost to identify characteristic biogeochemical signals. Catchments with higher spatial extents of permafrost were characterized by increasing nitrate concentration through the thaw season, an abrupt increase in nitrate and fluorescent dissolved organic matter (fDOM) and declining conductivity in late summer, and flushing of nitrate and fDOM during summer rainstorms. In contrast, these patterns were absent, of lower magnitude, or reversed in catchments with lower permafrost extent. Solute dynamics revealed a positive influence of permafrost on fDOM export and the role of shallow, seasonally dynamic flowpaths in delivering solutes from high-permafrost catchments to streams. Lower spatial extent of permafrost resulted in static delivery of nitrate and limited transport of fDOM to streams. Shifts in concentration-discharge relationships and seasonal trends in stream chemistry toward less temporally dynamic patterns might therefore indicate reorganized catchment hydrology and biogeochemistry due to permafrost thaw.

54 ENVIRONMENTAL SCIENCES↗

Spatio–Temporal Machine Learning for Regional to Continental Scale Terrestrial Hydrology

Integrated hydrologic models can simulate coupled surface and subsurface processes but are computationally expensive to run at high resolutions over large domains. Here we develop a novel deep learning model to emulate subsurface flows simulated by the integrated ParFlow–CLM model across the contiguous US. We compare convolutional neural networks like ResNet and UNet run autoregressively against our novel architecture called the Forced SpatioTemporal RNN (FSTR). The FSTR model incorporates separate encoding of initial conditions, static parameters, and meteorological forcings, which are fused in a recurrent loop to produce spatiotemporal predictions of groundwater. We evaluate the model architectures on their ability to reproduce 4D pressure heads, water table depths, and surface soil moisture over the contiguous US at 1 km resolution and daily time steps over the course of a full water year. The FSTR model shows superior performance to the baseline models, producing stable simulations that capture both seasonal and event–scale dynamics across a wide array of hydroclimatic regimes. The emulators provide over 1,000× speedup compared to the original physical model, which will enable new capabilities like uncertainty quantification and data assimilation for integrated hydrologic modeling that were not previously possible. Our results demonstrate the promise of using specialized deep learning architectures like FSTR for emulating complex process–based models without sacrificing fidelity.

54 ENVIRONMENTAL SCIENCES↗

Search for deterministic pulse trends in gamma ray burst temporal profiles

Most cosmic gamma-ray burst temporal profiles appear to be comprised of several individual pulses, many of which overlap. It is advantageous to deconvolve the temporal structures into their constituent pulses, and thereby investigate the shape, intensity and temporal distributions of the pulses as a function of energy. Such fundamental pulse descriptors would provide constraints for theoretical modeling of the burst emission process, such as indications of source size, optical depth and geometry, as a function of time. We have developed a deconvolution algorithm which treats sequences of pulse shapes that change deterministically. The algorithm, a generalization of autoregressive techniques, has been applied to a few bright bursts observed by BATSE. Results indicate that, even within short intervals, constituent pulses are not self-similarly shaped, nor do pulse shapes evolve in a simple manner throughout a burst. Hence, the direction of our future work on pulse deconvolution will focus on analysis methods which allow pulse shape to vary.

Norris, J. P.↗

Acceleration and Velocity Sensing from Measured Strain

A simple approach for computing acceleration and velocity of a structure from the strain is proposed in this study. First, deflection and slope of the structure are computed from the strain using a two-step theory. Frequencies of the structure are computed from the time histories of strain using a parameter estimation technique together with an Autoregressive Moving Average model. From deflection, slope, and frequencies of the structure, acceleration and velocity of the structure can be obtained using the proposed approach. shape sensing, fiber optic strain sensor, system equivalent reduction and expansion process.

shape sensing↗