Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Time Series Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

Quality-Controlled Meteorological Data from the Flood Control District of Maricopa County (FCDMC) Network, Phoenix, Arizona (1987-2024)

This dataset contains 15- or 30-minute interval meteorological data from the Flood Control District of Maricopa County (FCDMC), Arizona, USA, covering eight key variables across multiple sensor stations between 1987 and 2024. Each variable is stored as a separate CSV file, containing time-series data that have undergone rigorous quality control (QC) procedures and, where appropriate, short-gap interpolation for consistency. The quality control (QC) pipeline consisted of four sequential tests: (1) a range test to ensure all values fall within physically realistic limits, (2) a step test to identify abrupt and implausible changes between consecutive records, (3) a proximity test that validates flagged values from step test using data from nearby stations and exceedance probability thresholds, and (4) a persistence test to detect and remove periods of unrealistically constant readings. These thresholds were calibrated to Arizona’s environmental conditions and sensor specifications. After QC, short gaps (≤2 hours) were linearly interpolated to ensure consistent temporal resolution, except for wind variables. Due to a major upgrade in FCDMC’s data transmission system, only ALERT-2 protocol data (2016–2024) for wind variables are included; earlier ALERT-1 data were excluded because of irregular sampling and high missing rates. This dataset supports regional climate and infrastructure resilience studies by providing standardized, high-resolution meteorological data for the greater Phoenix metropolitan area.

54 ENVIRONMENTAL SCIENCES↗

Anomaly detection for MPC forecast in Fleet of Water Heaters

Among residential devices, water heaters consume 20% of home energy use in the United States. Water heaters possess the capability to store energy within their reservoirs, enabling the ability to decouple energy use from hot water use. This capability can be used to reduce energy usage and costs while also supporting grid services. This requires accurate forecasting of the parameters of the water heater such as upper and lower temperatures. In this study, we analyzed the performance and behavior of a water heater model used in the real-world to predict a control mechanism that is implemented in a smart residential neighborhood. The model forecasts are accurate in most cases but not all. In such scenarios, error correction of the model is necessary to further improve model predictive control accuracy. Anomaly detection is the first step of error correction. This study complements existing research by grouping time series data into two clusters one with anomalies and another without anomalies. To achieve this task, we explored and compared multiple unsupervised machine learning algorithms to perform clustering. Among these algorithms, Ward clustering has the lowest running time and identified the highest number of anomalies for the upper temperature limit. The proposed approach is tested based on the data collected in a neighborhood with 46 townhomes located in Atlanta, GA.

Lebakula, Viswadeep↗

Monitoring vegetation using Nimbus-7 scanning mutichannel microwave radiometer's data

Field studies and radiative transfer model calculations have shown that brightness temperature at high microwave frequencies is strongly affected by vegetation. The daytime observations for six consecutive years (1979 to 1984) over the Sahara, Senegalese Sahel, Burkina Fasso (Upper Volta), and U.S. Southern Great Plains at 37 GHz frequency of the Sanning Multichannel Microwave Radiometer (SMMR) on board the Nimbus-7 satellite are analyzed, and a high correlation with the normalized difference vegetation index derived from the Advanced Very High Resolution Radiometer on board the NOAA-7 satellite is found. The SMMR data appear to provide a valuable new long-term global data set for monitoring vegetation. In particular, the differing responses of vegetation (for example, annual grasses versus woody plants) to drought and the stability of the desert/steppe boundary of northern Africa might be studied using the time series data.

Choudhury, B. J.↗

Medical Device Recalls in Radiation Oncology: Analysis of US Food and Drug Administration Data, 2002-2015

Purpose: To analyze all recalls involving radiation oncology devices (RODs) from the US Food and Drug Administration (FDA)'s recall database, comparing these with non–radiation oncology device recalls to identify discipline-specific trends that may inform improvements in device safety. Methods and Materials: Recall data on RODs from 2002 to 2015 were sorted into 4 product categories (external beam, brachytherapy, planning systems, and simulation systems). Outcomes included determined cause of recall, recall class (severity), quantity in commerce, time until recall termination (date FDA determines recall is complete), and time since 510(k) approval. Descriptive statistics were performed with linear regression of time-series data. Results for RODs were compared with those for other devices by Pearson χ{sup 2} test for categorical data and 2-sample Kolmogorov-Smirnov test for distributions. Results: There were 502 ROD recalls and 9534 other class II device recalls during 2002 to 2015. Most recalls were for external beam devices (66.7%) and planning systems (22.9%), and recall events peaked in 2011. Radiation oncology devices differed significantly from other devices in all recall outcomes (P≤.04). Recall cause was commonly software related (49% vs 10% for other devices). Recall severity was more often moderate among RODs (97.6% vs 87.2%) instead of severe (0.2% vs 4.4%; P<.001). Time from 510(k) market approval to recall was shorter among RODs (P<.001) and progressively shortened over time. Radiation oncology devices had fewer recalled devices in commerce than other devices (P<.001). Conclusions: Compared with other class II devices, RODs experience recalls sooner after market approval and are trending sooner still. Most of these recalls were moderate in severity, and software issues are prevalent. Comprehensive analysis of recall data can identify areas for device improvement, such as better system design among RODs.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Machine Learning Based Cavity Fault Classification and Prediction

We report on the development of machine learning models for classifying C100 superconducting radio frequency (SRF) cavity faults in the Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Lab. CEBAF is a continuous-wave recirculating linac utilizing 418 SRF cavities to accelerate electrons up to 12 GeV through 5-passes. Of these, 96 cavities (12 cryomodules) are designed with a digital low-level RF system configured such that a cavity fault triggers waveform recordings of 17 RF signals for each of the eight cavities in the cryomodule. Subject matter experts (SME) can analyze the collected time-series data, identify which of the eight cavities faulted first, and classify the type of fault. This information is used to find trends and strategically deploy mitigations to problematic cryomodules. However, manually labeling the data is laborious and time-consuming. By leveraging machine learning, near real-time – rather than post-mortem – identification of the offending cavity and classification of the fault type has been implemented. We discuss the performance of the ML models during a recent physics run.

43 PARTICLE ACCELERATORS↗

A Flexible Framework for Building Occupancy Detection Using Spatiotemporal Pattern Networks: Preprint

This paper presents a reliable, scalable, and transferable framework to predict occupancy in a building utilizing diverse, multi-modal information. We propose a new methodology for learning-driven occupancy detection built on the concepts of probabilistic graphical modeling and observable Markov chain modeling. To capture the relationship between multi-sensor data and occupancy, we propose this Occ-STPN framework that is flexible to support both multivariate and univariate formulations. While the multivariate Occ-STPN performs feature-level fusion of multiple predictors and occupancy time-series data, the univariate Occ-STPN involves decision fusion of occupancy predictions using individual predictors based on a mutual information weighted fusion scheme. We also propose a new metric to evaluate the performance of occupancy prediction algorithms. Two popular datasets are used to validate our approach and demonstrate that our framework is scalable in terms of the number of information sources (e.g., sensors) as well as it is possible to transfer trained models from one building to another without significant reduction in performance. Reliability of the algorithm is also tested by injecting noise into the datasets.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

A Flexible Framework for Building Occupancy Detection Using Spatiotemporal Pattern Networks

This paper presents a reliable, scalable, and transferable framework to predict occupancy in a building utilizing diverse, multi-modal information. We propose a new methodology for learning-driven occupancy detection built on the concepts of probabilistic graphical modeling and observable Markov chain modeling. To capture the relationship between multi-sensor data and occupancy, we propose this Occ-STPN framework that is flexible to support both multivariate and univariate formulations. While the multivariate Occ-STPN performs feature-level fusion of multiple predictors and occupancy time-series data, the univariate Occ-STPN involves decision fusion of occupancy predictions using individual predictors based on a mutual information weighted fusion scheme. We also propose a new metric to evaluate the performance of occupancy prediction algorithms. Two popular datasets are used to validate our approach and demonstrate that our framework is scalable in terms of the number of information sources (e.g., sensors) as well as it is possible to transfer trained models from one building to another without significant reduction in performance. Reliability of the algorithm is also tested by injecting noise into the datasets.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Superconducting Radio-frequency Cavity Fault Classification Using Machine Learning at Jefferson Laboratory

We report on the development of machine learning models for classifying C100 superconducting radiofrequency (SRF) cavity faults in the Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Lab. Of the 418 SRF cavities in CEBAF, 96 are designed with a digital low-level RF system configured such that a cavity fault triggers recordings of RF signals for each of eight cavities in the cryomodule. Subject matter experts analyze the collected time-series data and identify which of the eight cavities faulted first and classify the type of fault. This information is used to find trends and strategically deploy mitigations to problematic cryomodules. However, manually labeling the data is laborious and time-consuming. By leveraging machine learning, near real-time - rather than postmortem - identification of the offending cavity and classification of the fault type has been implemented. We discuss the performance of the machine learning models during a recent physics run. We also discuss efforts for further insights into fault types through unsupervised learning techniques and present preliminary work on cavity and fault prediction using data collected prior to a failure event.

Tennant, C. D.↗

Alanna Conners and the Origins of Principled Data Analysis

Alanna was one of the most important pioneers in the development of not just sophisticated algorithms for analyzing astronomical data but more importantly an overall viewpoint emphasizing the use of statistically sound principles in place of blind application of cook-book recipes, or black boxes. I will outline some of the threads of this viewpoint, emphasizing time series data, with a focus on the importance of these developments for the Age of Digital Astronomy that we are entering.

age of digital astronomy↗

System and method for wave prediction

A method and system for prediction of wave properties include collecting time series data streams from one or more wave measurement devices and processing the data using a wave-prediction algorithm to identify the frequency components of the data and compute wave parameters. The wave-field is propagated in space and time to predict wave height, speed, and velocity at a target location. A sliding window approach is used to continuously update the prediction in real-time.

Previsic, Mirko↗

Spatiotemporal Characterization of Droughts and Vegetation Response in Northwest Africa From 1981 to 2020

Drought has become one of the most devastating natural risks of agricultural production and the environment in almost all climate regions. Thus, understanding the spatiotemporal characteristics of drought and its associated impacts is crucial in drought early warning management and adaptation efforts. In this study, we used the 3-month Standardized Precipitation Index (SPI-3) obtained from Climate Hazards Group InfraRed Precipitation with Station data (CHIRPS) to investigate the space–time characteristics of drought conditions. Also, this study examined the impact of the SPI-based drought on vegetation health conditions using the Advanced Very High Resolution Radiometer (AVHRR) Normalized Difference Vegetation Index (NDVI) time-series data from 1981 to 2020. The results revealed that the region experiences drought primarily between July and September, with the most prolonged drought events lasting up to five months. Morocco suffered from more frequent droughts than other countries in the region. The Mann-Kendall test showed that the trend of drought became drier over the last decade, whereas the period from 1981 to 2010 witnessed either wetting or no trends. This study also found that the response of crops and grasslands showed higher correlation with the SPI-3 and that the response of vegetation to droughts was higher during the dry season. The findings of this study provide useful information to support local and regional drought planning and adaptation programs and enhance the understanding of drought development in the region.

Nguyen Quang Thi↗

Development of a Annual Air Handling Unit Fault Dataset for FDD Tools: Lessons Learned and Considerations for FDD Developers

As energy management and information systems (e.g., automated fault detection and diagnostics [AFDD] tools) become more prevalent in the commercial building stock, it is important to determine the effectiveness of these technologies by benchmarking their performance. The authors have been working to develop the largest publicly available dataset of HVAC fault data for performance benchmarking applications, covering the most common HVAC systems and designs including chiller plants, rooftop packaged units, dual duct air handling units and single duct air handling units. This study covers the development, modeling, and validation of a synthetic fault dataset for a single duct air handling unit (AHU), one of the most common HVAC configurations found in the commercial building stock. Despite this being a common system, real-world time series data are scarce and usually do not span a wide range of weather conditions. Due to this limitation, a detailed AHU model was employed to carry out annual simulations of numerous common sensor and mechanical faults, which were then validated by comparing their effects on system performance to expected symptoms. We summarize the nature of each fault and their impacts under different weather and operation conditions. Finally, we highlight considerations for FDD developers that may want to use this dataset to assess their algorithms’ performance and their improvement over time.

Casillas, Armando↗

Atmospheric pressure, density, temperature and wind variations between 50 and 200 km

Data on atmospheric pressure, density, temperature and winds between 50 and 200 km were collected from sources including Meteorological Rocket Network data, ROBIN falling sphere data, grenade release and pitot tube data, meteor winds, chemical release winds, satellite data, and others. These data were analyzed by a daily difference method and results on the distribution statistics, magnitude, and spatial structure of the irregular atmospheric variations are presented. Time structures of the irregular variations were determined by the analysis of residuals from harmonic analysis of time series data. The observed height variations of irregular winds and densities are found to be in accord with a theoretical relation between these two quantities. The latitude variations (at 50 - 60 km height) show an increasing trend with latitude. A possible explanation of the unusually large irregular wind magnitudes of the White Sands MRN data is given in terms of mountain wave generation by the Sierra Nevada range about 1000 km west of White Sands. An analytical method is developed which, based on an analogy of the irregular motion field with axisymmetric turbulence, allows measured or model correlation or structure functions to be used to evaluate the effective frequency spectra of scalar and vector quantities of a spacecraft moving at any speed and at any trajectory elevation angle.

Justus, C. G.↗

High Performance Access to Archival Data Stored in HDF4 and HDF5 on Cloud Object Stores Without Reformatting the Files

Cloud computing offers numerous advantages for users of extensive Earth science data collections. These benefits encompass direct online access to data files and granules from any location, scalable access supporting parallel computing workflows, and flexible computing tools enabling innovative experimentation with processing techniques. However, older archival file formats designed for distinct computing systems hinder efficient access to decade-long time-series data when compared to data stored in modern cloud-optimized formats like Web Object Stores (WOS), exemplified by Amazon Web Services’ Simple Storage Service (S3). We describe DMR++ (Dataset Metadata Response plus plus), a technology facilitating efficient access to HDF5 (Hierarchical Data Format, version 5) and HDF4 files stored on WOS systems without requiring data reformatting. DMR++ achieves performance comparable to technologies like Zarr while preserving the original file structure, a substantial benefit considering the vast quantity of archival files held by organizations such as NASA. Moreover, DMR++ typically outperforms cloud-optimized versions of HDF5. Essentially an XML (Extensible Markup Language) document usually stored alongside the described data, DMR++ can also be generated on-the-fly but is generally created during data staging to the WOS. Archival files that use HDF4/5 often store large arrays of numerical data. The data in these files is often compressed, typically reducing their size by a factor of four or more. To achieve efficient access to portions of those arrays, they are 'chunked' into smaller sub-arrays, each individually compressed. The chunk size is a compromise, where spinning disks can efficiently access data in smaller chunks while S3 favors larger chunks. A simple optimization of aggregating smaller chunks that are stored adjacently, transferring them in a single access and then individually decompressing them will improve performance. NASA data pose an additional challenge: special Application Programmer Interface (API) libraries are often needed to compute some variables. These libraries are incompatible with WOS environments. Our solution involves storing computed values in the DMR++ document or a companion file, making them accessible like other variables and eliminating the need for specialized APIs. We outline specific optimizations for both satellite grid and swath data stored in HDF4-EOS2 (Earth Observing System).

James Gallagher↗

Using GIS for Processing, Analysis and Visualization of Hydrological Model Data - 16202

The Applied Research Center at Florida International University (ARC-FIU) is supporting the soil and groundwater remediation efforts of the U.S. Department of Energy's (DOE's) Savannah River Site (SRS) by developing a surface water model to simulate the hydrology and the fate and transport of contaminants and sediment in the Tims Branch watershed. Advances in ArcGIS software through the development of spatial GIS database (geo-database) technology, coupled with the development of data models such as ArcHydro which possesses a spatial relational database management system (RDMS) schema and relationship structure specific to hydrologic systems, provides modelers with tools and applications to assist in the processing, analysis and visualization of flow and contaminant transport data. In addition, the coupling of this type of geo-database structure with a numerical modeling package such as MIKE SHE/MIKE 11 can serve as an efficient tool that significantly reduces the time needed for data preparation. The MIKE modeling system has a geographic information systems (GIS) user interface built into its system that can directly use GIS data for model inputs. GIS-based hydrologic models can provide a spatial element that other hydrologic models lack. GIS is important for processing and analysis of large spatio-temporal datasets derived from multiple sources that are being used for setting up of models, model calibration and simulation of flow and contaminant fate and transport. GIS-based approaches in hydrological modeling can also provide the benefit of combining different layers of geographic data to create new integrated information which can be quite useful for creating dependent or independent hydrological variables. ARC-FIU has developed an ArcGIS geo-database to support the hydrological modeling work being conducted at SRS. The SRS geo-database was developed based on the ArcHydro and ArcGIS Base Map data models with modifications made for project specific input parameters. The significance of this approach was to ensure its replicability for potential application at other DOE sites. The SRS geo-database assists in the storage and management of GIS and time series data being used for hydrological model development at Savannah River Site. Other GIS tools such as ArcGIS Model Builder coupled with Python scripting have enabled the automation of many of the repetitive geo-processing tasks required for pre- and post-processing of the hydrological modeling data. Automation of many of the data processing tasks has facilitated faster and hence more complex analyses of field test data. The toolbox created is also a scalable and reusable application that can be implemented in other watersheds. This paper describes the use of ArcGIS tools for the pre-processing, spatial delineation and analysis of hydrological model data; for automation of repetitive geo-processing tasks; and for visualization of different hydrological components of the Tims Branch watershed. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

FAIRification, Quality Assessment, and Missingness Pattern Discovery for Spatiotemporal Photovoltaic Data

The growth of the photovoltaic market has pushed the demand for power forecasting and performance evaluation for a huge population of PV power plants. Many of these power plants have spatiotemporal coherence that can be utilized for improving model accuracy. We have demonstrated in this paper the FAIRification of spatiotemporal PV time series data. Through the creation of a solar power plant ontology, we propose standards for the naming and structure of metadata used to describe the data from these power plants. Using the structure from this ontology, we have developed both R and Python packages for the automation of the FAIRification process. Going further, we have also developed an R package that automates the analysis of the quality of a data set through the designation of letter grades. To solve the issue of data missingness, we propose the use of St-GNN autoencoders to detect and impute missing values from a data set by utilizing data from power plants nearby.

14 SOLAR ENERGY↗

Inferring HIV Escape Rates from Multi-Locus Genotype Data

Cytotoxic T-lymphocytes (CTLs) recognize viral protein fragments displayed by major histocompatibility complex molecules on the surface of virally infected cells and generate an anti-viral response that can kill the infected cells. Virus variants whose protein fragments are not efficiently presented on infected cells or whose fragments are presented but not recognized by CTLs therefore have a competitive advantage and spread rapidly through the population. We present a method that allows a more robust estimation of these escape rates from serially sampled sequence data. The proposed method accounts for competition between multiple escapes by explicitly modeling the accumulation of escape mutations and the stochastic effects of rare multiple mutants. Applying our method to serially sampled HIV sequence data, we estimate rates of HIV escape that are substantially larger than those previously reported. The method can be extended to complex escapes that require compensatory mutations. We expect our method to be applicable in other contexts such as cancer evolution where time series data is also available.

60 APPLIED LIFE SCIENCES↗

A Data-driven approach to Determining the Fidelity in the Hardware-in-the-loop Systems using Subspace Identification Method

One of the major questions in any Hardware-in-the-loop (HiL) simulation is to understand the fidelity of the HiL simulation itself which is most often indicated qualitatively as either high or low instead of quantifying it. Being cognizant of the level of fidelity forms the crux to assess the validity and credibility of the HiL simulation. In this work, we address this issue by developing a systematic, data-driven approach to assess the fidelity of any HiL simulation, and more specifically, the fidelity of the interface between the simulator and hardware in an HiL simulation. Applying a subspace identification method, a linear system representing the interface is obtained from the time-series data captured between the hardware and simulator in the HiL simulation. Finally, the fidelity is defined based on this linear system properties. The proposed data-driven fidelity quantification framework is illustrated on the IEEE 123 node feeder system running on HYPERSIM and interacting with virtual protection relays.

97 MATHEMATICS AND COMPUTING↗