Labeled Time Series Data Shifts
Labeled time series data set containing labeled data shifts/capacity changes for each time series.
14 SOLAR ENERGY↗
SEARCH · Engineering Papers
Results for “data shift”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Methods for evaluation and treatment of data shift
This is a code repository for a set of tools for detecting and mitigating data shifts in machine learning. The goal of the tools is to provide capabilities for determining when new data sets differ from training data sets and for adapting existing models to new data or correcting data shifts (via domain adaptation). The components will be written in Python, a high-level programming language that takes advantage of the Python ecosystem of high-quality open-source packages for machine learning and signal processing.
Parikh, Nidhi↗
Automated Shift Detection in Sensor-Based PV Power and Irradiance Time Series
PV power and irradiance sensor-based measurements are prone to error, resulting in issues such as abrupt time series data shifts. These shifts, which are usually unintentional, may be caused by software or hardware configuration changes on a PV system, and do not reflect an actual change in overall system performance. Locating these shifts and segmenting the associated time series aids in more accurate future PV analysis. In this research, an offline changepoint detection (CPD) algorithm that automatically detects these abrupt data shifts in sensor-based time series is introduced. Data shift periods in 101 daily PV power and irradiance time series were labeled manually by two solar experts. These data streams represent sensor-based measurements, and display a variety of data shift behaviors. A changepoint detection algorithm was tuned using the 101 labeled data streams, with each model configuration's ability to detect labeled changepoints benchmarked using metrics such as F1-score, recall, and Rand Index. Best performing models on seasonality-corrected data streams include the Pruned Exact Linear (PELT) method, the Binary Segmentation method, and the Bottom-Up method, all scoring an average F1-score of 0.76 or greater at detecting labeled changepoints within a 30-day window for the labeled data sets. To promote further research in this space, we are releasing the labeled data shift sets on U.S. Department of Energy's (DOE) DuraMAT Data Hub, and the associated algorithm in the Python PVAnalytics package.
changepoint detection↗
Automated Shift Detection in Sensor-Based PV Power and Irradiance Time Series: Preprint
PV power and irradiance sensor-based measurements are prone to error, resulting in issues such as abrupt time series data shifts. These shifts, which are usually unintentional, may be caused by software or hardware configuration changes on a PV system, and do not reflect an actual change in overall system performance. Locating these shifts and segmenting the associated time series aids in more accurate future PV analysis. In this research, an offline changepoint detection (CPD) algorithm that automatically detects these abrupt data shifts in sensor-based time series is introduced. Data shift periods in 101 daily PV power and irradiance time series were labeled manually by two solar experts. These data streams represent sensor-based measurements, and display a variety of data shift behaviors. A changepoint detection algorithm was tuned using the 101 labeled data streams, with each model configuration's ability to detect labeled changepoints benchmarked using metrics such as F1-score, recall, and Rand Index. Best performing models on seasonality-corrected data streams include the Pruned Exact Linear (PELT) method, the Binary Segmentation method, and the Bottom-Up method, all scoring an average F1-score of 0.76 or greater at detecting labeled changepoints within a 30-day window for the labeled data sets. To promote further research in this space, we are releasing the labeled data shift sets on U.S. Department of Energy's (DOE) DuraMAT Data Hub, and the associated algorithm in the Python PVAnalytics package.
changepoint detection↗
Automated Shift Detection in Sensor-Based PV Power and Irradiance Time Series
PV power and irradiance sensor-based measurements are prone to error, resulting in issues such as time series data shifts. In this research, a changepoint detection (CPD) algorithm that automatically detects data shifts in sensor-based time series is introduced. Data shift periods in 101 daily PV power and irradiance time series were labeled manually by two solar experts. These data streams represent sensor-based measurements, and display a variety of data shift behaviors. A changepoint detection algorithm was tuned using the 101 labeled data streams, with each model configuration's ability to detect labeled changepoints benchmarked using metrics such as F1-score, recall, and Rand Index. Best performing models on seasonality-corrected data streams include the Pruned Exact Linear (PELT) method, the Binary Segmentation method, and the Bottom-Up method, all scoring an average F1-score of 0.76 or greater at detecting labeled changepoints within a 30-day window across the labeled data sets. Pending approval, we plan to release the labeled data sets for this research on NREL's DuraMAT Data Hub, and the associated algorithm in the Python PVAnalytics package. By supplying the training sets and algorithm, we hope to encourage further development in this research space.
data shift↗
Decoding the shift-invariant data: applications for band-excitation scanning probe microscopy *
A shift-invariant variational autoencoder (shift-VAE) is developed as an unsupervised method for the analysis of spectral data in the presence of shifts along the parameter axis, disentangling the physically-relevant shifts from other latent variables. Using synthetic data sets, we show that the shift-VAE latent variables closely match the ground truth parameters. The shift VAE is extended towards the analysis of band-excitation piezoresponse force microscopy data, disentangling the resonance frequency shifts from the peak shape parameters in a model-free unsupervised manner. The extensions of this approach towards denoising of data and model-free dimensionality reduction in imaging and spectroscopic data are further demonstrated. This approach is universal and can also be extended to analysis of x-ray diffraction, photoluminescence, Raman spectra, and other data sets.
36 MATERIALS SCIENCE↗
Interpreting test temperature and loading rate effects on the fracture toughness of polymer-metal interfaces via time–temperature superposition
Here, in this letter, we present interfacial fracture toughness data for a polymer-metal interface where tests were conducted at various test temperatures T and loading rates $\dot{δ}$. An adhesively bonded asymmetric double cantilever beam (ADCB) specimen was utilized to measure toughness. ADCB specimens were created by bonding a thinner, upper adherend to a thicker, lower adherend (both 6061 T6 aluminum) using a thin layer of epoxy adhesive, such that the crack propagated along the interface between the thinner adherend and the epoxy layer. The specimens were tested at T from 25 to 65 °C and $\dot{δ}$ from 0.002 to 0.2 mm/s. The measured interfacial toughness Γ increased as both T and $\dot{δ}$ increased. For an ADCB specimen loaded at a constant $\dot{δ}$, the energy release rate G increases as the crack length a increases. For this reason, we defined rate effects in terms of the rate of change in the energy release rate $\dot{G}$. Although not rigorously correct, a formal application of time–temperature superposition (TTS) analysis to the Γ data provided useful insights on the observed dependencies. In the TTS-shifted data, Γ decreased and then increased for monotonically increasing $\dot{G}$. Thus, the TTS analysis suggests that there is a minimum value of Γ. This minimum value could be used to define a lower bound in Γ when designing critical engineering applications that are subjected to T and $\dot{δ}$ excursions.
36 MATERIALS SCIENCE↗
Calibration of Distributed Temperature Sensors Using Commercially Available SMF-28 Optical Fiber From 22 °C to 1000 °C
Optical frequency domain reflectometry (OFDR) is a family of optical techniques which can be used to produce distributed temperature measurements from the spectral shift of an interference pattern based on the Rayleigh backscatter signature of an optical fiber. Adaptive signal processing techniques have recently been used with OFDR to record meaningful spectral shift data from commercially available SMF-28 optical fibers heated beyond 950 °C. Furthermore, a correlation between the measured spectral shift and temperature has not yet been developed at these high temperatures. To extend the measurable temperature range of OFDR in SMF-28, this work describes the development of such a correlation from room temperature (22 °C) to 1000 °C. The relationship between spectral shift and temperature change over this range was found to be best characterized by the fourth-order polynomial ΔT=(-4.241 * 10 -11 )S 4 + (-2.017 * 10 -7 )S 3 + (-3.677 * 10 -4 )S 2 + (-0.8057)S, where ΔT represents the temperature difference compared to the reference temperature, and S represents the spectral shift measured by the fibers. The calibration developed in this work assumes that the fiber has been fully annealed by heating the fiber to 1000 °C for a few hours. This paper is the first to demonstrate the calibration and use of SMF-28 distributed optical fiber sensors up to 1000 °C, enabled using adaptive OFDR-based signal processing.
47 OTHER INSTRUMENTATION↗
Antarctic evidence for an abrupt northward shift of the Southern Hemisphere westerlies at 32 ka BP
High-resolution ice core records from coastal Antarctica are particularly useful to inform our understanding of environmental changes and their drivers. Here, we present a decadally resolved record of sea-salt sodium (a proxy for open-ocean area) and non-sea salt calcium (a proxy for continental dust) from the well-dated Roosevelt Island Climate Evolution (RICE) core, focusing on the time period between 40–26 ka BP. The RICE dust record exhibits an abrupt shift towards a higher mean dust concentration at 32 ka BP. Investigating existing ice-core records, we find this shift is a prominent feature across Antarctica. We propose that this shift is linked to an equatorward displacement of Southern Hemisphere westerly winds. Subsequent to the wind shift, data suggest a weakening of Southern Ocean upwelling and a decline of atmospheric CO 2 to lower glacial values, hence making this shift an important glacial climate event with potentially important insights for future projections.
54 ENVIRONMENTAL SCIENCES↗
Correlated Anion Disorder in Heteroanionic Cubic TiOF 2
Resolving anion configurations in heteroanionic materials is crucial for understanding and controlling their properties. For anion-disordered oxyfluorides, conventional Bragg diffraction cannot fully resolve the anionic structure, necessitating alternative structure determination methods. We have investigated the anionic structure of anion-disordered cubic (ReO 3 -type) TiOF 2 using X-ray pair distribution function (PDF), 19 F MAS NMR analysis, density functional theory (DFT), cluster expansion modeling, and genetic-algorithm structure prediction. Our computational data predict short-range anion ordering in TiOF 2 , characterized by predominant cis-[O 2 F 4 ] titanium coordination, resulting in correlated anion disorder at longer ranges. To validate our predictions, we generated partially disordered supercells using genetic-algorithm structure prediction and computed simulated X-ray PDF data and 19 F MAS NMR spectra, which we compared directly to experimental data. To construct our simulated 19 F NMR spectra, we derived new transformation functions for mapping calculated magnetic shieldings to predicted magnetic chemical shifts in titanium (oxy)fluorides, obtained by fitting DFT-calculated magnetic shieldings to previously published experimental chemical shift data for TiF 4 . We find good agreement between our simulated and experimental data, which supports our computationally predicted structural model and demonstrates the effectiveness of complementary experimental and computational techniques in resolving anionic structure in anion-disordered oxyfluorides. From additional DFT calculations, we predict that increasing anion disorder makes lithium intercalation more favorable by, on average, up to 2 eV, highlighting the significant effect of variations in short-range order on the intercalation properties of anion-disordered materials.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Data Centers and Digital Assurance Workshop 3 – Mitigations for Digital Assurance Risks
The third session of the TADA (Technical Assistance for Digital Assurance) Data Centers Cohort, held on November 18, 2025, focused on developing mitigation strategies for digital assurance risks identified in previous workshops. Hosted by Idaho National Laboratory (INL) and ScottMadden, the session emphasized the application of Cyber-Informed Engineering (CIE) to data center infrastructure, particularly at the utility–data center interface. Participants revisited and ranked key digital assurance risks, including architecture and interface weaknesses, governance gaps, and AI-enabled threats. The workshop introduced the 12 principles of CIE, advocating for consequence-focused design, engineered controls, and secure information architecture to proactively reduce cyber-physical vulnerabilities. These principles were applied to critical data center systems such as power distribution, UPS, cooling, SCADA/BMS, and grid-forming batteries. The session also addressed governance challenges at the interconnection boundary, highlighting the need for clear roles in telemetry sharing, firmware management, and trip settings. Special attention was given to emerging risks from behind-the-meter (BTM) generation, including reverse-power flow and the integration of small modular reactors (SMRs), which shift data centers from large loads to complex generation nodes. Participants explored how interconnection agreements can serve as enforceable instruments for digital assurance, and reviewed gaps in current standards such as NERC CIP, IEC 62443, and IEEE 1547. The workshop concluded with pathways to standardization, including model agreement language, state-level programs, and expanded NERC guidance. INL also presented tools and frameworks for secure procurement and supplier risk management, reinforcing the need for integrated engineering and policy solutions to secure the evolving data center–grid ecosystem. Session 3 of 3.
24 - POWER TRANSMISSION AND DISTRIBUTION↗
PVAnalytics: A Python Package for Automated Processing of Solar Time Series Data
Multiple publicly available software packages exist that analyze solar time series data, including RdTools and Solar Data Tools, among others. Several of these packages contain their own unique quality assurance (QA) and feature recognition algorithms. The python PVAnalytics package was developed to offer an internally consistent source for these analysis tools, making it easier for the end user to deploy these routines on his or her solar data. The PVAnalytics package currently contains routines for outlier detection, inverter clipping detection, irradiance and temperature checks, orientation checks, and data shift detection, among other functions. These functions have been aggregated from various sources including Solar Forecast Arbiter, RdTools, and the QA process developed by NREL's PV Fleets Initiative. We are continuously adding new functionality to the package, including documentation, examples and algorithms. By bundling QA functionality into a single software package, we hope to make PVAnalytics a comprehensive software library to support analysis of solar metadata and time series data.
data cleaning↗
Frustration-driven magnetic correlations in the spin-$\frac{5}{2}$ triangular lattice antiferromagnet RbFe(HPO 3 ) 2
Here, a detailed study of the structural and magnetic properties of a spin-5/2 triangular lattice antiferromagnet RbFe(HPO 3 ) 2 is presented using x-ray diffraction, magnetization, heat capacity, and 31 P nuclear magnetic resonance (NMR) experiments on a polycrystalline sample. The crystal structure features an equilateral triangular lattice of Fe 3+ ions. The thermodynamic measurements reveal the onset of a magnetic long-range order at 𝑇 N1 ≃7.8K in zero field, followed by another low-temperature field-induced ordering at 𝑇 N2 in higher fields. The transition at 𝑇 N1 is further confirmed from the NMR spin lattice relaxation measurements. The value of the frustration ratio (𝑓≃7) implies moderate spin frustration in the compound. The 31 P NMR spectra exhibit two distinct spectral lines corresponding to two inequivalent phosphorus sites (P1 and P2), consistent with the crystal structure. The P1 site is strongly coupled with an isotropic hyperfine coupling of 𝐴$^{iso}_{hf}$ = 0.55(2) T/𝜇 B while the P2 site is weakly coupled with 𝐴$^{iso}_{hf}$ = 0.25(3) T/𝜇 B with the Fe 3+ ions. The magnetic susceptibility and NMR shift data are described well assuming a spin-5/2 isotropic triangular lattice antiferromagnetic model with an average exchange coupling of 𝐽/𝑘 B = 2.8(2)K. Below 𝑇 N1 , the spectra evolve into a nearly rectangular powder pattern, indicating a commensurate antiferromagnetic type order. The 31 P spin-lattice relaxation rate well below 𝑇N1 follows a 𝑇 3 temperature dependence, implying a two-magnon Raman scattering mechanism in the ordered state. Three well-defined phase regimes are clearly ascertained in the 𝐻−𝑇 phase diagram, reflecting a weak magnetic anisotropy in the compound.
Nagpal, V. [Indian Institute of Science Education ↗
The Zwicky Transient Facility Bright Transient Survey. III. BTSbot: Automated Identification and Follow-up of Bright Transients with Deep Learning
Abstract The Bright Transient Survey (BTS) aims to obtain a classification spectrum for all bright ( m peak ≤ 18.5 mag) extragalactic transients found in the Zwicky Transient Facility (ZTF) public survey. BTS critically relies on visual inspection (“scanning”) to select targets for spectroscopic follow-up, which, while effective, has required a significant time investment over the past ∼5 yr of ZTF operations. We present BTSbot , a multimodal convolutional neural network, which provides a bright transient score to individual ZTF detections using their image data and 25 extracted features. BTSbot is able to eliminate the need for daily human scanning by automatically identifying and requesting spectroscopic follow-up observations of new bright transient candidates. BTSbot recovers all bright transients in our test split and performs on par with scanners in terms of identification speed (on average, ∼1 hr quicker than scanners). We also find that BTSbot is not significantly impacted by any data shift by comparing performance across a concealed test split and a sample of very recent BTS candidates. BTSbot has been integrated into Fritz and Kowalski , ZTF’s first-party marshal and alert broker, and now sends automatic spectroscopic follow-up requests for the new transients it identifies. Between 2023 December and 2024 May, BTSbot selected 609 sources in real time, 96% of which were real extragalactic transients. With BTSbot and other automation tools, the BTS workflow has produced the first fully automatic end-to-end discovery and classification of a transient, representing a significant reduction in the human time needed to scan.
Rehemtulla, Nabeel (ORCID:0000000256832389)↗
Storing Affordability: Battery Storage as an Asset to Reduce Data Center Cost Shifts
This report examines how battery energy storage systems (BESS) can help utilities accommodate large load growth while protecting affordability for existing ratepayers. Rapid growth in electricity demand from artificial intelligence (AI) data centers is straining the U.S. grid. Furthermore, many new data centers are entering rural markets, which could offer economic benefits but may also pose implementation challenges for smaller utilities. At the same time, retail electricity prices are increasing faster than inflation, elevating customer affordability as a key challenge. While data centers have not been the primary driver of increases in residential prices to date, they have pushed wholesale energy and capacity prices higher in several markets. Fundamental utility cost-allocation principles show that data center growth can be rate-positive for existing customers only if new peak demand grows faster than the costs a utility must incur to serve it. Several factors, including a utility’s degree of wholesale market exposure, forecast uncertainty and stranded-asset risk, and tariff design can determine the outcome of load growth on retail rates. Energy storage can make several affordability contributions in the face of this landscape of uncertainty and market volatility, including deferral of higher-cost grid investments through improved utilization of existing assets and flexibility of new large loads, insulation from volatile wholesale prices through peak shaving, and reliability support to address grid risks stemming from the behavior of AI data center loads. Different potential BESS deployment pathways—utility-scale front-of-the-meter systems, aggregated small-scale storage installations, and data center-sited behind-the-meter storage—are compared against each other and against conventional capacity alternatives. This framework is intended as a conceptual resource to utilities, particularly smaller public utilities with rural service territories, who may be considering the role that energy storage can play in insulating existing ratepayers from data center cost shifts.
25 ENERGY STORAGE↗
Methodological challenges and new perspectives of shifting vegetation phenology in eddy covariance data
While numerous studies report shifts in vegetation phenology, in this regard eddy covariance (EC) data, despite its continuous high-frequency observations, still requires further exploration. Furthermore, there is no general consensus on optimal methodologies for data smoothing and extracting phenological transition dates (PTDs). Here, we revisit existing methodologies and present new prospects to investigate phenological changes in gross primary productivity (GPP) from EC measurements. First, we present a smoothing technique of GPP time series through the derivative of its smoothed annual cumulative sum. Second, we calculate PTDs and their trends from a commonly used threshold method that identifies days with a fixed percentage of the annual maximum GPP. A systematic analysis is performed for various thresholds ranging from 0.1 to 0.7. Lastly, we examine the relation of PTDs trends to trends in GPP across the years on a weekly basis. Results from 47 EC sites with long time series (> 10 years) show that advancing trends in start of season (SOS) are strongest at lower thresholds but for the end of season (EOS) at higher thresholds. Moreover, the trends are variable at different thresholds for individual vegetation types and individual sites, outlining reasonable concerns on using a single threshold value. Relationship of trends in PTDs and weekly GPP reveal association of advanced SOS and delayed EOS to increase in immediate primary productivity, but not to the trends in overall seasonal productivity. Drawing on these analyses, we emphasise on abstaining from subjective choices and investigating relationship of PTDs trend to finer temporal trends of GPP. Our study examines existing methodological challenges and presents approaches that optimize the use of EC data in identifying vegetation phenological changes and their relation to carbon uptake.
54 ENVIRONMENTAL SCIENCES↗
Survey of Time Shift Detection Algorithms for Measured PV Data
In this research, three variations of time shift detection algorithms were tested for their ability to detect time shift issues (including daylight savings time and random time shifts) in measured PV data sets. Two algorithms from the Python PVAnalytics package were assessed, and one algorithm from the Solar-Data-Tools package was assessed. Each algorithm's ability to accurately detect and measure time shifts was assessed.
automated preprocessing↗
Ripening of Rh Nanoparticle Catalysts in Reverse Water–Gas Shift via a Data-Driven Model Combining Physics, Theory, and Experiment
Degradation via sintering is an ongoing challenge that impedes the broad commercial success of supported metallic nanoparticle catalysts. To mitigate degradation via informed catalyst design and process operations, here we aim to disambiguate the underlying mechanisms of sintering by combining theory and experiment in a quantitative framework. While mechanistic sintering models exist, they only model a single sintering pathway, even though multiple sintering mechanisms can occur simultaneously or dominate at different stages of the process. Data-driven machine learning models have emerged as a means to represent complex processes through data regression. However, machine learning models have very large data needs and lack mechanistic insights due to their black-box encoding. To develop an interpretive model of catalyst degradation via sintering, we constructed a hybrid model combining mechanistic “physics-based” models and data-driven methods to obtain both reliable predictions and mechanistic insights regarding experimentally observed sintering phenomena. Focusing on nanoparticle sintering in the Rh–TiO 2 catalyst for the reverse water–gas shift (RWGS) reaction, the hybrid model couples a mechanistic term for Ostwald ripening with energy values calculated via density functional theory (DFT) with a parametric, data-driven discrepancy function term for unmodeled mechanisms. The hybrid model is trained using Bayesian inference with data collected from small-angle X-ray scattering (SAXS) in situ experiments wherein average nanoparticle diameter versus time was measured at three relevant operating temperatures. The calibrated hybrid model results show that an Ostwald ripening-only model parameterized with fixed DFT energies does not fully capture the time and temperature dependence of the SAXS-observed sintering kinetics, and that an additional functional contribution, or DFT energy calibration, is required to reconcile simulation and experiment. Analysis of the hybrid-model error confirms that the hybrid model outperforms both the purely mechanistic and purely data-driven alternatives in terms of expected predictive accuracy for time-evolving average particle sizes. Furthermore, the results support the hypothesis that the Ostwald ripening mechanism is less important for explaining the sintering phenomena as operating temperature increases under an assumed fixed DFT parameterization. This could be explained in one of two ways: either latent, unmodeled sintering mechanisms dominate at higher temperatures, or the DFT uncertainty increases with temperature. The proposed modeling approach directly links theory to experiments and simulations via a statistical hybrid modeling framework and can be extended to other catalytic systems to improve predictive models and mechanistic understanding.
Bayesian hybrid modeling↗