Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “forecast error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Local-thermal-gradient and large-scale-circulation impacts on turbine-height wind speed forecasting over the Columbia River Basin

Abstract. We investigate the sensitivity of turbine-height wind speed forecast to initial condition (IC) uncertainties over the Columbia River Gorge (CRG) and Columbia River Basin (CRB) for two typical weather phenomena, i.e., local-thermal-gradient-induced marine air intrusion and a cold frontal passage. Four types of turbine-height wind forecast anomalies and their associated IC uncertainties related to local thermal gradients and large-scale circulations are identified using the self-organizing map (SOM) technique. The four SOM types are categorized into two patterns, each accounting for half of the ensemble members. The first pattern corresponds to IC uncertainties that alter the wind forecast through a modulating weather system, which produces the strongest wind anomalies in the CRG and CRB. In the second pattern, the moderate uncertainties in local thermal gradient and large-scale circulation jointly contribute to wind forecast anomaly. We analyze the cross section of wind and temperature anomalies through the gorge to explore the evolution of vertical features of each SOM type. The turbine-height wind anomalies induced by large-scale IC uncertainties are more concentrated near the front. In contrast, turbine-height wind anomalies induced by the local IC thermal uncertainties are found above the surface thermal anomalies. Moreover, the wind forecast accuracy in the CRG and CRB is limited by IC uncertainties in a few specific regions, e.g., the 2 m temperature within the basin and large-scale circulation over the northeast Pacific around 140∘ W.

17 WIND ENERGY↗
DOE OSTI2022

Coordinated Ramping Product and Regulation Reserve Procurements in CAISO and MISO using Multi-Scale Probabilistic Solar Power Forecasts (Pro2R)

How can probabilistic solar forecasts lower costs and improve reliability for independent system operator (ISO) markets? We tackle this question in three steps. First, we enhance an existing solar forecasting system to provide well-calibrated hours-ahead probabilistic forecasts. We then relate the degree of uncertainty in those forecasts to error distributions for net load ramps for the California ISO (CAISO) using statistical and machine learning methods. Projected net load errors conditioned on solar uncertainty are translated into flexible ramp requirements that therefore reflect real-time meteorological and solar conditions, improving on typical ISO procedures. Finally, a multi-period look-ahead production cost model quantifies how conditional ramp requirements can a) decrease operating costs by lowering requirements compared to often conservative unconditional methods, and b) reduce generation scarcity events and consequently improve reliability by increasing flexibility requirements at times when unconditional forecast-based requirements understate actual ramp uncertainty. In addition to the products just described (quantification of solar uncertainty, its translation into requirements for ramp capability product, and quantification of the benefits of more accurate ramp requirements), this project also developed a visualization system that alerts system operators of ramp and uncertainty conditions within the network based on solar forecasts. The system is called Resource Forecast and Ramp Visualization for Situational Awareness (RaVIS). These four products represent significant advances in the state-of-the-art of probabilistic solar forecasting, development of weather-informed reserve requirements, production costing methods for estimating the benefits of more accurate reserve requirements, and visualization of system status, respectively. Yet the products are also practical and can be immediately implemented, potentially enabling system operators to save millions of dollars in ramp product procurement costs per year.

14 SOLAR ENERGY↗
DOE OSTI2024

Redshift evolution and covariances for joint lensing and clustering studies with DESI Y1

ABSTRACT Galaxy–galaxy lensing (GGL) and clustering measurements from the Dark Energy Spectroscopic Instrument Year 1 (DESI Y1) data set promise to yield unprecedented combined-probe tests of cosmology and the galaxy–halo connection. In such analyses, it is essential to identify and characterize all relevant statistical and systematic errors. We forecast the covariances of DESI Y1 GGL + clustering measurements and the systematic bias due to redshift evolution in the lens samples. Focusing on the projected clustering and GGL correlations, we compute a Gaussian analytical covariance, using a suite of N-body and lognormal simulations to characterize the effect of the survey footprint. Using the DESI one percent survey data, we measure the evolution of galaxy bias parameters for the DESI luminous red galaxy (LRG) and bright galaxy survey (BGS) samples. We find mild evolution in the LRGs in $0.4 < z < 0.8$, subdominant to the expected statistical errors. For BGS, we find less evolution for brighter absolute magnitude cuts, at the cost of reduced sample size. We find that for a redshift bin width $\Delta z = 0.1$, evolution effects on DESI Y1 GGL is negligible across all scales, all fiducial selection cuts, all fiducial redshift bins. Galaxy clustering is more sensitive to evolution due to the bias squared scaling. Nevertheless the redshift evolution effect is insignificant for clustering above the 1-halo scale of $0.1h^{-1}$ Mpc. For studies that wish to reliably access smaller scales, additional treatment of redshift evolution is likely needed. This study serves as a reference for GGL and clustering studies using the DESI Y1 sample.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Short-term electricity load forecasting: Application-driven evaluation of machine learning models across spatial and temporal scales

As we transition towards a decarbonized economy, the integration of variable renewable energy resources and new demands (e.g., electric vehicles, heat pumps) into the electricity grid places unprecedented pressure on grid operators to effectively anticipate and manage peak load. In this context, machine learning algorithms are proving to be indispensable for accurate short-term load forecasting, a crucial task to address these challenges. This study benchmarks 6 machine learning algorithms, including three neural networks and three tree-based algorithms, across various levels of spatial aggregation and time horizons (1, 4, 8, 24, and 48 h). The central contribution of this work is the comparison and analysis of load forecasting models not only based on statistical metrics, but also based on a novel error metric, which evaluates the cost implications of forecast errors for power system stakeholders. Results show that tree-based models outperform neural networks, based on statistical metrics, and yield less skewed error distributions for most spatial scales. However, through the lens of the novel error metric, neural networks are the more competitive choice, especially for forecast horizons that exceed 8 h. The study concludes with actionable recommendations to grid operators and highlights the need for the development of error metrics that link forecasting accuracy to operational costs. To promote transparency and open science, the datasets and Python code are open-sourced via a supplementary repository.

Houben, Nikolaus↗
DOE OSTI2023

Reserve and energy scarcity pricing in United States power markets: A comparative review of principles and practices

Here, errors in forecasting load and renewable-based generation in restructured power systems mean that independent system operators (ISOs) must procure sufficient operating reserves to keep the real-time operation of the system reliable and secure. But when procured reserves turn out to be insufficient in real-time due to the lack of resource capacity or ramp capability, operators often set higher prices for reserves and energy to encourage more supply, and to motivate consumers to decrease usage or shift it to other times. This procedure, which is called scarcity or shortage pricing, is a core feature of U.S. electricity markets. It is receiving increased attention from market designers and stakeholders because scarcity will become more important for spot price formation in the future with the increased penetration of zero-marginal cost renewables, and the shrinking role of fuel costs in setting prices. Scarcity pricing is implemented in various ways by different ISOs. These differences have practical implications for the level of prices and incentives for investment, operations, and demand modification. In this paper, general approaches and specific calculation procedures for reserve and energy scarcity pricing practices and calculations across the seven ISO-based U.S. power markets are reviewed and compared. A consistent terminology is used to facilitate the comparison. Current scarcity pricing practices are grouped into three approaches: (1) imposing an adder after the spot market is run; (2) including stepwise demand curves within market clearing procedures for non-contingency reserve products (e.g., the novel flexiramp product), which tends to yield longer right tails for energy scarcity premium curves; and (3) having stepwise demand curves for traditional contingency reserve products only, which results in shorter right tails in energy scarcity curves. A generic numerical example is presented to highlight the large practical differences among the reserve scarcity pricing approaches and specific implementations. To further investigate factors that contribute the most to demand curves differences among ISOs, a sensitivity analysis is performed. This analysis shows that the largest source of differences among the curves is the scarcity prices assumed in the case of severe scarcity, while the number of steps used and whether flexiramp is considered also yields important differences in scarcity prices. As renewable penetration increases, it will become increasingly crucial to employ administrative demand curves so that spot prices more effectively motivate supply and demand adjustments exactly when and where they are needed. This study shows that the different assumptions yield very different scarcity premiums for reserves and energy, and are likely to provide divergent incentives for resources to respond to shortages. It is concluded that to promote market efficiency, a reserve shortage demand curve should have at least three features: inclusion of the marginal value of reserve products at each shortage level, consideration of the magnitude and probability of supply contingencies, and avoidance of abrupt price discontinuities that can cause excessively volatile market outcomes.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2023

Benefits and Cautions in Data Assimilation Strategies: An Example of Modeling Groundwater Recharge

Assimilating recent observations improves model outcomes for real-time assessments of groundwater processes. This is demonstrated in estimating time-varying recharge to a shallow fractured-rock aquifer in response to precipitation. Results from estimating the time-varying water-table altitude (h) and recharge, and their error covariances, are compared for forecasting, filtering, and fixed-lag smoothing (FLS), which are implemented using the Kalman Filter as applied to a data-driven, mechanistic model of recharge. Forecasting uses past observations to predict future states and is the current paradigm in most groundwater modeling investigations; filtering assimilates observations up to the current time to estimate current states; and FLS estimates states following a time lag over which additional observations are collected. Results for forecasting yield a large error covariance relative to the magnitude of the expected recharge. With assimilating recent observations of h, filtering and FLS produce estimates of recharge that better represent time-varying observations of h and reduce uncertainty in comparison to forecasting. Although model outcomes from applying data assimilation through filtering or FLS reduce model uncertainty, they are not necessarily mass conservative, whereas forecasting outcomes are mass conservative. Mass conservative outcomes from forecasting are not necessarily more accurate, because process errors are inherent in any model. Improvements in estimating real-time groundwater conditions that better represent observations need to be weighed for the model application against outcomes with inherent process deficiencies. In conclusion, results from data assimilation strategies discussed in this investigation are anticipated to be relevant to other groundwater processes models where system states are sensitive to system inputs.

aquifer recharge↗
DOE OSTI2020

Skill of Mesoscale Models in Forecasting Springtime Macrophysical Cloud Properties at the Savannah River Site in the Southeastern US

Predicting boundary layer clouds is important for the accurate modeling of pollutant dispersion. Higher resolution mesoscale models would be expected to produce better forecasts of cloud properties that affect dispersion. Using ceilometer observations, we assess the skill of two operational mesoscale models (RAMS and WRF) to forecast cloud base altitude and cloud fraction at the Savannah River Site in the southeastern US during the springtime. Verifications were performed at small spatial and temporal scales necessary for dispersion modeling. Both models were unreliable with a 50% (RAMS) and a 46% (WRF) rate of predicting clouds observed by the ceilometer which led to low cloud fraction predictions. Results indicated that WRF better predicted daytime cloud bases from convection that occurred frequently later in the period and RAMS better predicted nighttime cloud bases. Using root mean squared error (RMSE) to score the forecast periods also highlighted this diurnal dichotomy, with WRF scores better during the day and RAMS scores better at night. Analysis of forecast errors revealed divergent model cloud base biases—WRF low and RAMS high. A hybrid solution which weighs more heavily the RAMS nighttime forecasts and WRF daytime forecasts will likely provide the best prediction of cloud properties for dispersion.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Assimilation of X-band Phased-Array Radar Data with EnKF for the Analysis and Warning Forecast of a Tornadic storm

The impact of assimilating China's operational X-band Phased-Array radar's (X-PAR) data on the analysis and warning forecast of the vortex structure and intensity of the June 8, 2018 Foshan, Guangdong province, tornadic storm was investigated for the first time using an Ensemble Kalman Filter (EnKF) data assimilation system. Both radar radial velocity (Vr) and reflectivity (Z) from two S-band operational radars and one X-PAR were assimilated. Deterministic forecasts were launched every 6 min from 05:42 UTC (20 min before the tornado touched down) to 06:00 UTC from the EnKF mean analysis field. Five experiments were conducted to examine the added capability of Z assimilation of the EnKF system, and to investigate the impact of assimilating X-PAR data on the analysis and prediction of the tornadic storm. Compared to the experiment without Z assimilation, the assimilation of Z reduced the analysis error and greatly reduced the forecast error of Z. Furthermore, the assimilation of X-PAR data greatly improved the vortex structure of the tornadic storm at low levels, and improved the intensity of the rear inflow of the tornadic storm, especially with a higher assimilation frequency. Compared to the experiments without X-PAR data assimilation, assimilating X-PAR data improved the predictability of tornadic storm.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Strategies for microgrid operation under real-world conditions

Microgrids are an increasingly relevant technology for integrating renewable energy sources into electricity systems. Based on a microgrid implementation in California, in this study we investigate microgrid operation under real-world conditions. These conditions have not yet been considered in combination and encompass energy charges, demand charges, export limits, as well as uncertainty about future electricity demand and generation in the microgrid. Under these conditions, we evaluate the performance of two frequently applied groups of strategies for microgrid operation. The first group is composed of proactive strategies that optimize decisions based on forecasts of future electricity generation and demand. The second group includes reactive strategies that make operational decisions based exclusively on the current state of the microgrid. We evaluate the performance of the strategies under varying operational parameters, forecast accuracies, and microgrid configurations—well beyond our Californian showcase. Our results confirm the expectation that proactive strategies outperform reactive ones in the majority of settings. Yet, reactive strategies can perform better under short control intervals or under moderate prediction errors of PV generation or demand. Furthermore, the interplay between real-world conditions and operational strategies reveals several additional insights for research on microgrid operation. First, we find that demand charges and export limits decisively affect microgrid performance. Second, the impact of forecast errors is highly non-linear and non-monotonous. Third, escalating negative interactions between forecast errors and demand charges make proactive strategies benefit from longer control intervals. This result is contrary to existing best practice, which promotes short control intervals to minimize the impact of uncertainty.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Estimating Ocean Observation Impacts on Coupled Atmosphere‐Ocean Models Using Ensemble Forecast Sensitivity to Observation (EFSO)

Ensemble Forecast Sensitivity to Observation (EFSO) is a technique that can efficiently identify the beneficial/detrimental impacts of every observation in ensemble-based data assimilation (DA). While EFSO has been successfully employed on atmospheric DA, it has never been applied to ocean or coupled DA due to the lack of a suitable error norm for oceanic variables. This study introduces a new density-based error norm incorporating sea temperature and salinity forecast errors, making EFSO applicable to ocean DA for the first time. We implemented the oceanic EFSO on the CFSv2-LETKF and investigated the impact of ocean observations under a weakly coupled DA framework. By removing the detrimental ocean observations detected by EFSO, the CFSv2 forecasts were significantly improved, showing the validation of impact estimation and the great potential of EFSO to be extended as a data selection criterion.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Analyzing and Exploring Training Recipes for Large-Scale Transformer-Based Weather Prediction

Abstract The rapid rise of deep learning (DL) in numerical weather prediction (NWP) has led to a proliferation of models which forecast atmospheric variables with comparable or superior skill than traditional physics-based NWP. However, among these leading DL models, there is a wide variance in both the training settings and architecture used. Further, the lack of thorough ablation studies makes it hard to discern which components are most critical to success. In this work, we show that it is possible to attain high forecast skill even with relatively off-the-shelf architectures, simple training procedures, and moderate compute budgets. Specifically, we train a minimally modified Swin Transformer V2 (SwinV2) on ERA5 data and find that it attains superior skill in terms of mean-square errors of deterministic forecasts when compared against the European Centre for Medium-Range Weather Forecasts’ Integrated Forecasting System (IFS). Almost all DL–NWP systems share a core set of hyperparameters and design decisions. To aid and expedite future DL–NWP research, we present an in-depth, systematic exploration of different loss functions, model sizes and depths, patch sizes, and multistep training objectives. We also examine the model performance with metrics beyond the typical accuracy (ACC) and RMSE and investigate how the performance scales with model size. Through our open-source code, scoring pipelines, and models, we share our findings on key aspects of the training pipeline. These ablations reduce the necessity for expensive hyperparameter tuning and lower the barrier to entry for future DL–NWP research. Significance Statement This study investigates the potential of using large-scale transformer-based models for weather prediction, showing that it is possible to achieve high forecast accuracy with simpler, off-the-shelf architectures. By training a minimally modified SwinV2 transformer on ERA5 data, we show that the model achieves competitive forecast skill in terms of mean-square error for key variables, outperforming the European Centre for Medium-Range Weather Forecasts’ Integrated Forecasting System (IFS) at all lead times. Our findings suggest that effective training strategies, such as multistep fine-tuning and channel-weighted losses, significantly enhance the model’s performance. However, we also highlight that these improvements come with trade-offs in other areas, such as ensemble spread and high-frequency spatial detail. This work highlights the promise of deep learning in improving weather forecasts, which could lead to better preparedness and response to weather events, ultimately benefiting society by providing more reliable weather predictions.

Willard, Jared D. [Lawrence Berkeley National Labo↗
DOE OSTI2022

Investigation of Stochastic Unit Commitment to Enable Advanced Flexibility Measures for High Shares of Solar PV

As the share of solar photovoltaics (PV) in the power system increases, there is a growing need for flexibility from multiple, possibly interdependent sources to adjust to PV's variability, uncertainty, and diurnal dependence. This paper investigates how stochastic unit commitment leveraging probabilistic solar forecasts can support other flexibility measures under high solar shares. We consider two flexibility measures relevant to day-ahead scheduling: battery energy time-shifting and solar ancillary service provision. Unit commitment and economic dispatch simulations are conducted on a realistic test system based on Texas using day-ahead solar trajectories. The benefits of the two flexibility measures are pronounced when the instantaneous solar share is high, offering cost savings of 10%-20% in the spring. For a Texas-sized system, this translates to hundreds of millions of dollars in cost savings once the installed PV capacity enables instantaneous solar shares regularly exceeding 40%. Using probabilistic forecasts also greatly increases the reliability of upward reserve provision from solar PV, reducing unserved reserves by 50%-100%. Both day-ahead forecast resolution and errors can impact system reliability at high solar shares, but the stochastic formulation has significant value, mitigating reliability impacts on over-forecast days.

ancillary services↗
DOE OSTI2022

Occlusion-Perturbed Deep Learning for Probabilistic Solar Forecasting via Sky Images

Solar forecasting is shifting to the probabilistic paradigm due to the inherent uncertainty within the solar resource. Input uncertainty quantification is one of the widely used and best-performing ways to model solar uncertainty. However, compared to other sources of inputs, such as numerical weather prediction models, pure sky image-based probabilistic solar forecasting lags behind. In this research, an occlusion-perturbed convolutional neural network, named the PSolarNet, is developed. The PSolarNet provides very short-term deterministic forecasts, forecast scenarios, and probabilistic forecasts of the global horizontal irradiance from sky image sequences. Case studies based on 6 years of open-source data show that the developed PSolarNet is able to generate accurate 10-minute ahead deterministic forecasts with a 5.62% normalized root mean square error, realistic and diverse forecast scenarios with a 0.966 average correlation with the actual time series, and reliable and sharp probabilistic forecasts with a 2.77% normalized continuous ranked probability score.

Bayesian model averaging↗
DOE OSTI2022

Occlusion-Perturbed Deep Learning for Probabilistic Solar Forecasting via Sky Images: Preprint

Solar forecasting is shifting to the probabilistic paradigm due to the inherent uncertainty within the solar resource. Input uncertainty quantification is one of the widely used and best-performing ways to model solar uncertainty. However, compared to other sources of inputs, such as numerical weather prediction models, pure sky image-based probabilistic solar forecasting lags behind. In this research, an occlusion-perturbed convolutional neural network, named the PSolarNet, is developed. The PSolarNet provides very short-term deterministic forecasts, forecast scenarios, and probabilistic forecasts of the global horizontal irradiance from sky image sequences. Case studies based on 6 years of open-source data show that the developed PSolarNet is able to generate accurate 10-minute ahead deterministic forecasts with a 5.62% normalized root mean square error, realistic and diverse forecast scenarios with a 0.966 average correlation with the actual time series, and reliable and sharp probabilistic forecasts with a 2.77% normalized continuous ranked probability score.

Bayesian model averaging↗
DOE OSTI2021

Machine learning based algorithms for uncertainty quantification in numerical weather prediction models

Complex numerical weather prediction models incorporate a variety of physical processes, each described by multiple alternative physical schemes with specific parameters. The selection of the physical schemes and the choice of the corresponding physical parameters during model configuration can significantly impact the accuracy of model forecasts. There is no combination of physical schemes that works best for all times, at all locations, and under all conditions. It is therefore of considerable interest to understand the interplay between the choice of physics and the accuracy of the resulting forecasts under different conditions. This paper demonstrates the use of machine learning techniques to study the uncertainty in numerical weather prediction models due to the interaction of multiple physical processes. The first problem addressed herein is the estimation of systematic model errors in output quantities of interest at future times, and the use of this information to improve the model forecasts. The second problem considered is the identification of those specific physical processes that contribute most to the forecast uncertainty in the quantity of interest under specified meteorological conditions. In order to address these questions we employ two machine learning approaches, random forests and artificial neural networks. The discrepancies between model results and observations at past times are used to learn the relationships between the choice of physical processes and the resulting forecast errors. Numerical experiments are carried out with the Weather Research and Forecasting (WRF) model. The output quantity of interest is the model precipitation, a variable that is both extremely important and very challenging to forecast. The physical processes under consideration include various micro-physics schemes, cumulus parameterizations, short wave, and long wave radiation schemes. The experiments demonstrate the strong potential of machine learning approaches to aid the study of model errors.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Causes of Missing Snowmelt Following Drought

Abstract Water management in snowy mountainous regions hinges on forecasting snowmelt runoff. However, droughts are altering snowpack‐runoff relationships with ongoing debate about the driving mechanisms. For example, in 2021 in California, less than half of predicted streamflow arrived. Mechanisms proposed for this “missing” streamflow included changes in evapotranspiration ( ET ), rainfall, and subsurface moisture conditions. Here, we demonstrate that ET in drought years generates dry subsurface conditions that reduce runoff in subsequent years. A model including this legacy of depleted moisture storage reduced median error in 2021 forecasts from 60% to 20% at 15 minimally disturbed basins and from 18% to 2% at 6 water supply basins in the Sierra Nevada (basins range in area from 5 to 23,051 km 2 and mean annual precipitation from 814 to 1,549 mm). Our findings indicate that the relationship between snowpack and runoff will evolve as plant ecosystems respond to climate change and alter subsurface water storage dynamics.

Lapides, Dana A.↗
DOE OSTI2023

Improving the Representation of Hydropower in Production Cost Models

One of the challenges of a high-renewable, low-carbon system is that grid operators will no longer be able to count on fossil-fired generation should wind miss its forecast or modeling errors create shortage issues, and this change has operational and reliability implications that affect variable renewable energy integration. Hydropower can help in that it can provide many of the services that have been traditionally provided by fossil generation. However, getting the most from hydropower can be challenging in that its capabilities vary with time and are affected by other water users' needs, two complexities that are not well represented in most of today's production cost models (PCMs). This work is an early step in addressing these gaps and making the most of hydropower's capabilities.

cost model↗