Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “forecast error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Power generation forecasting for solar plants based on Dynamic Bayesian networks by fusing multi-source information

A Dynamic Bayesian network (DBN) model for solar power generation forecasting in solar plants is proposed in this paper. The key idea is to fuse sensor data, operational indicators, meteorological data, lagged output power information, and model errors for more accurate short-term (e.g., hours) and mid-term (e.g., days to weeks) power generation forecasting. The proposed DBN augments automated data-driven structure learning with expert knowledge encoding using continuous and categorical data given constraints to represent causal relationships within a solar inverter system. Additionally, an error compensation mechanism is proposed to capture temporal fluctuation. The effectiveness of the DBN on solar power generation forecasting was evaluated by rolling window analysis with one-year testing data collected from a local solar plant. The proposed DBN is compared with four state-of-art methods including support-vector regression (SVR), k-nearest neighbors (kNN), artificial neural network (ANN), and long short-term memory (LSTM) models. The result show that the proposed DBN achieves better accuracy in general, and it is not as data-hungry as some neural network-based models. The proposed DBN is also shown to have robust and consistent forecasting power with different forecasting horizons. The accuracy is 92% - 95% from one hour to one week ahead forecasting.

14 SOLAR ENERGY↗
DOE OSTI2022

Marginal unbiased score expansion and application to CMB lensing

Here, we present the marginal unbiased score expansion (MUSE) method, an algorithm for generic high-dimensional hierarchical Bayesian inference. MUSE performs approximate marginalization over arbitrary non-Gaussian latent parameter spaces, yielding Gaussianized asymptotically unbiased and near-optimal constraints on global parameters of interest. It is computationally much cheaper than exact alternatives like Hamiltonian Monte Carlo (HMC), excelling on funnel problems which challenge HMC, and does not require any problem-specific user supervision like other approximate methods such as variational inference or many simulation-based inference methods. MUSE makes possible the first joint Bayesian estimation of the delensed Cosmic Microwave Background (CMB) power spectrum and gravitational lensing potential power spectrum, demonstrated here on a simulated data set as large as the upcoming South Pole Telescope 3G 1500 deg 2 survey, corresponding to a latent dimensionality of ~6 million and of order 100 global bandpower parameters. On a subset of the problem where an exact but more expensive HMC solution is feasible, we verify that MUSE yields nearly optimal results. We also demonstrate that existing spectrum-based forecasting tools which ignore pixel-masking underestimate predicted error bars by only ~10%. This method is a promising path forward for fast lensing and delensing analyses which will be necessary for future CMB experiments such as SPT-3G, Simons Observatory, or CMB-S4, and can complement or supersede existing HMC approaches. The success of MUSE on this challenging problem strengthens its case as a generic procedure for a broad class of high-dimensional inference problems.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Surface temperature comparison of the Arctic winter MOSAiC observations, ERA5 reanalysis, and MODIS satellite retrieval

Atmospheric model systems, such as those used for weather forecast and reanalysis production, often have significant and systematic errors in their representation of the Arctic surface energy budget and its components. The newly available observation data of the Multidisciplinary drifting Observatory for the Study of Arctic Climate (MOSAiC) expedition (2019/2020) enable a range of model analyses and validation in order to advance our understanding of potential model deficiencies. In the present study, we analyze deficiencies in the surface radiative energy budget over Arctic sea ice in the ERA5 global atmospheric reanalysis by comparing against the winter MOSAiC campaign data, as well as, a pan-Arctic level-2 MODIS ice surface temperature remote sensing product. We find that ERA5 can simulate the timing of radiatively clear periods, though it is not able to distinguish the two observed radiative Arctic winter states, radiatively clear and opaquely cloudy, in the distribution of the net surface radiative budget. The ERA5 surface temperature over Arctic sea ice has a conditional error with a positive bias in radiatively clear conditions and a negative bias in opaquely cloudy conditions. The mean surface temperature error is 4°C for radiatively clear situations at MOSAiC and up to 15°C in some parts of the Arctic. The spatial variability of the surface temperature, given by 4 observation sites at MOSAiC, is not captured by ERA5 due to its spatial resolution but represented in the level-2 satellite product. The sensitivity analysis of possible error sources, using satellite products of snow depth and sea ice thickness, shows that the positive surface temperature errors during radiatively clear events are, to a large extent, caused by insufficient sea ice thickness and snow depth representation in the reanalysis system. A positive bias characterizes regions with ice thickness greater than 1.5 m, while the negative bias for thinner ice is partly compensated by the effect of snow.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1986

Toward the application of the Kalman filter to regional open ocean modeling

A partial differential equation model is defined for ocean meteorological prediction and synoptic analysis. The Kalman filter used for data assimilation is described and applied to the one-dimensional linear barotropic quasi-geostrophic model with periodic and open boundary conditions. The model accounts for eddy scale dynamics in the ocean. The assumptions made in the forecast model are discussed, along with comparisons of the error variances expected with the filter and from an objective analysis method. The effectiveness of the Kalman filter is demonstrated and subsequent efforts to extend the filter to two dimensions are indicated.

Miller, R. N.↗
NASA NTRS2000

Short-Term Retrospective Land Data Assimilation Schemes

Subsurface moisture and temperature and snow/ice stores exhibit persistence on various time scales that has important implications for the extended prediction of climatic and hydrologic extremes. Hence, to improve their specification of the land surface, many numerical weather prediction (NWP) centers have incorporated complex land surface schemes in their forecast models. However, because land storages are integrated states, errors in NWP forcing accumulates in these stores, which leads to incorrect surface water and energy partitioning. This has motivated the development of Land Data Assimilation Schemes (LDAS) that can be used to constrain NWP surface storages. An LDAS is an uncoupled land surface scheme that is forced primarily by observations, and is therefore less affected by NWP forcing biases. The implementation of an LDAS also provides the opportunity to correct the model's trajectory using remotely-sensed observations of soil temperature, soil moisture, and snow using data assimilation methods. The inclusion of data assimilation in LDAS will greatly increase its predictive capacity, as well as provide high-quality land surface assimilated data.

Houser, P. R.↗
NASA NTRS2019

Integrated Multi-Satellite Evaluation for the Global Precipitation Measurement: Impact of Precipitation Types on Spaceborne Precipitation Estimation

Integrated multi-sensor assessment is proposed as a novel approach to advance satellite precipitation validation in order to provide users and algorithm developers with an assessment adequately coping with the varying performances of merged satellite precipitation estimates. Gridded precipitation rates retrieved from space sensors with quasi-global coverage feed numerous applications ranging from water budget studies to forecasting natural hazards caused by extreme events. Characterizing the error structure of satellite precipitation products is recognized as a major issue for the usefulness of these estimates. The Global Precipitation Measurement (GPM) mission aims at unifying precipitation measurements from a constellation of low-earth orbiting (LEO) sensors with various capabilities to detect, classify and quantify precipitation. They are used in combination with geostationary observations to provide gridded precipitation accumulations. The GPM Core Observatory satellite serves as a calibration reference for consistent precipitation retrieval algorithms across the constellation. The propagation of QPE uncertainty from LEO active/passive microwave (PMW) precipitation estimates to gridded QPE is addressed in this study, by focusing on the impact of precipitation typology on QPE from the Level-2 GPM Core Observatory Dual-frequency Precipitation Radar (DPR) to the Microwave Imager (GMI) to Level-3 IMERG precipitation over the Conterminous U.S. A high-resolution surface precipitation used as a consistent reference across scales is derived from the ground radar-based Multi-Radar/Multi-Sensor. While the error structure of the DPR, GMI and subsequent IMERG is complex because of the interaction of various error factors, systematic biases related to precipitation typology are consistently quantified across products. These biases display similar features across Level-2 and Level-3, highlighting the need to better resolve precipitation typology from space and the room for improvement in global-scale precipitation estimates. The integrated analysis and framework proposed herein applies more generally to precipitation estimates from sensors and error sources affecting low-earth orbiting satellites and derived gridded products.

Rainfall↗
NASA NTRS2014

Analysis of Airport Ground Delay Program Decisions Using Data Mining Techniques

Air traffic service providers have to make decisions regarding changes to air traffic flow in the event of major weather disturbances and traffic congestions to maintain safety of the system. The behavior of the air traffic management system will be more predictable if consistent decisions are made under similar traffic and weather conditions. Consistency of deciding on control action depends on the weather and traffic conditions as well as accuracy in predicting these conditions. Weather parameters (defined in terms of forecast and actual weather and traffic conditions) on different days can be used to categorize days into days with little decision consistency, days with moderate decision consistency and days with high decision consistency. Four years of traffic, weather and ground delay program decisions data at major airports in the United States are used in the analysis. This paper examines performance of different data mining methods in the three regions of decision consistency. Not surprisingly, data mining methods have the best performance in the region of most decision consistency and have the poorest performance in the region of little decision consistency. In applications where data mining methods have differing performance in differing regions, it would be more useful to characterize the region specific performance instead of characterizing performance by a single parameter. Finally, the results show no significant variation in the performance of different data mining methods for this particular problem. The fact that different mining methods show no significant variation also provides further confidence in the results of data mining methods. Work in this abstract discusses initial results. This paper describes the results in terms of both forecast and actual environmental conditions and discusses how prediction errors impact decision consistency.

Kulkarni, Deepak↗
NASA NTRS2019

Observation of Absorbing Aerosols Above Clouds over the South-East Atlantic Ocean from the Geostationary Satellite SEVIRI – Part 1: Method Description and Sensitivity

High temporal resolution observations from satellites have a great potential for studying the impact of biomass burning aerosols and clouds over the South East Atlantic Ocean (SEAO). This paper presents a method developed to retrieve simultaneously aerosol and cloud properties in aerosol above cloud conditions from the geostationary instrument Meteosat Second Generation/Spinning Enhanced Visible and Infrared Imager (MSG/SEVIRI). The above-cloud Aerosol Optical Thickness (AOT), the Cloud Optical Thickness (COT) and the Cloud droplet Effective Radius (CER) are derived from the spectral contrast and the magnitude of the signal measured in three channels in the visible to shortwave infrared region. The impact of the absorption from atmospheric gases on the satellite signal is corrected by applying transmittances calculated using the water vapour profiles from a Met Office forecast model. The sensitivity analysis shows that a 10% error on the humidity profile leads to an 18.5% bias on the above-cloud AOT, which highlights the importance of an accurate atmospheric correction scheme. In situ measurements from the CLARIFY-2017 airborne field campaign are used to constrain the aerosol size distribution and refractive index that is assumed for the aforementioned retrieval algorithm. The sensitivities in the retrieved AOT, COT and CER to the aerosol model assumptions are assessed. Between 09:00-15:00 UTC, an uncertainty of 40% is estimated on the above-cloud AOT, which is dominated by the sensitivity of the retrieval to the single scattering albedo. The absorption AOT is less sensitive to the aerosol assumptions with an uncertainty generally lower than 17% between 09:00-15:00 UTC. Outside of that time range, as the scattering angle decreases, the sensitivity of the AOT and the absorption AOT to the aerosol model increases. The retrieved cloud properties are only weakly sensitive to the aerosol model assumptions throughout the day, with biases lower than 6% on the COT and 3% on the CER. The stability of the retrieval over time is analysed. For observations outside of the backscattering glory region, the time-series of the aerosol and cloud properties are physically consistent, which confirms the ability of the retrieval to monitor the temporal evolution of aerosol above cloud events over the SEAO.

Peers, Fanny↗
NASA NTRS2019

Sub-Picometer Dynamic Measurements of a Diffuse Surface

Future space observatory missions are forecasting the need for sensing and controlling wave front error and system alignment stability to picometer scale. Picometer stability performance demands precision knowledge of the mirror and metering structure materials to the same degree. A high-speed electronic speckle pattern interferometer was designed and built to demonstrate measurements of both static and dynamic responses of picometer amplitudes in materials of diffuse surface subjected to very low energy disturbances. This paper describes the details of a test to impart a dynamic disturbance of picometer scale and measure the response of a composite material.The results of the test are also reported and show conclusively that sub-picometer scale effects can be accurately measured in an open test environment outside a vacuum chamber.

Saif, Babak↗
NASA NTRS2019

Observation of absorbing aerosols above clouds over the south-east Atlantic Ocean from the geostationary satellite SEVIRI – Part 1: Method description and sensitivity

High-temporal-resolution observations from satellites have a great potential for studying the impact of biomass burning aerosols and clouds over the south-east Atlantic Ocean (SEAO). This paper presents a method developed to simultaneously retrieve aerosol and cloud properties in aerosol above-cloud conditions from the geostationary instrument Meteosat Second Generation/Spinning Enhanced Visible and Infrared Imager (MSG/SEVIRI). The above-cloud aerosol optical thickness (AOT), the cloud optical thickness (COT) and the cloud droplet effective radius (CER) are derived from the spectral contrast and the magnitude of the signal measured in three channels in the visible to shortwave infrared region. The impact of the absorption from atmospheric gases on the satellite signal is corrected by applying transmittances calculated using the water vapour profiles from a Met Office forecast model. The sensitivity analysis shows that a 10 % error on the humidity profile leads to an 18.5 % bias on the above-cloud AOT, which highlights the importance of an accurate atmospheric correction scheme. In situ measurements from the CLARIFY-2017 airborne field campaign are used to constrain the aerosol size distribution and refractive index that is assumed for the aforementioned retrieval algorithm. The sensitivities in the retrieved AOT, COT and CER to the aerosol model assumptions are assessed. Between 09:00 and 15:00 UTC, an uncertainty of 40 % is estimated on the above-cloud AOT, which is dominated by the sensitivity of the retrieval to the single-scattering albedo. The absorption AOT is less sensitive to the aerosol assumptions with an uncertainty generally lower than 17 % between 09:00 and 15:00 UTC. Outside of that time range, as the scattering angle decreases, the sensitivity of the AOT and the absorption AOT to the aerosol model increases. The retrieved cloud properties are only weakly sensitive to the aerosol model assumptions throughout the day, with biases lower than 6 % on the COT and 3 % on the CER. The stability of the retrieval over time is analysed. For observations outside of the backscattering glory region, the time series of the aerosol and cloud properties are physically consistent, which confirms the ability of the retrieval to monitor the temporal evolution of aerosol above-cloud events over the SEAO.

cloud properties↗
NASA NTRS2020

Integrated Multi-satellite Evaluation for the Global Precipitation Measurement: Impact of Precipitation Types on Spaceborne Precipitation Estimation

An integrated multi-sensor assessment is proposed as a novel approach to advance satellite precipitation validation in order to provide users and algorithm developers with an assessment adequately coping with the varying performances of merged satellite precipitation estimates. Gridded precipitation rates retrieved from space sensors with quasi-global coverage feed numerous applications ranging from water budget studies to forecasting natural hazards caused by extreme events. Characterizing the error structure of satellite precipitation products is recognized as a major issue for the usefulness of these estimates. The Global Precipitation Measurement (GPM) mission aims at unifying precipitation measurements from a constellation of low-earth orbiting (LEO) sensors with various capabilities to detect, classify and quantify precipitation. They are used in combination with geostationary observations to provide gridded precipitation accumulations. The GPM Core Obser­vatory satellite serves as a calibration reference for consistent precipitation retrieval algorithms across the constellation. The propagation of QPE uncertainty from LEO active/passive microwave (PMW) precipitation estimates to gridded QPE is addressed in this study, by focusing on the impact of precipitation typology on QPE from the Level-2 GPM Core Observatory Dual-frequency Precipitation Radar (DPR) to the Microwave Imager (GMI) to Level-3 IMERG precipitation over the Conterminous U.S. A high-resolution surface precipitation used as a consistent reference across scales is derived from the ground radar-based Multi-Radar/Multi­Sensor. While the error structure of the DPR, GMI and subsequent IMERG is complex because of the interaction of various error factors, systematic biases related to precipitation typology are consistently quantified across products. These biases display similar features across Level-2 and Level-3, highlighting the need to better resolve precipitation typology from space and the room for improvement in global­scale precipitation estimates. The integrated analysis and framework proposed herein applies more generally to precipitation estimates from sensors and error sources affecting low-earth orbiting satellites and derived gridded products.

Pierre-Emmanuel Kirstette↗
DOE OSTI2025

Evaluation of Drilling Performance at The Geysers with Machine Learning Methods Using Geologic Data

A recent well, GDC-36, was drilled in The Geysers Geothermal Field served in a Department of Energy-industry to demonstrate improved drilling performance with polycrystalline diamond compact (PDC) bits. Both PDC and roller cone drill bits were used to drill this well. Key challenges encountered during drilling included lost circulation in the mud-drilled section, and bit damage interfacial severity in the deeper, air-drilled section. The objective of this study is to evaluate the drilling performance in relation to the local geological characteristics using machine learning methods. By applying K-clustering to the sonic log data, we were able to identify areas correlated with measured lost circulation. Also, the boundaries defined by clustering of the mineralogical and lithological data from the mud logs correlate well with interfacial severity during drilling. A random forest model was employed to build correlation between drilling data and rock strength. The confined compressive strength (CCS) of the rock in the training of the machine learning model was inferred from the dipole sonic log. The R-squared of the testing data is 0.78, and the RMSE (Root Mean Squared Error) is 0.06. The trained model was used to forecast rock strength for the section where sonic log data are not available. CCS could also be inferred from mud logs provided the relationship between mineralogy and rock strength is established through core testing data.

15 GEOTHERMAL ENERGY↗
NASA NTRS2012

Comparison of TRMM 2A25 Products Version 6 and Version 7 with NOAA/NSSL Ground Radar-Based National Mosaic QPE

Characterization of the error associated to satellite rainfall estimates is a necessary component of deterministic and probabilistic frameworks involving spaceborne passive and active microwave measurements for applications ranging from water budget studies to forecasting natural hazards related to extreme rainfall events. We focus here on the error structure of Tropical Rainfall Measurement Mission (TRMM) Precipitation Radar (PR) quantitative precipitation estimation (QPE) at ground. The problem was addressed in a previous paper by comparison of 2A25 version 6 (V6) product with reference values derived from NOAA/NSSL's ground radar-based National Mosaic and QPE system (NMQ/Q2). The primary contribution of this study is to compare the new 2A25 version 7 (V7) products that were recently released as a replacement of V6. This new version is considered superior over land areas. Several aspects of the two versions are compared and quantified including rainfall rate distributions, systematic biases, and random errors. All analyses indicate V7 is an improvement over V6.

Kirstetter, Pierre-Emmanuel↗
NASA NTRS2011

Toward a Framework for Systematic Error Modeling of NASA Spaceborne Radar with NOAA/NSSL Ground Radar-Based National Mosaic QPE

Characterization of the error associated to satellite rainfall estimates is a necessary component of deterministic and probabilistic frameworks involving space-born passive and active microwave measurement") for applications ranging from water budget studies to forecasting natural hazards related to extreme rainfall events. We focus here on the error structure of NASA's Tropical Rainfall Measurement Mission (TRMM) Precipitation Radar (PR) quantitative precipitation estimation (QPE) at ground. The problem is addressed by comparison of PR QPEs with reference values derived from ground-based measurements using NOAA/NSSL ground radar-based National Mosaic and QPE system (NMQ/Q2). A preliminary investigation of this subject has been carried out at the PR estimation scale (instantaneous and 5 km) using a three-month data sample in the southern part of US. The primary contribution of this study is the presentation of the detailed steps required to derive trustworthy reference rainfall dataset from Q2 at the PR pixel resolution. It relics on a bias correction and a radar quality index, both of which provide a basis to filter out the less trustworthy Q2 values. Several aspects of PR errors arc revealed and quantified including sensitivity to the processing steps with the reference rainfall, comparisons of rainfall detectability and rainfall rate distributions, spatial representativeness of error, and separation of systematic biases and random errors. The methodology and framework developed herein applies more generally to rainfall rate estimates from other sensors onboard low-earth orbiting satellites such as microwave imagers and dual-wavelength radars such as with the Global Precipitation Measurement (GPM) mission.

Kirstettier, Pierre-Emmanual↗
NASA NTRS1986

Statistical corrections to numerical predictions. IV

The National Meteorological Center Barotropic-Mesh Model has been used to test a statistical correction procedure, designated as M-II, that was developed in Schemm et al. (1981). In the present application, statistical corrections at 12 h resulted in significant reductions of the mean-square errors of both vorticity and the Laplacian of thickness. Predictions to 48 h demonstrated the feasibility of applying corrections at every 12 h in extended forecasts. In addition to these improvements, however, the statistical corrections resulted in a shift of error from smaller to larger-scale motions, improving the smallest scales dramatically but deteriorating the largest scales. This effect is shown to be a consequence of randomization of the residual errors by the regression equations and can be corrected by spatially high-pass filtering the field of corrections before they are applied.

Schemm, Jae-Kyung↗
NASA NTRS2007

Recent Theoretical Advances in Analysis of AIRS/AMSU Sounding Data

AIRS was launched on EOS Aqua on May 4,2002, together with AMSU-A and HSB, to form a next generation polar orbiting infrared and microwave atmospheric sounding system. This paper describes the AIRS Science Team Version 5.0 retrieval algorithm. Starting in early 2007, the Goddard DAAC will use this algorithm to analyze near real time AIRS/AMSU observations. These products are then made available to the scientific community for research purposes. The products include twice daily measurements of the Earth's three dimensional global temperature, water vapor, and ozone distribution as well as cloud cover. In addition, accurate twice daily measurements of the earth's land and ocean temperatures are derived and reported. Scientists use this important set of observations for two major applications. They provide important information for climate studies of global and regional variability and trends of different aspects of the earth's atmosphere. They also provide information for researchers to improve the skill of weather forecasting. A very important new product of the AIRS Version 5 algorithm is accurate case-by-case error estimates of the retrieved products. This heightens their utility for use in both weather and climate applications. These error estimates are also used directly for quality control of the retrieved products. Version 5 also allows for accurate quality controlled AIRS only retrievals, called "Version 5 AO retrievals" which can be used as a backup methodology if AMSU fails. Examples of the accuracy of error estimates and quality controlled retrieval products of the AIRS/AMSU Version 5 and Version 5 AO algorithms are given, and shown to be significantly better than the previously used Version 4 algorithm. Assimilation of Version 5 retrievals are also shown to significantly improve forecast skill, especially when the case-by-case error estimates are utilized in the data assimilation process.

Susskind, Joel↗
NASA NTRS2017

Impact of Archeomagnetic Field Model Data on Modern Era Geomagnetic Forecasts

A series of geomagnetic data assimilation experiments have been carried out to demonstrate the impact of assimilating archeomagnetic data via the CALS3k.4 geomagnetic field model from the period between 10 and 1590 CE. The assimilation continues with the gufm1 model from 1590 to 1990 and CM4 model from 1990 to 2000 as observations, and comparisons between these models and the geomagnetic forecasts are used to determine an optimal maximum degree for the archeomagnetic observations, and to independently estimate errors for these observations. These are compared with an assimilation experiment that uses the uncertainties provided with CALS3k.4. Optimal 20 year forecasts in 1990 are found when the Gauss coefficients up to degree 3 are assimilated. In addition we demonstrate how a forecast and observation bias correction scheme could be used to reduce bias in modern era forecasts. Initial experiments show that this approach can reduce modern era forecast biases by as much as 50 percent.

Tangborn, Andrew↗