A Nowcasting Approach for Low-Earth-Orbiting Hyperspectral Infrared Soundings within the Convective Environment
Explore the source record for details and available documents.
54 ENVIRONMENTAL SCIENCES↗
SEARCH · Engineering Papers
Results for “Nowcasting”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Advanced Solar and Load Forecasting Incorporating HD Sky Imaging (Phase III)
Due to rapidly changing sky conditions, the available solar irradiance for energy generation is subject to wide swings in amplitude. As the market penetration of solar energy continues to increase rapidly, these variations in solar power generation are beginning to have an impact on grid stability and increased wear on power switches. Solar power forecasting plays a critical role in operations for Independent System Operators (ISOs) and utilities. Accurate forecasts help maintain grid reliability, optimize generation from renewables, and reduce operating costs. Of particular interest to the ISOs and utilities are sudden changes in solar irradiance, termed “ramp events,” due to the movement of clouds. One significant impact of ramp events on the grid is additional ancillary service requirements necessary to manage such variability. Ramp events can also cause voltage fluctuations in the distribution grids and trigger actions of automated line equipment (e.g., tap changers), leading to additional maintenance costs. In high penetration solar regions, forecasts must be made for both transmission-connected and distribution-connected resources – either behind the meter or in front of it. Particularly for distributed solar energy resources, forecasting can be a challenge due to the lack of visibility of the energy resource. Brookhaven National Laboratory (BNL) has been working towards nowcasting Global Horizontal Irradiance (GHI) using low-cost technologies for several years now. The Solar NowCasting technology being developed by BNL is a 0 – 30 min solar “nowcasting” technology applicable to a scale covering both large generating facilities and residential, distributed solar installations that relies on a network of ground-based high-definition (HD) cameras and surface pyranometers. GHI forecasts are being produced for both regions using 8 ground-based HD cameras and at least 2 surface pyranometers. When stitched together, the camera images can be used to forecast the impact of clouds on available solar irradiance over a domain of ~50 km 2 . Phase I of the project was the engineering scale up conceptual design stage. Phase II was the initial field test and demonstration, in which the technology was scaled up by a factor of 20 times and successfully demonstrated in eastern Long Island. In Phase III, an additional forecasting network was added in upstate NY and both networks are currently being operated until at least a full year of data is gathered by the Albany network to allow for the collection of sufficient data to evaluate performance against the persistence and smart persistence models using the Solar Arbiter.
14 SOLAR ENERGY↗
Resampling and data augmentation for short-term PV output prediction based on an imbalanced sky images dataset using convolutional neural networks
Integrating photovoltaics (PV) into electricity grids is challenged by potentially large fluctuations in power generation. In recent years, sky image-based PV output prediction using convolutional neural networks (CNNs) has emerged as a promising approach to forecasting fluctuations. A key challenge is imbalanced sky image datasets: because of the geography of solar PV system installations, sky image datasets are often rich in sunny condition data but deficient in cloudy condition data. This imbalance contrasts with the fact that model errors are dominated by cloudy condition performance. In this study, we attempt to remedy this by exploring the enrichment and augmentation of an imbalanced sky images dataset for two PV output prediction tasks: nowcasting (predicting concurrent PV output) and forecasting (predicting 15-minute-ahead future PV output). We empirically examine the efficacy of using different resampling and data augmentation approaches to create a rebalanced dataset for model development. A three-stage greedy search is used to determine the optimal resampling approach, data augmentation techniques and over-sampling rate. The results show that for the nowcast problem, resampling and data augmentation can effectively enhance the model performance, reducing overall root mean squared error (RMSE) by an average of 4%, or a 15 std. (standard deviation) of improvement compared to the variability of the baseline model. In contrast, the treatment RMSE for the forecast problem nearly always overlaps the baseline performance at the ± 2 std. level. The optimal resampling approach expands on the original dataset by over-sampling the minority cloudy data, with the best results from large over-sampling rate (e.g., 4 ~ 6 times over-sampling of cloudy images).
14 SOLAR ENERGY↗
PreMevE‐MEO: Predicting Ultra‐Relativistic Electrons Using Observations From GPS Satellites
Abstract Ultra‐relativistic electrons with energies greater than or equal to two megaelectron‐volt (MeV) pose a major radiation threat to spaceborne electronics, and thus specifying those highly energetic electrons has a significant meaning to space weather communities. Here we report the latest progress in developing our predictive model for MeV electrons in the outer radiation belt. The new version, primarily driven by electron measurements made along medium‐Earth‐orbits (MEO), is called PREdictive MEV Electron (PreMevE)‐MEO model that nowcasts ultra‐relativistic electron flux distributions across the whole outer belt. Model inputs include >2 MeV electron fluxes observed in MEOs by a fleet of GPS satellites as well as electrons measured by one Los Alamos satellite in the geosynchronous orbit. We developed an innovative Sparse Multi‐Inputs Latent Ensemble NETwork (SmileNet) which combines convolutional neural networks with transformers, and we used long‐term in situ electron data from NASA's Van Allen Probes mission to train, validate, optimize, and test the model. It is shown that PreMevE‐MEO can provide hourly nowcasts with high model performance efficiency and high correlation with observations. This prototype PreMevE‐MEO model demonstrates the feasibility of making high‐fidelity predictions driven by observations from longstanding space infrastructure in MEO, thus has great potential of growing into an invaluable space weather operational warning tool.
79 ASTRONOMY AND ASTROPHYSICS↗
Real-Time, Simultaneous Soil Water Content and Meteorological Data Measurement to Support TRACER over Harris County, Texas (Field Campaign Report: Part I)
The main purpose of this project was to provide ground-truth and satellite-based soil water content data in the Houston, Texas, area to support the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility’s 2022 field campaign, the Tracking Aerosol Convection Interactions ExpeRiment (TRACER). This involved installing four soil monitoring stations at different locations in the Houston area, two of which were part of the ARM facility and two were ancillary. The two stations associated directly with ARM were installed alongside other facilities managed and maintained by the TRACER research team during the intensive operational period (IOP), one located at the La Porte, Texas airport and the other near Guy, Texas. Data from these stations were assimilated with similar data collected by the Harris County Flood Control District (HCFCD) to improve the spatial coverage of the real-time monitoring network. The ground-truth data were compared to the satellite-based (National Aeronautics and Space Administration [NASA]’s Soil Moisture Active Passive [SMAP] mission) data in the TRACER area of interest, and analyzed further to nowcast gridded data over the Houston area. The nowcasted, gridded data product was made available to TRACER researchers for use in climate and land-atmosphere interaction modeling that can help understand formation and persistence of convective storms in dense urban areas, and predict possible environmental events such as floods.
54 ENVIRONMENTAL SCIENCES↗
Real-Time, Simultaneous Soil Water Content and Meteorological Data Measurement to Support TRACER over Harris County, Texas Field Campaign Report: Part II
The main purpose of this project was to provide ground-truth and satellite-based soil water content data in the Houston, Texas, area to support the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility’s 2022 field campaign, the Tracking Aerosol Convection Interactions ExpeRiment (TRACER). This involved installing four soil monitoring stations at different locations in the Houston area, two of which were part of the ARM facility and two were ancillary. The two stations associated directly with ARM were installed alongside other facilities managed and maintained by the TRACER research team during the intensive operational period (IOP), one located at the La Porte, Texas airport and the other located near Guy, Texas. Data from these stations were assimilated with similar data collected by the Harris County Flood Control District (HCFCD) to improve the spatial coverage of the real-time monitoring network. The ground-truth data were compared to the satellite-based (National Aeronautics and Space Administration [NASA]’s Soil Moisture Active Passive [SMAP] mission) data in the TRACER area of interest, and analyzed further to nowcast gridded data over the Houston area. The nowcasted, gridded data product was made available to TRACER researchers for use in climate and land-atmosphere interaction modeling that can help understand formation and persistence of convective storms in dense urban areas, and predict possible environmental events such as floods.
54 ENVIRONMENTAL SCIENCES↗
Solar Irradiance Ramp Forecasting Based on All-Sky Imagers
Solar forecasting constitutes a critical tool for operating, producing and storing generated power from solar farms. In the framework of the International Energy Agency’s Photovoltaic Power Systems Program Task 16, the solar irradiance nowcast algorithms, based on five all-sky imagers (ASIs), are used to investigate the feasibility of ASIs to foresee ramp events. ASIs 1–2 and ASIs 3–5 can capture the true ramp events by 26.0–51.0% and 49.0–92.0% of the cases, respectively. ASIs 1–2 provided the lowest (<10.0%) falsely documented ramp events while ASIs 3–5 recorded false ramp events up to 85.0%. On the other hand, ASIs 3–5 revealed the lowest falsely documented no ramp events (8.0–51.0%). ASIs 1–2 are developed to provide spatial solar irradiance forecasts and have been delimited only to a small area for the purposes of this benchmark, which penalizes these approaches. These findings show that ASI-based nowcasts could be considered as a valuable tool for predicting solar irradiance ramp events for a variety of solar energy technologies. The combination of physical and deep learning-based methods is identified as a potential approach to further improve the ramp event forecasts.
14 SOLAR ENERGY↗
Machine Learning–Adjusted WRF Forecasts to Support Wind Energy Needs in Black Start Operations
Abstract The push for increased capacity of renewable sources of electricity has led to the growth of wind-power generation, with a need for accurate forecasts of winds at hub height. Forecasts for these levels were uncommon until recently, and that, combined with the nocturnal collapse of the well-mixed boundary layer and daytime growth of the boundary layer through the levels important for energy generation, has contributed to errors in numerical modeling of wind generation resources. The present study explores several machine learning algorithms to both forecast and correct standard WRF Model forecasts of winds and temperature at hub height within wind turbine plants over several different time periods that are critical for the anticipation of potential blackouts and aiding in black start operations on the power grid. It was found that mean square error for day-2 wind forecasts from the WRF Model can be improved by over 90% with the use of a multioutput neural network, and that 60-min forecasts of WRF error, which can then be used to adjust forecasts, can be made with an LSTM with great accuracy. Nowcasting of temperature and wind speed over a 10-min period using an LSTM produced very low error and especially skillful forecasts of maximum and minimum values over the turbine plant area.
17 WIND ENERGY↗
Coastal-Fog Microphysics Using In-Situ Observations and GOES-R Retrievals
Here, the objective of this work is to evaluate GOES-R based fog conditions occurred during the C-FOG (Toward Improving Coastal Fog Prediction) field campaign. C-FOG is designed to advance understanding of fog formation, development, and dissipation over coastal environments to improve its predictability. The project took place along coastlines and open water environments of Eastern Canada (Nova Scotia, NS and the Island of Newfoundland, NL) during August-October of 2018 where environmental conditions play an important role for late season’s fog formation. During the C-FOG field campaign, instruments were located at the Ferryland Supersite, NL, with 2 main sites, and five satellite sites, as well as on the Research Vessel (R/V) Hugh R. Sharp. Key measurement instruments included a Gondola carrying a CDP (cloud droplet probe) and a BCP (backscatter cloud probe) was used for measuring droplet spectra from 1-75 µm on the R/V. A LPM (laser precipitation monitor) with 100 µm-1 cm size range and an OPC (optical particle counter) with 0.3-17 µm at 16 spectral channels provided information for fog and drizzle discrimination. Remote sensing platforms (e.g. profiling microwave radiometer, PMWR), Ceilometer, Lidar), meteorological towers, tethered balloons, and GOES-R products (e.g. fog coverage and droplet size and liquid water path) were used to evaluate fog over horizontal and vertical dimensions. Results suggest that effective radius, phase, liquid water path, and liquid water content values obtained from GOES-R and PMWR were comparable to ground based in-situ observations. It is concluded that integration of observations and nowcasting products may help improve short term local fog predictions.
54 ENVIRONMENTAL SCIENCES↗
Relating storm-snow avalanche instabilities to data collected from the Differential Emissivity Imaging Disdrometer (DEID)
Storm-snow avalanches are challenging to forecast due to complex alpine terrain and during rapidly changing weather conditions. They can result in loss of lives and significant economic impact. We describe how a new device that continuously measures with high-frequency snowflake mass, size, density, and type, the Differential Emissivity Imaging Disdrometer (DEID), and show how the DEID can be used to aid avalanche forecasting when coupled with a storm-snow stability model. DEID measurements of snow accumulation, snow water equivalent (SWE), and snow density obtained during seventeen storms taken at the mid-Collins Snow-Study Plot at Alta Ski Area in Utah's Central Wasatch mountain range during winter 2020–2021 show excellent agreement with infrequent manual measurements. Additionally, two new variables, the Shape Density Index (SDI) and Complexity, are proposed and used to classify snowflake habit and estimate storm-snow shear strength. We illustrate how these DEID-derived data can be used to identify layers of concern in the storm snow such as density inversions, in real-time without digging snow pits. Furthermore, the DEID-data are used to run four variations of the SNOw Slope Stability model (SNOSS) for the storms investigated. The results are evaluated with data collected from tilt-board tests, infrasound measurements, and visual observations of avalanches. For a total fourteen storms analyzed, the DEID-driven SNOSS-modeled minimum stability index predicts the general stability of the storm-snow as indicated by observed avalanches, both natural and of unknown cause. Finally, the results provide a promising approach for nowcasting instabilities within storm-snow layers with a single instrument.
54 ENVIRONMENTAL SCIENCES↗
Enhance Low Level Temperature and Moisture Profiles Through Combining NUCAPS, ABI Observations, and RTMA Analysis
Thermodynamic information from low levels in the atmosphere is crucial for operational weather forecasts and meteorological researchers. The NOAA Unique Combined Atmospheric Processing System (NUCAPS) sounding products have been proven beneficial to fill the data gap between synoptic radiosonde observations (RAOBs). However, compared with the upper troposphere, the accuracy of NUCAPS soundings in the low levels still needs improvement. In this study, a deep neural network (DNN) is applied to fuse multiple data sources to enhance the NUCAPS temperature and moisture profiles in the lower atmosphere. The network is developed by combining satellite observations, including NUCAPS sounding retrievals and high resolution geostationary satellite observations from the Advanced Baseline Imager, and surface analysis from the Real-Time Mesoscale Analysis (RTMA) as inputs, while collocated soundings from ECMWF re-analysis version 5 are used as the benchmark for the training. The performance of the model is evaluated by using the independent testing data set, data from a different year, as well as collocated RAOBs, showing improvement to the temperature and moisture profiles by reducing the root-mean-squared-error (RMSE) by more than 30% in the lower atmosphere (from 700 hPa to surface) in both clear sky and partially cloudy conditions. A convective event from June 18, 2017 is presented to illustrate the application of the enhanced low level soundings on high impact weather events. The enhanced soundings from fused data capture the large surface-based convective available potential energy structures in the preconvection environment, which is very useful for severe storm nowcasting and forecasting applications.
54 ENVIRONMENTAL SCIENCES↗
Medium Energy Electron Flux in Earth's Outer Radiation Belt (MERLIN): A Machine Learning Model
The radiation belts of the Earth, filled with energetic electrons, comprise complex and dynamic systems that pose a significant threat to satellite operation. While various models of electron flux both for low and relativistic energies have been developed, the behavior of medium energy (120–600 keV) electrons, especially in the MEO region, remains poorly quantified. At these energies, electrons are driven by both convective and diffusive transport, and their prediction usually requires sophisticated 4D modeling codes. In this paper, we present an alternative approach using the Light Gradient Boosting (LightGBM) machine learning algorithm. The Medium Energy electRon fLux In Earth's outer radiatioN belt (MERLIN) model takes as input the satellite position, a combination of geomagnetic indices and solar wind parameters including the time history of velocity, and does not use persistence. MERLIN is trained on >15 years of the GPS electron flux data and tested on more than 1.5 years of measurements. Tenfold cross validation yields that the model predicts the MEO radiation environment well, both in terms of dynamics and amplitudes o f flux. Evaluation on the test set shows high correlation between the predicted and observed electron flux (0.8) and low values of absolute error. The MERLIN model can have wide space weather applications, providing information for the scientific community in the form of radiation belts reconstructions, as well as industry for satellite mission design, nowcast of the MEO environment, and surface charging analysis.
79 ASTRONOMY AND ASTROPHYSICS↗
Probabilistic Diffusion Models Advance Extreme Flood Forecasting
Extreme floods pose escalating risks in a changing climate, yet forecasting remains challenging due to peak flow underestimation and high uncertainty. We introduce diffusion-based runoff model (DRUM), a probabilistic deep learning (DL) approach that advances extreme flood forecasting across representative basins in the contiguous United States. DRUM outperforms state-of-the-art benchmarks, enhancing nowcasting skill for the top 1‰ of flows in 72.3% of studied basins. Under operational scenarios, DRUM extends reliable lead times by nearly a full day for 20- and 50-year floods. When evaluated with measured precipitation, an ideal condition, recall improves by 0.3–0.4 and the early warning window extends by 2.3 days for 50-year floods. The enhancement potential varies regionally, with precipitation-driven flood zones in the eastern and northwestern US benefiting most, gaining 3–7 days in lead time. These findings highlight the transformative potential of diffusion models as a cutting-edge generative AI technique for advancing hydrology and broader Earth system sciences.
54 ENVIRONMENTAL SCIENCES↗
cheny-lanl/PreMevE-MEO-2024
This package contains R&D python codes that can nowcast >2 MeV electron distributions in the Earth's outer radiation belt, using LANL's GPS electron data (publicly available) as the primary driver. Functions of this PreMevE-MEO model has been described with details in a manuscript titled "PreMevE-MEO: Predicting Ultra-relativistic Electrons Using Observations from GPS Satellites" (LA-UR-24-23843), which has been submitted to Space Weather Journal.
Chen, Yue↗
An Analytic Formula for Entraining CAPE in Midlatitude Storm Environments
Abstract This article introduces an analytic formula for entraining convective available potential energy (ECAPE) with an entrainment rate that is determined directly from an environmental sounding, rather than prescribed by the formula user. Entrainment is connected to the background environment using an eddy diffusivity approximation for lateral mixing, updraft geometry assumptions, and mass continuity. These approximations result in a direct correspondence between the storm-relative flow and the updraft radius and an inverse scaling between the updraft radius squared and entrainment rate. The aforementioned concepts, combined with the assumption of adiabatic conservation of moist static energy, yield an explicit analytic equation for ECAPE that depends entirely on state variables in an atmospheric profile and a few constant parameters with values that are established in past literature. Using a simplified Bernoulli-like equation, the ECAPE formula is modified to account for updraft enhancement via kinetic energy extracted from the cloud’s background environment. CAPE and ECAPE can be viewed as predictors of the maximum vertical velocity w max in an updraft. Hence, these formulas are evaluated using w max from past numerical modeling studies. Both of the new formulas improve predictions of w max substantially over commonly used diagnostic parameters, including undiluted CAPE and ECAPE with a constant prescribed entrainment rate. The formula that incorporates environmental kinetic energy contribution to the updraft correctly predicts instances of exceedance of by w max , and provides a conceptual explanation for why such exceedance is rare among past simulations. These formulas are potentially useful in nowcasting and forecasting thunderstorms and as thunderstorm proxies in climate change studies. Significance Statement Substantial mixing occurs between the upward-moving air currents in thunderstorms (updrafts) and the surrounding comparatively dry environmental air, through a process called entrainment. Entrainment controls thunderstorm intensity via its diluting effect on the buoyancy of air within updrafts. A challenge to representing entrainment in forecasting and predictions of the intensity of updrafts in future climates is to determine how much entrainment will occur in a given thunderstorm environment without a computationally expensive high-resolution simulation. To address this gap, this article derives a new formula that computes entrainment from the properties of a single environmental profile. This formula is shown to predict updraft vertical velocity more accurately than past diagnostics, and can be used in forecasting and climate prediction to improve predictions of thunderstorm behavior and impacts.
54 ENVIRONMENTAL SCIENCES↗
Near-Real-Time Forecast of Satellite-Based Soil Moisture Using Long Short-Term Memory with an Adaptive Data Integration Kernel
Nowcasts, or near-real-time (NRT) forecasts, of soil moisture based on the Soil Moisture Active and Passive (SMAP) mission could provide substantial value for a range of applications including hazards monitoring and agricultural planning. To provide such a NRT forecast with high fidelity, we enhanced a time series deep learning architecture, long short-term memory (LSTM), with a novel data integration (DI) kernel to assimilate the most recent SMAP observations as soon as they become available. The kernel is adaptive in that it can accommodate irregular observational schedules. Testing over the CONUS, this NRT forecast product showcases predictions with unprecedented accuracy when evaluated against subsequent SMAP retrievals. It showed smaller error than NRT forecasts reported in the literature, especially at longer forecast latency. The comparative advantage was due to LSTM’s structural improvements, as well as its ability to utilize more input variables and more training data. The DI-LSTM was compared to the original LSTM model that runs without data integration, referred to as the projection model here. We found that the DI procedure removed the autocorrelated effects of forcing errors and errors due to processes not represented in the inputs, for example, irrigation and floodplain/lake inundation, as well as mismatches due to unseen forcing conditions. The effects of this purely data-driven DI kernel are discussed for the first time in the geosciences. Furthermore, this work presents an upper-bound estimate for the random component of the SMAP retrieval error.
54 ENVIRONMENTAL SCIENCES↗
Multisensor Agile Adaptive Sampling (MAAS): A Methodology to Collect Radar Observations of Convective Cell Life Cycle
Abstract Multisensor Agile Adaptive Sampling (MAAS), a smart sensing framework, was adapted to increase the likelihood of observing the vertical structure (with little to no gaps), spatial variability (at subkilometer scale), and temporal evolution (at ∼2-min resolution) of convective cells. This adaptation of MAAS guided two mechanically scanning C-band radars (CSAPR2 and CHIVO) by automatically analyzing the latest NEXRAD data to identify, characterize, track, and nowcast the location of all convective cells forming in the Houston domain. MAAS used either a list of predetermined rules or real-time user input to select a convective cell to be tracked and sampled by the C-band radars. The CSAPR2 tracking radar was first tasked to collect three sector plan position indicator (PPI) scans toward the selected cell. Edge computer processing of the PPI scans was used to identify additional targets within the selected cell. In less than 2 min, both the CSAPR2 and CHIVO radars were able to collect bundles of three to six range–height indicator (RHI) scans toward different targets of interest within the selected cell. Bundles were successively collected along the path of cell advection for as long as the cell met a predetermined set of criteria. Between 1 June and 30 September 2022 over 315 000 vertical cross-section observations were collected by the C-band radars through ∼1300 unique isolated convective cells, most of which were observed for over 15 min of their life cycle. To the best of our knowledge, this dataset, collected primarily through automatic means, constitutes the largest dataset of its kind.
54 ENVIRONMENTAL SCIENCES↗
Quantitative Ceilometer-Radar Studies of Clouds Field Campaign Report
The Long Island Solar Farm (LISF) is a 32-megawatt solar photovoltaic power plant built through a collaboration including BP Solar, the Long Island Power Authority (LIPA), and the U.S. Department of Energy (DOE). The LISF, located on the Brookhaven National Laboratory site, began delivering power to the LIPA grid in November 2011, and is currently one of the largest solar photovoltaic power plants in the eastern United States. It is generating enough renewable energy to power approximately 4,500 homes and is helping New York State meet its clean energy and carbon reduction goals. Brookhaven National Laboratory (BN)L has the LISF instrumented and is capturing solar insolation and power data that can be used for research purposes. Additional information on LISF is available here: https://www.bnl.gov/lisf/. A network of nine high-definition (HD cameras) and 32 pyranometers are deployed within the LISF for the purpose of monitoring the location and characteristics of clouds and available global horizontal irradiance at the surface across the region. Information collected by this network every 30 s is currently used in BNL’s solar NowCasting algorithm, which forecasts near-term solar energy availability accounting for the behavior of clouds. The most important parameters for forecasting the near-term solar energy availability are accurate estimations of their horizontal extend, horizontal motion, and cloud base height. Currently, images from nearly located cameras are used to estimate the cloud base height. In order to evaluate the potential of this method for estimating the cloud base height, we decided to deploy a Vaisala ceilometer at the LISF.
14 SOLAR ENERGY↗