Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Model evaluation/performance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

31 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Are North Atlantic Tropical Cyclones Modulated by the Madden–Julian Oscillation in HighResMIP AGCMs?

This study assesses the representation of the observed relationship between Atlantic tropical cyclones (TCs) and the Madden–Julian oscillation (MJO) across nine models participating in CMIP6 High Resolution Model Intercomparison Project (HighResMIP). Most models struggle to faithfully reproduce the observed impacts of the MJO on Atlantic TCs, with the primary issue being the underestimated TC activity over the Atlantic main development region (MDR). The negative biases in genesis frequency within the MDR can be further attributed to weaker-than-observed African easterly wave (AEW) activity south of ∼12°N. Errors in the diabatic heating profile within the Atlantic intertropical convergence zone lead to insufficient potential vorticity production in the lower troposphere and constrain the amplification of AEWs. In addition to the biased TC climatology, the eastward-propagating power of the MJO is consistently underestimated across all models. Nevertheless, in models with a higher eastward–westward power ratio, the simulated MJO demonstrates a stronger capacity to modulate subseasonal TC activity. Models with relatively realistic eastward propagation of the MJO also exhibit greater variance in tropical intraseasonal convection. Stronger contrasts in convective heating over the North Atlantic between phases 2–3 and phases 6–7 drive larger fluctuations in MDR shear and AEW activity over the Gulf of Mexico and West Africa, resulting in a more pronounced TC response to the MJO. Overall, our findings suggest that improved MDR TC climatology and MJO propagation are essential for models to accurately capture the observed modulations of Atlantic TCs by the MJO.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Performance of New Near-Real-Time PERSIANN Product (PDIR-Now) for Atmospheric River Events over the Russian River Basin, California

Most heavy precipitation events and extreme flooding over the U.S. Pacific coast can be linked to prevalent atmospheric river (AR) conditions. Thus, reliable quantitative precipitation estimation with a rich spatiotemporal resolution is vital for water management and early warning systems of flooding and landslides over these regions. At the same time, high-quality near-real-time measurements of AR precipitation remain challenging due to the complex topographic features of land surface and meteorological conditions of the region: specifically, orographic features occlude radar measurements while infrared-based algorithms face challenges, differentiating between both cold brightband (BB) precipitation and the warmer nonbrightband (NBB) precipitation. It should be noted that the latter precipitation is characterized by greater orographic enhancement. In this study, we evaluate the performance of a recently developed near-real-time satellite precipitation algorithm: Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks (PERSIANN) Dynamic Infrared–Rain Rate-Now (PDIR-Now). This model is primarily dependent on infrared information from geostationary satellites as input; consequently, PDIR-Now has the advantage of short data latency, 15–60-min delay between observation to precipitation product delivery. The performance of PDIR-Now is analyzed with a focus on AR-related events for cases dominated by NBB and BB precipitation over the Russian River basin. In our investigations, we utilize S-band (3-GHz) precipitation profilers with Joss/Parsivel disdrometer measurements at the Middletown and Santa Rosa stations to classify BB and NBB precipitation events. In general, our analysis shows that PDIR-Now is more skillful in retrieving precipitation rates over both BB and NBB events across the topologically complex study area as compared to PERSIANN-Cloud Classification System (CCS). Also, we discuss the performance of well-known operational near-real-time precipitation products from 2017 to 2019. Conventional categorical and volumetric categorical indices, as well as continuous statistical metrics, are used to show the differences between various high-resolution precipitation products such as Multi-Radar Multi-Sensor (MRMS).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Evaluation of CMIP6 Streamflow in the Arctic

Earth system models (ESMs) serve as the primary basis for projecting future streamflow changes, but they are biased in terms of the their ability to reproduce historical observations of streamflow. Rigorous downscaling and bias-correction procedures, which are time-consuming and introduce uncertainties, are commonly used to address these biases in streamflow or runoff. However, the applicable limits of streamflow projections remain largely untested through direct comparisons with observations. In this work, we compare historical streamflow time series observations with a suite of Coupled Model Intercomparison Project phase 6 (CMIP6) ESMs to assess the space and time scales at which each ESM represents historic streamflow observations. We focus our analysis on streamflow across the Arctic given the higher degree of warming experienced over this region, underscoring its importance for understanding the future changes. A series of metrics—volume, seasonality, extreme capturing streamflow variability, drought, and flooding is tested to assess ESM simulation skill for total events and overall distributions. Our results indicate that although improvements are necessary in ESM runoff and streamflow projections, the use of temporal averaging and targeting the best-fit model for comparison to observations when a suite of ESMs is considered can provide robust streamflow projections. Moreover, this work provides a basis for contextualizing the limits of ESM applicability for studies aiming to assess streamflow in the future.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Evaluation of coupled wind-wave model simulations of offshore winds in the Mid-Atlantic Bight using lidar-equipped buoys.

From 2014 to 2017, two Department of Energy buoys equipped with Doppler lidar were deployed off the U.S. East Coast to provide long term measurements of hub-height wind speed in the marine environment. In this study, we performed simulations of selected cases from the deployment using a 5-km configuration of the Weather Research and Forecasting (WRF) model, to see if simulated hub height speeds could produce closer agreement with the observations than existing reanalysis products. For each case we performed two additional simulations: one in which marine surface roughness height was one-way coupled to forecast wave parameters from a standalone WaveWatch III (WW3) simulation, and another in which WRF and WW3 were two-way coupled using the Coupled-Ocean-Atmosphere-Wave-Sediment-Transport (COAWST) framework. It was found that all the 5-km WRF simulations improved 90-m wind speed statistics for the tropical cyclone case of 08 May 2015 and the cold frontal case of 25 Mar 2016, but not the nor-easter of 18 Jan 2016. The impact of wave coupling on buoy-level (4 m) wind speed was modest and case dependent, but when present, the impact was typically seen at 90 m as well, being as large as 10% in stable conditions. One-way wave coupling consistently reduced wind speeds, improving biases for 25 Mar 2016 but worsening them for 08 May 2015. Two-way wave coupling mitigated these negative biases, improved wave field representation and statistics, and mostly improved 4-m wind field correlation coefficients, at least at the VA buoy, largely due to greater self-consistency between wind and wave fields.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Effects of Horizontal Resolution, Domain Size, Boundary Conditions, and Surface Heterogeneity on Coarse LES of a Convective Boundary Layer

Atmospheric properties in a convective boundary layer vary over a wide range of spatial scales and are commonly studied using large-eddy simulations (LES) in various configurations. We examine how the boundary layer depth and distribution of variability across scales are affected by LES grid spacing, domain size, inhomogeneity of surface properties, and external forcing. Two different setups of the Weather Research and Forecasting (WRF) model are analyzed. A semi-idealized configuration uses a periodic domain, flat surface, prescribed homogeneous surface heat fluxes, and horizontally uniform profiles of large-scale advective tendencies. A nested LES setup employs a larger domain and realistic initial and boundary conditions, including an interactive land surface model with representative topography and vegetation and soil types. Subdomains of identical size are analyzed for all simulations. Characteristic structure sizes are quantified using the variability scales L 50 and L 95 , defined such that features smaller than that contain 50% and 95% of the total variance, respectively. Progressive increase in L 50 from vertical velocity to temperature and moisture structures is systematically reproduced in all simulation configurations. This dependence of L 50 on the considered variable complicates the development of scale-aware parameterizations for models with grid spacing in the “terra incognita”. In simulations using a larger domain with heterogeneous surface properties, the development of internal mesoscale patterns significantly affects variance distributions inside analyzed subdomains. Sizes of boundary layer structures also strongly depend on the LES grid spacing and, in case of heterogeneous surface and topography, on location of the subdomain inside a larger computational domain.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Evaluation of the Fitch Wind-Farm Wake Parameterization with Large-Eddy Simulations of Wakes Using the Weather Research and Forecasting Model

Abstract Wind-farm parameterizations in weather models can be used to predict both the power output and farm effects on the flow; however, their correctness has not been thoroughly assessed. We evaluate the wind-farm parameterization of the Weather Research and Forecasting Model with large-eddy simulations (LES) of the wake performed with the same model. We study the impact on the velocity and turbulence kinetic energy (TKE) of inflow velocity, roughness, resolution, number of turbines (one or two), and inversion height and strength. We compare the mesoscale with the LES by spatially averaging the LES within areas correspondent to the mesoscale horizontal spacing: one covering the turbine area and two downwind. We find an excellent agreement of the velocity within the turbine area between the two types of simulations. However, within the same area, we find the largest TKE discrepancies because in mesoscale simulations, the turbine-added TKE has to be highest at the turbine position to be advected downwind. Within the downwind areas, differences between velocities increase as the wake recovers faster in the LES, whereas for the TKE both types of simulations show similar levels. From the various configurations, the impact of inversion height and strength is small for these heights and inversion levels. The highest impact for the one-turbine simulations appears under the low-speed case due to the higher thrust, whereas the impact of resolution is low for the large-eddy simulations but high for the mesoscale simulations. Our findings demonstrate that higher-fidelity simulations are needed to validate wind-farm parameterizations.

17 WIND ENERGY↗
DOE OSTI2023

A Moving-Wave Implementation in WRF to Study the Impact of Surface Water Waves on the Atmospheric Boundary Layer

Abstract While numerous modeling studies have focused on the interaction of ocean surface waves with the atmospheric boundary layer, most employ idealized waves that are either monochromatic or synthetically generated from a theoretical wave spectrum, and the atmospheric solvers are typically incompressible. To study wind–wave coupling in real-world scenarios, a model that can simulate both realistic meteorological and wave conditions is necessary. In this paper we describe the implementation of a moving bottom boundary condition into the Weather Research and Forecasting Model for large-eddy simulation applications. We first describe the moving bottom boundary conditions within WRF’s pressure-based vertical coordinate system. We then validate our code with idealized test cases that have analytical solutions, including flow over a monochromatic wave with and without viscosity. Finally, we present results from turbulent flows over a moving monochromatic wave with different wave ages, and demonstrate satisfactory agreement of the wave growth rate with results from the literature. We also compare atmospheric stress and wind parameters from two physically equivalent cases. The first specifies a wind moving in the same direction as a propagating wave, while the second involves a stationary wave with the wind adjusted such that the wind relative to the wave is the same as in the first case. Results indicate that the velocity and Reynolds stress profiles for the two cases match, further validating the moving bottom implementation.

17 WIND ENERGY↗
DOE OSTI2023

CFL Optimized Forward–Backward Runge–Kutta Schemes for the Shallow-Water Equations

Abstract We present the formulation and optimization of a Runge–Kutta-type time-stepping scheme for solving the shallow-water equations, aimed at substantially increasing the effective allowable time step over that of comparable methods. This scheme, called FB-RK(3,2), uses weighted forward–backward averaging of thickness data to advance the momentum equation. The weights for this averaging are chosen with an optimization process that employs a von Neumann–type analysis, ensuring that the weights maximize the admittable Courant number. Through a simplified local truncation error analysis and numerical experiments, we show that the method is at least second-order in time for any choice of weights and exhibits low dispersion and dissipation errors for well-resolved waves. Further, we show that an optimized FB-RK(3,2) can take time steps up to 2.8 times as large as a popular three-stage, third-order strong stability-preserving Runge–Kutta method in a quasi-linear test case. In fully nonlinear shallow-water test cases relevant to oceanic and atmospheric flows, FB-RK(3,2) outperforms SSPRK3 in admittable time step by factors roughly between 1.6 and 2.2, making the scheme approximately twice as computationally efficient with little to no effect on solution quality. Significance Statement The purpose of this work is to develop and optimize time-stepping schemes for models relevant to oceanic and atmospheric flows. Specifically, for the shallow-water equations we optimize for schemes that can take time steps as large as possible while retaining solution quality. We find that our optimized schemes can take time steps between 1.6 and 2.2 times larger than schemes that cost the same number of floating point operations, translating directly to a corresponding speedup. Our ultimate goal is to use these schemes in climate-scale simulations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Influences of the South American Low-Level Jet on the Convective Environment in Central Argentina Using a Convection-Permitting Simulation

This study documents the spatial and temporal distribution of the South American low-level jet (SALLJ) and quantifies its impact on the convective environment using a 6.5-month convection-permitting simulation during the Remote Sensing of Electrification, Lightning, And Mesoscale/Microscale Processes with Adaptive Ground Observations and Clouds, Aerosols, and Complex Terrain Interactions (RELAMPAGO-CACTI) campaigns. Overall, the simulation reproduces the observed SALLJ characteristics in central Argentina near the Sierras de Córdoba (SDC), a focal point for terrain-focused upscale growth. SALLJs most frequently occur in the summer with maxima to the northwest and east of the SDC and minima over the higher terrain. The shallower SALLJs (<1750 m) have a strong overnight skew, while the elevated jets are more equally spread throughout the day. SALLJ periods often have higher amounts of low-level moisture and instability compared to non-SALLJ periods, with these impacts increasing over time when the SALLJ is present and decreasing afterward. The SALLJ may enhance low-level wind shear magnitudes (particularly when accounting for the jet height); however, enhancement is somewhat limited due to the presence of speed shear in most situations. SALLJ periods are associated with low-level directional shear favorable for organized convection and an orientation of cloud-layer wind shear parallel to the terrain, which could favor upscale growth. A case study is shown in which the SALLJ influenced both the magnitude and direction of wind shear concurrent with convective upscale growth near the SDC. This study highlights the complex relationship between the SALLJ and its impacts during periods of widespread convection.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Evaluation of Model Summertime Boundary Layer Cloud Development over Complex Terrain in New York State

Weather forecasting over complex terrain with diverse land cover is challenging. Utilizing the high-resolution observations from New York State Mesonet (NYSM), we are able to evaluate the surface processes of the Weather Research Forecast (WRF) Model in a detailed, scale-dependent manner. Here, in the study, possible impacts of land–atmosphere interaction on surface meteorology and boundary layer cloud development are investigated with different model resolutions, land surface models (LSMs), and planetary boundary layer (PBL) physical parameterizations. The High-Resolution Rapid Refresh, version 3 (HRRR), forecasting model is used as a reference for the sensitivity evaluation. Results show that over complex terrain, the high-resolution simulations (1 km × 60 vertical levels) generally perform better compared to low-resolution (3 km × 50 levels) in both surface meteorology and cloud fields. LSMs play a more important role in surface meteorology compared to PBL schemes. The NoahMP land surface model exhibits daytime warmer and drier biases compared to the Rapid Update Cycle (RUC) due to better prediction of the Bowen ratio in RUC. The PBL schemes would affect the convective strength in the boundary layer. The Shin–Hong (SH) scale-aware scheme tends to produce the strongest convective strength in the PBL, while the ACM2 PBL scheme rarely resolved convection even at 1-km resolution. By considering the radiation effect of subgrid-scale (SGS) clouds, the Mellor–Yamada–Nakanishi–Niino eddy diffusivity mass flux (MYNN-EDMF) predicted the highest cloud coverage and lowest surface solar radiation bias. The configuration of SGS clouds in MYNN-EDMF would not only significantly reduce shortwave radiation bias, but also affect the convection behaviors through land surface–cloud–radiation interaction.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Model Evaluation by Measurements from Collocated Remote Sensors in Complex Terrain

Model improvement efforts involve an evaluation of changes in model skill in response to changes in model physics and parameterization. When using wind measurements from various remote sensors to determine model forecast accuracy, it is important to understand the effects of measurement-uncertainty differences among the sensors resulting from differences in the methods of measurement, the vertical and temporal resolution of the measurements, and the spatial variability of these differences. Here we quantify instrument measurement variability in 80-m wind speed during WFIP2 and its impact on the calculated errors and the change in error from one model version to another. Here the model versions tested involved updates in model physics from HRRRv1 to HRRRv4, and reductions in grid interval from 3 km to 750 m. Model errors were found to be 2–3 m s –1 . Differences in errors as determined by various instruments at each site amounted to about 10% of this value, or 0.2–0.3 m s –1 . Changes in model skill due to physics or grid-resolution updates also differed depending on the instrument used to determine the errors; most of the instrument-to-instrument differences were ~0.1 m s –1 , but some reached 0.3 m s –1 . All instruments at a given site mostly showed consistency in the sign of the change in error. In two examples, though, the sign changed, illustrating a consequence of differences in measurements: errors determined using one instrument may show improvement in model skill, whereas errors determined using another instrument may indicate degradation. This possibility underscores the importance of having accurate measurements to determine the model error.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2014

Assessment of NASA GISS CMIP5 and Post-CMIP5 Simulated Clouds and TOA Radiation Budgets Using Satellite Observations. Part I: Cloud Fraction and Properties

Although many improvements have been made in phase 5 of the Coupled Model Intercomparison Project (CMIP5), clouds remain a significant source of uncertainty in general circulation models (GCMs) because their structural and optical properties are strongly dependent upon interactions between aerosol/cloud microphysics and dynamics that are unresolved in such models. Recent changes to the planetary boundary layer (PBL) turbulence and moist convection parameterizations in the NASA GISS Model E2 atmospheric GCM(post-CMIP5, hereafter P5) have improved cloud simulations significantly compared to its CMIP5 (hereafter C5) predecessor. A study has been performed to evaluate these changes between the P5 and C5 versions of the GCM, both of which used prescribed sea surface temperatures. P5 and C5 simulated cloud fraction (CF), liquid water path (LWP), ice water path (IWP), cloud water path (CWP), precipitable water vapor (PWV), and relative humidity (RH) have been compared to multiple satellite observations including the Clouds and the Earth's Radiant Energy System-Moderate Resolution Imaging Spectroradiometer (CERES-MODIS, hereafter CM), CloudSat- Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO; hereafter CC), Atmospheric Infrared Sounder (AIRS), and Advanced Microwave Scanning Radiometer for Earth Observing System (AMSR-E). Although some improvements are observed in the P5 simulation on a global scale, large improvements have been found over the southern midlatitudes (SMLs), where correlations increased and both bias and root-mean-square error (RMSE) significantly decreased, in relation to the previous C5 simulation, when compared to observations. Changes to the PBL scheme have resulted in improved total column CFs, particularly over the SMLs where marine boundary layer (MBL) CFs have increased by nearly 20% relative to the previous C5 simulation. Globally, the P5 simulated CWPs are 25 gm22 lower than the previous C5 results. The P5 version of the GCM simulates PWV and RH higher than its C5 counterpart and agrees well with the AMSR-E and AIRS observations. The moister atmospheric conditions simulated by P5 are consistent with the CF comparison and provide a strong support for the increase in MBL clouds over the SMLs. Over the tropics, the P5 version of the GCM simulated total column CFs and CWPs are slightly lower than the previous C5 results, primarily as a result of the shallower tropical boundary layer in P5 relative to C5 in regions outside the marine stratocumulus decks.

General circulation models↗
NASA NTRS2022

Advances in the prediction of MJO-Teleconnections in the S2S forecast systems

This study evaluates the ability of state-of-the-art subseasonal to seasonal (S2S) forecasting systems to represent and predict the teleconnections of the Madden Julian Oscillations and their effects on weather in terms of midlatitude weather patterns and North Atlantic tropical cyclones. This evaluation of forecast systems applies novel diagnostics developed to track teleconnections along their preferred pathways in the troposphere and stratosphere, and to measure the global and regional responses induced by teleconnections across both the Northern and Southern Hemispheres. Results of this study will help the modeling community understand to what extent the potential to predict the weather on S2S time scales is achieved by the current generation of forecasting systems, while informing where to focus further development efforts. The findings of this study will also provide impact modelers and decision makers with a better understanding of the potential of S2S predictions related to MJO teleconnections.

Forecasting↗