Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,297 records · Page 72

Potential New Lidar Observations for Cloud Studies

The response of clouds to global warming represents a major uncertainty in estimating climate sensitivity. These uncertainties have been tracked to shallow marine clouds in the tropics and subtropics. CALIOP observations have already been used extensively to evaluate model predictions of shallow cloud fraction and top height (Leahy et al. 2013; Nam et al 2012). Tools are needed to probe the lowest levels of the troposphere. The large footprint of satellite lidars gives large multiple scattering from clouds which presents new possibilities for cloud retrievals to constrain model predictions.

Winker, Dave↗

Vertical Variation of Ice Particle Size in Convective Cloud Tops

A novel technique is used to estimate derivatives of ice effective radius with respect to height near convective cloud tops (dr(sub e)/dz) from airborne shortwave reflectance measurements and lidar. Values of dr(sub e)/dz are about −6 micrometer/km for cloud tops below the homogeneous freezing level, increasing to near 0 micrometer/km above the estimated level of neutral buoyancy. Retrieved dr(sub e)/dz compares well with previously documented remote sensing and in situ estimates. Effective radii decrease with increasing cloud top height, while cloud top extinction increases. This is consistent with weaker size sorting in high, dense cloud tops above the level of neutral buoyancy where fewer large particles are present and with stronger size sorting in lower cloud tops that are less dense. The results also confirm that cloud top trends of effective radius can generally be used as surrogates for trends with height within convective cloud tops. These results provide valuable observational targets for model evaluation.

cloud height indicators↗

Effect of Satellite Formations and Imaging Modes on Global Albedo Estimation

We confirm the applicability of using small satellite formation flight for multi-angular earth observation to retrieve global, narrow band, narrow field-of-view albedo. The value of formation flight is assessed using a coupled systems engineering and science evaluation model, driven by Model Based Systems Engineering and Observing System Simulation Experiments. Albedo errors are calculated against bi-directional reflectance data obtained from NASA airborne campaigns made by the Cloud Absorption Radiometer for the seven major surface types, binned using MODIS' land cover map - water, forest, cropland, grassland, snow, desert and cities. A full tradespace of architectures with three to eight satellites, maintainable orbits and imaging modes (collective payload pointing strategies) are assessed. For an arbitrary 4-sat formation, changing the reference, nadir-pointing satellite dynamically reduces the average albedo error to 0.003, from 0.006 found in the static reference case. Tracking pre-selected waypoints with all the satellites reduces the average error further to 0.001, allows better polar imaging and continued operations even with a broken formation. An albedo error of 0.001 translates to 1.36 W/sq m or 0.4% in Earth's outgoing radiation error. Estimation errors are found to be independent of the satellites' altitude and inclination, if the nadir-looking is changed dynamically. The formation satellites are restricted to differ in only right ascension of planes and mean anomalies within slotted bounds. Three satellites in some specific formations show average albedo errors of less than 2% with respect to airborne, ground data and seven satellites in any slotted formation outperform the monolithic error of 3.6%. In fact, the maximum possible albedo error, purely based on angular sampling, of 12% for monoliths is outperformed by a five-satellite formation in any slotted arrangement and an eight satellite formation can bring that error down four fold to 3%. More than 70% ground spot overlap between the satellites is possible with 0.5deg of pointing accuracy, 2 Km of GPS accuracy and commands uplinked once a day. The formations can be maintained at less than 1 m/s of monthly (Delta)V per satellite.

BRDF↗

Enabling Research Tools for Sustained Climate Assessment

The U.S. Global Change Research Program Sustained Assessment process benefits from long-term investments in Earth science research that enable the scientific community to conduct assessment-relevant science. To this end, NASA initiated several research programs over the past five years to support the Earth observation community in developing indicators, datasets, research products, and tools to support ongoing and future National Climate Assessments. These activities complement NASA's ongoing Earth science research programs. One aspect of the assessment portfolio funds four "enabling tools" projects at NASA research centers. Each tool leverages existing capacity within the center, but has developed tailored applications and products for National Climate Assessments. The four projects build on the capabilities of a global atmospheric reanalysis (MERRA-2), a continental U.S. land surface reanalysis (NCA-LDAS), the NASA Earth Exchange (NEX), and a Regional Climate Model Evaluation System (RCMES). Here, we provide a brief overview of each enabling tool, highlighting the ways in which it has advanced assessment science to date. We also discuss how the assessment community can access and utilize these tools for National Climate Assessments and other sustained assessment activities.

Research↗

Predicting the Operational Acceptability of Route Advisories

NASA envisions a future Air Traffic Management system that allows safe, efficient growth in global operations, enabled by increasing levels of automation and autonomy. In a safety-critical system, the introduction of increasing automation and autonomy has to be done in stages, making human-system integrated concepts critical in the foreseeable future. One example where this is relevant is for tools that generate more efficient flight routings or reroute advisories. If these routes are not operationally acceptable, they will be rejected by human operators, and the associated benefits will not be realized. Operational acceptance is therefore required to enable the increased efficiency and reduced workload benefits associated with these tools. In this paper, the authors develop a predictor of operational acceptability for reroute advisories. Such a capability has applications in tools that identify more efficient routings around weather and congestion and that better meet airline preferences. The capability is based on applying data mining techniques to flight plan amendment data reported by the Federal Aviation Administration and data on requested reroutes collected from a field trial of the NASA developed Dynamic Weather Routes tool, which advised efficient route changes to American Airlines dispatchers in 2014. 10-Fold cross validation was used for feature, model and parameter selection, while nested cross validation was used to validate the model. The model performed well in predicting controller acceptance or rejection of a route change as indicated by chosen performance metrics. Features identified as relevant to controller acceptance included the historical usage of the advised route, the location of the maneuver start point relative to the boundaries of the airspace sector containing the maneuver start (the maneuver start sector), the reroute deviation from the original flight plan, and the demand level in the maneuver start sector. A random forest with forty trees was the best performing of the five models evaluated in this paper.

route advisories↗

Understanding Arctic Surface Temperature Differences in Reanalyses

Reanalyses in the Arctic are widely used for model evaluation and for understanding contemporary climate change. Nevertheless, differences among reanalyses in fundamental meteorological variables including surface air temperature are large. A review of surface temperature differences is presented with a particular focus on differences in contemporary reanalyses. An important consideration is the significant differences in Arctic surfaces, including the central Arctic Ocean, the Greenland Ice Sheet, and non-glaciated land. While there is significant correlation among reanalyses in annual time series, there is substantial disagreement in mean values. For the period 1980-2013, the trend in annual temperature ranges from 0.3 to 0.7K per decade. Over the central Arctic Ocean, differences in mean values and trends are larger. Most of the uncertainty is associated with winter months. This is likely associated with the constraint imposed by melting processes (i.e. 0 deg. Celsius), rather than seasonal changes to the observing system.

Sea Ic↗

Predicting the Operational Acceptability of Route Advisories

NASA envisions a future Air Traffic Management system that allows safe, efficient growth in global operations, enabled by increasing levels of automation and autonomy. In a safety-critical system, the introduction of increasing automation and autonomy has to be done in stages, making human-system integrated concepts critical in the foreseeable future. One example where this is relevant is for tools that generate more efficient flight routings or reroute advisories. If these routes are not operationally acceptable, they will be rejected by human operators, and the associated benefits will not be realized. Operational acceptance is therefore required to enable the increased efficiency and reduced workload benefits associated with these tools. In this paper, the authors develop a predictor of operational acceptability for reroute advisories. Such a capability has applications in tools that identify more efficient routings around weather and congestion and that better meet airline preferences. The capability is based on applying data mining techniques to flight plan amendment data reported by the Federal Aviation Administration and data on requested reroutes collected from a field trial of the NASA developed Dynamic Weather Routes tool, which advised efficient route changes to American Airlines dispatchers in 2014. 10-Fold cross validation was used for feature, model and parameter selection, while nested cross validation was used to validate the model. The model performed well in predicting controller acceptance or rejection of a route change as indicated by chosen performance metrics. Features identified as relevant to controller acceptance included the historical usage of the advised route, the location of the maneuver start point relative to the boundaries of the airspace sector containing the maneuver start (the maneuver start sector), the reroute deviation from the original flight plan, and the demand level in the maneuver start sector. A random forest with forty trees was the best performing of the five models evaluated in this paper.

operational acceptability↗

An A-Train Climatology of Extratropical Cyclone Clouds

Extratropical cyclones (ETCs) are the main purveyors of precipitation in the mid-latitudes, especially in winter, and have a significant radiative impact through the clouds they generate. However, general circulation models (GCMs) have trouble representing precipitation and clouds in ETCs, and this might partly explain why current GCMs disagree on to the evolution of these systems in a warming climate. Collectively, the A-train observations of MODIS, CloudSat, CALIPSO, AIRS and AMSR-E have given us a unique perspective on ETCs: over the past 10 years these observations have allowed us to construct a climatology of clouds and precipitation associated with these storms. This has proved very useful for model evaluation as well in studies aimed at improving understanding of moist processes in these dynamically active conditions. Using the A-train observational suite and an objective cyclone and front identification algorithm we have constructed cyclone centric datasets that consist of an observation-based characterization of clouds and precipitation in ETCs and their sensitivity to large scale environments. In this presentation, we will summarize the advances in our knowledge of the climatological properties of cloud and precipitation in ETCs acquired with this unique dataset. In particular, we will present what we have learned about southern ocean ETCs, for which the A-train observations have filled a gap in this data sparse region. In addition, CloudSat and CALIPSO have for the first time provided information on the vertical distribution of clouds in ETCs and across warm and cold fronts. We will also discuss how these observations have helped identify key areas for improvement in moist processes in recent GCMs. Recently, we have begun to explore the interaction between aerosol and cloud cover in ETCs using MODIS, CloudSat and CALIPSO. We will show how aerosols are climatologically distributed within northern hemisphere ETCs, and how this relates to cloud cover.

clouds↗

Winter Precipitation in North America and the Pacific-North America Pattern in GEOS-S2Sv2 Seasonal Hindcast

Reliable prediction of precipitation remains one of the most pivotal and complex challenges in seasonal forecasting. Previous studies show that various large-scale climate modes, such as ENSO, PNA and NAO play significant role in winter precipitation variability over the Northern America. The influences are most pronounced in years of strong indices of such climate modes. This study evaluates model bias, predictability and forecast skills of monthly winter precipitation in GEOS5-S2S 2.0 retrospective forecast from 1981 to 2016, with emphasis on the forecast skill of precipitation over North America during the extreme events of ENSO, PNA and NAO by applying EOF and composite analysis.

Li, Zhao↗

The SUMup Dataset: Compiled Measurements of Surface Mass Balance Components over Ice Sheets and Sea Ice with Analysis over Greenland

Increasing atmospheric temperatures over ice cover affect surface processes, including melt, snowfall, and snow density. Here, we present the Surface Mass Balance and Snow on Sea Ice Working Group (SUMup) dataset, a standardized dataset of Arctic and Antarctic observations of surface mass balance components. The July 2018 SUMup dataset consists of three subdatasets, snow/firn density (https://doi.org/10.18739/A2JH3D23R), at least near-annually resolved snow accumulation on land ice (https://doi.org/10.18739/A2DR2P790), and snow depth on sea ice (https://doi.org/10.18739/A2WS8HK6X), to monitor change and improve estimates of surface mass balance. The measurements in this dataset were compiled from field notes, papers, technical reports, and digital files. SUMup is a compiled, community-based dataset that can be and has been used to evaluate modeling efforts and remote sensing retrievals. Active submission of new or past measurements is encouraged. Analysis of the dataset shows that Greenland Ice Sheet density measurements in the top 1m do not show a strong relationship with annual temperature. At Summit Station, Greenland, accumulation and surface density measurements vary seasonally with lower values during summer months. The SUMup dataset is a dynamic, living dataset that will be updated and expanded for community use as new measurements are taken and new processes are discovered and quantified.

Montgomery, Lynn↗

Overview of SatCORPS Satellite-Derived Cloud and Radiation Retrieval Coverage of ARM Domains

The ARM Climate Research Facility program maintains a number of climatically representative sites, which provide long-term cloud- and climate-monitoring records via ground-based instrumentation. These measurements provide a valuable data record over a localized region, but can be greatly enhanced by use of satellite monitoring. Satellite analyses over larger domains can provide parameters helpful for monitoring climate and evaluating models. The NASA/Langley Cloud group routinely derives such cloud and radiative parameters, from various geostationary and polar-orbiting satellite coverage over ARM sites; the group provides near-realtime analyses covering the 3 ARM fixed sites, as well as the GO-Amazon AMF site. This is accomplished by employing a suite of algorithms including VISST (Visible Infrared Solar Split-Window Technique), SIST (Solar Infrared Split-Window Technique), and SINT (Solar-infrared Infrared Near-Infrared Technique), now collectively called SatCORPS (Satellite Cloud Observations and Radiative Property retrieval System). An overview and catalog of SatCORPS-derived datasets processed for ASR, and available from both the ARM archive and the NASA/Langley Cloud group website, is provided. Specific improvements included in recently added datasets such as GO-Amazon and Azores are highlighted, including an improved cloud-detection mask, as well as improvements in derived Top-of-Atmosphere (TOA) SW albedoes and LW fluxes. New narrowband-to-broadband (NB-BB) fits and corrections for improved TOA fluxes are illustrated, including MTSAT-1/CERES Aqua NB-BB fits for the TWPICE field campaign, as well as new fits covering the Azores region which incorporate GERB TOA fluxes (Geostationery Earth Radiation Budget). Finally, validation of the reprocessed SatCORPS datasets is shown.

M M Khader↗

Earth system modeling: can we stay ahead of what's actually happening?

Over the last forty years, Earth System Models have become more and more comprehensive and detailed. They have assimilated previously independent modeling efforts in ocean circulation, atmospheric composition, the stratosphere, land surface, and the cryosphere, and are today demonstrably more skillful than earlier efforts. Yet the structural uncertainty in the latest model projections is actually increasing. Meanwhile, impacts of climate change on, for instance, wildfires, tropical cyclones and sea level, appear to be increasing faster than we have been able to predict. I will address some of the current issues at the bleeding edges of model development and the continuing importance of model evaluation.

Gavin A Schmidt↗

Unveiling Urban Pollution Patterns from a Bird’s-Eye View

Low-earth orbiting satellites have been observing the global distribution of nitrogen dioxide (NO2) since the 1990s and these observations have been used to identify pollution source regions, temporal trends, and links to health effects and community disparities. However, historically, space-based NO2 observations have been coarse in spatial and temporal resolution resulting in challenges in data interpretation and product validation; therefore, NASA developed airborne capabilities and measurements strategies to observe ultraviolet-visible (UV-VIS) trace gases at high temporal and spatial resolutions to unveiling never before seen spatiotemporal patterns over major urban hubs during recent air quality field studies. In this talk, Dr. Judd will share the story of these field studies with datasets collected from NASA’s airborne spectrometers, GeoTASO and GCAS, and how they integrate in with other space- and ground-based instrumentation to illustrate the conceptual model of air quality episodes in coastal urban environments, contribute to satellite validation and regional chemical transport model evaluation. These imagers have the capability to observe UV-VIS absorbing species, typically NO2 and HCHO, at spatial resolutions as fine as 250 m when observing from the upper-troposphere. Flight strategies allowed for the collection of repeated raster samples over urban areas multiple times per flight day in cities like Chicago, Los Angeles, and New York City, capturing never before seen pictures of the diurnal evolution of emissions, chemistry, and meteorology associated with NO2 heterogeneity. These airborne efforts help local air quality management better understand their air quality challenges as well as benefit the larger air quality community in preparation for geostationary air quality observations and the development of the global air quality observing system.

Laura Judd↗

Polar Vortex Outbreak Air Transport: Observation using Satellite IR Sounder Derived Ozone Product and Comparison with Model

The Single Field of View (SFOV) Sounder Atmospheric Products (SiFSAP) derived from Cross-track Infrared Sounder (CrIS) on SNPP and JPSS have a spatial resolution ( ~14.5 km) better than most global weather and climate models. Most recent significant improvement in the quality of these products enables us to use these high-resolution observation-based sounding data for weather studies and model evaluation. The aim of this study is to explore the value to use these SFOV products for studying the dynamic transport associated with polar vortex outbreak. In a few cases studies, we used SiFSAP products, especially ozone, as well as the data of wind, geopotential height (GPH) and potential velocity (PV) from the fifth-generation ECMWF reanalysis (ERA5) data, to analyze the variation of total ozone, relative humidity and ozone in different layers, as well as their correlation with GPH and PV. Some comparison of the retrieved temperature and water vapor with ERA5 data, and ozone with the Ozone Mapping and Profiler Suite (OMPS) measurements have also been made. It is found (1) the transport of O3-riched polar air accompanying the polar vortex breakup to lower latitude and lower atmosphere, and (2) the transport path of O3 agreeing well with the path of polar cold air (illustrated using the retrieved RH, wind fields and GPH). These results demonstrate the 3-D structure of O3 distribution as derived from CrIS measurements provides some insights of the cold air transport, and can be used to track the dynamic transport of polar cold air following the outbreak of polar vortex. The observed enhancement of O3 following the polar vortex outbreak also suggests some possible linkage between the tropospheric cold air advected with stratospheric air source.

Xiaozhen Xiong↗

Optimizing GRACE/GRACE-FO Data and A priori Hydrological Knowledge for Improved Global Terrestial Water Storage Component Estimates

The comprehensive information of global terrestrial water storage (TWS) components (soil moisture, groundwater, snow, surface water) is essential for effective assessment of water resource availability, climate variation, and disaster mitigation measures. Observational data provided by the Gravity Recovery And Climate Experiment (GRACE) and GRACE Follow-On satellite missions offer global TWS variation (ΔTWS) in terms of an integrated water column. However, GRACE spatial resolution is relatively coarse (i.e., 3°), and the vertically integrated value cannot be separated into ΔTWS components directly. This study demonstrates the feasibility to estimate ΔTWS components at any desired spatial-vertical resolution by effectively maintaining the native resolution of the employed hydrological knowledge. It utilizes a least-squares with constraints (LSC) approach to rigorously incorporate GRACE and GRACE-FO data and a priori hydrological knowledge, with the aim to improve global ΔTWS components’ accuracy and spatial resolution. The 3°×3° GRACE mascon derived ΔTWS data is disaggregated into the 0.5°×0.5° anomalous soil moisture storage (ΔSMS), groundwater storage (ΔGWS), snow water equivalent (ΔSWE), and surface water storage (ΔSWS) based on the covariance information obtained from the Community Atmosphere Biosphere Land Exchange (CABLE) and the PCRaster Global Water Balance (PCR-GLOBWB) models. Evaluation with different ground measurements and satellite products between 2002 and 2019 exhibits significantly improved accuracy in all individual ΔTWS components. This improvement is of particular note in ΔGWS and ΔSWS, where the LSC approach increases the globally averaged correlation values by approximately 0.13 and 0.05, respectively. Reliable prior knowledge leads to a more accurate ΔTWS component estimate, and the use of ensemble-mean knowledge yields the best result.

GRACE↗

Methodology Report of Cost Benefit Analysis of Space Based Solar Power

Space-based solar power (SBSP) is the concept of gathering power in space and transmitting it wirelessly to users on Earth or elsewhere in space. SBSP has seen renewed interest from governments, businesses, and universities for reasons including persistent operations in space, achieving carbon neutrality targets, and more. Several major government agencies in Europe, Asia, and the Americas have renewed their SBSP investigations, and hundreds of millions of dollars have been donated to universities to advance the technical state of the art. Given the renewed interest, this study will identify and update some key system parameters to current performance levels. This paper presents the methodology and approach to identify the fiscal and environmental costs and benefits of SBSP when compared to other sustainable and grid-scale energy sources. From these divergent angles the models evaluate fiscal and environmental impacts, identify the state of the most sensitive parameters of the designs, and seek to provide context to leadership on the relative value of pursuing SBSP related technologies.

Space-based solar power↗

Soil Carbon Losses Reduce Soil Moisture in Global Climate Model Simulations

Most agricultural soils have experienced substantial soil organic carbon losses in time. These losses motivate recent calls to restore organic carbon in agricultural lands to improve biogeochemical cycling and for climate change mitigation. Declines in organic carbon also reduce soil infiltration and water holding capacity, which may have important effects on regional hydrology and climate. To explore the regional hydroclimate impacts of soil organic carbon changes, we conduct new global climate model experiments with NASA Goddard Institute for Space Studies ModelE that include spatially explicit soil organic carbon concentrations associated with different human land management scenarios. Compared to a “no land use” case, a year 2010 soil degradation scenario, in which organic carbon content (OCC; weight %) is reduced by a factor of ∼0.12 on average across agricultural soils, resulted in soil moisture losses between 0.5 and 1 temporal standard deviations over eastern Asia, northern Europe, and the eastern United States. In a more extreme idealized scenario where OCC is reduced uniformly by 0.66 across agricultural soils, soil moisture losses exceed one standard deviation in both hemispheres. Within the model, these soil moisture declines occur primarily due to reductions in porosity (and to a lesser extent infiltration) that overall soil water holding capacity. These results demonstrate that changes in soil organic carbon can have meaningful, large-scale effects on regional hydroclimate and should be considered in climate model evaluations and developments. Further, this also suggests that soil restoration efforts targeting the carbon cycle are likely to have additional benefits for improving drought resilience.

Atmosphere-land interaction↗

Heterogeneity and Chemical Reactivity of the Remote Troposphere Defined By Aircraft Measurements - Corrected

The NASA Atmospheric Tomography (ATom) mission built a photochemical climatology of air parcels based on in situ measurements with the NASA DC-8 aircraft along objectively planned profiling transects through the middle of the Pacific and Atlantic oceans. In this paper we present and analyze a data set of 10 s (2 km) merged and gap-filled observations of the key reactive species driving the chemical budgets of O3 and CH4 (O3, CH4, CO, H2O, HCHO, H2O2, CH3OOH, C2H6, higher alkanes, alkenes, aromatics, NOx, HNO3, HNO4, peroxyacetyl nitrate, and other organic nitrates), consisting of 146 494 distinct air parcels from ATom deployments 1 through 4. Six models calculated the O3 and CH4 photochemical tendencies from this modeling data stream for ATom 1. We find that 80 %–90 % of the total reactivity lies in the top 50 % of the parcels and 25 %–35 % in the top 10 %, supporting previous model-only studies that tropospheric chemistry is driven by a fraction of all the air. Surprisingly, the probability densities of species and reactivities averaged on a model scale (100 km) differ only slightly from the 2 km ATom 10 s data, indicating that much of the heterogeneity in tropospheric chemistry can be captured with current global chemistry models. Comparing the ATom reactivities over the tropical oceans with climatological statistics from six global chemistry models, we find generally good agreement with the reactivity rates for O3 and CH4. Models distinctly underestimate O3 production below 2 km relative to the mid-troposphere, and this can be traced to lower NOx levels than observed. Attaching photochemical reactivities to measurements of chemical species allows for a richer, yet more constrained-to-what-matters, set of metrics for model evaluation.

Tropospheric Chemistry↗