Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model skill”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS1997

Distortion Representation of Forecast Errors for Model Skill Assessment and Objective Analysis

We proposed a novel characterization of errors for numerical weather predictions. In its simplest form we decompose the error into a part attributable to phase errors and a remainder. The phase error is represented in the same fashion as a velocity field and is required to vary slowly and smoothly with position. A general distortion representation allows for the displacement and amplification or bias correction of forecast anomalies. Characterizing and decomposing forecast error in this way has two important applications, which we term the assessment application and the objective analysis application. For the assessment application, our approach results in new objective measures of forecast skill which are more in line with subjective measures of forecast skill and which are useful in validating models and diagnosing their shortcomings. With regard to the objective analysis application, meteorological analysis schemes balance forecast error and observational error to obtain an optimal analysis. Presently, representations of the error covariance matrix used to measure the forecast error are severely limited. For the objective analysis application our approach will improve analyses by providing a more realistic measure of the forecast error. We expect, a priori, that our approach should greatly improve the utility of remotely sensed data which have relatively high horizontal resolution, but which are indirectly related to the conventional atmospheric variables. In this project, we are initially focusing on the assessment application, restricted to a realistic but univariate 2-dimensional situation. Specifically, we study the forecast errors of the sea level pressure (SLP) and 500 hPa geopotential height fields for forecasts of the short and medium range. Since the forecasts are generated by the GEOS (Goddard Earth Observing System) data assimilation system with and without ERS 1 scatterometer data, these preliminary studies serve several purposes. They (1) provide a testbed for the use of the distortion representation of forecast errors, (2) act as one means of validating the GEOS data assimilation system and (3) help to describe the impact of the ERS 1 scatterometer data.

Hoffman, Ross N.↗
NASA NTRS1998

Distortion Representation of Forecast Errors for Model Skill Assessment and Objective Analysis

We proposed a novel characterization of errors for numerical weather predictions. A general distortion representation allows for the displacement and amplification or bias correction of forecast anomalies. Characterizing and decomposing forecast error in this way has several important applications, including the model assessment application and the objective analysis application. In this project, we have focused on the assessment application, restricted to a realistic but univariate 2-dimensional situation. Specifically, we study the forecast errors of the sea level pressure (SLP), the 500 hPa geopotential height, and the 315 K potential vorticity fields for forecasts of the short and medium range. The forecasts are generated by the Goddard Earth Observing System (GEOS) data assimilation system with and without ERS-1 scatterometer data. A great deal of novel work has been accomplished under the current contract. In broad terms, we have developed and tested an efficient algorithm for determining distortions. The algorithm and constraints are now ready for application to larger data sets to be used to determine the statistics of the distortion as outlined above, and to be applied in data analysis by using GEOS water vapor imagery to correct short-term forecast errors.

Hoffman, Ross N.↗
NASA NTRS2001

Distortion Representation of Forecast Errors for Model Skill Assessment and Objective Analysis

We completed the formulation of the smoothness penalty functional this past quarter. We used a simplified procedure for estimating the statistics of the FCA solution spectral coefficients from the results of the unconstrained, low-truncation FCA (stopping criterion) solutions. During the current reporting period we have completed the calculation of GEOS-2 model-equivalent brightness temperatures for the 6.7 micron and 11 micron window channels used in the GOES imagery for all 10 cases from August 1999. These were simulated using the AER-developed Optimal Spectral Sampling (OSS) model.

Hoffman, Ross N.↗
NASA NTRS2023

Assimilation of Blended Satellite Soil Moisture Data Products to Further Improve Noah-MP Model Skills

Microwave satellite remote sensing has enabled observations of soil moisture (SM) at the global scale, and multiple SM data products have been developed in the past decades. However, single-sensor-based measurements are insufficient for continuous spatiotemporal coverage. In the context of its climate program, the Climate Change Initiative, the European Space Agency (ESA) has developed robust, long term, global scale, multi instrument satellite derived time series of climate data record for key component of the climate system, including soil moisture (CCI), while the Soil Moisture Operational Product System (SMOPS) was specifically developed by National Oceanic and Atmospheric Administration (NOAA) to offer the real time blended SM datasets through merging all available individual products. Before combining, all individual SM data ingested into both SMOPS and CCI blended products are scaled to Global Land Data Assimilation System (GLDAS) 0-10 cm SM climatology. Benefiting from land surface model evolution and the availability of high-quality forcing data, GLDAS has become more comprehensive to track SM changes and dynamic trends. The development of GLDAS and the scaling procedure in CCI and SMOPS leave an open scientific and operational question: do the blended satellite SM data products have added value comparing to the GLDAS product? This study clearly reveals that both CCI and SMOPS can provide the reliable SM observations with independent information, although their climatology matches well with GLDAS. Relative to assimilation of GLDAS 0-10 cm SM data, Noah-MP model can be further improved by assimilating the blended satellite SM observations with respect to the quality-controlled in situ measurements. The strong consistency of results presented in this paper proves that the blended satellite SM data products are more useful than the GLDAS product in terms of improving Noah-MP model performance.

Jifu Yin↗
NASA NTRS1996

Distortion Representation of Forecast Errors for Model Skill Assessment and Objective Analysis

We study a novel characterization of errors for numerical weather predictions. In its simplest form we decompose the error into a part attributable to phase errors and a remainder. The phase error is represented in the same fashion as a velocity field and will be required to vary slowly and smoothly with position. A general distortion representation allows for the displacement and a bias correction of forecast anomalies. In brief, the distortion is determined by minimizing the objective function by varying the displacement and bias correction fields. In the present project we use a global or hemispheric domain, and spherical harmonics to represent these fields. In this project we are initially focusing on the assessment application, restricted to a realistic but univariate 2-dimensional situation. Specifically we study the forecast errors of the 500 hPa geopotential height field for forecasts of the short and medium range. The forecasts are those of the Goddard Earth Observing System data assimilation system. Results presented show that the methodology works, that a large part of the total error may be explained by a distortion limited to triangular truncation at wavenumber 10, and that the remaining residual error contains mostly small spatial scales.

Hoffman, Ross N.↗
NASA NTRS2013

Modeling the Skills and Practices of Scientists through an “All-Inclusive” Comparative Planetology Student Research Investigation

To effectively prepare the nation's future Science, Technology, Engineering, and Mathematics (STEM) workforce, students in today's classrooms need opportunities to engage in authentic experiences that model skills and practices used by STEM professionals. Relevant, real-world authentic research experiences allow students to behave as scientists as they model the process of science. This enables students to get a true sense of STEM-related professions and also allows them to develop the requisite knowledge, skills, curiosity, and creativity necessary for success in STEM careers. Providing professional development and opportunities to help teachers infuse research in the classroom is one of the primary goals of the Expedition Earth and Beyond (EEAB) program. EEAB, facilitated by the Astromaterials Research and Exploration Science (ARES) Directorate at the NASA Johnson Space Center, is an Earth and planetary science education program designed to inspire, engage, and educate teachers and students in grades 5-12 by getting them actively involved with exploration, discovery, and the process of science. The program combines the expertise of scientists and educators to ensure the professional development provided to classroom teachers is scientifically valid and also recognizes classroom constraints. For many teachers, facilitating research in the classroom can be challenging. In addition to addressing required academic standards and dealing with time constraints, challenges include structuring a research investigation the entire class can successfully complete. To build educator confidence, foster positive classroom research experiences, and enable teachers to help students model the skills and practices of scientists, EEAB has created an "allinclusive" comparative planetology research investigation activity. This activity addresses academic standards while recognizing students (and teachers) potentially lack experience with scientific practices involved in conducting research. Designed as an entry level research engagement investigation, the activity introduces, illustrates, and teaches the skills involved in each step of the research process. Students use astronaut photos, provided through the ARES Crew Earth Observations (CEO) payload on the International Space Station (ISS) as well as remote sensing imagery of other planetary worlds. By including all the necessary tools to complete the investigation, students can focus on gaining experience in the process of science. Additionally, students are able to extend their experience of modeling the skills and practices of scientists through the opportunity to request new data of Earth from the ISS. Professional development offered through in-person and webinar trainings, along with the resources provided, enable educators to gain first-hand experience implementing a structured research investigation in the classroom. Through data and feedback collected from teachers, this type of "all-inclusive" investigation activity aims to become a model that can be utilized for other research topics and STEM disciplines.

Graff, Paige↗
NASA NTRS1985

Planetary wave prediction: Benefits of tropical data and global models

Skillful numerical predictions of midlatitude atmospheric planetary waves generally require both tropical data for the initial conditions and a global domain for the forecast model. The lack of either adequate tropical observations or a global domain typically leads to a significant degradation of forecast skill in middle latitudes within the first one to three days of the forecast period. These effects were first discovered by numerical experimentation. They were subsequently explained theoretically, and their importance for practical forecasting was confirmed in a series of prediction experiments using FGGE data.

Somerville, R. C. J.↗
NASA NTRS2015

Model Forecast Skill and Sensitivity to Initial Conditions in the Seasonal Sea Ice Outlook

We explore the skill of predictions of September Arctic sea ice extent from dynamical models participating in the Sea Ice Outlook (SIO). Forecasts submitted in August, at roughly 2 month lead times, are skillful. However, skill is lower in forecasts submitted to SIO, which began in 2008, than in hindcasts (retrospective forecasts) of the last few decades. The multimodel mean SIO predictions offer slightly higher skill than the single-model SIO predictions, but neither beats a damped persistence forecast at longer than 2 month lead times. The models are largely unsuccessful at predicting each other, indicating a large difference in model physics and/or initial conditions. Motivated by this, we perform an initial condition sensitivity experiment with four SIO models, applying a fixed −1 m perturbation to the initial sea ice thickness. The significant range of the response among the models suggests that different model physics make a significant contribution to forecast uncertainty.

Arctic Sea↗
NASA NTRS2012

Drought Indicators Based on Model Assimilated GRACE Terrestrial Water Storage Observations

The Gravity Recovery and Climate Experiment (GRACE) twin satellites observe time variations in Earth's gravity field which yield valuable information about changes in terrestrial water storage (TWS). GRACE is characterized by low spatial (greater than 150,000 square kilometers) and temporal (greater than 10 day) resolution but has the unique ability to sense water stored at all levels (including groundwater) systematically and continuously. The GRACE Data Assimilation System (GRACE-DAS), based on the Catchment Land Surface Model (CLSM) enhances the value of the GRACE water storage data by enabling spatial and temporal downscaling and vertical decomposition into moisture 39 components (i.e. groundwater, soil moisture, snow), which individually are more useful for scientific applications. In this study, GRACE-DAS was applied to North America and GRACE-based drought indicators were developed as part of a larger effort that investigates the possibility of more comprehensive and objective identification of drought conditions by integrating spatially, temporally and vertically disaggregated GRACE data into the U.S. and North American Drought Monitors. Previously, the Drought Monitors lacked objective information on deep soil moisture and groundwater conditions, which are useful indicators of drought. Extensive datasets of groundwater storage from USGS monitoring wells and soil moisture from the Soil Climate Analysis Network (SCAN) were used to assess improvements in the hydrological modeling skill resulting from the assimilation of GRACE TWS data. The results point toward modest, but statistically significant, improvements in the hydrological modeling skill across major parts of the United States, highlighting the potential value of GRACE assimilated water storage field for improving drought detection.

Houborg, Rasmus↗
NASA NTRS1989

Sensitivity of mesoscale-model forecast skill to some initial-data characteristics, data density, data position, analysis procedure and measurement error

The effects of horizontal and vertical data resolution, data density, data location, different objective analysis algorithms, and measurement error on mesoscale-forecast accuracy are studied with observing-system simulation experiments. Domain-averaged errors are shown to generally decrease with time. It is found that the vertical distribution of error growth depends on the initial vertical distribution of the error itself. Larger gravity-inertia wave noise is produced in forecasts with coarser vertical data resolution. The use of a low vertical resolution observing system with three data levels leads to more forecast errors than moderate and high vertical resolution observing systems with 8 and 14 data levels. Also, with poor vertical resolution in soundings, the initial and forecast errors are not affected by the horizontal data resolution.

Warner, Thomas T.↗
NASA NTRSDate not supplied

Machine Learning Emulators and Empirical Models Combining Climate and Global Crop Models for Seasonal Agricultural Production

We present results from several connected efforts to apply machine learning methods to estimates of seasonal agricultural production anomalies around the world. First, we apply the XGBoost Random Forest method to fit emulators that mimic global crop models participating in the Agricultural Model Intercomparison and Improvement Project (AgMIP) Global Gridded Crop Model Intercomparison (GGCMI). These are the same models used in the agricultural sector simulations of the Inter-Sectoral Impacts Model Intercomparison Project (ISIMIP). These emulators use 8 climate variables split across 5 sub-seasonal representations of the growing season for each ½ degree grid cell around the world for maize, wheat, rice and soybeans. Emulators are useful for estimating conditions that have not already been simulated by GGCMI (e.g., in a seasonal prediction model) and also to diagnose model differences and capabilities. For example, emulators of the pDSSAT maize model tend to be more reliant on mean temperatures than the LPJmL model, and few models have strong responses to cold extremes. Second, we use a similar XGBoost approach to fit empirical models for national production data for the top 20 producing countries according to the United Nations Food and Agricultural Organization (FAO). Models utilize both climate observations and the GGCM models as predictors, resulting in skillful models for many (but not all) top producing-countries. The patterns of climate and crop model features selected indicate regions and systems that are better or worse simulated by the GGCMs. For example, information in cold extreme predictors is often combined with GGCM output predictors to provide sensitivity that models may underrepresent.

machine learning↗
NASA NTRS2017

Global Gridded Crop Model Evaluation: Benchmarking, Skills, Deficiencies and Implications.

Crop models are increasingly used to simulate crop yields at the global scale, but so far there is no general framework on how to assess model performance. Here we evaluate the simulation results of 14 global gridded crop modeling groups that have contributed historic crop yield simulations for maize, wheat, rice and soybean to the Global Gridded Crop Model Intercomparison (GGCMI) of the Agricultural Model Intercomparison and Improvement Project (AgMIP). Simulation results are compared to reference data at global, national and grid cell scales and we evaluate model performance with respect to time series correlation, spatial correlation and mean bias. We find that global gridded crop models (GGCMs) show mixed skill in reproducing time series correlations or spatial patterns at the different spatial scales. Generally, maize, wheat and soybean simulations of many GGCMs are capable of reproducing larger parts of observed temporal variability (time series correlation coefficients (r) of up to 0.888 for maize, 0.673 for wheat and 0.643 for soybean at the global scale) but rice yield variability cannot be well reproduced by most models. Yield variability can be well reproduced for most major producing countries by many GGCMs and for all countries by at least some. A comparison with gridded yield data and a statistical analysis of the effects of weather variability on yield variability shows that the ensemble of GGCMs can explain more of the yield variability than an ensemble of regression models for maize and soybean, but not for wheat and rice. We identify future research needs in global gridded crop modeling and for all individual crop modeling groups. In the absence of a purely observation-based benchmark for model evaluation, we propose that the best performing crop model per crop and region establishes the benchmark for all others, and modelers are encouraged to investigate how crop model performance can be increased. We make our evaluation system accessible to all crop modelers so that other modeling groups can also test their model performance against the reference data and the GGCMI benchmark.

wheat↗
NASA NTRS2019

Evaluation of the Impacts of Biomass Burning Aerosols on the NASA GEOS Sub-Seasonal Climate Forecasts

Emission sources of trace gases and aerosol particles in the South American (SA)and African (Af) continents have a strong seasonal and space variability associated with the extensive vegetation fires activities. In both continents, during the austral winter, the fires affect mainly tropical forest and savannah-type biomes and are mostly associated with deforestation and agricultural/pasture land management. Smoke aerosol particles, on average, contribute to at least 90% of the total aerosol optical depth (AOD) in the visible spectrum in the case of the South America regional smoke. Smoke aerosols also act as cloud condensation nuclei affecting cloud microphysics properties and therefore, changing the radiation budget, hydrological cycle and global circulation patterns over disturbed areas (Kaufman, 1995; Rosenfeld, 1999; Andreae,et al., 2004; Koren et al., 2004, Zhang, 2008; Ott et al., 2010; Randles et al., 2013). This study aims to evaluate and quantify the impact of including a comprehensive emission field of biomass burning aerosol on the performance of a seasonal climate forecast system, not only regarding the AOD itself but mainly on the meteorological state variable (e.g., precipitation and temperature). To address the questions put above, we designed two numerical experiments: 1- named"AERO_CTL" which applies the Quick Fire Emissions Dataset (QFED) emissions estimated with intra-diurnal variation (hereafter, BBE), and 2- named "AERO_CLM" where the sourcee mission is based on a climatology of the QFED emissions, with only monthly variation(hereafter, BBCLIM). Hindcast simulations were produced using the Goddard Earth ObservingSystem global circulation model, version 5, sub-seasonal to seasonal (GEOS5-S2S) system with a nominal spatial resolution of 56km (Rienecker et al., 2008). In both experiments, the aerosol feedbacks from cloud developments and radiation interactions were accounted. The two experiments consisted of 4 members each and ran from June to November spanning over the years 2000 to 2015. Model performance was evaluated by calculating statistical metrics on the mean area of SA and Af. Our results demonstrated that the skill model in predicting AOD is significantly improve when BBE source emission is applied over SA, but not over the Afcontinent. Over SA, the correlation between the AERO_CTL model configuration and MERRA-2 is 0.93 (R2= 0.86, RMS=0.02, BIAS=0.01), while the AERO_CLM model presents a value of0.81 (R2= 0.65, RMS=0.04, BIAS=0.06). However, the AERO_CTL experiment better represents the inter-annual variability of the AOS in both regions. The gain of the skill in predicting the AOD by the AERO_CTL experiment is also seen in some meteorological variables. We observed an increase in the model skill in predicting the 2-meter temperature and precipitation of up to 0.3 for the AERO_CTL experiment in comparison to the AERO_CLM. AERO_CLM. According to the analyzed hindcast, we inferred that representing the BBE more realistically implies in a significant gain of skills in the seasonal climate forecasting over SA and Af continents.

Aerosol↗
NASA NTRS2013

Interactive Ozone and Methane Chemistry in GISS-E2 Historical and Future Climate Simulations

The new generation GISS climate model includes fully interactive chemistry related to ozone in historical and future simulations, and interactive methane in future simulations. Evaluation of ozone, its tropospheric precursors, and methane shows that the model captures much of the largescale spatial structure seen in recent observations. While the model is much improved compared with the previous chemistry-climate model, especially for ozone seasonality in the stratosphere, there is still slightly too rapid stratospheric circulation, too little stratosphere-to-troposphere ozone flux in the Southern Hemisphere and an Antarctic ozone hole that is too large and persists too long. Quantitative metrics of spatial and temporal correlations with satellite datasets as well as spatial autocorrelation to examine transport and mixing are presented to document improvements in model skill and provide a benchmark for future evaluations. The difference in radiative forcing (RF) calculated using modeled tropospheric ozone versus tropospheric ozone observed by TES is only 0.016W/sq. m. Historical 20th Century simulations show a steady increase in whole atmosphere ozone RF through 1970 after which there is a decrease through 2000 due to stratospheric ozone depletion. Ozone forcing increases throughout the 21st century under RCP8.5 owing to a projected recovery of stratospheric ozone depletion and increases in methane, but decreases under RCP4.5 and 2.6 due to reductions in emissions of other ozone precursors. RF from methane is 0.05 to 0.18W/ sq. m higher in our model calculations than in the RCP RF estimates. The surface temperature response to ozone through 1970 follows the increase in forcing due to tropospheric ozone. After that time, surface temperatures decrease as ozone RF declines due to stratospheric depletion. The stratospheric ozone depletion also induces substantial changes in surface winds and the Southern Ocean circulation, which may play a role in a slightly stronger response per unit forcing during later decades. Tropical precipitation shifts south during boreal summer from 1850 to 1970, but then shifts northward from 1970 to 2000, following upper tropospheric temperature gradients more strongly than those at the surface.

Ozone↗
NASA NTRS2020

Enabling Assurance in the MBSE Environment

A number of specific benefits that fit within the hallmarks of effective development are realized with implementation of model-based approaches to systems and assurance. Model Based Systems Engineering (MBSE) enabled by standardized modeling languages (e.g., SysML®) is at the core. These benefits in the context of spaceflight system challenges can include [1]: • Improved management of complex development • Reduced risk in the development process • Improved cost management • Improved design decisions With appropriate modeling techniques the assurance community also can improve early oversight and insight into project development. NASA has shown the basic constructs of SysML in an MBSE environment offer several key advantages, within a Model Based Mission Assurance (MBMA) initiative [2, 3]. These include the following: • Model viewpoints that promote rapid and systematic assessment of requirements coverage, hazard tagging and risk management • Embedded safety assessments for launch vehicles • Deployment of model assisted development of reliability products - Failure Modes and Effects Analyses (FMEAs) and Fault Trees • Test Planning • Validation and Verification of complex functions • Support of Assurance Case development for complex systems In addition, while there are benefits to be harvested, there is a realization that these do not come without effort and cost. Enabling model-based approaches requires structure, not only in an organizational context, but in a modeling context as well. There can be a steep learning curve and costs associated to train skilled modelers. But, on the other hand, not all of the assurance community need to be modelers. Models themselves must conform to ontologies that enable assurance. This places constraints upon the models and modelers. Optimums have yet to be developed where resources and constraints on modeling must be traded off in the organization and modeling efforts for projects. A number of barriers need to be overcome, as well, which pose challenges to the developers of the software that supports MBSE/MBMA. Information and data must be made to flow seamlessly through the life cycle. Because there is a wide variety of tools used in the community, to avoid the problems of the past of silos, delays, and diverging interests, information should flow among these tools to support the “single source of truth” paradigm of MBSE. This will greatly facilitate MBMA and advancement of assurance functions.

Evans, John W.↗