Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

A model to evaluate 100-year energy mix scenarios to facilitate deep decarbonization in the southeastern United States

The Southeast United States consumes approximately one billion megawatt-hours of electricity annually; roughly two-thirds from carbon dioxide (CO 2 ) emitting sources. The balance is produced by non-CO 2 emitting sources: nuclear power, hydroelectric power, and other renewables. Approximately 40% of the total CO 2 emissions come from the electric grid. The CO 2 emitting sources, coal, natural gas, and petroleum, produce approximately 372 million metric tons of CO 2 annually. The rest is divided between the transportation sector (36%), the industrial sector (20%), the residential sector (3%), and the commercial sector (2%). An Energy Mix Modeling Analysis (EMMA) tool was developed to evaluate 100-year energy mix strategies to reduce CO 2 emissions in the southeast. Current energy sector data was gathered and used to establish a 2016 reference baseline. The spreadsheet-based calculation runs 100-year scenarios based on current nuclear plant expiration dates, assumed electrical demand changes from the grid, assumed renewable power increases and efficiency gains, and assumed rates of reducing coal generation and deployment of new nuclear reactors. Within the model, natural gas electrical generation is calculated to meet any demand not met by other sources. Thus, natural gas is viewed as a transitional energy source that produces less CO 2 than coal until non-CO 2 emitting sources can be brought online. The annual production of CO 2 and spent nuclear fuel and the natural gas consumed are calculated and summed. A progression of eight preliminary scenarios show that nuclear power can substantially reduce or eliminate demand for natural gas within 100 years if it is added at a rate of only 1000 MWe per year. Any increases in renewable energy or efficiency gains can offset the need for nuclear power. However, using nuclear power to reduce CO 2 will result in significantly more spent fuel. More efficient advanced reactors can only marginally reduce the amount of spent fuel generated in the next 100 years if they are assumed to be available beginning around 2040. Thus closing the nuclear fuel cycle to reduce nuclear spent fuel inventories should be considered. Future work includes the incorporation of economic features into the model and the extension of the evaluation to the industrial sector. It will also be necessary to identify suitable sites for additional reactors.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

An evaluation of the variable-resolution CESM for modeling California's climate

In this paper, the recently developed variable-resolution option within the Community Earth System Model (VR-CESM) is assessed for long-term regional climate modeling of California at 0.25° (~ 28 km) and 0.125° (~ 14 km) horizontal resolutions. The mean climatology of near-surface temperature and precipitation is analyzed and contrasted with reanalysis, gridded observational data sets, and a traditional regional climate model (RCM)—the Weather Research and Forecasting (WRF) model. Statistical metrics for model evaluation and tests for differential significance have been extensively applied. VR-CESM tended to produce a warmer summer (by about 1–3°C) and overestimated overall winter precipitation (about 25%–35%) compared to reference data sets when sea surface temperatures were prescribed. Increasing resolution from 0.25° to 0.125° did not produce a statistically significant improvement in the model results. By comparison, the analogous WRF climatology (constrained laterally and at the sea surface by ERA-Interim reanalysis) was ~1–3°C colder than the reference data sets, underestimated precipitation by ~20%–30% at 27 km resolution, and overestimated precipitation by ~ 65–85% at 9 km. Overall, VR-CESM produced comparable statistical biases to WRF in key climatological quantities. Moreover, this assessment highlights the value of variable-resolution global climate models (VRGCMs) in capturing fine-scale atmospheric processes, projecting future regional climate, and addressing the computational expense of uniform-resolution global climate models.

54 ENVIRONMENTAL SCIENCES↗

A model-data intercomparison of CO2 exchange across North America: Results from the North American Carbon Program site synthesis

Our current understanding of terrestrial carbon processes is represented in various models used to integrate and scale measurements of CO2 exchange from remote sensing and other spatiotemporal data. Yet assessments are rarely conducted to determine how well models simulate carbon processes across vegetation types and environmental conditions. Using standardized data from the North American Carbon Program we compare observed and simulated monthly CO2 exchange from 44 eddy covariance flux towers in North America and 22 terrestrial biosphere models. The analysis period spans 220 site-years, 10 biomes, and includes two large-scale drought events, providing a natural experiment to evaluate model skill as a function of drought and seasonality. We evaluate models' ability to simulate the seasonal cycle of CO2 exchange using multiple model skill metrics and analyze links between model characteristics, site history, and model skill. Overall model performance was poor; the difference between observations and simulations was 10 times observational uncertainty, with forested ecosystems better predicted than nonforested. Model-data agreement was highest in summer and in temperate evergreen forests. In contrast, model performance declined in spring and fall, especially in ecosystems with large deciduous components, and in dry periods during the growing season. Models used across multiple biomes and sites, the mean model ensemble, and a model using assimilated parameter values showed high consistency with observations. Models with the highest skill across all biomes all used prescribed canopy phenology, calculated NEE as the difference between GPP and ecosystem respiration, and did not use a daily time step.

0793↗

Review of Models to Evaluate and Guide the Development of Low-Thermal-Conductivity Materials

Developing insulation materials with low–thermal conductivity values is appealing for building envelope retrofits and new construction. Therefore, it is a priority research area for the US Department of Energy Building Technologies Office (BTO). Multiple institutions are exploring various methods to develop new insulation materials with a thermal resistivity ≥97 m·K/W (14 h·ft2·°F/Btu/inch). Current materials that provide such high thermal resistivity use micro- or nanoporous structures, as well as low–thermal conductivity gases and reduced pressure. Efforts to develop these new materials would benefit greatly from a tool that can numerically evaluate various options to optimize material design and achieve the desired thermal resistivity. The available models do not capture all modes of heat transfer through the complex structures required to achieve very high thermal resistivity. This paper presents a literature review of the existing models that have been used in the study of heat transfer mechanisms of insulation materials such as aerogels, foams, and fibrous materials. The review shows most of the models make some assumptions to simplify the analysis. Further, such models have not included important factors, such as the size effect and the interfacial effect, which could significantly affect heat transfer through insulation materials. Moreover, there is a lack of a database for the thermal accommodation coefficients of various insulation materials that are used to calculate gaseous thermal conductivity. In addition, this paper presents details regarding the development of new models. The new models will capture various factors that have been neglected in existing models (such as nanoscale heat transfer, interfacial thermal resistance). They will allow parametric studies that identify optimum design options for insulation materials and thus accelerate the achievement of BTO’s goal of developing insulation technologies with thermal resistivity greater than ≥97 m·K/W (14 h·ft2·°F/Btu/inch).

Shrestha, Som↗

Improved Geothermometry Through Multivariate Reaction-path Modeling and Evaluation of Geomicrobiological Influences on Geochemical Temperature Indicators: Final Report

The project was aimed at demonstrating that the geothermometric predictions can be improved through the application of multi-element reaction path modeling that accounts for lithologic and tectonic settings, while also accounting for biological influences on geochemical temperature indicators. The limited utilization of chemical signatures by individual traditional geothermometer in the development of reservoir temperature estimates may have been constraining their reliability for evaluation of potential geothermal resources. This project, however, was intended to build a geothermometry tool which can integrate multi-component reaction path modeling with process-optimization capability that can be applied to dilute, low-temperature water samples to consistently predict reservoir temperature within ±30 °C. The project was also intended to evaluate the extent to which microbiological processes can modulate the geochemical signals in some thermal waters and influence the geothermometric predictions.

15 GEOTHERMAL ENERGY↗

A model-data intercomparison of CO2 exchange across North America: Results from the North American Carbon Program Site Synthesis

Our current understanding of terrestrial carbon processes is represented in various models used to integrate and scale measurements of CO{sub 2} exchange from remote sensing and other spatiotemporal data. Yet assessments are rarely conducted to determine how well models simulate carbon processes across vegetation types and environmental conditions. Using standardized data from the North American Carbon Program we compare observed and simulated monthly CO{sub 2} exchange from 44 eddy covariance flux towers in North America and 22 terrestrial biosphere models. The analysis period spans {approx}220 site-years, 10 biomes, and includes two large-scale drought events, providing a natural experiment to evaluate model skill as a function of drought and seasonality. We evaluate models' ability to simulate the seasonal cycle of CO{sub 2} exchange using multiple model skill metrics and analyze links between model characteristics, site history, and model skill. Overall model performance was poor; the difference between observations and simulations was {approx}10 times observational uncertainty, with forested ecosystems better predicted than nonforested. Model-data agreement was highest in summer and in temperate evergreen forests. In contrast, model performance declined in spring and fall, especially in ecosystems with large deciduous components, and in dry periods during the growing season. Models used across multiple biomes and sites, the mean model ensemble, and a model using assimilated parameter values showed high consistency with observations. Models with the highest skill across all biomes all used prescribed canopy phenology, calculated NEE as the difference between GPP and ecosystem respiration, and did not use a daily time step.

54 ENVIRONMENTAL SCIENCES↗

A parameterized ray-tracing model for evaluation of hemispherical lensing of a uniform, telecentric object aperture -- performance improvement estimates for segmented scintillators with miniature lens arrays

A parametric model describing the collimation of radiation from a uniform circular aperture by a hemispherical lens of the same radius has been developed for rapid estimation of the potential benefit of miniature dielectric lens arrays in lens-coupled gamma ray cameras (GRCs) with segmented scintillators. A simplified ray-tracing approach is adopted that makes use of small angle approximations relevant to GRCs used in nuclear science at Los Alamos National Laboratory and elsewhere. Application of this analysis technique in a first-pass design optimization process results in the optimistic estimate that miniature lenses fabricated from typical optical quality glass might boost optical coupling by as much as a factor of 4 (6 dB gain). However, this is not likely to be economically attractive compared to other techniques, such as reduction of the GRC lens f-number.

47 OTHER INSTRUMENTATION↗

Watershed and Hydrodynamic Modeling for Evaluating the Impact of Land Use Change on Submerged Aquatic Vegetation and Seagrasses in Mobile Bay

There is a continued need to understand how human activities along the northern Gulf of Mexico coast are impacting the natural ecosystems. The gulf coast is experiencing rapid population growth and associated land cover/land use change. Mobile Bay, AL is a designated pilot region of the Gulf of Mexico Alliance (GOMA) and is the focus area of many current NASA and NOAA studies, for example. This is a critical region, both ecologically and economically to the entire United States because it has the fourth largest freshwater inflow in the continental USA, is a vital nursery habitat for commercially and recreational important fisheries, and houses a working waterfront and port that is expanding. Watershed and hydrodynamic modeling has been performed for Mobile Bay to evaluate the impact of land use change in Mobile and Baldwin counties on the aquatic ecosystem. Watershed modeling using the Loading Simulation Package in C++ (LSPC) was performed for all watersheds contiguous to Mobile Bay for land use Scenarios in 1948, 1992, 2001, and 2030. The Prescott Spatial Growth Model was used to project the 2030 land use scenario based on observed trends. All land use scenarios were developed to a common land classification system developed by merging the 1992 and 2001 National Land Cover Data (NLCD). The LSPC model output provides changes in flow, temperature, sediments and general water quality for 22 discharge points into the Bay. These results were inputted in the Environmental Fluid Dynamics Computer Code (EFDC) hydrodynamic model to generate data on changes in temperature, salinity, and sediment concentrations on a grid with four vertical profiles throughout the Bay s aquatic ecosystems. The models were calibrated using in-situ data collected at sampling stations in and around Mobile bay. This phase of the project has focused on sediment modeling because of its significant influence on light attenuation which is a critical factor in the health of submerged aquatic vegetation. The impact of land use change on sediment concentrations was evaluated by analyzing the LSPC and EFDC sediment simulations for the four land use scenarios. Such analysis was also performed for storm and non-storm periods. In- situ data of total suspended sediments (TSS) and light attenuation were used to develop a regression model to estimate light attenuation from TSS. This regression model was used to derive marine light attenuation estimates throughout Mobile bay using the EFDC TSS outputs. The changes in sediment concentrations and associated impact on light attenuation in the aquatic ecosystem were used to perform an ecological analysis to evaluate the impact on seagreasses and Submerged Aquatic Vegetation (SAV) habitat. This is the key product benefiting the Mobile Bay coastal environmental managers that integrates the influences of sediments due to land use driven flow changes with the restoration potential of SAVs.

Estes, Maurice G.↗

Evaluating SWAT + model uncertainties for human and natural outcomes: Application in a Great Lakes agricultural watershed

Nutrient exports from agricultural lands in the Great Lakes Region pose significant threats to water quality and ecological health through eutrophication, hypoxia, and harmful algal blooms. Climate change and agricultural adaptation practices complicate future nutrient loading due to intensified hydrologic cycles and land use decisions. Our research focuses on evaluating the Soil and Water Assessment Tool (SWAT) plus model parametric uncertainties for human and natural outcomes across different scales. These factors are integral to ensuring a balance between productive agricultural practices and maintaining the health of watershed hydrology. However, uncertainties in modeling such complex interactions pose significant challenges, limiting our ability to precisely determine critical factors that influence crop yield and soil moisture. Our analysis employs Sobol global sensitivity analysis to evaluate first-order, second order, and total-order indices for SWAT crop growth parameter, ensuring comprehensive assessment of individual and interactive effects on model outputs. The objective is to identify the parameters that significantly affect model outputs for crop yield and soil moisture and improve our understanding of their interactions at the basin and hydrological response unit (HRU) scale. Our case study, the Portage River Watershed, which drains into Lake Erie, is chosen to better capture finer scale interactions crucial for predicting nutrient loading under future climate scenarios. This foundational work is aimed at setting the stage for the future development of an agent-based model (ABM). The ABM model would incorporate SWAT outputs to dynamically simulate decision-making processes.

Bunyon, Enock↗

Ad Hoc modeling, expert problem solving, and R&T program evaluation

A simplified cost and time (SCAT) analysis program utilizing personal-computer technology is presented and demonstrated in the case of the NASA-Goddard end-to-end data system. The difficulties encountered in implementing complex program-selection and evaluation models in the research and technology field are outlined. The prototype SCAT system described here is designed to allow user-friendly ad hoc modeling in real time and at low cost. A worksheet constructed on the computer screen displays the critical parameters and shows how each is affected when one is altered experimentally. In the NASA case, satellite data-output and control requirements, ground-facility data-handling capabilities, and project priorities are intricately interrelated. Scenario studies of the effects of spacecraft phaseout or new spacecraft on throughput and delay parameters are shown. The use of a network of personal computers for higher-level coordination of decision-making processes is suggested, as a complement or alternative to complex large-scale modeling.

Silverman, B. G.↗

Development of aerosol activation in the double-moment Unified Model and evaluation with CLARIFY measurements

Representing the number and mass of cloud and aerosol particles independently in a climate, weather prediction or air quality model is important in order to simulate aerosol direct and indirect effects on radiation balance. Here we introduce the first configuration of the UK Met Office Unified Model in which both cloud and aerosol particles have “double-moment” representations with prognostic number and mass. The GLObal Model of Aerosol Processes (GLOMAP) aerosol microphysics scheme, already used in the Hadley Centre Global Environmental Model version 3 (HadGEM3) climate configuration, is coupled to the Cloud AeroSol Interacting Microphysics (CASIM) cloud microphysics scheme. We demonstrate the performance of the new configuration in high-resolution simulations of a case study defined from the CLARIFY aircraft campaign in 2017 near Ascension Island in the tropical southern Atlantic. We improve the physical basis of the activation scheme by representing the effect of existing cloud droplets on the activation of new aerosol, and we also discuss the effect of unresolved vertical velocities. We show that neglect of these two competing effects in previous studies led to compensating errors but realistic droplet concentrations. While these changes lead only to a modest improvement in model performance, they reinforce our confidence in the ability of the model microphysics code to simulate the aerosol–cloud microphysical interactions it was designed to represent. Capturing these interactions accurately is critical to simulating aerosol effects on climate.

54 ENVIRONMENTAL SCIENCES↗

Toward a chemical reanalysis in a coupled chemistry-climate model: An evaluation of MOPITT CO assimilation and its impact on tropospheric composition

Here, we examine in detail a 1 year global reanalysis of carbon monoxide (CO) that is based on joint assimilation of conventional meteorological observations and Measurement of Pollution in The Troposphere (MOPITT) multispectral CO retrievals in the Community Earth System Model (CESM). Our focus is to assess the impact to the chemical system when CO distribution is constrained in a coupled full chemistry-climate model like CESM. To do this, we first evaluate the joint reanalysis (MOPITT Reanalysis) against four sets of independent observations and compare its performance against a reanalysis with no MOPITT assimilation (Control Run). We then investigate the CO burden and chemical response with the aid of tagged sectoral CO tracers. We estimate the total tropospheric CO burden in 2002 (from ensemble mean and spread) to be 371 ± 12% Tg for MOPITT Reanalysis and 291 ± 9% Tg for Control Run. Our multispecies analysis of this difference suggests that (a) direct emissions of CO and hydrocarbons are too low in the inventory used in this study and (b) chemical oxidation, transport, and deposition processes are not accurately and consistently represented in the model. Increases in CO led to net reduction of OH and subsequent longer lifetime of CH 4 (Control Run: 8.7 years versus MOPITT Reanalysis: 9.3 years). Yet at the same time, this increase led to 5-10% enhancement of Northern Hemisphere O 3 and overall photochemical activity via HO x recycling. Such nonlinear effects further complicate the attribution to uncertainties in direct emissions alone. This has implications to chemistry-climate modeling and inversion studies of longer-lived species.

54 ENVIRONMENTAL SCIENCES↗

An Overview of Atmospheric Chemistry and Air Quality Modeling

This presentation will include my personal research experience and an overview of atmospheric chemistry and air quality modeling to the participants of the NASA Student Airborne Research Program (SARP 2017). The presentation will also provide examples on ways to apply airborne observations for chemical transport (CTM) and air quality (AQ) model evaluation. CTM and AQ models are important tools in understanding tropospheric-stratospheric composition, atmospheric chemistry processes, meteorology, and air quality. This presentation will focus on how NASA scientist currently apply CTM and AQ models to better understand these topics. Finally, the importance of airborne observation in evaluating these topics and how in situ and remote sensing observations can be used to evaluate and improve CTM and AQ model predictions will be highlighted.

Atmospheric↗

Modern sensitivity analysis - 15651

Sensitivity analysis has been applied to complex models, such as performance assessment models, since at least the advent of computational tools to support such an approach to model evaluation. Initially, sensitivity analysis methods were applied to deterministic models and focused on changing one input variable at a time. Probabilistic modeling opened the door to evaluating changes in all input variables simultaneously. Standardized rank regression methods were primarily used in the last century to evaluate probabilistic models. These methods assume linearity in the rank response of the model to the inputs. At the very least this assumes monotonicity in the response, but for the most part the response needs to be near-linear. This is highly unlikely for performance assessment models that are based on differential equations, multiplication and division of variables that are usually not correlated, and conservatism that is not well understood. There are better statistical methods for evaluating highly non-linear and dynamic probabilistic models. Some of these methods were founded in the data mining industry. This century has seen a rapid development of the internet that led directly to the development of data mining techniques for massive databases. Although the goals were different, these techniques can be adapted to sensitivity analysis for complex probabilistic models, which, also, form large, or even massive, databases. These modern sensitivity analysis procedures assess the importance of every explanatory variable (input parameters) used in a probabilistic model. In practice, this means that for each endpoint of interest, every explanatory variable in the model is associated with a quantitative measure of importance. For a given explanatory variable, the importance value depends on the endpoint of interest. This global approach to sensitivity analysis allows all levels of interactions to be evaluated, so that the variation in the output can be measured at every level of possible interaction. The effects are collected together during sensitivity analysis processing, and then separated out to attribute the overall contribution of each input parameter to the model output. These global sensitivity analysis procedures have the ability to characterize non-linear and non-monotonic relationships between explanatory variables and endpoints of interest. This is critically important because of the need to characterize complex interactions among multiple explanatory variables in PA models that often have non-linear and non-monotonic effects. The options for sensitivity analysis are fully explored, including an evaluation of several methods when applying them to a well-known complex non-linear non-monotonic model. The results clearly demonstrate the relative benefit of using data mining methods over simple statistical approaches such as standardized rank regression. (authors)

97 MATHEMATICS AND COMPUTING↗

Evaluation and Improvement of Cloud and Convective Parameterizations from Analyses of ARM Observations and Models

Over this period the PI and his performed a broad range of data analysis, model evaluation, and model improvement studies using ARM data. These included cloud regimes in the TWP and their evolution over the MJO; M-PACE IOP SCM-CRM intercomparisons; simulations of convective updraft strength and depth during TWP-ICE; evaluation of convective entrainment parameterizations using TWP-ICE simulations; evaluation of GISS GCM cloud behavior vs. long-term SGP cloud statistics; classification of aerosol semi-direct effects on cloud cover; depolarization lidar constraints on cloud phase; preferred states of the winter Arctic atmosphere, surface, and sub-surface; sensitivity of convection to tropospheric humidity; constraints on the parameterization of mesoscale organization from TWP-ICE WRF simulations; updraft and downdraft properties in TWP-ICE simulated convection; insights from long-term ARM records at Manus and Nauru.

54 ENVIRONMENTAL SCIENCES↗

Integrated Modeling Driven Evaluation of Opportunities for Climate‐Resilient Perennial Biomass Crop Plantings in Flood‐Prone Agricultural Landscapes

Adapting to future climate change in flood-prone landscapes will require climate-resilient agricultural systems. Planting perennial crops, like switchgrass and willow, along river corridors can mitigate future flooding while supporting bioenergy markets. We developed an integrated assessment linking climate, hydrologic, and inundation model results to assess future flood risk to river-adjacent agricultural lands in the Mid-Atlantic Region (MAR) and explore this opportunity. We produced ensemble streamflow projections for every MAR stream using a hydrologic model driven by a suite of downscaled and bias-corrected Coupled Model Intercomparison Project Phase 6 climate projections. We then conducted high-resolution inundation mapping based on projected flood frequencies for baseline and future periods. Results show that in the near-term future, at least two-thirds of the streams will experience 100-year floods more severe than the baseline 200-year floods. Riparian zones are projected to face a median rise of inundation by 9.5%–24.1%. Results show that there is an opportunity to mitigate flooding in over half of MAR's counties with the quantities of switchgrass and willow plantings anticipated for mature bioenergy markets, even under the most extreme (200-year) flood events. Our integrated modeling framework can guide similar regions to evaluate opportunities for flood-resilient agricultural systems under climate change.

60 APPLIED LIFE SCIENCES↗

Computer simulation incorporating a helicopter model for evaluation of aircraft avionics systems

A computer program was developed to integrate avionics research in navigation, guidance, controls, and displays with a realistic aircraft model. A user oriented program is described that allows a flexible combination of user supplied models to perform research in any avionics area. A preprocessor technique for selecting various models without significantly changing the memory storage is included. Also included are mathematical models for several avionics error models and for the CH-47 helicopter used in this program.

Ostroff, A. J.↗