Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46

Evaluation of cloud shading effects on the generation and modification of mesoscale circulations

This study presents scale analysis and numerical model evaluations of the impact of cloud shading on (1) the development of sea breeze and thermally induced upslope flows and (2) the generation of mesoscale circulations between cloudy areas adjacent to clear areas. Based on the assumption of modifications of solar and longwave radiation which are typical for some overcast conditions, it was found that the reduction in the first type of circulation is significant and most noticeable in the vertical velocities. In the second case, thermally induced circulations in favorable conditions appear to approach the typical intensity of the sea breeze.

Segal, M.↗

Studies in Software Cost Model Behavior: Do We Really Understand Cost Model Performance?

While there exists extensive literature on software cost estimation techniques, industry practice continues to rely upon standard regression-based algorithms. These software effort models are typically calibrated or tuned to local conditions using local data. This paper cautions that current approaches to model calibration often produce sub-optimal models because of the large variance problem inherent in cost data and by including far more effort multipliers than the data supports. Building optimal models requires that a wider range of models be considered while correctly calibrating these models requires rejection rules that prune variables and records and use multiple criteria for evaluating model performance. The main contribution of this paper is to document a standard method that integrates formal model identification, estimation, and validation. It also documents what we call the large variance problem that is a leading cause of cost model brittleness or instability.

data mining↗

On the Performance of Global Magnetohydrodynamic Models in the Earth's Magnetosphere

We study the performance of four magnetohydrodynamic models (BATS-R-US, GUMICS, LFM, Open GGCM) in the Earths magnetosphere. Using the Community Coordinated Modeling Centers Run-on-Request system, we compare model predictions with magnetic field measurements of the Cluster, Geotail and Wind spacecraft during a multiple substorm event. We also compare model cross polar cap potential results to those obtained from the Super Dual Auroral Radar Network(SuperDARN) and the model magnetopause standoff distances to an empirical magnetopause model. The correlation coefficient (CC) and prediction efficiency (PE) metrics are used to objectively evaluate model performance quantitatively. For all four models, the best performance outside geosynchronous orbit is found on the dayside. Generally, the performance of models decreases steadily down stream from the Earth. On the dayside most CCs are above 0.5 with CCs for Bx and Bz close to 0.9 for three out of four models. In the magnetotail at a distance of about -130 Earth radii from Earth, the prediction efficiency of all models is below that of using an average value for the prediction with the exception of Bz. Bx is most often best predicted and correlated both on the dayside and the nightside close to the Earth whereas in the far tail the CC and PE for Bz are substantially higher than other components in all models. We also find that increasing the resolution or coupling an additional physics module does notautomatically increase the model performance in the magnetosphere.

geomagnetic tail↗

WaterSense Program: Methodology for National Water Savings Analysis Model Indoor Residential Water Use

The U.S. Environmental Protection Agency (EPA) influences the market for plumbing fixtures and fittings by encouraging consumers to purchase products that carry the WaterSense label, which certifies those products as performing at low flow rates compared to unlabeled fixtures and fittings. As consumers decide to purchase water-efficient products, water consumption will decline nationwide. Decreased water consumption should prolong the operating life of water and wastewater treatment facilities.This report describes the method used to calculate national water savings attributable to EPA?s WaterSense program. A Microsoft Excel spreadsheet model, the National Water Savings (NWS) analysis model, accompanies this methodology report. Version 1.0 of the NWS model evaluates indoor residential water consumption. Two additional documents, a Users? Guide to the spreadsheet model and an Impacts Report, accompany the NWS model and this methodology document. Altogether, these four documents represent Phase One of this project. The Users? Guide leads policy makers through the spreadsheet options available for projecting the water savings that result from various policy scenarios. The Impacts Report shows national water savings that will result from differing degrees of market saturation of high-efficiency water-using products.This detailed methodology report describes the NWS analysis model, which examines the effects of WaterSense by tracking the shipments of products that WaterSense has designated as water-efficient. The model estimates market penetration of products that carry the WaterSense label. Market penetration is calculated for both existing and new construction. The NWS model estimates savings based on an accounting analysis of water-using products and of building stock. Estimates of future national water savings will help policy makers further direct the focus of WaterSense and calculate stakeholder impacts from the program.Calculating the total gallons of water the WaterSense program saves nationwide involves integrating two components, or modules, of the NWS model. Module 1 calculates the baseline national water consumption of typical fixtures, fittings, and appliances prior to the program (as described in Section 2.0 of this report). Module 2 develops trends in efficiency for water-using products both in the business-as-usual case and as a result of the program (Section 3.0). The NWS model combines the two modules to calculate total gallons saved by the WaterSense program (Section 4.0). Figure 1 illustrates the modules and the process involved in modeling for the NWS model analysis.The output of the NWS model provides the base case for each end use, as well as a prediction of total residential indoor water consumption during the next two decades. Based on the calculations described in Section 4.0, we can project a timeline of water savings attributable to the WaterSense program. The savings increase each year as the program results in the installation of greater numbers of efficient products, which come to compose more and more of the product stock in households throughout the United States.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Evaluating Global Aerosol Models and Aerosol and Water Vapor Properties Near Clouds

The 'Evaluating Global Aerosol Models and Aerosol and Water Vapor Properties Near Clouds' project focused extensively on the analysis and utilization of water vapor and aerosol profiles derived from the ARM Raman lidar at the Southern Great Plains ARM site. A wide range of different tasks were performed during this project, all of which improved quality of the data products derived from the lidar or advanced the understanding of atmospheric processes over the site. These activities included: upgrading the Raman lidar to improve its sensitivity; participating in field experiments to validate the lidar aerosol and water vapor retrievals; using the lidar aerosol profiles to evaluate the accuracy of the vertical distribution of aerosols in global aerosol model simulations; examining the correlation between relative humidity and aerosol extinction, and how these change, due to horizontal distance away from cumulus clouds; inferring boundary layer turbulence structure in convective boundary layers from the high-time-resolution lidar water vapor measurements; retrieving cumulus entrainment rates in boundary layer cumulus clouds; and participating in a field experiment that provided data to help validate both the entrainment rate retrievals and the turbulent profiles derived from lidar observations.

54 ENVIRONMENTAL SCIENCES↗

SciDAC's Earth System Grid Center for Enabling Technologies Semi-Annual Progress Report for the Period October 1, 2009 through March 31, 2010

This report summarizes work carried out by the ESG-CET during the period October 1, 2009 through March 31, 2009. It includes discussion of highlights, overall progress, period goals, collaborations, papers, and presentations. To learn more about our project, and to find previous reports, please visit the Earth System Grid Center for Enabling Technologies (ESG-CET) website. This report will be forwarded to the DOE SciDAC program management, the Office of Biological and Environmental Research (OBER) program management, national and international collaborators and stakeholders (e.g., the Community Climate System Model (CCSM), the Intergovernmental Panel on Climate Change (IPCC) 5th Assessment Report (AR5), the Climate Science Computational End Station (CCES), the SciDAC II: A Scalable and Extensible Earth System Model for Climate Change Science, the North American Regional Climate Change Assessment Program (NARCCAP), and other wide-ranging climate model evaluation activities).

99 GENERAL AND MISCELLANEOUS↗

Development of a Human Motor Model for the Evaluation of an Integrated Alerting and Notification Flight Deck System

A human motor model was developed on the basis of performance data that was collected in a flight simulator. The motor model is under consideration as one component of a virtual pilot model for the evaluation of NextGen crew alerting and notification systems in flight decks. This model may be used in a digital Monte Carlo simulation to compare flight deck layout design alternatives. The virtual pilot model is being developed as part of a NASA project to evaluate multiple crews alerting and notification flight deck configurations. Model parameters were derived from empirical distributions of pilot data collected in a flight simulator experiment. The goal of this model is to simulate pilot motor performance in the approach-to-landing task. The unique challenges associated with modeling the complex dynamics of humans interacting with the cockpit environment are discussed, along with the current state and future direction of the model.

Daiker, Ron↗

Evaluating alternative ebullition models for predicting peatland methane emission and its pathways via data–model fusion

Abstract. Understanding the dynamics of peatland methane (CH4) emissions and quantifying sources of uncertainty in estimating peatland CH4 emissions are critical for mitigating climate change. The relative contributions of CH4 emission pathways through ebullition, plant-mediated transport, and diffusion, together with their different transport rates and vulnerability to oxidation, determine the quantity of CH4 to be oxidized before leaving the soil. Notwithstanding their importance, the relative contributions of the emission pathways are highly uncertain. In particular, the ebullition process is more uncertain and can lead to large uncertainties in modeled CH4 emissions. To improve model simulations of CH4 emission and its pathways, we evaluated two model structures: (1) the ebullition bubble growth volume threshold approach (EBG) and (2) the modified ebullition concentration threshold approach (ECT) using CH4 flux and concentration data collected in a peatland in northern Minnesota, USA. When model parameters were constrained using observed CH4 fluxes, the CH4 emissions simulated by the EBG approach (RMSE = 0.53) had a better agreement with observations than the ECT approach (RMSE = 0.61). Further, the EBG approach simulated a smaller contribution from ebullition but more frequent ebullition events than the ECT approach. The EBG approach yielded greatly improved simulations of pore water CH4 concentrations, especially in the deep soil layers, compared to the ECT approach. When constraining the EBG model with both CH4 flux and concentration data in model–data fusion, uncertainty of the modeled CH4 concentration profiles was reduced by 78 % to 86 % in comparison to constraints based on CH4 flux data alone. The improved model capability was attributed to the well-constrained parameters regulating the CH4 production and emission pathways. Our results suggest that the EBG modeling approach better characterizes CH4 emission and underlying mechanisms. Moreover, to achieve the best model results both CH4 flux and concentration data are required to constrain model parameterization.

59 BASIC BIOLOGICAL SCIENCES↗

Influence of Precipitation Forcing Uncertainty on Hydrological Simulations with the NASA South Asia Land Data Assimilation System

Accurate meteorological estimates are critical for process-based hydrological simulationand prediction. This presents a significant challenge in mountainous Asia where in situmeteorological stations are limited and major river basins cross international borders. In thiscontext, remotely sensed and model-derived meteorological estimates are often necessary inputsfor distributed hydrological analysis. However, these datasets are difficult to evaluate on accountof limited access to ground data. In this case, the implications of uncertainty associated withprecipitation forcing for hydrological simulations is explored by driving the South Asia Land DataAssimilation System (South Asia LDAS) using a range of meteorological forcing products.MERRA2, GDAS, and CHIRPS produce a wide range of estimates for rainfall, which causes awidespread simulated streamflow and evapotranspiration. A combination of satellite-derived andlimited in situ data are applied to evaluate model simulations and, by extension, to constrain theestimates of precipitation. The results show that available gridded precipitation estimates based onin situ data may systematically underestimate precipitation in mountainous regions and thatperformance of gridded satellite-derived or modeled precipitation estimates varies systematicallyacross the region. Since no station-based data or product including station data is satisfactoryeverywhere, our results suggest that the evaluation of the hydrological simulation of streamflowand ET can be used as an indirect evaluation of precipitation forcing based on ground-basedproducts or in-situ data. South Asia LDAS produces reasonable evapotranspiration and streamflowwhen forced with appropriate meteorological forcing and the choice of meteorological forcingshould be made based on the geographical location as well as on the purpose of the simulations.

South Asia land data assimilation system (South As↗

A progress update on selected AREVA NP advanced BWR methodologies

This article presents an overview of select advances in AREVA NP methods concerning 3 topics: Analytical fuel rod model, Transient and accident simulation, and ATWSi post dryout treatment. Concerning the first topic, the analytical fuel rod model relies heavily on first-principles closed-form solutions instead of ever finer nodalization and higher dimensions of standard numerical methods such as finite differences or finite elements. This analytical fuel rod model takes full account of the complications that made numerical analysis necessary, e.g. variation of pellet thermal conductivity with temperature and burnup, and the radial dependence of fission power deposition in pellets. The model also calculates thermo-mechanical parameters such as clad stress and strain, fission gas release, pellet sintering and swelling, and pellet-clad gap size and thermal resistance. The model is suited for innovative fuel designs such as doped pellets. Concerning the second topic, the BWR evaluation model, AURORA-B, is an advanced transient and accident simulator including SRELAP5, a two fluid thermal hydraulic code comprehensively reviewed by international safety authorities for BWR and PWR applications; RODEX4, a best estimate fuel performance code used to evaluate the local thermal-mechanical behavior of fuel rods during postulated transients and accidents; and MB2-K, a neutron kinetics extension of the steady state core simulator MICROBURN-B2. The analytical domains will include transients as well as control rod drop and loss of coolant accidents. Extension of RODEX4 have been made to accommodate analysis of chromium-doped fuel. Concerning the third issue,The new AREVA post-dryout models can accurately predict dryout/re-wet followed by an ultimate failure to re-wet under oscillatory ATWSi (Anticipated Transient Without SCRAM with core instability) conditions. These models, while not a boiling curve based method, can qualitatively reproduce the characteristics of the boiling curve.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Precipitable water vapor over oceans from the Maritime Aerosol Network: Evaluation of global models and satellite products under clear sky conditions

We present results from an evaluation of precipitable water vapor (W) over remote oceanic areas as derived from global reanalysis models and from satellites against observations from the Maritime Aerosol Network (MAN) for cloudless skies during the period of 2004–2017. They cover polar, mid latitude and tropical oceanic regions and represent a first effort to use MAN observations for such evaluation. The global reanalysis model products evaluated in this study are from the Modern-Era Retrospective analysis for Research and Applications Version 2 (MERRA-2), the European Centre for Medium-Range Weather Forecasts (ECMWF) Interim Reanalysis (ERA I), and the Climate Forecast System Reanalysis (CFSR) model. The satellite products evaluated are from the Moderate Resolution Imaging Spectroradiometer (MODIS), the Polarization and Directionality of the Earth's Reflectances (POLDER), the Global Ozone Monitoring Experiment (GOME-2), the Scanning Imaging Absorption Spectrometer for Atmospheric Chartography (SCIAMACHY), and the Atmospheric Infra-red Sounder (AIRS). Satellite retrievals of W are based on the attenuation of solar reflected light by water vapor absorption bands, except those from AIRS that rely on brightness temperature measurements. A very good agreement is observed between the model estimates and MAN, with mean differences of ~5% and standard deviations of ~15%. These results are within the uncertainties associated with the models and the measurements, indicating the skill of the reanalysis models to estimate W over oceans under clear sky conditions. Mean differences of W between the satellite and MAN products are ~11, 6.7, 12, −7, and 3% for MODIS, POLDER, GOME-2, SCIAMACHY and AIRS respectively, while their standard deviations are 31, 29, 28, 20 and 17%. These differences reveal the need to address inconsistencies among different satellite sensors and ground-based measurements to reduce the uncertainties associated with the retrievals.

Daniel Pérez-Ramírez↗

Toward integrating high-fidelity CFD approaches in the thermal-hydraulic analysis of turbulent dry cask systems

Nuclear power plants have been supplying resilient and reliable electricity for decades, contributing to energy independence of the U.S.. However, nuclear waste management remains one of the most significant challenges in the industry. The safety of dry cask storage systems relies heavily on their thermal-hydraulic performance. Computational Fluid Dynamics (CFD) simulations are often used to demonstrate this performance and ensure that the system design meets safety standards. This study presents reduced numerical models for various types of dry cask systems. These numerical models can produce efficient and fast results based on the employed modeling strategies. Additionally, the study uses a novel approach to high-fidelity simulations to evaluate modeling assumptions in dry cask modeling. Large Eddy Simulations (LES) are used for this purpose, particularly in regions where fluid velocity is relatively high and the turbulence characteristics become important. Furthermore, the results of these high-fidelity simulations will enhance the interpretation of outcomes produced from a lower-fidelity CFD model.

CFD↗

Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction

Two prime bottlenecks to the solution of large-scale Bayesian inverse problems are the scaling of posterior sampling algorithms to high-dimensional parameter spaces and the computational cost of forward model evaluations. However, incomplete or noisy data, the state variation and parameter dependence of the forward model, and correlations in the prior collectively provide useful structure that can be exploited for dimension reduction in this setting—both in the parameter space of the inverse problem and in the state space of the forward model. To this end, we show how to jointly construct low-dimensional subspaces of the parameter space and the state space in order to accelerate the Bayesian solution of the inverse problem. As a byproduct of state dimension reduction, we also show how to identify low-dimensional subspaces of the data in problems with high-dimensional observations. These subspaces enable approximation of the posterior as a product of two factors: (i) a projection of the posterior onto a low-dimensional parameter subspace, wherein the original likelihood is replaced by an approximation involving a reduced model; and (ii) the marginal prior distribution on the high-dimensional complement of the parameter subspace. We introduce and compare several strategies for constructing these subspaces using only a limited number of forward and adjoint model simulations. The resulting posterior approximations can rapidly be characterized using standard sampling techniques, e.g., Markov chain Monte Carlo. Two numerical examples demonstrate the accuracy and efficiency of our approach: inversion of an integral equation in atmospheric remote sensing, where the data dimension is very high; and the inference of a heterogeneous transmissivity field in a groundwater system, which involves a partial differential equation forward model with high dimensional state and parameters.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Data, scripts, and figures associated with a manuscript studying impact of climate and topography on post-fire vegetation recovery.

This data package is associated with the publication “Impact of Topography and Climate on Post-fire Vegetation Recovery Across Different Burn Severity and Land Cover Types through Machine Learning” submitted to Remote Sensing of Environment (Zahura et al. 2023). In this research, a machine learning algorithm, random forest (RF), was utilized to examine the impact of climate and topography on post-fire vegetation recovery. We used enhanced vegetation index (EVI) to examine varying burn severity and land cover types. The data package includes the input files for RF model training, outputs from model predictions and analysis, and python scripts to run the model, analyze the results to understand model performance and interpretability, and plot manuscript figures. This data package contains three folders (Data, Scripts, and Figures), a file-level metadata (FLMD) csv, and a data dictionary (dd) csv. Please see Postfire_recovery_flmd.csv for a list of all files contained in this data package and descriptions for each. The data dictionary (Postfire_recovery_dd.csv) describes the csv column headers. The “Data” folder provides all the inputs and outputs to train the RF model, evaluate performance, and interpret predictions. The “Scripts” folder contains python scripts and jupyter notebooks for model training and result analysis. The “Figures” folder includes the figures used in the manuscript in “.png” and “.jpg” format.

54 ENVIRONMENTAL SCIENCES↗

Residential Mechanical Precooling

This research conducted by the Alliance for Residential Building Innovation team evaluated mechanical air conditioner pre-cooling strategies in homes throughout the United States. EnergyPlus modeling evaluated two homes with different performance characteristics in seven climates. Results are applicable to new construction homes and most existing homes built in the last 10 years, as well as fairly efficient retrofitted homes.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Arctic Cloud Microphysical Processes. Final report

The primary goal of our team’s effort under this grant was to quantify Arctic lower-troposphere processes by tying data from the large suite of NSA instruments to modeling efforts indirectly through data assimilation and directly through remote sensing and cloud modeling efforts. We maintained a distinct focus on how the ARM measurements may be used evaluate model simulations and/or physical processes. We will separate this report into two distinct sections: our data assimilation effort and the use of NSA instruments to improve our understanding and modeling of model physics to better characterize the critical surface energy budget in polar regimes.

58 GEOSCIENCES↗

A first large-scale flood inundation forecasting model

At present continental to global scale flood forecasting focusses on predicting at a point discharge, with little attention to the detail and accuracy of local scale inundation predictions. Yet, inundation is actually the variable of interest and all flood impacts are inherently local in nature. This paper proposes a first large scale flood inundation ensemble forecasting model that uses best available data and modeling approaches in data scarce areas and at continental scales. The model was built for the Lower Zambezi River in southeast Africa to demonstrate current flood inundation forecasting capabilities in large data-scarce regions. The inundation model domain has a surface area of approximately 170k km2. ECMWF meteorological data were used to force the VIC (Variable Infiltration Capacity) macro-scale hydrological model which simulated and routed daily flows to the input boundary locations of the 2-D hydrodynamic model. Efficient hydrodynamic modeling over large areas still requires model grid resolutions that are typically larger than the width of many river channels that play a key a role in flood wave propagation. We therefore employed a novel sub-grid channel scheme to describe the river network in detail whilst at the same time representing the floodplain at an appropriate and efficient scale. The modeling system was first calibrated using water levels on the main channel from the ICESat (Ice, Cloud, and land Elevation Satellite) laser altimeter and then applied to predict the February 2007 Mozambique floods. Model evaluation showed that simulated flood edge cells were within a distance of about 1 km (one model resolution) compared to an observed flood edge of the event. Our study highlights that physically plausible parameter values and satisfactory performance can be achieved at spatial scales ranging from tens to several hundreds of thousands of km2 and at model grid resolutions up to several km2. However, initial model test runs in forecast mode revealed that it is crucial to account for basin-wide hydrological response time when assessing lead time performances notwithstanding structural limitations in the hydrological model and possibly large inaccuracies in precipitation data.

hydrodynamic, hydrology, forecast, flood inundatio↗

Resolving Mesoscale Convective Systems: Grid Spacing Sensitivity in the Tropics and Midlatitudes

Abstract Mesoscale convective systems (MCSs) are a critical global water cycle component and drive extreme precipitation events in tropical and midlatitude regions. However, simulating deep convection remains challenging for modern numerical weather and climate models due to the complex interactions of processes from microscales to synoptic scales. Recent models with kilometer‐scale horizontal grid spacings offer notable improvements in simulating deep convection compared to coarser‐resolution models. Still, deficiencies in representing key physical processes, such as entrainment, lead to systematic biases. Additionally, evaluating model outputs using process‐oriented observational data remain difficult. This study presents an ensemble of MCS simulations with spanning the deep convective gray zone ( from 12 km to 125 m) in the Southern Great Plains of the U.S. and the Amazon Basin. Comparing these simulations with Atmospheric Radiation Measurement (ARM) wind profiler observations, we find greater sensitivity in the Amazon Basin compared to the Great Plains. Convective drafts converge structurally at sub‐kilometer scales, but some deficiencies remain. In both regions, simulated up and downdrafts are too deep and extreme downdrafts are not strong enough. Furthermore, Amazonian updrafts are too strong. Overall, we observe higher sensitivity in the tropics, including an artificial buildup in vertical kinetic energy at scales of , suggesting a need for 250 m in this region. Nevertheless, bulk convergence—agreement of storm‐average statistics—is achievable with kilometer‐scale simulations within a 10% error margin with 1 km providing a good balance between accuracy and computational cost.

54 ENVIRONMENTAL SCIENCES↗