Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Kinetics of Uranium(VI) Desorption from Contaminated Sediments: Effect of Geochemical Conditions and Model Evaluation

Stirred-flow cell experiments were performed to investigate the kinetics of uranyl [U(VI)] desorption from a contaminated sediment collected from the Hanford 300 Area at the US Department of Energy (DOE) Hanford Site, Washington. Three influent solutions of variable pH, Ca and carbonate concentrations that affected U(VI) aqueous and surface speciation were used under dynamic flow conditions to evaluate the effect of geochemical conditions on the rate of U(VI) desorption. The measured rate of U(VI) desorption varied with solution chemical composition that evolved as a result of thermodynamic and kinetic interactions between the influent solutions and sediment. The solution chemical composition that led to a lower equilibrium U(VI) sorption to the solid phase yielded a faster desorption rate. The experimental results were used to evaluate a multi-rate, surface complexation model (SCM) that has been proposed to describe U(VI) desorption kinetics in the Hanford sediment that contained complex sorbed U(VI) species in mass transfer limited domains. The model was modified and supplemented by including multi-rate, ion exchange reactions to describe the geochemical interactions between the solutions and sediment. With the same set of model parameters, the modified model reasonably well described the evolution of major ions and the rates of U(VI) desorption under variable geochemical and flow conditions, implying that the multi-rate SCM is an effective way to describe U(VI) desorption kinetics in subsurface sediments.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Model Evaluation Protocol for Fire Models Involving Fuels at Liquefied Natural Gas Facilities (Version 2)

This document provides a description of the model evaluation protocol (MEP) for pool fires, jet fires, and fireballs involving liquefied natural gas (LNG) and processing fuels at LNG facilities. The purpose of the MEP is to provide procedures regarding the assessment of a model’s suitability to predict heat flux from fires. Three components, namely, a scientific assessment, model verification, and model validation comprise the MEP. The evaluation of a model satisfying these three components is to be documented in the form of a model evaluation report (MER). Discussion of models for the prediction of fire, detailed information on each of the three MEP components, the MEP procedure regarding new versions of previously approved models, and the format of the model evaluation report (MER) are provided.

03 NATURAL GAS↗

Model Evaluation Protocol for Fire Models Involving Fuels at Liquefied Natural Gas Facilities.

This document provides a description of the model evaluation protocol (MEP) for pool fires, jet fires, and fireballs involving liquefied natural gas (LNG), refrigerant fluids, and byproducts at LNG facilities. The purpose of the MEP is to provide procedures regarding the assessment of a model's suitability to predict heat flux from fires. Three components, namely, a scientific assessment, model verification, and model validation comprise the MEP. The evaluation of a model satisfying these three components is to be documented in the form of a model evaluation report (MER). Discussion of models for the prediction of fire, detailed information on each of the three MEP components, the MEP procedure regarding new versions of previously approved models, and the format of the model evaluation report (MER) are provided.

03 NATURAL GAS↗

Application of an Online-Coupled Regional Climate Model, WRF-CAM5, over East Asia for Examination of Ice Nucleation Schemes: Part I. Comprehensive Model Evaluation and Trend Analysis for 2006 and 2011

Online-coupled climate and chemistry models are necessary to realistically represent the interactions between climate variables and chemical species and accurately simulate aerosol direct and indirect effects on cloud, precipitation, and radiation. In this Part I of a two-part paper, simulations from the Weather Research and Forecasting model coupled with the physics package of Community Atmosphere Model (WRF-CAM5) are conducted with the default heterogeneous ice nucleation parameterization over East Asia for two full years: 2006 and 2011. A comprehensive model evaluation is performed using satellite and surface observations. The model shows an overall acceptable performance for major meteorological variables at the surface and in the boundary layer, as well as column variables (e.g., precipitation, cloud fraction, precipitating water vapor, downward longwave and shortwave radiation). Moderate to large biases exist for cloud condensation nuclei over oceanic areas, cloud variables (e.g., cloud droplet number concentration, cloud liquid and ice water paths, cloud optical depth, longwave and shortwave cloud forcing). These biases indicate a need to improve the model treatments for cloud processes, especially cloud droplets and ice nucleation, as well as to reduce uncertainty in the satellite retrievals. The model simulates well the column abundances of chemical species except for column SO 2 but relatively poor for surface concentrations of several species such as CO, NO 2 , SO 2 , PM2.5, and PM10. Several reasons could contribute to the underestimation of major chemical species in East Asia including underestimations of anthropogenic emissions and natural dust emissions, uncertainties in the spatial and vertical distributions of the anthropogenic emissions, as well as biases in meteorological, radiative, and cloud predictions. Despite moderate to large biases in the chemical predictions, the model performance is generally consistent with or even better than that reported for East Asia with only a few exceptions. The model generally reproduces the observed seasonal variations and the difference between 2006 and 2011 for most variables or chemical species. Overall, these results demonstrate promising skills of WRF-CAM5 for long-term simulations at a regional scale and suggest several areas of potential improvements.

54 ENVIRONMENTAL SCIENCES↗

A Surface Radiation Balance Dataset from Siple Dome in West Antarctica for Atmospheric and Climate Model Evaluation

Abstract A field campaign at Siple Dome in West Antarctica during the austral summer 2019/20 offers an opportunity to evaluate climate model performance, particularly cloud microphysical simulation. Over Antarctic ice sheets and ice shelves, clouds are a major regulator of the surface energy balance, and in the warm season their presence occasionally induces surface melt that can gradually weaken an ice shelf structure. This dataset from Siple Dome, obtained using transportable and solar-powered equipment, includes surface energy balance measurements, meteorology, and cloud remote sensing. To demonstrate how these data can be used to evaluate model performance, comparisons are made with meteorological reanalysis known to give generally good performance over Antarctica (ERA5). Surface albedo measurements show expected variability with observed cloud amount, and can be used to evaluate a model’s snowpack parameterization. One case study discussed involves a squall with northerly winds, during which ERA5 fails to produce cloud cover throughout one of the days. A second case study illustrates how shortwave spectroradiometer measurements that encompass the 1.6- μ m atmospheric window reveal cloud phase transitions associated with cloud life cycle. Here, continuously precipitating mixed-phase clouds become mainly liquid water clouds from local morning through the afternoon, not reproduced by ERA5. We challenge researchers to run their various regional or global models in a manner that has the large-scale meteorology follow the conditions of this field campaign, compare cloud and radiation simulations with this Siple Dome dataset, and potentially investigate why cloud microphysical simulations or other model components might produce discrepancies with these observations. significance statement Antarctica is a critical region for understanding climate change and sea level rise, as the great ice sheets and the ice shelves are subject to increasing risk as global climate warms. Climate models have difficulties over Antarctica, particularly with simulation of cloud properties that regulate snow surface melting or refreezing. Atmospheric and climate-related field work has significant challenges in the Antarctic, due to the small number of research stations that can support state-of-the-art equipment. Here we present new data from a suite of transportable and solar-powered instruments that can be deployed to remote Antarctic sites, including regions where ice shelves are most at risk, and we demonstrate how key components of climate model simulations can be evaluated against these data.

54 ENVIRONMENTAL SCIENCES↗

Aerosol transport and wet scavenging in deep convective clouds: a case study and model evaluation using a multiple passive tracer analysis approach

The effect of wet scavenging on ambient aerosols in deep, continental convective clouds in the mid-latitudes is studied for a severe storm case in Oklahoma during the Deep Convective Clouds and Chemistry (DC3) field campaign. A new passive-tracer based transport analysis framework is developed to characterize the convective transport based on the vertical distribution of several slowly reacting and nearly insoluble trace gases. The passive gas concentration in the upper troposphere convective outflow results from a mixture of 47% from the lower level (0-3 km), 21% entrained from the upper troposphere, and 32% from mid-atmosphere based on observations. The transport analysis framework is applied to aerosols to estimate aerosol transport and wet-scavenging efficiency. Observations yield high overall scavenging efficiencies of 81% and 68% for aerosol mass (Dp < 1μm) and aerosol number (0.03< Dp < 2.5μm), respectively. Little chemical selectivity to wet scavenging is seen among observed submicron sulfate (84%), organic (82%), and ammonium (80%) aerosols, while nitrate has a much lower scavenging efficiency of 57% likely due to the uptake of nitric acid. Observed larger size particles (0.15 - 2.5μm) are scavenged more efficiently (84%) than smaller particles (64%; 0.03 - 0.15μm). The storm is simulated using the chemistry version of the WRF model. Compared to the observation based analysis, the standard model underestimates the wet scavenging efficiency for both mass and number concentrations with low biases of 31% and 40%, respectively. Adding a new treatment of secondary activation significantly improves simulation results, so that the bias in scavenging efficiency in mass and number concentrations is reduced to <10%. This supports the hypothesis that secondary activation is an important process for wet removal of aerosols in deep convective storms.

Yang, Qing↗

Observational Data for Next-Generation Climate Model Evaluation: Requirements, Considerations, and Best Practices

Climate model simulations are an important source of information about our planet’s climate system and also enable informed decision-making under different future scenarios. As a new archive of results from the next generation of climate models is anticipated to become available with the Coupled Model Intercomparison Project phase 7 (CMIP7), the need to develop efficient and robust methods to evaluate models is paramount. Observations are an integral part of model evaluation, providing a means to quantify and understand the degree to which climate models can faithfully reproduce Earth system processes. Such analysis is critical for constraining climate projections, identifying areas of focus for model development, and assisting analysts in deciphering the utility of models for specific applications. Observations of Earth system come from a diversity of sources, span different space–time domains, and are produced by different communities, and each dataset features different data structures and formats, metadata standards, and its own unique uncertainties. Uncertainties in an observational dataset may stem from gaps in temporal and spatial coverage, instrumentation errors, or assumptions in retrieval and processing methods. How then does one ensure that observational data are ready for use and utilized in the most appropriate way for robust, rapid, and routine climate model evaluation? The CMIP7 Model Benchmarking Task Team with input from the broader climate modeling, model evaluation, and observational data communities present a vision and considerations for best practices toward the optimal and appropriate use of observational data to support next-generation climate model evaluation.

Climate models↗

Empirical Validation of Multi-Zone HVAC System Model: Evaluation of Existing Infiltration Models used in Building Energy Simulation

Infiltration can have a significant impact on building loads. Studies have shown that infiltration can account for 15-40% of annual space conditioning needs in commercial buildings (Emmerich et al. 2019; Younes et al. 2012). The driving force of infiltration is the pressure difference across the building envelope caused by wind, the stack effect (known as buoyancy effect), and the operation of ventilation equipment. Wind pressure is governed by wind direction, speed, building shape, and other structures around the building. The stack effect is a function of the building height and air density differences of ambient air (Han, 2015). The effect of wind is dominant in low-rise residential buildings, and the stack effect is dominant in highrise buildings (ASHRAE 2017). In building energy simulation programs (e.g., EnergyPlus), various empirical infiltration models (e.g., the effective leakage area model, the flow coefficient model) are available to simulate infiltration rates. To help users in selecting a proper infiltration model for modeling of the two-story Flexible Research Platform (FRP), the team evaluates the existing infiltration models in EnergyPlus based on field measurements from the FRP. The blower door and tracer gas decay tests were performed in the FRP. The blower door test result was used to estimate input parameters required in the infiltration models. The actual infiltration rates were estimated with the tracer gas decay test results.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Development of evaluation models of manpower needs for dismantling the dry conversion process-related equipment in uranium refining and conversion plant (URCP)

Evaluation models for determining the manpower needs for dismantling various types of equipment in uranium refining and conversion plant (URCP) have been developed. The models are widely applicable to other uranium handling facilities. Additionally, a simplified model was developed for easily and accurately calculating the manpower needs for dismantling dry conversion process-related equipment (DP equipment). It is important to evaluate beforehand project management data such as manpower needs to prepare an optimized decommissioning plan and implement effective dismantling activity. The Japan Atomic Energy Agency (JAEA) has developed the project management data evaluation system for dismantling activities (PRODIA code), which can generate project management data using evaluation models. For preparing an optimized decommissioning plan, these evaluation models should be established based on the type of nuclear facility and actual dismantling data. In URCP, the dry conversion process of reprocessed uranium and others was operated until 1999, and the equipment related to the main process was dismantled from 2008 to 2011. Actual data such as manpower for dismantling were collected during the dismantling activities, and evaluation models were developed using the collected actual data on the basis of equipment classification considering the characteristics of uranium handling facility. (authors)

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Nevada National Security Site (NNSS) Environmental Remediation Using EPA's Model Evaluation Guidance to Move to Closure - 15588

The National Nuclear Security Administration Nevada Field Office (NNSA/NFO) has adapted the Environmental Protection Agency's guidance on environmental model evaluation as a framework for evaluating corrective action investigation models. Model evaluation is an important confidence building activity, which is needed to support closure of the Nevada National Security Site (NNSS) Corrective Action Units (CAUs) associated with underground nuclear testing. Through this process, site-investigation models of groundwater flow and contaminant transport are reviewed and compared to new site-specific data to determine if the results can be used to support regulatory decisions required for closure and to achieve NNSA/NFO site management objectives. The regulatory decisions include identifying and establishing CAU regulatory boundaries; identifying institutional controls, including use-restriction boundaries; and developing a long-term closure monitoring program. A successful case study from Frenchman Flat, Nevada is used to illustrate the model evaluation process. Groundwater flow and transport from two of the 10 underground nuclear tests in Frenchman Flat had the largest potential for contaminating groundwater, and were selected for evaluation. Wells were drilled, logged, hydraulically tested, sampled, and analyzed for radionuclides and the data was evaluated with respect to previous site conceptualization and numerical model results. The approach and results of this process as applied to the PIN STRIPE test and how it helped build confidence to close the Frenchman Flat CAU is described in this case study. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Model Evaluation and Hindcasting: An Experiment with an Integrated Assessment Model

Integrated assessment models have been extensively used for analyzing long term energy and greenhouse emissions trajectories and have influenced key policies on this subject. Though admittedly these models are focused on the long term trajectories, how well these models are able to capture historical dynamics is an open question. In a first experiment of its kind, we present a framework for evaluation of such integrated assessment models. We use Global Change Assessment Model for this zero order experiment, and focus on the building sector results for USA. We calibrate the model for 1990 and run it forward up to 2095 in five year time steps. This gives us results for 1995, 2000, 2005 and 2010 which we compare to observed historical data at both fuel level and service level. We focus on bringing out the key insights for the wider process of model evaluation through our experiment with GCAM. We begin with highlighting that creation of an evaluation dataset and identification of key evaluation metric is the foremost challenge in the evaluation process. Our analysis highlights that estimation of functional form of the relationship between energy service demand, which is an unobserved variable, and its drivers is a significant challenge in the absence of adequate historical data for both the dependent and driver variables. Historical data availability for key metrics is a serious limiting factor in the process of evaluation. Interestingly, service level data against which such models need to be evaluated are itself a result of models. Thus for energy services, the best we can do is compare our model results with other model results rather than observed and measured data. We show that long term models, by the nature of their construction, will most likely underestimate the rapid growth in some services observed in a short time span. Also, we learn that modeling saturated energy services like space heating is easier than modeling unsaturated services like space cooling and understanding that how far a service is from its saturation level is a key question which we probably don’t have an answer to. Finally and most importantly, even if long term models partially miss the short term dynamics, the long term insights provides by these models is fairly robust. We conclude by highlighting that our work is the first step in the much wider process of integrated assessment model evaluation and will hence have its own limitations. Future evaluation research work should build upon this zero order experiment for improving our modeling of human and coupled earth systems.

building energy↗

Completion Report for Model Evaluation Well ER-11-2: Corrective Action Unit 98: Frenchman Flat

Model Evaluation Well ER-11-2 was drilled for the U.S. Department of Energy, National Nuclear Security Administration Nevada Site Office in support of Nevada Environmental Management Operations at the Nevada National Security Site (formerly known as the Nevada Test Site). The well was drilled in August 2012 as part of a model evaluation program in the Frenchman Flat area of Nye County, Nevada. The primary purpose of the well was to provide detailed geologic, hydrogeologic, chemical, and radionuclide data that can be used to test and build confidence in the applicability of the Frenchman Flat Corrective Action Unit flow and transport models for their intended purpose. In particular, this well was designed to provide data to evaluate the uncertainty in model forecasts of contaminant migration from the upgradient underground nuclear test PIN STRIPE, conducted in borehole U-11b in 1966. Well ER-11-2 will provide information that can be used to refine the Phase II Frenchman Flat hydrostratigraphic framework model if necessary, as well as to support future groundwater flow and transport modeling. The main 31.1-centimeter (cm) hole was drilled to a total depth of 399.6 meters (m). A completion casing string was not set in Well ER-11-2. However, a piezometer string was installed in the 31.1-cm open hole. The piezometer is composed of 7.3-cm stainless-steel tubing hung on 6.0-cm carbon-steel tubing via a crossover sub. The piezometer string was landed at 394.5 m, for monitoring the lower tuff confining unit. Data collected during and shortly after hole construction include composite drill cuttings samples collected every 3.0 m, various geophysical logs, water quality (including tritium and other test-related radionuclides) measurements, and water level measurements. The well penetrated 42.7 m of Quaternary and Tertiary alluvium and 356.9 m of Tertiary volcanic rock. The water-level measured in the piezometer string on September 25, 2012, was 353.8 m below ground surface. No tritium above levels detectable by field methods were encountered in this hole. No well development or hydrologic testing was conducted in this well immediately after completion, and future well development, sampling, and hydrologic testing planned for this well will be limited due to the diameter of the piezometer string. The stratigraphy, general lithology, and the water level are as expected, but the section of geology encountered is higher than expected due to faulting. No tritium above the minimum detection limit of the field equipment was detected because the target aquifer (the Topopah Spring aquifer) at Well ER-11-2 is structurally higher than expected and thus unsaturated.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Completion Report for Model Evaluation Well ER-5-5: Corrective Action Unit 98: Frenchman Flat

Model Evaluation Well ER-5-5 was drilled for the U.S. Department of Energy, National Nuclear Security Administration Nevada Site Office in support of Nevada Environmental Management Operations at the Nevada National Security Site (formerly known as the Nevada Test Site). The well was drilled in July and August 2012 as part of a model evaluation well program in the Frenchman Flat area of Nye County, Nevada. The primary purpose of the well was to provide detailed geologic, hydrogeologic, chemical, and radiological data that can be used to test and build confidence in the applicability of the Frenchman Flat Corrective Action Unit flow and transport models for their intended purpose. In particular, this well was designed to obtain data to evaluate the uncertainty in model forecasts of contaminant migration from the upgradient underground nuclear test MILK SHAKE, conducted in Emplacement Hole U-5k in 1968, which were considered to be uncertain due to the unknown extent of a basalt lava-flow aquifer present in this area. Well ER-5-5 is expected to provide information to refine the Phase II Frenchman Flat hydrostratigraphic framework model, if necessary, as well as to support future groundwater flow and transport modeling. The 31.1-centimeter (cm) diameter hole was drilled to a total depth of 331.3 meters (m). The completion string, set at the depth of 317.2 m, consists of 16.8-cm stainless-steel casing hanging from 19.4-cm carbon-steel casing. The 16.8-cm stainless-steel casing has one slotted interval open to the basalt lava-flow aquifer and limited intervals of the overlying and underlying alluvial aquifer. A piezometer string was also installed in the annulus between the completion string and the borehole wall. The piezometer is composed of 7.3-cm stainless-steel tubing suspended from 6.0-cm carbon-steel tubing. The piezometer string was landed at 319.2 m, to monitor the basalt lava-flow aquifer. Data collected during and shortly after hole construction include composite drill cuttings samples collected every 3.0 m, various geophysical logs, preliminary water quality measurements, and water-level measurements. The well penetrated 331.3 m of Quaternary–Tertiary alluvium, including an intercalated layer of saturated basalt lava rubble. No well development or hydrologic testing was conducted in this well immediately after completion; however, a preliminary water level was measured in the piezometer string at the depth of 283.4 m on September 25, 2012. No tritium above the minimum detection limit of the field instruments was detected in this hole. Future well development, sampling, and hydrologic testing planned for this well will provide more accurate hydrologic information for this site. The stratigraphy, general lithology, and water level were as expected, though the expected basalt lava-flow aquifer is basalt rubble and not the dense, fractured lava as modeled. The lack of tritium transport is likely due to the difference in hydraulic properties of the basalt lava-flow rubble encountered in the well, compared to those of the fractured aquifer used in the flow and transport models.

58 GEOSCIENCES↗

Incorporation of aerosol into the COSPv2 satellite lidar simulator for climate model evaluation

Abstract. Atmospheric aerosol has substantial impacts on climate, air quality and biogeochemical cycles, and its concentrations are highly variable in space and time. A key variability to evaluate within models that simulate aerosol is the vertical distribution, which influences atmospheric heating profiles and aerosol–cloud interactions, to help constrain aerosol residence time and to better represent the magnitude of simulated impacts. To ensure a consistent comparison between modeled and observed vertical distribution of aerosol, we implemented an aerosol lidar simulator within the Cloud Feedback Model Intercomparison Project (CFMIP) Observation Simulator Package version 2 (COSPv2). We assessed the attenuated total backscattered (ATB) signal and the backscatter ratios (SRs) at 532 nm in the U.S. Department of Energy's Energy Exascale Earth System Model version 1 (E3SMv1). The simulator performs the computations at the same vertical resolution as the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP), making use of aerosol optics from the E3SMv1 model as inputs and assuming that aerosol is uniformly distributed horizontally within each model grid box. The simulator applies a cloud masking and an aerosol detection threshold to obtain the ATB and SR profiles that would be observed above clouds by CALIOP with its aerosol detection capability. Our analysis shows that the aerosol distribution simulated at a seasonal timescale is generally in good agreement with observations. Over the Southern Ocean, however, the model does not produce the SR maximum as observed in the real world. Comparison between clear-sky and all-sky SRs shows little differences, indicating that the cloud screening by potentially incorrect model clouds does not affect the mean aerosol signal averaged over a season. This indicates that the differences between observed and simulated SR values are due not to sampling errors, but to deficiencies in the representation of aerosol in models. Finally, we highlight the need for future applications of lidar observations at multiple wavelengths to provide insights into aerosol properties and distribution and their representation in Earth system models.

54 ENVIRONMENTAL SCIENCES↗