Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

Status of yield estimation technology: A review of second-generation model development and evaluation

Multiple regression models were studied in order to determine their yield estimation capability for any arbitrary unit area and to obtain greater responsiveness and accuracy through the use of additional data sources applied at smaller spatial and temporal scales. It was concluded that data base inadequacy was the factor limiting performance in the models studied and that each of the models has more yield predicting capability than was reached during LACIE.

Stuff, R. G.↗

A simple model for straggling evaluation

Some straggling models had largely been abandoned in favor of Monte Carlo simulations of straggling which are accurate but time consuming, limiting their application in practice. The difficulty of simple analytic models is the failure to give accurate values past 85% of the particle range. A simple model is derived herein based on a second order approximation upon which rapid analysis tools are developed for improved understanding of material charged particle transmission properties. Published by Elsevier Science B.V.

Models, Statistical↗

A Simple Model for Straggling Evaluation

Simple straggling models had largely been abandoned in favor of Monte Carlo simulations of straggling which are accurate but time consuming, limiting their application in practice. The difficulty of simple analytic models is the failure to give accurate values past 85% of the particle range. A simple model is derived herein based on a second order approximation upon which rapid analysis tools are developed for improved understanding of material charged particle transmission properties.

Wilson, J. W.↗

Bridging the Gap Between Requirements and Model Analysis: Evaluation on Cyber-Physical Challenge Problems

We present a framework for introducing high-level requirement specifications in the automated analysis of dataflow models. By integrating the FRET requirements elicitation tool with the CoCoSim analyzer, our framework enables the analysis of hierarchical Simulink models against requirements written in a restricted English language. More precisely, we support: automatic extraction of Simulink model information and association of high-level requirements with target model signals and components; translation of temporal logic formulas into synchronous dataflow CoCoSpec specifications as well as Simulink monitors; and interpretation of counterexamples produced by the analysis both at the requirement and model level. The features provided by our framework are generic and can be used to integrate other requirements elicitation and Simulink/Lustre analysis tools. We report on the lessons learned from the application of our approach to the Lockheed Martin Cyber-Physical, aerospace-inspired challenge problems. For the analysis, we used the Kind2, Zustre, and Simulink Design Verifier (SLDV) tools.

Simulink↗

Development and Implementation of Dynamic Scripts to Support Local Model Verification at National Weather Service Weather Forecast Offices

Local modeling with a customized configuration is conducted at National Weather Service (NWS) Weather Forecast Offices (WFOs) to produce high-resolution numerical forecasts that can better simulate local weather phenomena and complement larger scale global and regional models. The advent of the Environmental Modeling System (EMS), which provides a pre-compiled version of the Weather Research and Forecasting (WRF) model and wrapper Perl scripts, has enabled forecasters to easily configure and execute the WRF model on local workstations. NWS WFOs often use EMS output to help in forecasting highly localized, mesoscale features such as convective initiation, the timing and inland extent of lake effect snow bands, lake and sea breezes, and topographically-modified winds. However, quantitatively evaluating model performance to determine errors and biases still proves to be one of the challenges in running a local model. Developed at the National Center for Atmospheric Research (NCAR), the Model Evaluation Tools (MET) verification software makes performing these types of quantitative analyses easier, but operational forecasters do not generally have time to familiarize themselves with navigating the sometimes complex configurations associated with the MET tools. To assist forecasters in running a subset of MET programs and capabilities, the Short-term Prediction Research and Transition (SPoRT) Center has developed and transitioned a set of dynamic, easily configurable Perl scripts to collaborating NWS WFOs. The objective of these scripts is to provide SPoRT collaborating partners in the NWS with the ability to evaluate the skill of their local EMS model runs in near real time with little prior knowledge of the MET package. The ultimate goal is to make these verification scripts available to the broader NWS community in a future version of the EMS software. This paper provides an overview of the SPoRT MET scripts, instructions for how the scripts are run, and example use cases.

Zavordsky, Bradley↗

Development and Implementation of Dynamic Scripts to Support Local Model Verification at National Weather Service Weather Forecast Offices

Local modeling with a customized configuration is conducted at National Weather Service (NWS) Weather Forecast Offices (WFOs) to produce high-resolution numerical forecasts that can better simulate local weather phenomena and complement larger scale global and regional models. The advent of the Environmental Modeling System (EMS), which provides a pre-compiled version of the Weather Research and Forecasting (WRF) model and wrapper Perl scripts, has enabled forecasters to easily configure and execute the WRF model on local workstations. NWS WFOs often use EMS output to help in forecasting highly localized, mesoscale features such as convective initiation, the timing and inland extent of lake effect snow bands, lake and sea breezes, and topographically-modified winds. However, quantitatively evaluating model performance to determine errors and biases still proves to be one of the challenges in running a local model. Developed at the National Center for Atmospheric Research (NCAR), the Model Evaluation Tools (MET) verification software makes performing these types of quantitative analyses easier, but operational forecasters do not generally have time to familiarize themselves with navigating the sometimes complex configurations associated with the MET tools. To assist forecasters in running a subset of MET programs and capabilities, the Short-term Prediction Research and Transition (SPoRT) Center has developed and transitioned a set of dynamic, easily configurable Perl scripts to collaborating NWS WFOs. The objective of these scripts is to provide SPoRT collaborating partners in the NWS with the ability to evaluate the skill of their local EMS model runs in near real time with little prior knowledge of the MET package. The ultimate goal is to make these verification scripts available to the broader NWS community in a future version of the EMS software. This paper provides an overview of the SPoRT MET scripts, instructions for how the scripts are run, and example use cases.

Zavodsky, Bradley↗

Chapter 10 - Remote Sensing Measurements of Aerosol Properties

Satellite instruments have proven especially capable at monitoring the quantity of airborne particles in columns of atmosphere, globally. This chapter describes the principles of satellite measurements and retrieval algorithms, and surveys current instruments and their capabilities. We outline the issues associated with retrieval algorithms, such as surface characterization and aerosol proximity to clouds, and the challenges with interpretation of the results. The relationship between measured aerosol properties and climate-relevant aerosol properties simulated in models is outlined, as well as how measurements are used to evaluate models. Most space-based aerosol instruments are passive sensors that measure reflected sunlight at multiple wavelengths, some at multiple viewing angles. A few are active sensors that send out their own laser light and measure the returned signal. Except when clouds are present, the excess amount of light scattered back to space, beyond that expected from the surface and atmospheric gas, is attributed to aerosol. Satellite measurements are used in many ways in aerosol research. They often provide the only method for monitoring hazardous phenomena such as major wildfire and volcanic eruption plumes, especially in remote areas. Stable, long-term, near-global-scale satellite data records make it possible to identify regional and global aerosol trends. Aerosol radiative effects on climate can be quantified on a near-global scale and used to estimate the strength of aerosol–radiation and aerosol–cloud interactions as well as to evaluate climate model simulations of these interactions. Aerosol-type mapping from satellite imagery is helpful for source attribution, model validation, and to constrain particle light-absorption properties that are essential for radiative forcing calculations. The range of aerosol properties retrieved from satellite observations has grown considerably since the first global estimates of aerosol optical depth (τ a) over ocean were made in the late 1970s. Methods for retrieving particle size and light-absorption properties were explored in the 1990s using multispectral, multi-angle observations, and polarization in visible and near-infrared wavelengths. Sensitivity to particle light absorption, primarily from black or brown carbon content, improved with the inclusion of UV channels, and sensitivity to very thin aerosol layers in the upper troposphere and lower stratosphere was advanced with the use of limb-sounding instruments and active sensors. There are limitations to every measurement technique, including satellite aerosol remote sensing. For wide-swath, passive instruments, aerosol retrievals near clouds can present substantial challenges as far as 15 km away due to cloud-scattered light contaminating the signal. In nearly all cases, retrievals over bright snow and ice surfaces are precluded because surface reflectance uncertainties can overwhelm the aerosol signal. Similarly, meteorological cloud is identified and masked out where possible. Data from passive sensors also lack vertical resolution except those that view toward the limb or where multi-angle imagery is acquired over plumes from wildfires, erupting volcanoes, and wind-blown dust. Yet, passive sensors provide vastly more coverage than the active instruments that mitigate these issues. Particle microphysical information is qualitative from all remote sensing techniques, relying on proxies to infer particle composition, hygroscopicity, and the amount of light-absorbing material. Further, particles smaller than about 200 nm diameter cannot be distinguished from atmospheric gas molecules with remote sensing, which hinders studies of cloud condensation nuclei and their effects on clouds. Most satellite instruments dedicated to aerosol observations are in low-Earth, near-polar, sun-synchronous orbits, which means they cross the equator at the same local time each day. Most are set on cycles that repeat approximately every 16 days, which makes it difficult to monitor aerosol evolution locally. Geostationary satellites make it possible to observe changes occurring from minutes to hours over regions up to 8000 km in size, but lack coverage of high latitudes, and often provide more limited constraints on aerosol properties. Ground-truth data are vital for satellite aerosol-retrieval validation. The AErosol RObotic NETwork (AERONET) of sun photometers was created in 1993 and has become an established global network of over 350 instruments for validating satellite measurements. The network, as well as global networks of ground-based lidars, solar flux radiometers and other sun photometers, are widely used for evaluating global satellite retrievals and model simulations. NASA's Earth Observing System (EOS) program beginning in 1999 led to improvements in reliability, spatial resolution, and spectral resolution (and hence, to improved particle size discrimination and light absorption properties). Satellite payloads include advanced broad-swath and multi-angle imagers, along with the first space-based active sensor focused largely on long-term aerosol monitoring. Since about 2002, Europe's SENTINEL and operational meteorological satellite fleets are also providing sustained aerosol observations, with planned continuation until at least 2030. Satellite remote sensing instruments offer valuable data for evaluating aerosol representations in global climate models. They have been used to assess aerosol optical and physical properties, trends and distributions, and are applied increasingly as direct model constraints in data assimilation to create global aerosol reanalysis products. Aerosol optical depth is the most common quantity adopted for routine model evaluation, including multiwavelength data to loosely constrain particle-size distributions. These evaluations of multiple models have revealed general biases in their regional aerosol amounts and seasonal patterns of transport and removal. Although satellite measurements have near-global coverage, substantial errors can be introduced into the model observation comparison unless attention is paid to spatial and temporal collocation, cloud screening, subgrid-scale variability, and measurement uncertainties that vary with retrieval conditions.

aerosol properties↗

Modeling and evaluating 239 Pu and 235 U PFNS and average prompt-neutron multiplicity [Slides]

The following are currently in VIII.1 LANL and IAEA test files: 239 Pu nu-bar including CGMF modeling and CEA data, 239 Pu PFNS including Chi-Nu & CEA data, 239 Pu(n,f) cross section including fission TPC data. The following are currently being tested: 235 U nu-bar including CGMF modeling and 235 U PFNS including Chi-Nu data. Upcoming tasks include the correction of 235 U Chi-Nu PFNS at higher E inc , benchmarking 235 U PFNS and nu-bar evaluations, and getting 238 U nu-bar.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

TPSAS-NF1676L-33231-DND

It is well established that clouds have a profound influence on Earth's energy budget and that cloud feedbacks are responsible for much of the uncertainty in climate model projections of global warming in response to radiative forcing by greenhouse gases. While most model evaluation efforts focus on model representation of the observed mean state. For example, the regional mean climatology of key climate variables there is also tremendous value in testing how a model represents observed internal variations of the climate system. Here we use CERES observations during and after the so-called global warming hiatus to evaluate a subset of CMIP6 climate models. The CERES data show a marked 0.83 Wm-2 reduction in global mean reflected SW TOA flux during the three years following the hiatus that results in an increase in net energy into the climate system. The primary driver of the decrease in SW TOA flux is a reduction in low cloud cover over the eastern Pacific Ocean that occurs in response to an unprecedented increase in SSTs in that region following the hiatus. The CMIP6 simulations consist of AMIP-style experiments in which SSTs and sea-ice boundary conditions are constrained by observations. For each model, the AMIP runs were extended through the end of 2017 from the official CMIP6 AMIP end period in 2014. Results show that the models do a good job at capturing the pattern of observed changes in TOA flux observed by CERES, but tend to underestimate the magnitude.

Norman G Loeb↗

Use of the SWAT model to evaluate the sustainability of bioenergy production at a National scale

As the US begins to integrate biomass crops and residues into its mix of energy feedstocks, tools are needed to measure the long-term sustainability of these feedstocks. Two aspects of sustainability are long-term potential for profitably producing energy and protection of ecosystems influenced by energy-related activities. The Soil and Water Assessment Tool (SWAT) is an important model used in the efforts to quantify both aspects. To quantify potential feedstock production, they used SWAT to estimate switchgrass yields at a national scale. The results from this analysis produced a map of the potential switchgrass yield along its natural eastern range. To quantify ecological protection, they are using the SWAT model to forecast changes in water quality and fish richness as a result of landscape alterations due to incorporating bioenergy crops. They have implemented the SWAT model in the Arkansas-Red-White region, which drains into the Mississippi River, and they present their methods here. They identified two sub-watersheds for sensitivity analysis and calibration of the water quality results, and then, explored ways to apply the calibration results to the whole region and validate the model setup. They also present an overview of their research in which results from the calibrated regional SWAT model were used to analyze potential changes in fish biodiversity. Only by evaluating the energy and environmental implications of landscape changes can we make informed decisions about bioenergy at the national scale, and the SWAT model will enable us to reach that goal.

09 BIOMASS FUELS↗

Multi-model Mean Nitrogen and Sulfur Deposition from the Atmospheric Chemistry and Climate Model Intercomparison Project (ACCMIP): Evaluation of Historical and Projected Future Changes

We present multi-model global datasets of nitrogen and sulfate deposition covering time periods from 1850 to 2100, calculated within the Atmospheric Chemistry and Climate Model Intercomparison Project (ACCMIP). The computed deposition fluxes are compared to surface wet deposition and ice-core measurements. We use a new dataset of wet deposition for 2000-2002 based on critical assessment of the quality of existing regional network data. We show that for present-day (year 2000 ACCMIP time-slice), the ACCMIP results perform similarly to previously published multi-model assessments. The analysis of changes between 1980 and 2000 indicates significant differences between model and measurements over the United States, but less so over Europe. This difference points towards misrepresentation of 1980 NH3 emissions over North America. Based on ice-core records, the 1850 deposition fluxes agree well with Greenland ice cores but the change between 1850 and 2000 seems to be overestimated in the Northern Hemisphere for both nitrogen and sulfur species. Using the Representative Concentration Pathways to define the projected climate and atmospheric chemistry related emissions and concentrations, we find large regional nitrogen deposition increases in 2100 in Latin America, Africa and parts of Asia under some of the scenarios considered. Increases in South Asia are especially large, and are seen in all scenarios, with 2100 values more than double 2000 in some scenarios and reaching >1300 mgN/m2/yr averaged over regional to continental scale regions in RCP 2.6 and 8.5, ~30-50% larger than the values in any region currently (2000). Despite known issues, the new ACCMIP deposition dataset provides novel, consistent and evaluated global gridded deposition fields for use in a wide range of climate and ecological studies.

ACCMIP↗

Introduction to the Special Section on Fast Physics in Climate Models: Parameterization, Evaluation and Observation

Many physical processes that influence Earth’s climate occur at scales of space and time that are too small to be resolved by typical global climate models (GCMS). These processes – e.g., aerosol/cloud physics, convection, boundary layer processes, turbulence, radiation, and land-atmosphere interactions–must be parameterized in GCMs, and are collectively called fast physics or fast processes for convenience.

54 ENVIRONMENTAL SCIENCES↗

Final Technical Report of ASR project entitled “ARM Observations for the Development and Evaluation of Models and Parameterizations of Cloudy Boundary Layers” (DE-SC0000825)

This project aims to elucidate the processes governing boundary layer clouds and improve the treatment of cloud processes in Global Climate Models (GCMs). Specifically, we have made research effort in following areas: (1) Developing novel numerical approach of using multiple scale Weather Research & Forecasting (WRF) model simulations for boundary layer cloud research; (2) Addressing issues of PDF schemes for parameterizing sub-grid scale cloud radiative properties; (3) Investigating the impact of mesoscale cloud organizations on the evolution of boundary layer clouds; (4) Evaluating parameterizations of the cumulus induced vertical transport; (5) Limited area model (LAM) intercomparison study of TWP-ICE convective case; (6) Investigating convective invigoration processes at shallow cumulus cold poll boundaries; and (7) Investigating vertical transport processes in moist convection.

54 ENVIRONMENTAL SCIENCES↗

Reduction of a detailed soot model for simulations of pyrolysing solid fuels

Here, we introduce a computationally inexpensive soot formation model for use in simulating large solid-complex fuel systems. The proposed model evaluates three variables: number density of tars, bulk mass density of soot particles, and particle number density. Each of these variables are influenced by soot formation phenomena, and this model evaluates the most common of these phenomena: soot nucleation, surface reactions, and coagulation. Two separate simulations were carried out using this model and results were compared against simulations using a more detailed and computationally expensive model. Findings are promising and show a meaningful reduction in model complexity and computational cost while still giving comparable results.

58 GEOSCIENCES↗

An economic model for evaluating high-speed aircraft designs

A Class 1 method for determining whether further development of a new aircraft design is desirable from all viewpoints is presented. For the manufacturer the model gives an estimate of the total cost of research and development from the preliminary design to the first production aircraft. Using Wright's law of production, one can derive the average cost per aircraft produced for a given break-even number. The model will also provide the airline with a good estimate of the direct and indirect operating costs. From the viewpoint of the passenger, the model proposes a tradeoff between ticket price and cruise speed. Finally all of these viewpoints are combined in a Comparative Aircraft Seat-kilometer Economic Index.

Vandervelden, Alexander J. M.↗

Detailed Modeling and Evaluation of a Scalable Multilevel Checkpointing System

High-performance computing (HPC) systems are growing more powerful by utilizing more components. As the system mean time before failure correspondingly drops, applications must checkpoint frequently to make progress. But, at scale, the cost of checkpointing becomes prohibitive. A solution to this problem is multilevel checkpointing, which employs multiple types of checkpoints in a single run. Moreover, lightweight checkpoints can handle the most common failure modes, while more expensive checkpoints can handle severe failures. We designed a multilevel checkpointing library, the Scalable Checkpoint/Restart (SCR) library, that writes lightweight checkpoints to node-local storage in addition to the parallel file system. We present probabilistic Markov models of SCR's performance. We show that on future large-scale systems, SCR can lead to a gain in machine efficiency of up to 35 percent, and reduce the load on the parallel file system by a factor of two. In addition, we predict that checkpoint scavenging, or only writing checkpoints to the parallel file system on application termination, can reduce the load on the parallel file system by 20 × on today's systems and still maintain high application efficiency.

97 MATHEMATICS AND COMPUTING↗

Integrated analytical method to demonstrate 10 CFR 50.46c compliance

Large break loss-of-coolant accident (LOCA) evaluation models (EMs) have historically been focused on fresh fuel at or near beginning-of-life in water cooled reactors. However, these evaluation models did not account for fuel thermal conductivity degradation (TCD). With the explicit modeling of fuel TCD, it is possible that the limiting LOCA conditions could occur for higher burnup fuel. NRC-sponsored research conducted by ANL as part of the 10 CFR 50.46c rulemaking identified mechanisms for cladding embrittlement that exist in higher burnup fuel which were not previously well understood. Additionally, the explicit consideration of fuel pellet thermal conductivity degradation results in more severe fuel rod / assembly considerations for higher burnup fuel in the large break LOCA scenario. In aggregate, these embrittlement mechanisms and phenomena necessitate the consideration of all fuel in the core which undergoes a postulated LOCA, as the limiting fuel cannot be determined a priori for each of the analysis acceptance criteria. The Full Spectrum LOCA EM accounts for all assemblies in the core at the time of a postulated LOCA by defining representative hot assemblies which combine the limiting elements of the fuel assemblies which they represent. By doing so, it is ensured that the LOCA analysis result represents the more limiting assembly in the core for each criterion. Changes in the LOCA boundary conditions (fuel rod initialization, axial power distributions, etc.) that occur during a cycle of operation are also appropriately accounted for in the FSLOCA EM by integrating fuel rod design and nuclear design methods into the LOCA EM. As such, the FSLOCA EM is well-positioned to show compliance with the 10 CFR 50.46c rulemaking as the rule package is nearing completion.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗