Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Human Modeling Evaluations in Microgravity Workstation and Restraint Development

The International Space Station (ISS) will provide long-term missions which will enable the astronauts to live and work, as well as, conduct research in a microgravity environment. The dominant factor in space affecting the crew is "weightlessness" which creates a challenge for establishing workstation microgravity design requirements. The crewmembers will work at various workstations such as Human Research Facility (HRF), Microgravity Sciences Glovebox (MSG) and Life Sciences Glovebox (LSG). Since the crew will spend considerable amount of time at these workstations, it is critical that ergonomic design requirements are integral part of design and development effort. In order to achieve this goal, the Space Human Factors Laboratory in the Johnson Space Center Flight Crew Support Division has been tasked to conduct integrated evaluations of workstations and associated crew restraints. Thus, a two-phase approach was used: 1) ground and microgravity evaluations of the physical dimensions and layout of the workstation components, and 2) human modeling analyses of the user interface. Computer-based human modeling evaluations were an important part of the approach throughout the design and development process. Human modeling during the conceptual design phase included crew reach and accessibility of individual equipment, as well as, crew restraint needs. During later design phases, human modeling has been used in conjunction with ground reviews and microgravity evaluations of the mock-ups in order to verify the human factors requirements. (Specific examples will be discussed.) This two-phase approach was the most efficient method to determine ergonomic design characteristics for workstations and restraints. The real-time evaluations provided a hands-on implementation in a microgravity environment. On the other hand, only a limited number of participants could be tested. The human modeling evaluations provided a more detailed analysis of the setup. The issues identified during the real-time testing were investigated in the human modeling analyses. In some cases, the opposite was true where preliminary human modeling analyses provided the design engineers with critical issues that needed to be addressed further. This extensive approach provided an effective means to fully address ergonomic design considerations and accurately identify critical issues.

Whitmore, Mihriban↗

Comparative Model Evaluation Studies of Biogenic Trace Gas Fluxes in Tropical Forests

Simulation modeling can play a number of important roles in large-scale ecosystem studies, including synthesis of patterns and changes in carbon and nutrient cycling dynamics, scaling up to regional estimates, and formulation of testable hypotheses for process studies. Recent comparative studies have shown that ecosystem models of soil trace gas exchange with the atmosphere are evolving into several distinct simulation approaches. Different levels of detail exist among process models in the treatment of physical controls on ecosystem nutrient fluxes and organic substrate transformations leading to gas emissions. These differences are is in part from distinct objectives of scaling and extrapolation. Parameter requirements for initialization scalings, boundary conditions, and time-series driven therefore vary among ecosystem simulation models, such that the design of field experiments for integration with modeling should consider a consolidated series of measurements that will satisfy most of the various model requirements. For example, variables that provide information on soil moisture holding capacity, moisture retention characteristics, potential evapotranspiration and drainage rates, and rooting depth appear to be of the first order in model evaluation trials for tropical moist forest ecosystems. The amount and nutrient content of labile organic matter in the soil, based on accurate plant production estimates, are also key parameters that determine emission model response. Based on comparative model results, it is possible to construct a preliminary evaluation matrix along categories of key diagnostic parameters and temporal domains. Nevertheless, as large-scale studied are planned, it is notable that few existing models age designed to simulate transient states of ecosystem change, a feature which will be essential for assessment of anthropogenic disturbance on regional gas budgets, and effects of long-term climate variability on biosphere-atmosphere exchange.

Potter, C. S.↗

Model Evaluation Report for Corrective Action Unit 98: Frenchman Flat, Nevada National Security Site, Nye County, Nevada, Revision 1

Model evaluation focused solely on the PIN STRIPE and MILK SHAKE underground nuclear tests’ contaminant boundaries (CBs) because they had the largest extent, uncertainty, and potential consequences. The CAMBRIC radionuclide migration experiment also had a relatively large CB, but because it was constrained by transport data (notably Well UE-5n), there was little uncertainty, and radioactive decay reduced concentrations before much migration could occur. Each evaluation target and the associated data-collection activity were assessed in turn to determine whether the new data support, or demonstrate conservatism of, the CB forecasts. The modeling team—in this case, the same team that developed the Frenchman Flat geologic, source term, and groundwater flow and transport models—analyzed the new data and presented the results to a PER committee. Existing site understanding and its representation in numerical groundwater flow and transport models was evaluated in light of the new data and the ability to proceed to the CR stage of long-term monitoring and institutional control.

54 ENVIRONMENTAL SCIENCES↗

Solid rocket booster performance evaluation model. Volume 3: Sample case

The solid rocket booster performance evaluation model (SRB-11) is used to predict internal ballistics in a sample motor. This motor contains a five segmented grain. The first segment has a 14 pointed star configuration with a web which wraps partially around the forward dome. The other segments are circular in cross-section and are tapered along the interior burning surface. Two of the segments are inhibited on the forward face. The nozzle is not assumed to be submerged. The performance prediction is broken into two simulation parts: the delivered end item specific impulse and the propellant properties which are required as inputs for the internal ballistics module are determined; and the internal ballistics for the entire burn duration of the motor are simulated.

Source record↗

Greenland Surface Mass Balance as Simulated by the Community Earth System Model. Part I: Model Evaluation and 1850–2005 Results

The modeling of the surface mass balance (SMB) of the Greenland Ice Sheet (GIS) requires high-resolution models in order to capture the observed large gradients in the steep marginal areas. Until now, global climate models have not been considered suitable to model ice sheet SMB owing to model biases and insufficient resolution. This study analyzes the GIS SMB simulated for the period 1850–2005 by the Community Earth System Model (CESM), which includes a new ice sheet component with multiple elevation classes for SMB calculations. The model is evaluated against observational data and output from the regional model Regional Atmospheric Climate Model version 2 (RACMO2). Because of a lack of major climate biases, a sophisticated calculation of snow processes (including surface albedo evolution) and an adequate downscaling technique, CESM is able to realistically simulate GIS surface climate and SMB. CESM SMB agrees reasonably well with in situ data from 475 locations ($r$ = 0.80) and output from RACMO2 ($r$ = 0.79). The simulated mean SMB for 1960–2005 is 359 ± 120 Gt yr -1 in the range of estimates from regional climate models. The simulated seasonal mass variability is comparable with mass observations from the Gravity Recovery and Climate Experiment (GRACE), with synchronous annual maximum (May) and minimum (August–September) and similar amplitudes of the seasonal cycle. CESM is able to simulate the bands of precipitation maxima along the southeast and northwest margins, but absolute precipitation rates are underestimated along the southeastern margin and overestimated in the high interior. The model correctly simulates the major ablation areas. Total refreezing represents 35% of the available liquid water (the sum of rain and melt).

54 ENVIRONMENTAL SCIENCES↗

Using Atmospheric Trace Gas Vertical Profiles to Evaluate Model Fluxes: A Case Study of Arctic-Cap Observations and Geos Simulations for the ABoVE Domain

Accurate estimates of carbon–climate feedbacks require an independent means for evaluating surface flux models at regional scales. The altitude-integrated enhancement (AIE) derived from the Arctic Carbon Atmospheric Profiles (Arctic-CAP) project demonstrates the utility of this bulk quantity for surface flux model evaluation. This bulk quantity leverages background mole fraction values from the middle free troposphere, is agnostic to uncertainties in boundary layer height, and can be derived from model estimates of mole fractions and vertical gradients. To demonstrate the utility of the bulk quantity, six airborne profiling surveys of atmospheric carbon dioxide (CO2), methane (CH4), and carbon monoxide (CO) throughout Alaska and northwestern Canada between April and November 2017 were completed as part of NASA’s Arctic–Boreal Vulnerability Experiment (ABoVE). The Arctic-CAP sampling strategy involved acquiring vertical profiles of CO2, CH4, and CO from the surface to 5 km altitude at 25 sites around the ABoVE domain on a 4- to 6-week time interval. All Arctic-CAP measurements were compared to a global simulation using the Goddard Earth Observing System (GEOS) modeling system. Comparisons of the AIE bulk quantity from aircraft observations and GEOS simulations of atmospheric CO2, CH4, and CO highlight the fidelity of the modeled surface fluxes. The model–data comparison over the ABoVE domain reveals that while current state-of-the-art models and flux estimates are able to capture broad-scale spatial and temporal patterns in near-surface CO2 and CH4 concentrations, more work is needed to resolve fine-scale flux features that are captured in CO observations

CO2↗

Geothermal Electricity Technologies Evaluation Model DOE Tool for Assessing Impact of Research on Cost of Power

The U.S. Department of Energy (DOE) has developed a spreadsheet model to provide insight as to how its research activities can impact of cost of producing power from geothermal energy. This model is referred to as GETEM, which stands for “Geothermal Electricity Technologies Evaluation Model”. Based on user input, the model develops estimates of costs associated with exploration, well field development, and power plant construction that are used along with estimated operating costs to provide a predicted power generation cost. The model allows the user to evaluate how reductions in cost, or increases in performance or productivity will impact the predicted power generation cost. This feature provides a means of determining how specific technology improvements can impact generation costs, and as such assists DOE in both prioritizing research areas and identifying where research is needed.

15 GEOTHERMAL ENERGY↗

Process-level model evaluation: a snow and heat transfer metric

Land models require evaluation in order to understand results and guide future development. Examining functional relationships between model variables can provide insight into the ability of models to capture fundamental processes and aid in minimizing uncertainties or deficiencies in model forcing. This study quantifies the proficiency of land models to appropriately transfer heat from the soil through a snowpack to the atmosphere during the cooling season (Northern Hemisphere: October–March). Using the basic physics of heat diffusion, we investigate the relationship between seasonal amplitudes of soil versus air temperatures due to insulation from seasonal snow. Observations demonstrate the anticipated exponential relationship of attenuated soil temperature amplitude with increasing snow depth and indicate that the marginal influence of snow insulation diminishes beyond an effective snow depth of about 50 cm. A snow and heat transfer metric (SHTM) is developed to quantify model skill compared to observations. Land models within the CMIP5 experiment vary widely in SHTM scores, and deficiencies can often be traced to model structural weaknesses. The SHTM value for individual models is stable over 150 years of climate, 1850–2005, indicating that the metric is insensitive to climate forcing and can be used to evaluate each model's representation of the insulation process.

58 GEOSCIENCES↗

The winter central Arctic surface energy budget: A model evaluation using observations from the MOSAiC campaign

This study evaluates the simulation of wintertime (15 October, 2019, to 15 March, 2020) statistics of the central Arctic near-surface atmosphere and surface energy budget observed during the MOSAiC campaign with short-term forecasts from 7 state-of-the-art operational and experimental forecast systems. Five of these systems are fully coupled ocean-sea ice-atmosphere models. Forecast systems need to simultaneously simulate the impact of radiative effects, turbulence, and precipitation processes on the surface energy budget and near-surface atmospheric conditions in order to produce useful forecasts of the Arctic system. This study focuses on processes unique to the Arctic, such as, the representation of liquid-bearing clouds at cold temperatures and the representation of a persistent stable boundary layer. It is found that contemporary models still struggle to maintain liquid water in clouds at cold temperatures. Given the simple balance between net longwave radiation, sensible heat flux, and conductive ground flux in the wintertime Arctic surface energy balance, a bias in one of these components manifests as a compensating bias in other terms. This study highlights the different manifestations of model bias and the potential implications on other terms. Three general types of challenges are found within the models evaluated: representing the radiative impact of clouds, representing the interaction of atmospheric heat fluxes with sub-surface fluxes (i.e., snow and ice properties), and representing the relationship between stability and turbulent heat fluxes.

54 ENVIRONMENTAL SCIENCES↗

Model Evaluation Report for Corrective Action Unit 97: Yucca Flat/Climax Mine, Nevada National Security Site, Nye County, Nevada (Revision 1)

This Model Evaluation Report (MER) has been prepared for Corrective Action Unit (CAU) 97, Yucca Flat/Climax Mine (YF/CM), Nevada National Security Site (NNSS), Nevada. The YF/CM CAU is located in the northeastern portion of the NNSS and comprises 720 corrective action sites. A total of 747 underground nuclear detonations took place within this CAU between 1957 and 1992 and resulted in the release of radionuclides (RNs) in the subsurface in the vicinity of the detonation cavities.

54 ENVIRONMENTAL SCIENCES↗

Climate Model Evaluation using New Datasets from the Clouds and the Earth's Radiant Energy System (CERES)

There are some in the science community who believe that the response of the climate system to anthropogenic radiative forcing is unpredictable and we should therefore call off the quest . The key limitation in climate predictability is associated with cloud feedback. Narrowing the uncertainty in cloud feedback (and therefore climate sensitivity) requires optimal use of the best available observations to evaluate and improve climate model processes and constrain climate model simulations over longer time scales. The Clouds and the Earth s Radiant Energy System (CERES) is a satellite-based program that provides global cloud, aerosol and radiative flux observations for improving our understanding of cloud-aerosol-radiation feedbacks in the Earth s climate system. CERES is the successor to the Earth Radiation Budget Experiment (ERBE), which has widely been used to evaluate climate models both at short time scales (e.g., process studies) and at decadal time scales. A CERES instrument flew on the TRMM satellite and captured the dramatic 1998 El Nino, and four other CERES instruments are currently flying aboard the Terra and Aqua platforms. Plans are underway to fly the remaining copy of CERES on the upcoming NPP spacecraft (mid-2010 launch date). Every aspect of CERES represents a significant improvement over ERBE. While both CERES and ERBE measure broadband radiation, CERES calibration is a factor of 2 better than ERBE. In order to improve the characterization of clouds and aerosols within a CERES footprint, we use coincident higher-resolution imager observations (VIRS, MODIS or VIIRS) to provide a consistent cloud-aerosol-radiation dataset at climate accuracy. Improved radiative fluxes are obtained by using new CERES-derived Angular Distribution Models (ADMs) for converting measured radiances to fluxes. CERES radiative fluxes are a factor of 2 more accurate than ERBE overall, but the improvement by cloud type and at high latitudes can be as high as a factor of 5. Diurnal cycles are explicitly resolved by merging geostationary satellite observations with CERES and MODIS. Atmospheric state data are provided from a frozen version of the Global Modeling and Assimilation Office- Data Assimilation System at the NASA Goddard Space Flight Center. In addition to improving the accuracy of top-of-atmosphere (TOA) radiative fluxes, CERES also produces radiative fluxes at the surface and at several levels in the atmosphere using radiative transfer modeling, constrained at the TOA by CERES (ERBE was limited to the TOA). In all, CERES uses 11 instruments on 7 spacecraft all integrated to obtain climate accuracy in TOA to surface fluxes. This presentation will provide an overview of several new CERES datasets of interest to the climate community (including a new adjusted TOA flux dataset constrained by estimates of heat storage in the Earth system), show direct comparisons between CERES ad ERBE, and provide a detailed error analysis of CERES fluxes at various time and space scales. We discuss how observations can be used to reduce uncertainties in cloud feedback and climate sensitivity and strongly argue why we should NOT "call off the quest".

Loeb, Norman G.↗

Solid rocket booster performance evaluation model. Volume 1: Engineering description

The space shuttle solid rocket booster performance evaluation model (SRB-II) is made up of analytical and functional simulation techniques linked together so that a single pass through the model will predict the performance of the propulsion elements of a space shuttle solid rocket booster. The available options allow the user to predict static test performance, predict nominal and off nominal flight performance, and reconstruct actual flight and static test performance. Options selected by the user are dependent on the data available. These can include data derived from theoretical analysis, small scale motor test data, large motor test data and motor configuration data. The user has several options for output format that include print, cards, tape and plots. Output includes all major performance parameters (Isp, thrust, flowrate, mass accounting and operating pressures) as a function of time as well as calculated single point performance data. The engineering description of SRB-II discusses the engineering and programming fundamentals used, the function of each module, and the limitations of each module.

Source record↗

R&D for computational cognitive and social models : foundations for model evaluation through verification and validation (final LDRD report).

Sandia National Laboratories is investing in projects that aim to develop computational modeling and simulation applications that explore human cognitive and social phenomena. While some of these modeling and simulation projects are explicitly research oriented, others are intended to support or provide insight for people involved in high consequence decision-making. This raises the issue of how to evaluate computational modeling and simulation applications in both research and applied settings where human behavior is the focus of the model: when is a simulation 'good enough' for the goals its designers want to achieve? In this report, we discuss two years' worth of review and assessment of the ASC program's approach to computational model verification and validation, uncertainty quantification, and decision making. We present a framework that extends the principles of the ASC approach into the area of computational social and cognitive modeling and simulation. In doing so, we argue that the potential for evaluation is a function of how the modeling and simulation software will be used in a particular setting. In making this argument, we move from strict, engineering and physics oriented approaches to V&V to a broader project of model evaluation, which asserts that the systematic, rigorous, and transparent accumulation of evidence about a model's performance under conditions of uncertainty is a reasonable and necessary goal for model evaluation, regardless of discipline. How to achieve the accumulation of evidence in areas outside physics and engineering is a significant research challenge, but one that requires addressing as modeling and simulation tools move out of research laboratories and into the hands of decision makers. This report provides an assessment of our thinking on ASC Verification and Validation, and argues for further extending V&V research in the physical and engineering sciences toward a broader program of model evaluation in situations of high consequence decision-making.

97 MATHEMATICS AND COMPUTING↗

p-values for model evaluation

Deciding whether a model provides a good description of data is often based on a goodness-of-fit criterion summarized by a p-value. Although there is considerable confusion concerning the meaning of p-values, leading to their misuse, they are nevertheless of practical importance in common data analysis tasks. We motivate their application using a Bayesian argumentation. We then describe commonly and less commonly known discrepancy variables and how they are used to define p-values. The distribution of these are then extracted for examples modeled on typical data analysis tasks, and comments on their usefulness for determining goodness-of-fit are given.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Taking Climate Model Evaluation to the Next Level

Earth system models are complex and represent a large number of processes, resulting in a persistent spread across climate projections for a given future scenario. Owing to different model performances against observations and the lack of independence among models, there is now evidence that giving equal weight to each available model projection is suboptimal. This Perspective discusses newly developed tools that facilitate a more rapid and comprehensive evaluation of model simulations with observations, process-based emergent constraints that are a promising way to focus evaluation on the observations most relevant to climate projections, and advanced methods for model weighting. These approaches are needed to distil the most credible information on regional climate changes, impacts, and risks for stakeholders and policy-makers.

Eyring, Veronika↗

Solid rocket booster performance evaluation model. Volume 2: Users manual

This users manual for the solid rocket booster performance evaluation model (SRB-II) contains descriptions of the model, the program options, the required program inputs, the program output format and the program error messages. SRB-II is written in FORTRAN and is operational on both the IBM 370/155 and the MSFC UNIVAC 1108 computers.

Source record↗