Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,657 records · Page 92

Using ARM observations to evaluate NWP results of radiation and cloud fraction

This poster will present the results from: (1) Evaluating NWP model performances in simulating surface cloud radiative forcing measured by effective cloud albedo and total cloud fraction by comparing three major reanalysis datasets (ERAInterim, NCEP/NCAR Reanalysis and NCEP/DOE Reanalysis) against decade-long ARM observations; (2) Examining the relationships between the model biases of the cloud properties and those of nearsurface relative humidity and temperature; (3) Evaluating NWP model performances in simulating vertical profiles of cloud fraction; and, (4) Evaluating NWP model skill scores of predicting cloud fraction.

54 ENVIRONMENTAL SCIENCES↗

Evaluation of Data-Driven Models for Predicting Solar Photovoltaics Power Output

This research was undertaken to evaluate different inverse models for predicting power output of solar photovoltaic (PV) systems under different practical scenarios. In particular, we have investigated whether PV power output prediction accuracy can be improved if module/cell temperature was measured in addition to climatic variables, and also the extent to which prediction accuracy degrades if solar irradiation is not measured on the plane of array but only on a horizontal surface. We have also investigated the significance of different independent or regressor variables, such as wind velocity and incident angle modifier in predicting PV power output and cell temperature. The inverse regression model forms have been evaluated both in terms of their goodness-of-fit, and their accuracy and robustness in terms of their predictive performance. Given the accuracy of the measurements, expected CV-RMSE of hourly power output prediction over the year varies between 3.2% and 8.6% when only climatic data are used. Depending on what type of measured climatic and PV performance data is available, different scenarios have been identified and the corresponding appropriate modeling pathways have been proposed. The corresponding models are to be implemented on a controller platform for optimum operational planning of microgrids and integrated energy systems.

14 SOLAR ENERGY↗

The Strengths and Limitations of Satellite Data for Evaluating Tropospheric Processes in Chemistry-Climate Models

There is now a wealth of satellite data products available with which to evaluate a model fs simulation of tropospheric composition and other model processes. All of these data products have their strengths and limitations that need to be considered for this purpose. For example, uncertainties are introduced into a data product when 1) converting a slant column to a vertical column and 2) estimating the amount of a total column of a trace gas (e.g., ozone, nitrogen dioxide) that resides in the troposphere. Oftentimes, these uncertainties are not well quantified and the satellite data products are not well evaluated against in situ observations. However, these limitations do not preclude us from using these data products to evaluate our model processes if we understand these strengths and limitations when developing diagnostics. I will show several examples of how satellite data products are being used to evaluate particular model processes with a focus on the strengths and limitations of these data products. In addition, I will introduce the goals of a newly formed team to address issues on the topic of "satellite data for improved model evaluation and process studies" that is established in support of the IGAC/SPARC Global Chemistry ]Climate Modeling and Evaluation Workshop.

Duncan, Bryan↗

Evaluation of a stochastic model of particle dispersion in a turbulent round jet

A stochastic model of particle dispersion by turbulence, proposed by Gosman and Ioannides (1981), is evaluated. The method employs a k-epsilon model to estimate turbulence properties. Dispersion is determined by computing particle motion, with random sampling to obtain instantaneous flow properties for a statistically significant number of particle trajectories. The stochastic model yields good results particularly when eddy lifetimes are evaluated. The method allows the effects of large relative velocities between the particles and the flow, drag properties at Reynolds numbers greater than the Stokes flow regime, and the variations of local turbulence properties to be readily handled, at least for boundary layer flows.

Shuen, J.-S.↗

A model for interphase chromosomes and evaluation of radiation-induced aberrations

We have developed a theoretical model for evaluating radiation-induced chromosomal exchanges by explicitly taking into account interphase (G(0)/G(1)) chromosome structure, nuclear organization of chromosomes, the production of double-strand breaks (DSBs), and the subsequent rejoinings in a faithful or unfaithful manner. Each of the 46 chromosomes for human lymphocytes (40 chromosomes for mouse lymphocytes) is modeled as a random polymer inside a spherical volume. The chromosome spheres are packed randomly inside a spherical nucleus with an allowed overlap controlled by a parameter Omega. The rejoining of DSBs is determined by a Monte Carlo procedure using a Gaussian proximity function with an interaction range parameter sigma. Values of Omega and sigma have been found which yield calculated results of interchromosomal aberration frequencies that agree with a wide range of experimental data. Our preferred solution is one with an interaction range of 0.5 microm coupled with a relatively small overlap parameter of 0.675 microm, which more or less confirms previous estimates. We have used our model with these parameter values and with resolution or detectability limits to calculate yields of translocations and dicentrics for human lymphocytes exposed to low-LET radiation that agree with experiments in the dose range 0.09 to 4 Gy. Five different experimental data sets have been compared with the theoretical results. Essentially all of the experimental data fall between theoretical curves corresponding to resolution limits of 1 Mbp and 20 Mbp, which may reflect the fact that different investigators use different limits for sensitivity or detectability. Translocation yields for mouse lymphocytes have also been calculated and are in good agreement with experimental data from 1 cGy to 10 cGy. There is also good agreement with recent data on complex aberrations. Our model is expected to be applicable to both low- and high-LET radiation, and we include a sample prediction of the yield of interchromosomal rejoining in the dose range 0.22 Gy to 2 Gy of 1000 MeV/nucleon iron particles. This dose range corresponds to average particle traversals per nucleus ranging from 1.0 to 9.12.

Non-NASA Center↗

Modelling evapotranspiration during precipitation deficits: Identifying critical processes in a land surface model

Surface fluxes from land surface models (LSMs) have traditionally been evaluated against monthly, seasonal or annual mean states. The limited ability of LSMs to reproduce observed evaporative fluxes under water-stressed conditions has been previously noted, but very few studies have systematically evaluated these models during rainfall deficits. We evaluated latent heat fluxes simulated by the Community Atmosphere Biosphere Land Exchange (CABLE) LSM across 20 flux tower sites at sub-annual to inter-annual timescales, in particular focusing on model performance during seasonal-scale rainfall deficits. The importance of key model processes in capturing the latent heat flux was explored by employing alternative representations of hydrology, leaf area index, soil properties and stomatal conductance. We found that the representation of hydrological processes was critical for capturing observed declines in latent heat during rainfall deficits. By contrast, the effects of soil properties, LAI and stomatal conductance were highly site-specific. Whilst the standard model performs reasonably well at annual scales as measured by common metrics, it grossly underestimates latent heat during rainfall deficits. A new version of CABLE, with a more physically consistent representation of hydrology, captures the variation in the latent heat flux during seasonal-scale rainfall deficits better than earlier versions, but remaining biases point to future research needs. Lastly, our results highlight the importance of evaluating LSMs under water-stressed conditions and across multiple plant functional types and climate regimes.

54 ENVIRONMENTAL SCIENCES↗

Comparing multi-model ensemble simulations with observations and decadal projections of upper atmospheric variations following the Hunga eruption

The Hunga Tonga–Hunga Ha'apai Model–Observation Comparison (HTHH–MOC) project aims to comprehensively investigate the evolution of volcanic water vapor and sulfur emissions and their subsequent atmospheric impacts and underlying response mechanisms using state-of-the-art global climate models. This study evaluates multi-model ensemble simulations participating in the HTHH–MOC free-run experiment with climate projections for 10 years (2022–2032). Model results are evaluated against satellite observations to assess their ability to reproduce the observed evolution of stratospheric water vapor, aerosols, temperature, and ozone from 2022 to 2024. The participating models accurately capture the observed distribution patterns and associated upper atmospheric responses, providing confidence for their future projections. Model simulations suggest that the Hunga eruption-induced stratospheric water vapor anomaly lasts 4–7 years, with a water vapor e-folding time of 31–43 months. This prolonged water vapor perturbation leads to significant stratospheric and mesospheric cooling, resulting in significant ozone loss in the upper stratosphere and lower mesosphere for 7–10 years. Comparisons between simulations with both SO 2 and H 2 O emissions and those with H 2 O-only emissions indicate that the pronounced dipole response with upper-stratospheric cooling and lower-stratospheric warming is driven by the combined effects of SO 2 and H 2 O injections. These results highlight the prolonged atmospheric impacts of the Hunga eruption and the potential critical role of stratospheric water vapor in modulating long-term atmospheric chemistry and dynamics.

Zhuo, Zhihong [Univ. of Quebec, Montreal, QC (Cana↗

Climate Model Diagnostic Analyzer

The comprehensive and innovative evaluation of climate models with newly available global observations is critically needed for the improvement of climate model current-state representation and future-state predictability. A climate model diagnostic evaluation process requires physics-based multi-variable analyses that typically involve large-volume and heterogeneous datasets, making them both computation- and data-intensive. With an exploratory nature of climate data analyses and an explosive growth of datasets and service tools, scientists are struggling to keep track of their datasets, tools, and execution/study history, let alone sharing them with others. In response, we have developed a cloud-enabled, provenance-supported, web-service system called Climate Model Diagnostic Analyzer (CMDA). CMDA enables the physics-based, multivariable model performance evaluations and diagnoses through the comprehensive and synergistic use of multiple observational data, reanalysis data, and model outputs. At the same time, CMDA provides a crowd-sourcing space where scientists can organize their work efficiently and share their work with others. CMDA is empowered by many current state-of-the-art software packages in web service, provenance, and semantic search.

cloud computing↗

ML Pipeline (Machine Learning Pipeline) [SWR-22-35]

The Machine Learning Pipeline (ML Pipeline) allows the user to fit a model to predict a dependent variable, y, based on a feature set, X. ML Pipeline gives the ability to automatically generate features with its Feature Engineering function, automatically select the most important features with its Feature Selection function, fit the model with its Fit function, and evaluate the model with its Evaluate function. ML Pipeline's functionality can not only help the user predict values for the desired variable it also gives the user a better understanding of which features are important and how effective the model is. ML Pipeline accomplishes all this, while remaining simple to use and interpret.

Schiek, Andrew↗

Multi-Model Mean Nitrogen and Sulfur Deposition from the Atmospheric Chemistry and Climate Model Intercomparison Project (ACCMIP): Evaluation of Historical and Projected Future Changes

We present multi-model global datasets of nitrogen and sulfate deposition covering time periods from 1850 to 2100, calculated within the Atmospheric Chemistry and Climate Model Intercomparison Project (ACCMIP). The computed deposition fluxes are compared to surface wet deposition and ice core measurements.We use a new dataset of wet deposition for 2000-2002 based on critical assessment of the quality of existing regional network data. We show that for present day (year 2000 ACCMIP time slice), the ACCMIP results perform similarly to previously published multi-model assessments. For this time slice, we find a multimodel mean deposition of approximately 50 Tg(N) yr−1 from nitrogen oxide emissions, 60 Tg(N) yr−1 from ammonia emissions, and 83 Tg(S) yr−1 from sulfur emissions. The analysis of changes between 1980 and 2000 indicates significant differences between model and measurements over the United States but less so over Europe. This difference points towards a potential misrepresentation of 1980 NH3 emissions over North America. Based on ice core records, the 1850 deposition fluxes agree well with Greenland ice cores, but the change between 1850 and 2000 seems to be overestimated in the Northern Hemisphere for both nitrogen and sulfur species. Using the Representative Concentration Pathways (RCPs) to define the projected climate and atmospheric chemistry related emissions and concentrations, we find large regional nitrogen deposition increases in 2100 in Latin America, Africa and parts of Asia under some of the scenarios considered. Increases in South Asia are especially large, and are seen in all scenarios, with 2100 values more than double their 2000 counterpart in some scenarios and reaching >1300 mg(N)m−2 yr−1 averaged over regional to continental-scale regions in RCP 2.6 and 8.5, 30-50% larger than the values in any region currently (circa 2000). However, sulfur deposition rates in 2100 are in all regions lower than in 2000 in all the RCPs. The new ACCMIP multimodel deposition dataset provides state-of-the-science, consistent and evaluated time slice (spanning 1850-2100) global gridded deposition fields for use in a wide range of climate and ecological studies.

Lamarque, J.-F.↗

Validation of Atmospheric Correction of Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) Radiance Data Based on Radiative Transfer Modeling

An evaluation of atmospheric correction of AVIRIS data using radiative transfer codes LOWTRAN7 and MODTRAN is presented. The algorithm employed is based on a simple model of radiance L at each wavelength at the sensor that can be written approximately LAV=Lp+Tp, where subscript AV referes to AVIRIS, Lp is the path radiance and Tp is the diffuse + direct transmitted radiance of the atmosphere at AVIRIS.

Infrared Mapping↗

Applying Human-performance Models to Designing and Evaluating Nuclear Power Plants: Review Guidance and Technical Basis

Human performance models (HPMs) are simulations of human behavior with which we can predict human performance. Designers use them to support their human factors engineering (HFE) programs for a wide range of complex systems, including commercial nuclear power plants. Applicants to U.S. Nuclear Regulatory Commission (NRC) can use HPMs for design certifications, operating licenses, and license amendments. In the context of nuclear-plant safety, it is important to assure that HPMs are verified and validated, and their usage is consistent with their intended purpose. Using HPMs improperly may generate misleading or incorrect information, entailing safety concerns. The objective of this research was to develop guidance to support the NRC staff's reviews of an applicant's use of HPMs in an HFE program. The guidance is divided into three topical areas: (1) HPM Verification, (2) HPM Validation, and (3) User Interface Verification. Following this guidance will help ensure the benefits of HPMs are achieved in a technically sound, defensible manner. During the course of developing this guidance, I identified several issues that could not be addressed; they also are discussed.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Evaluation of the Predictive Accuracy of Five Whole Building Baseline Models

This report documents the relative and absolute performance of five baseline models used to characterize whole­building energy consumption. The Pulse Adaptive Model1, multi-­parameter change-­point, mean-week, day-­time-emperature, and LBNL models were evaluated according to a number of statistical ‘goodness of fit’ metrics, to determine their accuracy in characterizing the energy consumption of a set of 29 buildings. The baseline training period, prediction horizon, and predicted energy quantity (daily, weekly, and monthly energy consumption) were varied, and model predictions were compared to interval meter data to determine the accuracy of each model. Three combinations of baseline training periods and prediction horizons were considered: 6 months of training to generate a 12-­month prediction; 9 months of training to generate a 7-month prediction; and 12 months of training to generate a 6- month prediction.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Radiation Pneumonitis After Hypofractionated Radiotherapy: Evaluation of the LQ(L) Model and Different Dose Parameters

Purpose: To evaluate the linear quadratic (LQ) model for hypofractionated radiotherapy within the context of predicting radiation pneumonitis (RP) and to investigate the effect if a linear (L) model in the high region (LQL model) is used. Methods and Materials: The radiation doses used for 128 patients treated with hypofractionated radiotherapy were converted to the equivalent doses given in fractions of 2 Gy for a range of {alpha}/{beta} ratios (1 Gy to infinity) according to the LQ(L) model. For the LQL model, different cut-off values between the LQ model and the linear component were used. The Lyman model parameters were fitted to the events of RP grade 2 or higher to derive the normal tissue complication probability (NTCP). The lung dose was calculated as the mean lung dose and the percentage of lung volume (V) receiving doses higher than a threshold dose of xGy (V{sub x}). Results: The best NTCP fit was found if the mean lung dose, or V{sub x}, was calculated with an {alpha}/{beta} ratio of 3 Gy. The NTCP fit of other {alpha}/{beta} ratios and the LQL model were worse but within the 95% confidence interval of the NTCP fit of the LQ model with an {alpha}/{beta} ratio of 3 Gy. The V{sub 50} NTCP fit was better than the NTCP fit of lower threshold doses. Conclusions: For high fraction doses, the LQ model with an {alpha}/{beta} ratio of 3 Gy was the best method for converting the physical lung dose to predict RP.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Evaluation of Global Climate Models for Use in Energy Analysis

The interplay between energy, climate, and weather is becoming more complex due to increasing contributions of renewable energy generation, energy storage, electrified end uses, and the increasing frequency of extreme weather events. Energy system analyses commonly rely on meteorological inputs to estimate renewable energy generation and energy demand; however, these inputs rarely represent the estimated impacts of future climate change. Climate models and publicly available climate change datasets can be used for this purpose, but the selection of inputs from the myriad of available models and datasets is a nuanced and subjective process. In this work, we assess datasets from various global climate models (GCMs) from the Coupled Model Intercomparison Project Phase 6 (CMIP6). We present evaluations of their skills with respect to the historical climate and comparisons of their future projections of climate change for two climate change scenarios. We present the results for different climatic and energy system regions and include interactive figures in the accompanying software repository. Previous work has presented similar GCM evaluations, but none have presented variables and metrics specifically intended for comprehensive energy systems analysis including impacts on energy demand, thermal cooling, hydropower, water availability, solar energy generation, and wind energy generation. We focus on GCM output meteorological variables that directly affect these energy system components including the representation of extreme values that can drive grid resilience events. The objective of this work is not to recommend the best climate model and dataset for a given analysis, but instead to provide a reference to facilitate the selection of climate models and scenarios in subsequent work.

14 SOLAR ENERGY↗

Sensitivity of Turbine-Height Wind Speeds to Parameters in Planetary Boundary-Layer and Surface-Layer Schemes in the Weather Research and Forecasting Model

We evaluate the sensitivity of simulated turbine-height winds to 26 parameters applied in a planetary boundary layer (PBL) scheme and a surface layer scheme of the Weather Research and Forecasting (WRF) model over an area of complex terrain during the Columbia Basin Wind Energy Study. An efficient sampling algorithm and a generalized linear model are used to explore the multiple-dimensional parameter space and quantify the parametric sensitivity of modeled turbine-height winds. The results indicate that most of the variability in the ensemble simulations is contributed by parameters related to the dissipation of the turbulence kinetic energy (TKE), Prandtl number, turbulence length scales, surface roughness, and the von Kármán constant. The relative contributions of individual parameters are found to be dependent on both the terrain slope and atmospheric stability. The parameter associated with the TKE dissipation rate is found to be the most important one, and a larger dissipation rate can produce larger hub-height winds. A larger Prandtl number results in weaker nighttime winds. Increasing surface roughness reduces the frequencies of both extremely weak and strong winds, implying a reduction in the variability of the wind speed. All of the above parameters can significantly affect the vertical profiles of wind speed, the altitude of the low-level jet and the magnitude of the wind shear strength. The wind direction is found to be modulated by the same subset of influential parameters. Remainder of abstract is in attachment.

54 ENVIRONMENTAL SCIENCES↗