Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,369 records · Page 76

Evaluation of ADAM/1 model for advanced coal extraction concepts

Several existing computer programs for estimating life cycle cost of mining systems were evaluated. A commercially available program, ADAM/1 was found to be satisfactory in relation to the needs of the advanced coal extraction project. Two test cases were run to confirm the ability of the program to handle nonconventional mining equipment and procedures. The results were satisfactory. The model, therefore, is recommended to the project team for evaluation of their conceptual designs.

Deshpande, G. K.↗

Evaluation of Full Reynolds Stress Turbulence Models in FUN3D

Full seven-equation Reynolds stress turbulence models are a relatively new and promising tool for todays aerospace technology challenges. This paper uses two stress-omega full Reynolds stress models to evaluate challenging flows including shock-wave boundary layer interactions, separation and mixing layers. The Wilcox and the SSG/LRR full second-moment Reynolds stress models have been implemented into the FUN3D (Fully Unstructured Navier-Stokes Three Dimensional) unstructured Navier-Stokes code and are evaluated for four problems: a transonic two-dimensional diffuser, a supersonic axisymmetric compression corner, a compressible planar shear layer, and a subsonic axisymmetric jet. Simulation results are compared with experimental data and results using the more commonly used Spalart-Allmaras (SA) one-equation and the Menter Shear Stress Transport (SST-V) two-equation turbulence models.

reynolds averaging↗

(GO) 2 -SIM: a GCM-oriented ground-observation forward-simulator framework for objective evaluation of cloud and precipitation phase

General circulation model (GCM) evaluation using ground-based observations is complicated by inconsistencies in hydrometeor and phase definitions. Here we describe (GO) 2 -SIM, a forward simulator designed for objective hydrometeor-phase evaluation, and assess its performance over the North Slope of Alaska using a 1-year GCM simulation. For uncertainty assessment, 18 empirical relationships are used to convert model grid-average hydrometeor (liquid and ice, cloud, and precipitation) water contents to zenith polarimetric micropulse lidar and Ka-band Doppler radar measurements, producing an ensemble of 576 forward-simulation realizations. Sensor limitations are represented in forward space to objectively remove from consideration model grid cells with undetectable hydrometeor mixing ratios, some of which may correspond to numerical noise. Phase classification in forward space is complicated by the inability of sensors to measure ice and liquid signals distinctly. However, signatures exist in lidar–radar space such that thresholds on observables can be objectively estimated and related to hydrometeor phase. The proposed phase-classification technique leads to misclassification in fewer than 8 % of hydrometeor-containing grid cells. Such misclassifications arise because, while the radar is capable of detecting mixed-phase conditions, it can mistake water- for ice-dominated layers. However, applying the same classification algorithm to forward-simulated and observed fields should generate hydrometeor-phase statistics with similar uncertainty. Alternatively, choosing to disregard how sensors define hydrometeor phase leads to frequency of occurrence discrepancies of up to 40 %. So, while hydrometeor-phase maps determined in forward space are very different from model reality they capture the information sensors can provide and thereby enable objective model evaluation.

54 ENVIRONMENTAL SCIENCES↗

(GO)2-SIM: a GCM-Oriented Ground-Observation Forward-Simulator Framework for Objective Evaluation of Cloud and Precipitation Phase

General circulation model (GCM) evaluation using ground-based observations is complicated by inconsistencies in hydrometeor and phase definitions. Here we describe (GO)2-SIM, a forward simulator designed for objective hydrometeor-phase evaluation, and assess its performance over the North Slope of Alaska using a 1-year GCM simulation. For uncertainty assessment, 18 empirical relationships are used to convert model grid-average hydrometeor (liquid and ice, cloud, and precipitation) water contents to zenith polarimetric micropulse lidar and Ka-band Doppler radar measurements, producing an ensemble of 576 forward-simulation realizations. Sensor limitations are represented in forward space to objectively remove from consideration model grid cells with undetectable hydrometeor mixing ratios, some of which may correspond to numerical noise.Phase classification in forward space is complicated by the inability of sensors to measure ice and liquid signals distinctly. However, signatures exist in lidar–radar space such that thresholds on observables can be objectively estimated and related to hydrometeor phase. The proposed phase-classification technique leads to misclassification in fewer than 8% of hydrometeor-containing grid cells. Such misclassifications arise because, while the radar is capable of detecting mixed-phase conditions, it can mistake water- for ice-dominated layers. However, applying the same classification algorithm to forward-simulated and observed fields should generate hydrometeor-phase statistics with similar uncertainty. Alternatively, choosing to disregard how sensors define hydrometeor phase leads to frequency of occurrence discrepancies of up to 40%. So, while hydrometeor-phase maps determined in forward space are very different from model "reality" they capture the information sensors can provide and thereby enable objective model evaluation.

Lamer, K.↗

Global distribution and climate forcing of marine organic aerosol: 1. Model improvements and evaluation

Marine organic aerosol emissions have been implemented and evaluated within the National Center of Atmospheric Research (NCAR)'s Community Atmosphere Model (CAM5) with the Pacific Northwest National Laboratory's 7-mode Modal Aerosol Module (MAM-7). Emissions of marine primary organic aerosols (POA), phytoplanktonproduced isoprene- and monoterpenes-derived secondary organic aerosols (SOA) and methane sulfonate (MS{sup -}) are shown to affect surface concentrations of organic aerosols in remote marine regions. Global emissions of submicron marine POA is estimated to be 7.9 and 9.4 Tg yr{sup -1}, for the Gantt et al. (2011) and Vignati et al. (2010) emission parameterizations, respectively. Marine sources of SOA and particulate MS{sup -} (containing both sulfur and carbon atoms) contribute an additional 0.2 and 5.1 Tg yr{sup -1}, respectively. Widespread areas over productive waters of the Northern Atlantic, Northern Pacific, and the Southern Ocean show marine-source submicron organic aerosol surface concentrations of 100 ngm{sup -3}, with values up to 400 ngm{sup -3} over biologically productive areas. Comparison of long-term surface observations of water insoluble organic matter (WIOM) with POA concentrations from the two emission parameterizations shows that despite revealed discrepancies (often more than a factor of 2), both Gantt et al. (2011) and Vignati et al. (2010) formulations are able to capture the magnitude of marine organic aerosol concentrations, with the Gantt et al. (2011) parameterization attaining better seasonality. Model simulations show that the mixing state of the marine POA can impact the surface number concentration of cloud condensation nuclei (CCN). The largest increases (up to 20 %) in CCN (at a supersaturation (S) of 0.2 %) number concentration are obtained over biologically productive ocean waters when marine organic aerosol is assumed to be externally mixed with sea-salt. Assuming marine organics are internally-mixed with sea-salt provides diverse results with increases and decreases in the concentration of CCN over different parts of the ocean. The sign of the CCN change due to the addition of marine organics to seasalt aerosol is determined by the relative significance of the increase in mean modal diameter due to addition of mass, and the decrease in particle hygroscopicity due to compositional changes in marine aerosol. Based on emerging evidence for increased CCN concentration over biologically active surface ocean areas/periods, our study suggests that treatment of sea spray in global climate models (GCMs) as an internal mixture of marine organic aerosols and sea-salt will likely lead to an underestimation in CCN number concentration.

03 NATURAL GAS↗

Automating and Evaluating Large Language Models for Accurate Text Summarization Under Zero-Shot Conditions

Automated text summarization (ATS) is crucial for collecting specialized, domain-specific information. Zero-shot learning (ZSL) allows large language models (LLMs) to respond to prompts on information not included in their training, playing a vital role in this process. This study evaluates LLMs' effectiveness in generating accurate summaries under ZSL conditions and explores using retrieval augmented generation (RAG) and prompt engineering to enhance factual accuracy and understanding. We combined LLMs with summarization modeling, prompt engineering, and RAG, evaluating the summaries using the METEOR metric and keyword frequencies through word clouds. Results indicate that LLMs are generally well-suited for ATS tasks, demonstrating an ability to handle specialized information under ZSL conditions with RAG. However, web scraping limitations hinder a single generalized retrieval mechanism. While LLMs show promise for ATS under ZSL conditions with RAG, challenges like goal misgeneralization and web scraping limitations need addressing. Future research should focus on solutions to these issues.

Priebe Mendes Rocha, Maria Eduarda [ORNL]↗

Bayesian performance evaluation of evapotranspiration models based on eddy covariance systems in an arid region

Abstract. Evapotranspiration (ET) is a major component of the land surface processinvolved in energy fluxes and energy balance, especially in the hydrologicalcycle of agricultural ecosystems. While many models have been developed aspowerful tools to simulate ET, there is no agreement on which model bestdescribes the loss of water to the atmosphere. This study focuses on twoaspects, evaluating the performance of four widely used ET models andidentifying parameters, and the physical mechanisms that havesignificant impacts on the model performance. The four tested models arethe Shuttleworth–Wallace (SW) model, Penman–Monteith (PM) model,Priestley–Taylor and Flint–Childs (PT–FC) model, and advection–aridity (AA)model. By incorporating the mathematically rigorous thermodynamicintegration algorithm, the Bayesian model evidence (BME) approach is adoptedto select the optimal model with half-hourly ET observations obtained at aspring maize field in an arid region. Our results reveal that the SW model has thebest performance, and the extinction coefficient is not merely partitioningthe total available energy into the canopy and surface but also includingthe energy imbalance correction. The extinction coefficient is wellconstrained in the SW model and poorly constrained in the PM model but notconsidered in PT–FC and AA models. This is one of the main reasons that theSW model outperforms the other models. Meanwhile, the good fitting of SWmodel to observations can counterbalance its higher complexity. In addition,the detailed analysis of the discrepancies between observations and modelsimulations during the crop growth season indicate that explicit treatmentof energy imbalance and energy interaction will be the primary way offurther improving ET model performance.

54 ENVIRONMENTAL SCIENCES↗

Simulation-modeling tool for evaluation of space telerobotic control strategies

An object-oriented software graphic simulation and modeling tool is developed to explore human/autonomous control allocation for varied operational scenarios. The system provides designers with LISP workstations through which to simulate telerobotic operations, model human performance in those operations, examine task performance profiles, and investigate control and diagnostic strategies. It is suggested that the provision of such analytic tools along with the presentation of performance projections can assist designers in rapidly and conveniently exploring design alternatives for telerobotic system control.

Corker, Kevin↗

An Exploratory Approach Using Regression and Machine Learning in the Analysis of Mass Absorption Cross Section of Black Carbon Aerosols: Model Development and Evaluation

Mass absorption cross-section of black carbon (MAC BC ) describes the absorptive cross-section per unit mass of black carbon, and is, thus, an essential parameter to estimate the radiative forcing of black carbon. Many studies have sought to estimate MAC BC from a theoretical perspective, but these studies require the knowledge of a set of aerosol properties, which are difficult and/or labor-intensive to measure. We therefore investigate the ability of seven data analytical approaches (including different multivariate regressions, support vector machine, and neural networks) in predicting MAC BC for both ambient and biomass burning measurements. Our model utilizes multi-wavelength light absorption and scattering as well as the aerosol size distributions as input variables to predict MAC BC across different wavelengths. We assessed the applicability of the proposed approaches in estimating MAC BC using different statistical metrics (such as coefficient of determination (R 2 ), mean square error (MSE), fractional error, and fractional bias). Overall, the approaches used in this study can estimate MAC BC appropriately, but the prediction performance varies across approaches and atmospheric environments. Based on an uncertainty evaluation of our models and the empirical and theoretical approaches to predict MAC BC , we preliminarily put forth support vector machine (SVM) as a recommended data analytical technique for use. We provide an operational tool built with the approaches presented in this paper to facilitate this procedure for future users.

54 ENVIRONMENTAL SCIENCES↗

A model propulsion simulator for evaluating counter rotating blade characteristics

Three Model Propulsion Simulators (MPS) were designed and built to evaluate candidate counterrotation Ultra bypass fan model blade designs of nominally 2-ft (0.61 m) tip diameter for an advanced 'pusher-type' aircraft engine. These propulsion simulators (nominally 1/5 engine size) are capable of operation over a wide range of subsonic conditions and can deliver up to 750 shaft horsepower per rotor at rotor speeds of 10,000 rpm. The rotor thrust and torque, dynamic blade stresses, and system temperature data are transmitted through an integral telemetry system to facilitate data acquisition. Salient features of the design, instrumentation, and operation of these simulators are described in this paper.

Delaney, B. R.↗

Global Modeling of Tropospheric Chemistry with Assimilated Meteorology: Model Description and Evaluation

We present a first description and evaluation of GEOS-CHEM, a global three-dimensional (3-D) model of tropospheric chemistry driven by assimilated meteorological observations from the Goddard Earth Observing System (GEOS) of the NASA Data Assimilation Office (DAO). The model is applied to a 1-year simulation of tropospheric ozone-NOx-hydrocarbon chemistry for 1994, and is evaluated with observations both for 1994 and for other years. It reproduces usually to within 10 ppb the concentrations of ozone observed from the worldwide ozonesonde data network. It simulates correctly the seasonal phases and amplitudes of ozone concentrations for different regions and altitudes, but tends to underestimate the seasonal amplitude at northern midlatitudes. Observed concentrations of NO and peroxyacetylnitrate (PAN) observed in aircraft campaigns are generally reproduced to within a factor of 2 and often much better. Concentrations of HNO3 in the remote troposphere are overestimated typically by a factor of 2-3, a common problem in global models that may reflect a combination of insufficient precipitation scavenging and gas-aerosol partitioning not resolved by the model. The model yields an atmospheric lifetime of methylchloroform (proxy for global OH) of 5.1 years, as compared to a best estimate from observations of 5.5 plus or minus 0.8 years, and simulates H2O2 concentrations observed from aircraft with significant regional disagreements but no global bias. The OH concentrations are approximately 20% higher than in our previous global 3-D model which included an UV-absorbing aerosol. Concentrations of CO tend to be underestimated by the model, often by 10-30 ppb, which could reflect a combination of excessive OH (a 20% decrease in model OH could be accommodated by the methylchloroform constraint) and an underestimate of CO sources (particularly biogenic). The model underestimates observed acetone concentrations over the South Pacific in fall by a factor of 3; a missing source from the ocean may be implicated.

Bey, Isabelle↗

Evaluation of joint probability density function models for turbulent nonpremixed combustion with complex chemistry

Two types of mixing sub-models are evaluated in connection with a joint-scalar probability density function method for turbulent nonpremixed combustion. Model calculations are made and compared to simulation results for homogeneously distributed methane-air reaction zones mixing and reacting in decaying turbulence within a two-dimensional enclosed domain. The comparison is arranged to ensure that both the simulation and model calculations a) make use of exactly the same chemical mechanism, b) do not involve non-unity Lewis number transport of species, and c) are free from radiation loss. The modified Curl mixing sub-model was found to provide superior predictive accuracy over the simple relaxation-to-mean submodel in the case studied. Accuracy to within 10-20% was found for global means of major species and temperature; however, nitric oxide prediction accuracy was lower and highly dependent on the choice of mixing sub-model. Both mixing submodels were found to produce non-physical mixing behavior for mixture fractions removed from the immediate reaction zone. A suggestion for a further modified Curl mixing sub-model is made in connection with earlier work done in the field.

Smith, N. S. A.↗

An eddy-permitting ocean-sea ice general circulation model (E3SMv0-HiLAT03): Description and evaluation

We document modeling setups and present results for simulated ocean and sea ice climate from a recently developed, intermediate-resolution global ocean-sea ice model [Energy Exascale Earth System Model version 0 configured for High-Latitude Application and Testing at 0.3° resolution (E3SMv0-HiLAT03)]. The horizontal resolution of HiLAT03 ranges from 33 km in the tropics to 8.5 km at high latitudes, with 100 vertical levels. This resolution allows the explicit representation of some mesoscale eddies, particularly at lower latitudes, therefore being named as an “eddy-permitting model”. Analyses are presented based on the output of two 186-yr control simulations forced by modern atmospheric conditions of climatological and inter-annually varying data sets, respectively. Results are compared to available observational data sets and the 1° NCAR model, which has somewhat similar ocean physics but a coarser resolution and an earlier version of the sea ice component. Analyses focus on ocean temperature and salinity fields, ocean dynamics and circulations, sea ice concentrations and thicknesses. The HiLAT03 model does reasonably well in most oceanic aspects evaluated here, especially in ocean meridional heat transport, despite the fact that no explicit eddy parameterization is applied. Many Arctic sea ice features are much improved compared with the 1° NCAR model, but the Antarctic summer sea ice is still generally low in concentrations and thicknesses. In addition, we discuss the prolonged deep convection (“Antarctic prolonged polynya”) over the Southern Ocean in the inter-annually forced case.

54 ENVIRONMENTAL SCIENCES↗

Evaluation of Trapped Radiation Model Uncertainties for Spacecraft Design

The standard AP8 and AE8 models for predicting trapped proton and electron environments have been compared with several sets of flight data to evaluate model uncertainties. Model comparisons are made with flux, dose, and activation measurements made on various U.S. low-Earth orbit satellites (APEX, CRRES, DMSP. LDEF, NOAA) and Space Shuttle flights, on Russian satellites (Photon-8, Cosmos-1887, Cosmos-2044), and on the Russian Mir space station. This report gives a summary of the model-data given in a companion report. Results from the model comparisons with flight data show, for example, that the AP8 model underpredicts the trapped proton flux at low altitudes by a factor of about two (independent of proton energy and solar cycle conditions), and that the AE8 model overpredict the flux in the outer electron belt be an order of magnitude or more.

Armstrong, T. W.↗

Evaluation of Trapped Radiation Model Uncertainties for Spacecraft Design

The standard AP8 and AE8 models for predicting trapped proton and electron environments have been compared with several sets of flight data to evaluate model uncertainties. Model comparisons are made with flux, dose, and activation measurements made on various U.S. low-Earth orbit satellites (APEX, CRRES, DMSP, LDEF, NOAA) and Space Shuttle flights, on Russian satellites (Photon-8, Cosmos-1887, Cosmos-2044), and on the Russian Mir Space Station. This report gives a summary of the model-data comparisons-detailed results are given in a companion report. Results from the model comparisons with flic,ht data show, for example, the AP8 model underpredicts the trapped proton flux at low altitudes by a factor of about two (independent of proton energy and solar cycle conditions), and that the AE8 model overpredicts the flux in the outer electron belt by an order of magnitude or more.

Armstrong, T. W.↗

Evaluation of Turbulence Models for Unsteady Flows of an Oscillating Airfoil

Unsteady flowfields of a two-dimensional oscillating airfoil are calculated using an implicit, finite-difference, Navier Stokes numerical scheme. Five widely used turbulence models are used with the numerical scheme to assess the accuracy and suitability of the models for simulating the retreating blade stall of helicopter rotor in forward flight. Three unsteady flow conditions corresponding to an essentially attached flow, light-stall, and deep-stall cases of an oscillating NACA 0015 wing experiment were chosen as test cases for computations. Results of unsteady airloads hysteresis curves, harmonics of unsteady pressures, and instantaneous flowfield patterns are presented. Some effects of grid density, time-step size, and numerical dissipation on the unsteady solutions relevant to the evaluation of turbulence models are examined. Comparison of unsteady airloads with experimental data show that all models tested are deficient in some sense and no single model predicts airloads consistently and in agreement with experiment for the three flow regimes. The chief findings are that the simple algebraic model based on the renormalization group theory (RNG) offers some improvement over the Baldwin Lomax model in all flow regimes with nearly same computational cost. The one-equation models provide significant improvement over the algebraic and the half-equation models but have their own limitations. The Baldwin-Barth model overpredicts separation and underpredicts reattachment. In contrast, the Spalart-Allmaras model underpredicts separation and overpredicts reattachment.

Srinivasan, G. R.↗

Novel reusable animal model for comparative evaluation of in vivo growth and protein-expression of Escherichia coli O157 strains in the bovine rumen

Previously, we had demonstrated that Escherichia coli O157:H7 (O157) strain 86–24 expresses proteins involved in survival rather than virulence in vitro in rumen fluid from dairy cattle limit fed a maintenance diet. Here, we verified if this observation would be true for different O157 strains grown in vitro in rumen fluid from, and in vivo in the rumen of, animals on contrasting maintenance (high fiber) and lactation (high energy-protein) diets usually limit fed to dairy cattle. For the in vivo studies, an economical, novel, reusable and non-terminal rumen-fistulated animal model permitting simultaneous evaluation of multiple bacterial strains in the bovine rumen was developed. All experiments were conducted in duplicate using different animals to account for host-related variations. The O157 strains included, 86–24, EDL933 and the super shed SS-17. E. coli Nal R (#5735), derived from a bovine intestinal commensal E. coli, was included as a control. As expected, diet influenced ruminal pH and volatile fatty acid (VFA) composition. The pH ranged from 6.2–7.0 and total VFA concentrations from 109–141 μM/ml, in animals fed the maintenance diet. In comparison, animals fed the lactation diet had a ruminal pH ranging between 5.18–6.0, and total VFA of 125–219 μM/ml. Strain dependent differences in O157 recovery from the rumen fluid of cattle fed either diet was observed, both in vitro and in vivo, with O157 strains 86–24 and EDL933 demonstrating similar survival patterns. Analysis of the O157 proteomes expressed in the rumen fluid/rumen verified previous observations of adaptive responses. Any difference in the adaptive response was mainly influenced by the animal’s diet and growth conditions (in vitro and in vivo) and not the O157 strain. These new insights into the O157 responses could help formulate modalities to control O157 across strains in cattle at all stages of husbandry.

59 BASIC BIOLOGICAL SCIENCES↗