Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,621 records · Page 90

The June 2012 North American Derecho: A Testbed for Evaluating Regional and Global Climate Modeling Systems at Cloud‐Resolving Scales

Abstract In this paper, we introduce a testbed for evaluating and comparing climate modeling systems at cloud resolving scales using hindcasts of the June 2012 North American derecho. To demonstrate its utility for model intercomparison, the testbed is applied to two models: the regionally‐refined Simple Cloud‐Resolving E3SM Atmosphere Model (SCREAM) at 6.5, 3.25 and 1.625 km grid spacing and the Weather Research and Forecasting (WRF) model with 3.2 and 1.6 km grid spacing. We find the simulation results to be highly sensitive to the initial conditions (ICs), initialization time, and model configurations, with ICs from the Rapid Refresh producing the best simulation. Significant improvement is identified in both models as horizontal grid spacing is refined. While a propagation delay of approximately 2 hr is found in both models, SCREAM at 1.625 km simulates the observed bow echo structure of the derecho well and predicts strong surface gusts that exceed 30 m/s. In comparison, WRF has difficulty producing surface wind over 25 m/s, with wind gusts in WRF 42%–46% lower than in SCREAM. However, WRF has a lower bias in simulating cloud top temperature and extent, but overestimates precipitation intensity. Both models reproduce the observed outgoing longwave radiation spatial patterns well (Pearson correlation >0.88), but, compared with NEXRAD observations, simulate generally larger areas of composite radar reflectivity >40 dBZ and underestimate the precipitating area by ∼47%.

2012 North American derecho↗

A Modified Vegetation Photosynthesis and Respiration Model (VPRM) for the Eastern USA and Canada, Evaluated With Comparison to Atmospheric Observations and Other Biospheric Models

Atmospheric CO 2 measurements from a dense surface network can help to evaluate terrestrial biosphere model (TBM) simulations of Net Ecosystem Exchange (NEE) with two key benefits. First, gridded CO 2 flux estimates can be evaluated over regional scales, not possible using flux tower observations at discrete locations for model evaluation. Second, TBM ability to explain atmospheric CO 2 fluctuations due to the biosphere can be directly tested, an important objective for anthropogenic emissions monitoring using atmospheric observations. Here, we customize the Vegetation Photosynthesis and Respiration Model (VPRM) for an eastern North American domain with strong biological activity upwind of urban areas. Parameters are optimized using flux tower observations from a historical database with sites in (and near) the domain. In addition, the respiration model (originally a linear function of temperature) is modified to account for impacts of changing foliage, non-linear temperature, and water stress. Flux estimates from VPRM, the Carnegie-Ames-Stanford Approach (CASA) model and the Simple Biosphere Model v4 (SiB4), are convolved with footprints from atmospheric transport models for evaluation with CO 2 observations at 21 towers in the domain, with roughly half of the towers used here for the first time. Results show that the new respiration model in VPRM helps to correct a growing season sink bias in the atmosphere associated with underestimated summertime respiration using the original model with annual parameters. The new VPRM also better explains fine-scale atmospheric CO 2 variability compared to other TBMs, due to higher resolution diagnostic phenology, the new respiration model, domain-specific parameters, and high-quality input data sets.

54 ENVIRONMENTAL SCIENCES↗

Chemical kinetic and photochemical data for use in stratospheric modelling

An evaluated set of rate constants and photochemical cross sections were compiled for use in modelling stratospheric processes. The data are primarily relevant to the ozone layer, and its possible perturbation by anthropogenic activities. The evaluation is current to, approximately, January, 1979.

Demore, W. B.↗

Deep learning categorization of infrasound array data

Here we develop a deep learning-based infrasonic detection and categorization methodology that uses convolutional neural networks with self-attention layers to identify stationary and non-stationary signals in infrasound array processing results. Using features extracted from the coherence and direction-of-arrival information from beamforming at different infrasound arrays, our model more reliably detects signals compared with raw waveform data. Using three infrasound stations maintained as part of the International Monitoring System, we construct an analyst-reviewed data set for model training and evaluation. We construct models using a 4-category framework, a generalized noise vs non-noise detection scheme, and a signal-of-interest (SOI) categorization framework that merges short duration stationary and non-stationary categories into a single SOI category. We evaluate these models using a combination of k-fold cross-validation, comparison with an existing “state-of-the-art” detector, and a transportability analysis. Although results are mixed in distinguishing stationary and non-stationary short duration signals, f-scores for the noise vs non-noise and SOI analyses are consistently above 0.96, implying that deep learning-based infrasonic categorization is a highly accurate means of identifying signals-of-interest in infrasonic data records.

47 OTHER INSTRUMENTATION↗

Building a Wide-Area File Transfer Performance Predictor: An Empirical Study

Wide-area data transfer is central to geographically distributed scientific workflows. Faster delivery of data is important for these workflows. Predictability is equally (or even more) important. With the goal of providing a reasonably accurate estimate of data transfer time to improve resource allocation & scheduling for workflows and enable end-to-end data transfer optimization, we apply machine learning methods to develop predictive models for data transfer times over a variety of wide area networks. To build and evaluate these models, we use 201,388 transfers, involving 759 million files totaling 9 PB transferred, over 115 heavily used source-destination pairs (“edges”) between 135 unique endpoints. We evaluate the models for different retraining frequencies and different window size of history data. In the best case, the resulting models have a median prediction error of ≤21% for 50% of the edges, and ≤32% for 75% of the edges. We present a detailed analysis of these results that provides insights into the cause of some of the high errors. We envision that the performance predictor will be informative for scheduling geo-distributed workflows. The insights also suggest obvious directions for both further analysis and transfer service optimization.

Liu, Zhengchun↗

Evaluation of Deep Learning Model Architectures for Point-of-Care Ultrasound Diagnostics

Point-of-care ultrasound imaging is a critical tool for patient triage during trauma for diagnosing injuries and prioritizing limited medical evacuation resources. Specifically, an eFAST exam evaluates if there are free fluids in the chest or abdomen but this is only possible if ultrasound scans can be accurately interpreted, a challenge in the pre-hospital setting. In this effort, we evaluated the use of artificial intelligent eFAST image interpretation models. Widely used deep learning model architectures were evaluated as well as Bayesian models optimized for six different diagnostic models: pneumothorax (i) B- or (ii) M-mode, hemothorax (iii) B- or (iv) M-mode, (v) pelvic or bladder abdominal hemorrhage and (vi) right upper quadrant abdominal hemorrhage. Models were trained using images captured in 27 swine. Using a leave-one-subject-out training approach, the MobileNetV2 and DarkNet53 models surpassed 85% accuracy for each M-mode scan site. The different B-mode models performed worse with accuracies between 68% and 74% except for the pelvic hemorrhage model, which only reached 62% accuracy for all model architectures. These results highlight which eFAST scan sites can be easily automated with image interpretation models, while other scan sites, such as the bladder hemorrhage model, will require more robust model development or data augmentation to improve performance. With these additional improvements, the skill threshold for ultrasound-based triage can be reduced, thus expanding its utility in the pre-hospital setting.

47 OTHER INSTRUMENTATION↗

Evaluating backreaction with the peak model of structure formation

We evaluate the average expansion rate of a universe which contains a realistic evolving ensemble of non-linear structures. We use the peak model of structure formation to obtain the number density of structures, and take the individual structures to be spherical. The expansion rate increases relative to the Friedmann-Robertson-Walker value on a timescale of 10-100 billion years, because the universe becomes dominated by fast expanding voids. However, the increase is not rapid enough to correspond to acceleration. We discuss how to improve our treatment. We also consider various qualitative issues related to backreaction.

79 ASTRONOMY AND ASTROPHYSICS↗

Development of a Random-Forest Cloud-Regime Classification Model Based on Surface Radiation and Cloud Products

Various methods have been developed to characterize cloud type, otherwise referred to as cloud regime. These include manual sky observations, combining radiative and cloud vertical properties observed from satellite, surface-based remote sensing, and digital processing of sky imagers. While each method has inherent advantages and disadvantages, none of these cloud-typing methods actually includes measurements of surface shortwave or longwave radiative fluxes. Here, a method that relies upon detailed, surface-based radiation and cloud measurements and derived data products to train a random-forest machine-learning cloud classification model is introduced. Measurements from five years of data from the ARM Southern Great Plains site were compiled to train and independently evaluate the model classification performance. A cloud-type accuracy of approximately 80% using the random-forest classifier reveals that the model is well suited to predict climatological cloud properties. Furthermore, an analysis of the cloud-type misclassifications is performed. While physical cloud types may be misreported, the shortwave radiative signatures are similar between misclassified cloud types. From this, we assert that the cloud-regime model has the capacity to successfully differentiate clouds with comparable cloud–radiative interactions. Therefore, we conclude that the model can provide useful cloud-property information for fundamental cloud studies, inform renewable energy studies, and be a tool for numerical model evaluation and parameterization improvement, among many other applications.

54 ENVIRONMENTAL SCIENCES↗

Evaluation of a Thermodynamic Ice Crystal Icing Model Using Experimental Ice Accretion Data

This paper presents the evaluation of a thermodynamic ice crystal icing model, previously presented to describe the possible mechanisms of icing within the core of a turbofan jet engine. It has been proposed that there are two types of distinct ice accretions based on a surface energy balance: freeze-dominated icing and melt-dominated icing. In the former, ice accretion occurs where a freeze fraction (0 to 1) of melted ice crystals freezes on a surface, along with the existing ice of the impinging water and ice mass. This freeze-dominated icing is characterized by having strong adhesion to the surface. In the latter, icing occurs from accumulated unmelted ice on a surface, where a melt fraction (0 to 1) dictates the amount of unmelted impinged ice. This melt-dominated icing is characterized by weakly bonded surface adhesion. The experimentally observed ice growth rates suggest that only a small fraction of the impinging ice remains on the surface, implying a mass loss mechanism such as splash, runback, bounce, or erosion. This mass loss parameter must be determined in conjunction with the fraction of freezing liquid water or fraction of melting ice on an icing surface. This loss parameter, however, along with the freeze and melt fraction, are the only experimental parameters that are currently not measured directly. Using reported icing growth rates from published ice crystal icing experiments, a methodology is proposed to determine these unknown parameters. This work takes reported ice accretion data from tests conducted by the National Aeronautics and Space Administration (NASA) in 2016 and tests NASA collaborated on with the National Research Council (NRC) of Canada in 2012 that examined the fundamental physics of ice crystal icing. Those research efforts sought to generate icing conditions representative of those that occur inside a jet engine when ingesting ice crystals. This paper presents the fundamental equations of the thermodynamic model, the methodology used to determine the aforementioned unknown icing parameters, and results from model evaluation using experimental data. In addition, this paper builds on the previously proposed model by adding a transient conduction term to explain ice growth behavior at the onset of experimental tests that was observed to be different from steady-state ice growth that occurred later in the test run.With the addition of this energy term, this becomes a quasi-steady model. A key finding from this work suggests that mass loss fractions can exceed 0.90 for steady ice growth periods. In addition, due to conductive heat fluxes when using a warmer-than-freezing airfoil, lower mass loss fraction values were calculated during the initial transient period.

Ice-Crystal Ice Accretion↗

Optical modeling and physical performances evaluations for the JT-60SA ECRF antenna

The purpose of this work is the optical modeling and physical performances evaluations of the JT-60SA ECRF launcher system. The beams have been simulated with the electromagnetic code GRASP® and used as input for ECCD calculations performed with the beam tracing code GRAY, capable of modeling propagation, absorption and current drive of an EC Gaussion beam with general astigmatism. Full details of the optical analysis has been taken into account to model the launched beams. Inductive and advanced reference scenarios has been analysed for physical evaluations in the full poloidal and toroidal steering ranges for two slightly different layouts of the launcher system.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Variability of building environmental assessment tools on evaluating carbon emissions

With an increasing importance of sustainability in construction, more and more clients and designers employ building environmental assessment (BEA) tools to evaluate the environmental friendliness of their building facilities, and one important aspect of evaluation in the BEA models is the assessment of carbon emissions. However, in the absence of any agreed framework for carbon auditing and benchmarking, the results generated by the BEA tools might vary significantly which could lead to confusion or misinterpretation on the carbon performance of a building. This study thus aims to unveil the properties of and the standard imposed by the current BEA models on evaluating the life cycle carbon emissions. The analyses cover the (i) weighting of energy efficiency and emission levels among various environmental performance indicators; (ii) building life cycle stages in which carbon is taken into consideration; (iii) objectiveness of assessment; (iv) baseline set for carbon assessment; (v) mechanism for benchmarking the emission level; and (v) limitations of the carbon assessment approaches. Results indicate that the current BEA schemes focus primarily on operational carbon instead of the emissions generated throughout the entire building life cycle. Besides, the baseline and benchmark for carbon evaluation vary significantly among the BEA tools based on the analytical results of a hypothetical building. The findings point to the needs for a more transparent framework for carbon auditing and benchmarking in BEA modeling. - Highlights: Black-Right-Pointing-Pointer Carbon emission evaluation in building environmental assessment schemes are studied. Black-Right-Pointing-Pointer Simulative carbon emission is modeled for building environmental assessment schemes. Black-Right-Pointing-Pointer Carbon assessments focus primarily on operational stage instead of entire lifecycle. Black-Right-Pointing-Pointer Baseline and benchmark of carbon assessment vary greatly among BEA schemes. Black-Right-Pointing-Pointer A more transparent and comprehensive framework for carbon assessment is required.

54 ENVIRONMENTAL SCIENCES↗

Evaluating and Improving Cloud Processes in the Multi-Scale Modeling Framework

The research performed under this grant was intended to improve the embedded cloud model in the Multi-scale Modeling Framework (MMF) for convective clouds by using a 2-moment microphysics scheme rather than the single moment scheme used in all the MMF runs to date. The technical report and associated documents describe the results of testing the cloud resolving model with fixed boundary conditions and evaluation of model results with data. The overarching conclusion is that such model evaluations are problematic because errors in the forcing fields control the results so strongly that variations in parameterization values cannot be usefully constrained

54 ENVIRONMENTAL SCIENCES↗

Building a Business Case for Compressed Natural Gas in Fleet Applications

Natural gas is a clean-burning, abundant, and domestically produced source of energy. Compressed natural gas (CNG) has recently garnered interest as a transportation fuel because of these attributes and because of its cost savings and price stability compared to conventional petroleum fuels. The National Renewable Energy Laboratory (NREL) developed the Vehicle Infrastructure and Cash-Flow Evaluation (VICE) model to help businesses and fleets evaluate the financial soundness of CNG vehicle and CNG fueling infrastructure projects.

03 NATURAL GAS↗

Evaluating the GPSS simulation model for the Viking batch computer system

In anticipation of extremely heavy loading requirements by the Viking mission during the post-landing periods, a GPSS model has been developed for the purpose of simulating these requirements on the Viking batch computer system. This paper presents the effort pursued in evaluating such a model and results thereby obtained. The evaluation effort consists of selecting the evaluation approach, collecting actual test run data, making comparisons and deriving conclusions.

Lee, J.-J.↗

Development of a two-equation turbulence model for hypersonic flows: Evaluation of a low Reynolds number correction to the Kappa - epsilon two equation compressible turbulence model - Volume 1

The objective of the current research is the development of an improved k-epsilon two-equation compressible turbulence model for turbulent boundary layer flows experiencing strong viscous-inviscid interactions. The development of an improved model is important in the design of hypersonic vehicles such as the National Aerospace Plane (NASP) and the High Speed Civil Transport (HSCT). Improvements have been made to the low Reynolds number functions in the eddy viscosity and dissipation of solenoidal dissipation of the k-epsilon turbulence mode. These corrections offer easily applicable modifications that may be utilized for more complex geometries. The low Reynolds number corrections are functions of the turbulent Reynolds number and are therefore independent of the coordinate system. The proposed model offers advantages over some current models which are based upon the physical distance from the wall, that modify the constants of the standard model, or that make more corrections than are necessary to the governing equations. The code has been developed to solve the Favre averaged, boundary layer equations for mass, momentum, energy, turbulence kinetic energy, and dissipation of solenoidal dissipation using Keller's box scheme and the Newton spatial marching method. The code has been validated by removing the turbulent terms and comparing the solution with the Blasius solution, and by comparing the turbulent solution with an existing k-epsilon model code using wall function boundary conditions. Excellent agreement is seen between the computed solution and the Blasius solution, and between the two codes. The model has been tested for both subsonic and supersonic flat-plate turbulent boundary layer flow by comparing the computed skin friction with the Van Driest II theory and the experimental data of Weighardt; by comparing the transformed velocity profile with the data of Weighardt, and the Law of the Wall and the Law of the Wake; and by comparing the computed results of an adverse pressure gradient with the experimental data of Fernando and Smits. Good agreement is obtained with the experimental correlations for all flow conditions.

Knight, Doyle D.↗

Shuttle/Payload Contamination Evaluation /SPACE/ - A systems level contamination model

The Shuttle/Payload Contamination Evaluation (SPACE) computer program was designed to three dimensionally synthesize the dynamics of the induced on-orbit molecular contaminant environment of the Shuttle Orbiter and a number of Spacelab payload carrier configurations. SPACE has the capability to accept any instrument or spacecraft configuration for contamination evaluation through manipulation of program input data. It represents a significant advancement over previous analytical or modeling techniques in that it is the first integrated systems level model of its type to collectively consider geometry (emphasizing surface shadowing), any arbitrary contaminant source(s) and the contaminant transport mechanisms of: (1) direct source-to-surface; (2) return flux due to molecular collisions with the ambient atmosphere and other contaminant molecules; and (3) reflection/reemission from structural surfaces.

Jarossy, F. J.↗

Feeding Ten Billion People Is Possible Within Four Terrestrial Planetary Boundaries

Global agriculture puts heavy pressure on planetary boundaries, posing the challenge to achieve future food security without compromising Earth system resilience. On the basis of process-detailed, spatially explicit representation of four interlinked planetary boundaries (biosphere integrity, land-system change, freshwater use, nitrogen flows) and agricultural systems in an internally consistent model framework, we here show that almost half of current global food production depends on planetary boundary transgressions. Hotspot regions, mainly in Asia, even face simultaneous transgression of multiple underlying local boundaries. If these boundaries were strictly respected, the present food system could provide a balanced diet (2,355 kcal per capita per day) for 3.4 billion people only. However, as we also demonstrate, transformation towards more sustainable production and consumption patterns could support 10.2 billion people within the planetary boundaries analysed. Key prerequisites are spatially redistributed cropland, improved water–nutrient management, food waste reduction and dietary changes. Adoption of the Sustainable Development Goals by all nations in 2015 is the first ever commitment to a world development path that safeguards the stability of the Earth system as a prerequisite for meeting universal human standards1. The longstanding challenge of achieving food security through sustainable agriculture is particularly acute in this context as world agriculture is a leading cause for the current transgressions of multiple planetary boundaries (PBs) globally and regionally2–5. The PB framework is a comprehensive scientific attempt to synoptically define our planet’s biogeophysical limits to anthropogenic interference. It suggests bounds to nine interacting processes that together delineate a Holocene-like Earth system state. The Holocene is chosen as the reference state as it is the only period known to provide a safe operating space for a world population of several billion people, and according to a precautionary principle, the PBs are set in sufficient distance from processes that may critically undermine Earth system resilience and global sustainability. A challenging question, thus, is whether human development goals such as food security can be met while maintaining multiple PBs along with their subglobal manifestations. Further PB transgressions could jeopardize the chances of providing sufficient food for a world population projected to be wealthier and reach >9 billion by 2050. This conundrum portrays a tradeoff between Earth’s biophysical carrying capacity and humankind’s rising food demand, calling in response for radical rethinking of food production and consumption patterns6–9. Yield gap closures, avoidance of excessive input use, shifts towards less resource-demanding diets, food waste reductions and efficient international trade are crucial options for sustainably increasing the food supply10–15. For example, enhancing water-use efficiency on irrigated and rain-fed farms can triple or quadruple crop yields in low-performing systems, suggesting possible global gains of >20% (ref. 16). Even higher gains appear feasible through globally optimized configurations of the land-use pattern17, and cutting food losses by half could generate food for another billion people18. Thus, collective large-scale implementation of such options could sustain food for a further growing world population19. Yet achieving this within a safe operating space as defined by PBs requires not only a halt to but actually a reversal of existing PB transgressions. Previous studies suggest that such a reconciliation might be possible, but these were based on aggregate representations of PBs (not accounting for the spatial patterns of limits, transgressions and interactions) or considered only one boundary in isolation17,20–23. Here, we systematically quantify to what extent current food production depends on local to global transgressions of the PBs for biosphere integrity, land-system change, freshwater use and nitrogen (N) flows, along with the potential of a range of solutions to avoid these transgressions and still increase food supply (Table 1). To this end, we configured an internally consistent process-based model of the terrestrial biosphere including agriculture (LPJmL) with multiple spatially distributed PBs and their interactions. LPJmL is among the longest-established and best-evaluated biosphere models, showing robust performance regarding simulation of, for example, carbon, water and crop yield dynamics (Supplementary Figs. 1 and 2 and Supplementary Table 1; see ref. 24 for a comprehensive benchmarking and Supplementary Methods for more detail on model evaluations). In principle following established definitions4, we refine the computation of some PBs with respect to their regional patterns and interactions (Methods), providing globally gridded precautionary limits to human interference with the Earth system at a level of great detail. In particular, we account for the evidence that many PBs need to be represented spatially explicitly4 to cover their

Gerten, Dieter↗