Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,765 records · Page 98

Diagnosing the Dynamics of Observed and Simulated Ecosystem Gross Primary Productivity with Time Causal Information Theory Quantifiers

Data analysis and model-data comparisons in the environmental sciences require diagnostic measures that quantify time series dynamics and structure, and are robust to noise in observational data. This paper investigates the temporal dynamics of environmental time series using measures quantifying their information content and complexity. The measures are used to classify natural processes on one hand, and to compare models with observations on the other. The present analysis focuses on the global carbon cycle as an area of research in which model-data integration and comparisons are key to improving our understanding of natural phenomena. We investigate the dynamics of observed and simulated time series of Gross Primary Productivity (GPP), a key variable in terrestrial ecosystems that quantifies ecosystem carbon uptake. However, the dynamics, patterns and magnitudes of GPP time series, both observed and simulated, vary substantially on different temporal and spatial scales. Here we demonstrate that information content and complexity, or Information Theory Quantifiers (ITQ) for short, serve as robust and efficient data-analytical and model benchmarking tools for evaluating the temporal structure and dynamical properties of simulated or observed time series at various spatial scales. At continental scale, we compare GPP time series simulated with two models and an observations-based product. This analysis reveals qualitative differences between model evaluation based on ITQ compared to traditional model performance metrics, indicating that good model performance in terms of absolute or relative error does not imply that the dynamics of the observations is captured well. Furthermore, we show, using an ensemble of site-scale measurements obtained from the FLUXNET archive in the Mediterranean, that model-data or model-model mismatches as indicated by ITQ can be attributed to and interpreted as differences in the temporal structure of the respective ecological time series. At global scale, our understanding of C fluxes relies on the use of consistently applied land models. Here, we use ITQ to evaluate model structure: The measures are largely insensitive to climatic scenarios, land use and atmospheric gas concentrations used to drive them, but clearly separate the structure of 13 different land models taken from the CMIP5 archive and an observations-based product. In conclusion, diagnostic measures of this kind provide data-analytical tools that distinguish different types of natural processes based solely on their dynamics, and are thus highly suitable for environmental science applications such as model structural diagnostics.

54 ENVIRONMENTAL SCIENCES↗

A multiscale anisotropic polymer network model coupled with phase field fracture

The study of polymers has continued to gain substantial attention due to their expanding range of applications, spanning essential engineering fields to emerging domains like stretchable electronics, soft robotics, and implantable sensors. These materials exhibit remarkable properties, primarily stemming from their intricate polymer chain network, which, in turn, increases the complexity of precisely modeling their behavior. Especially for modeling elastomers and their fracture behavior, accurately accounting for the deformations of the polymer chains is vital for predicting the rupture in highly stretched chains. Despite the importance, many robust multiscale continuum frameworks for modeling elastomer fracture tend to simplify network deformations by assuming uniform behavior among chains in all directions. Recognizing this limitation, our study proposes a multiscale fracture model that accounts for the anisotropic nature of elastomer network responses. At the microscale, damage in the chains is assumed to be driven by both the chain's entropy and the internal energy due to molecular bond distortions. In order to bridge the stretching in the chains to the macroscale deformation, we employ the maximal advance path constraint network model, inherently accommodating anisotropic network responses. As a result, chains oriented differently can be predicted to exhibit varying stretch and, consequently, different damage levels. To drive macroscale fracture based on damages in these chains, we utilize the micromorphic regularization theory, which involves the introduction of dual local-global damage variables at the macroscale. The macroscale local damage variable is obtained through the homogenization of the chain damage values, resulting in the prediction of an isotropic material response. The macroscale global damage variable is subjected to nonlocal effects and boundary conditions in a thermodynamically consistent phase field continuum formulation. Moreover, the total dissipation in the system is considered to be mainly due to the breaking of the molecular bonds at the microscale. To validate our model, we employ the double-edge notched tensile test as a benchmark, comparing simulation predictions with existing experimental data. Additionally, to enhance our understanding of the fracturing process, we conduct uniaxial tensile experiments on a square film made up of polydimethylsiloxane (PDMS) rubber embedded with a hole and notches and then compare our simulation predictions with the experimental observations. Furthermore, we visualize the evolution of stretch and damage values in chains oriented along different directions to assess the predictive capacity of the model. In conclusion, the results are also compared with another existing model to evaluate the utility of our model in accurately simulating the fracture behavior of rubber-like materials.

42 ENGINEERING↗

EJECTION AND CAPTURE DYNAMICS IN RESTRICTED THREE-BODY ENCOUNTERS

We study the tidal disruption of binaries by a massive point mass (e.g., the black hole at the Galactic center), and we discuss how the ejection and capture preference between unequal-mass binary members depends on which orbit they approach the massive object. We show that the restricted three-body approximation provides a simple and clear description of the dynamics. The orbit of a binary with mass m around a massive object M should be almost parabolic with an eccentricity of |1 - e| {approx}< (m/M){sup 1/3} << 1 for a member to be captured, while the other is ejected. Indeed, the energy change of the members obtained for a parabolic orbit can be used to describe non-parabolic cases. If a binary has an encounter velocity much larger than (M/m){sup 1/3} times the binary rotation velocity, it would be abruptly disrupted, and the energy change at the encounter can be evaluated in a simple disruption model. We evaluate the probability distributions for the ejection and capture of circular binary members and for the final energies. In principle, for any hyperbolic (elliptic) orbit, the heavier member has more chance to be ejected (captured), because it carries a larger fraction of the orbital energy. However, if the orbital energy is close to zero, the difference between the two members becomes small, and there is practically no ejection and capture preferences. The preference becomes significant when the orbital energy is comparable to the typical energy change at the encounter. We discuss its implications to hypervelocity stars and irregular satellites around giant planets.

79 ASTRONOMY AND ASTROPHYSICS↗

Evaluating algorithmic bias on biomarker classification of breast cancer pathology reports

Objectives: This work evaluated algorithmic bias in biomarkers classification using electronic pathology reports from female breast cancer cases. Bias was assessed across 5 subgroups: cancer registry, race, Hispanic ethnicity, age at diagnosis, and socioeconomic status. Materials and Methods: We utilized 594 875 electronic pathology reports from 178 121 tumors diagnosed in Kentucky, Louisiana, New Jersey, New Mexico, Seattle, and Utah to train 2 deep-learning algorithms to classify breast cancer patients using their biomarkers test results. We used balanced error rate (BER), demographic parity (DP), equalized odds (EOD), and equal opportunity (EOP) to assess bias. Results: We found differences in predictive accuracy between registries, with the highest accuracy in the registry that contributed the most data (Seattle Registry, BER ratios for all registries >1.25). BER showed no significant algorithmic bias in extracting biomarkers (estrogen receptor, progesterone receptor, human epidermal growth factor receptor 2) for race, Hispanic ethnicity, age at diagnosis, or socioeconomic subgroups (BER ratio <1.25). DP, EOD, and EOP all showed insignificant results. Discussion: We observed significant differences in BER by registry, but no significant bias using the DP, EOD, and EOP metrics for socio-demographic or racial categories. This highlights the importance of employing a diverse set of metrics for a comprehensive evaluation of model fairness. Conclusion: A thorough evaluation of algorithmic biases that may affect equality in clinical care is a critical step before deploying algorithms in the real world. We found little evidence of algorithmic bias in our biomarker classification tool. Artificial intelligence tools to expedite information extraction from clinical records could accelerate clinical trial matching and improve care.

60 APPLIED LIFE SCIENCES↗

Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions

Models that predict brain responses to stimuli provide one measure of understanding of a sensory system and have many potential applications in science and engineering. Deep artificial neural networks have emerged as the leading such predictive models of the visual system but are less explored in audition. Prior work provided examples of audio-trained neural networks that produced good predictions of auditory cortical fMRI responses and exhibited correspondence between model stages and brain regions, but left it unclear whether these results generalize to other neural network models and, thus, how to further improve models in this domain. We evaluated model-brain correspondence for publicly available audio neural network models along with in-house models trained on 4 different tasks. Most tested models outpredicted standard spectromporal filter-bank models of auditory cortex and exhibited systematic model-brain correspondence: Middle stages best predicted primary auditory cortex, while deep stages best predicted non-primary cortex. However, some state-of-the-art models produced substantially worse brain predictions. Models trained to recognize speech in background noise produced better brain predictions than models trained to recognize speech in quiet, potentially because hearing in noise imposes constraints on biological auditory representations. The training task influenced the prediction quality for specific cortical tuning properties, with best overall predictions resulting from models trained on multiple tasks. The results generally support the promise of deep neural networks as models of audition, though they also indicate that current models do not explain auditory cortical responses in their entirety.

59 BASIC BIOLOGICAL SCIENCES↗

Model verification of mixed dynamic systems

A parameter-estimation method is described for verifying the mathematical model of mixed (combined interactive components from various engineering fields) dynamic systems against pertinent experimental data. The model verification problem is divided into two separate parts: defining a proper model and evaluating the parameters of that model. The main idea is to use differences between measured and predicted behavior (response) to adjust automatically the key parameters of a model so as to minimize response differences. To achieve the goal of modeling flexibility, the method combines the convenience of automated matrix generation with the generality of direct matrix input. The equations of motion are treated in first-order form, allowing for nonsymmetric matrices, modeling of general networks, and complex-mode analysis. The effectiveness of the method is demonstrated for an example problem involving a complex hydraulic-mechanical system.

Chrostowski, J. D.↗

Crop weather models of corn and soybeans for Agrophysical Units (APU's) in Iowa using monthly meteorological predictors

Models based on multiple regression were developed to estimate corn and soybean yield from weather data for agrophysical units (APU) in Iowa. The predictor variables are derived from monthly average temperature and monthly total precipitation data at meteorological stations in the cooperative network. The models are similar in form to the previous models developed for crop reporting districts (CRD). The trends and derived variables were the same and the approach to select the significant predictors was similar to that used in developing the CRD models. The APU's were selected to be more homogeneous with respect crop to production than the CRDs. The APU models are quite similar to the CRD models, similar explained variation and number of predictor variables. The APU models are to be independently evaluated and compared to the previously evaluated CRD models. That comparison should indicate the preferred model area for this application, i.e., APU or CRD.

Leduc, S.↗

Crop weather models of barley and spring wheat yield for agrophysical units in North Dakota

Models based on multiple regression were developed to estimate barley yield and spring wheat yield from weather data for Agrophysical units(APU) in North Dakota. The predictor variables are derived from monthly average temperature and monthly total precipitation data at meteorological stations in the cooperative network. The models are similar in form to the previous models developed for Crop Reporting Districts (CRD). The trends and derived variables were the same and the approach to select the significant predictors was similar to that used in developing the CRD models. The APU models show sight improvements in some of the statistics of the models, e.g., explained variation. These models are to be independently evaluated and compared to the previously evaluated CRD models. The comparison will indicate the preferred model area for this application, i.e., APU or CRD.

Leduc, S.↗

X-ray induced stellar mass loss near active galactic nuclei

The effects of UV and X-ray radiation on stars in active galactic nuclei (AGN) are critically evaluated. Mass loss rates in X-ray-induced winds are evaluated for realistic red giant models, and the effects of the ablation of stellar envelopes by radiation pressure are considered. The importance of X-ray-induced mass loss in the standard quasar model is evaluated and whether it can provide a source of accretion fuel or emission-line clouds is discussed. It is concluded that thermal winds driven by X-ray heating are a minor total supply of mass to AGN, but that thermal plus line-driven winds and stellar ablation may increase the mass loss and improve the chances for supplying a fraction of the necessary mass supply to the central object. It is speculated that when steady winds are inefficient, complex time-dependent processes due to X-ray energy injection deep into a stellar atmosphere could still release significant mass from stars.

Voit, G. Mark↗

Evolution of the 1991-1992 Arctic vortex and comparison with the Geophysical Fluid Dynamics Laboratory SKYHI general circulation model

Nitrous oxide (N2O) measured on board the ER-2 aircraft during the Airborne Arctic Stratospheric Expedition 2 (AASE 2) has been used to monitor descent of air inside the Arctic vortex between October 1991 and March 1992. Monthly mean N2O fields are calculated from the flight data and then compared with mean fields calculated from the high-resolution Geophysical Fluid Dynamics Laboratory general circulation model SKYHI in order to evaluate the model's simulation of the polar vortex. From late fall through winter the model vortex evolves in much the same way as the 1991-1992 vortex, with N2O gradients at the edge becoming progressively steeper. The October to March trends in N2O profiles inside the vortex are used to verify daily net heating rates in the vortex that were computed from clear sky radiative heating rates and National Meteorological Center temperature observations. The computed heating rates successfully estimate the descent of vortex air from December through February but suggest that before December, air at high latitudes may not be isolated from the midlatitudes. SKYHI heating rates are in good agreement with the computed rates but tend to be slightly higher (i.e., less cooling) due to meteorological differences between SKYHI and the 1991-1992 winter. Three ER-2 flights measured N2O just north of the subtropical jet. These low-midlatitude profiles show only slight differences from the high-midlatitude profiles (45 deg - 60 deg N), indicating strong meridional mixing in the midlatitude 'surf zone.' Mean midwinter N2O profiles inside and outside the vortex calculated from AASE 2 data are shown to be nearly identical to 1989 AASE profiles, pointing to the N2O/potential temperature relationship as an excellent marker for vortex air.

Strahan, S. E.↗

Comparison of simulated cloud cover with satellite obsrvations over the Western United States

Satellite imagery datasets and regional climate model results are intercompared for evaluation of model accuracy in the simulation of cloud cover. Both monthly average individual simulation times are analyzed. To provide a consistent comparison, satellite data are first mapped into the model's geographic projection, grid domain, and resolution. It is found that September 1988 monthly average cloud fraction results from the modeled simulations correspond to observations, in both spatial pattern and magnitude, with bias less than +/- 20% cloud fraction over the entire inland West. Agreement in the pattern of cloud fraction also is evident for monthly average cloud fraction in July, but there is no negative bias of 10%-30% cloud fraction in the model diagnosis of cloud cover. Correlations between the spatial distributions of model-derived and observed cloud fractions are found to exceed 0.80 for certain geographic regions of the West, and these correlations are largest over mountainous areas during summer. Case studies of a series of daily cloud cover demonstrate the ability of the model to simulate the effects of frontal passage on cloud distribution. The ability of the RegCM1 to simulate daily cloud fraction and diurnal cloud evolution is somewhat weak for the summer convective season. It is anticipated that a more recent version of the regional climate model may improve the simulation of summer season cloud cover, through changes in cloud parameterization and improvements in model resolution.

Wetzel, Melanie A.↗

Variations and Defects: Influence on Performance and Safety of Aerospace Missions

The overall goal of this project is to determine sources of normal and abnormal statistical variation for helicopter transmission vibrations. One current application is HUMS technology for helicopters. The design and manufacturing research hypothesis is that statistical variation due to design and manufacturing is a significant source of unreliability and eventual failures of systems. The Systems Health and Safety group research objectives include the following: (1) to develop empirical database for modeling and evaluation; (2) to identify and model factors that influence vibrations; (3) to develop algorithms for damage detection; (4) to determine 'hit' and 'false alarm' rates; and (5) to understand the fundamentals of transmission vibrations.

Turner, Irem Y.↗

Aerosol Source Attributions and Source-Receptor Relationships Across the Northern Hemisphere

Emissions and long-range transport of air pollution pose major concerns on air quality and climate change. To better assess the impact of intercontinental transport of air pollution on regional and global air quality, ecosystems, and near-term climate change, the UN Task Force on Hemispheric Transport of Air Pollution (HTAP) is organizing a phase II activity (HTAP2) that includes global and regional model experiments and data analysis, focusing on ozone and aerosols. This study presents the initial results of HTAP2 global aerosol modeling experiments. We will (a) evaluate the model results with surface and aircraft measurements, (b) examine the relative contributions of regional emission and extra-regional source on surface PM concentrations and column aerosol optical depth (AOD) over several NH pollution and dust source regions and the Arctic, and (c) quantify the source-receptor relationships in the pollution regions that reflect the sensitivity of regional aerosol amount to the regional and extra-regional emission reductions.

HTAP2↗

Probabilistic Forecasting of Ground Magnetic Perturbation Spikes at Mid-Latitude Stations

The prediction of large fluctuations in the ground magnetic field (dB/dt) is essential for preventing damage from Geomagnetically Induced Currents. Directly forecasting these fluctuations has proven difficult, but accurately determining the risk of extreme events can allow for the worst of the damage to be prevented. Here we trained Convolutional Neural Network models for eight mid-latitude magnetometers to predict the probability that dB/dt will exceed the 99th percentile threshold 30–60 min in the future. Two model frameworks were compared, a model trained using solar wind data from the Advanced Composition Explorer (ACE) satellite, and another model trained on both ACE and SuperMAG ground magnetometer data. The models were compared to examine if the addition of current ground magnetometer data significantly improved the forecasts of dB/dt in the future prediction window. A bootstrapping method was employed using a random split of the training and validation data to provide a measure of uncertainty in model predictions. The models were evaluated on the ground truth data during eight geomagnetic storms and a suite of evaluation metrics are presented. The models were also compared to a persistence model to ensure that the model using both datasets did not over-rely on dB/dt values in making its predictions. Overall, we find that the models using both the solar wind and ground magnetometer data had better metric scores than the solar wind only and persistence models, and was able to capture more spatially localized variations in the dB/dt threshold crossings.

Michael Coughlan↗

Modeling the Effect of Evaporative Flux and Colloidal Aggregation on Crud Deposition

During the last few years lot of research has been conducted to investigate and predict the Crud deposition in PWRs. One of the common findings of various experimental and computational studies is that thickness or amount of crud deposits is directly related to evaporative flux maps. Evaporation driven deposition profile on the surfaces takes different forms, i.e. from ring to disk shapes during initial stages and in the form of porous to impervious continuous layers covering the hot fuel surface at the later stages. The amount and thickness of these deposits although are directly related to evaporative flux, the relationship is much complex and can be dependent upon several other factors. A popular problem relating evaporative flux to particulate deposition has been well studied in the form of drying of sessile colloidal drops. It was proposed that, this phenomenon is due to a geometrical constraint: the free surface, constrained by a pinned contact line squeezes the fluid outwards to compensate for evaporative losses and thereby transporting particles to the contact line. Later on a quantitative relationship between evaporative flux and deposition rate was proposed and validated by the same group, which showed power-law relationship. Deposition studies conducted with controlled concentration of particles with subcooled nucleate boiling also showed a power-law relationship rather than a simpler linear relationship whereas physics of evaporation would tend to concentrate particles at linear rate. In a typical subcooled nucleate boiling system, bubble growth and departure time is of the order 1-2 ms, measured experimentally. Diffusion time constant for dilute spherical metallic colloids such as crud particulates, generally present as particulate in process piping, range from 0.0025-2.5 sec in water for a size range 0.1-1.0 μm. Colloidal behavioral characteristics during the growth of bubbles can be computed with escape time constant at collision, which is dependent on effective Hamaker constant. For minimum effective Hamaker constant of Hematite is -2.5 kT and diffusion time constant τ = 0.0025 sec, escape time constant is computed as Kramer rule, τexp[A/kT] magnitude is 0.03 sec or 30 ms. This clearly indicates that evaporation will cause particles to come together and they cannot escape at a fast enough rate which would lead them to form agglomerates. Therefore both effects of evaporative flux in conjunction with the agglomeration kinetics must be considered for modeling the crud deposition if the evaporative flux is known. In this paper, we propose a phenomenological model of colloidal transport and aggregation with evaporative flux. The dynamic of particles interactions while drying of sessile colloidal drops is developed by coupling general transport equation with Smoluchowshki Coagulation equation. Crud deposition on the nuclear fuel has been a serious problem with severe consequences in nuclear power plants. In several studies it has been consistently reported that crud deposition is directly related to evaporative flux on the nuclear fuel surfaces. However, it has not been established how to quantify the deposition rates of crud type particulates if the evaporative flux is known. This complex multiphysics problem is similar to coffee-ring effect, where the governing physics is understood more clearly. In this work, a simplified geometry is used to model the effects of evaporative flux acting at a boundary on the colloidal concentration. The effect of this condition on change in concentration is shown numerically. Further, agglomeration is suspected to play a significant role in high evaporative flux such as bubble nucleation. The coagulation kinetics is modeled using uniform particle distribution and the effective total concentration of colloids including original particles and agglomerates is studied. Once this model is validated with a simple separate effect experiments, it can be integrated to large scale PWR models of evaluating evaporative flux and chemistry models developed by Electrical Power Research Institute (EPRI). (authors)

42 ENGINEERING↗

Comparison of nuclear mass models at nuclearmasses.org

Nuclear masses are crucial in many astrophysics studies, as well as other areas of basic and applied nuclear science. There are now many different global theoretical models of nuclear masses - but choosing the best model for a given application can be challenging. The suite of codes online at nuclearmasses.org greatly facilitates the comparison of nuclear models with evaluated masses as well as the intercomparison of theoretical models. We demonstrate the utility of this suite by comparing the RMS deviations of 13 different theoretical mass models from the AME2003 evaluated masses over mass ranges appropriate for astrophysics.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Introducing Multisensor Satellite Radiance-Based Evaluation for Regional Earth System Modeling

Earth System modeling has become more complex, and its evaluation using satellite data has also become more difficult due to model and data diversity. Therefore, the fundamental methodology of using satellite direct measurements with instrumental simulators should be addressed especially for modeling community members lacking a solid background of radiative transfer and scattering theory. This manuscript introduces principles of multisatellite, multisensor radiance-based evaluation methods for a fully coupled regional Earth System model: NASA-Unified Weather Research and Forecasting (NU-WRF) model. We use a NU-WRF case study simulation over West Africa as an example of evaluating aerosol-cloud-precipitation-land processes with various satellite observations. NU-WRF-simulated geophysical parameters are converted to the satellite-observable raw radiance and backscatter under nearly consistent physics assumptions via the multisensor satellite simulator, the Goddard Satellite Data Simulator Unit. We present varied examples of simple yet robust methods that characterize forecast errors and model physics biases through the spatial and statistical interpretation of various satellite raw signals: infrared brightness temperature (Tb) for surface skin temperature and cloud top temperature, microwave Tb for precipitation ice and surface flooding, and radar and lidar backscatter for aerosol-cloud profiling simultaneously. Because raw satellite signals integrate many sources of geophysical information, we demonstrate user-defined thresholds and a simple statistical process to facilitate evaluations, including the infrared-microwave-based cloud types and lidar/radar-based profile classifications.

Planetary Boundary Layer↗

Assessing damping uncertainty in space structures with fuzzy sets

NASA has been interested in the development of methods for evaluating the predictive accuracy of structural dynamic models. This interest stems from the use of mathematical models in evaluating the structural integrity of all spacecraft prior to flight. Space structures are often too large and too weak to be tested fully assembled in a ground test lab. The predictive accuracy of a model depends on the nature and extent of its experimental verification. The further the test conditions depart from in-service conditions, the less accurate the model will be. Structural damping is known to be one source of uncertainty in models. The uncertainty in damping is explored in order to evaluate the accuracy of dynamic models. A simple mass-spring-dashpot system is used to illustrate a comparison among three methods for propagating uncertainty in structural dynamics models: the First Order Method, the Numerical Simulation Method, and the Fuzzy Set Method. The Fuzzy Set Method is shown to bound the range of possible responses and thus to provide a valuable limiting check on the First Order Method near resonant conditions. Fuzzy Methods are a relative inexpensive alternative to numerical simulation.

Ross, Timothy J.↗