Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Model uncertainty”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,279 records · Page 71
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

A Primer on Dose-Response Data Modeling in Radiation Therapy

An overview of common approaches used to assess a dose response for radiation therapy–associated endpoints is presented, using lung toxicity data sets analyzed as a part of the High Dose per Fraction, Hypofractionated Treatment Effects in the Clinic effort as an example. Each component presented (eg, data-driven analysis, dose-response analysis, and calculating uncertainties on model prediction) is addressed using established approaches. Specifically, the maximum likelihood method was used to calculate best parameter values of the commonly used logistic model, the profile-likelihood to calculate confidence intervals on model parameters, and the likelihood ratio to determine whether the observed data fit is statistically significant. The bootstrap method was used to calculate confidence intervals for model predictions. Correlated behavior of model parameters and implication for interpreting dose response are discussed.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2023

Climate response to off-equatorial stratospheric sulfur injections in three Earth system models – Part 2: Stratospheric and free-tropospheric response

The paper constitutes Part 2 of a study performing a first systematic inter-model comparison of the atmospheric responses to stratospheric aerosol injection (SAI) at various single latitudes in the tropics, as simulated by three state-of-the-art Earth system models – CESM2-WACCM6, UKESM1.0, and GISS-E2.1-G. Building on Part 1 (Visioni et al., 2023) we demonstrate the role of biases in the climatological circulation and specific aspects of the model microphysics in driving the inter-model differences in the simulated sulfate distributions. We then characterize the simulated changes in stratospheric and free-tropospheric temperatures, ozone, water vapor, and large-scale circulation, elucidating the role of the above aspects in the surface SAI responses discussed in Part 1. We show that the differences in the aerosol spatial distribution can be explained by the significantly faster shallow branches of the Brewer–Dobson circulation in CESM2, a relatively isolated tropical pipe and older tropical age of air in UKESM, and smaller aerosol sizes and relatively stronger horizontal mixing (thus very young stratospheric age of air) in the two GISS versions used. We also find a large spread in the magnitudes of the tropical lower-stratospheric warming amongst the models, driven by microphysical, chemical, and dynamical differences. These lead to large differences in stratospheric water vapor responses, with significant increases in stratospheric water vapor under SAI in CESM2 and GISS that were largely not reproduced in UKESM. For ozone, good agreement was found in the tropical stratosphere amongst the models with more complex microphysics, with lower stratospheric ozone changes consistent with the SAI-induced modulation of the large-scale circulation and the resulting changes in transport. In contrast, we find a large inter-model spread in the Antarctic ozone responses that can largely be explained by the differences in the simulated latitudinal distributions of aerosols as well as the degree of implementation of heterogeneous halogen chemistry on sulfate in the models. The use of GISS runs with bulk microphysics demonstrates the importance of more detailed treatment of aerosol processes, with contrastingly different stratospheric SAI responses to the models using the two-moment aerosol treatment; however, some problems in halogen chemistry in GISS are also identified that require further attention. Overall, our results contribute to an increased understanding of the underlying physical mechanisms as well as identifying and narrowing the uncertainty in model projections of climate impacts from SAI.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Assessing the representation of the Australian carbon cycle in global vegetation models

Australia plays an important role in the global terrestrial carbon cycle on inter-annual timescales. While the Australian continent is included in global assessments of the carbon cycle such as the global carbon budget, the performance of dynamic global vegetation models (DGVMs) over Australia has rarely been evaluated. We assessed simulations of net biome production (NBP) and the carbon stored in vegetation between 1901 to 2018 from 13 DGVMs (TRENDY v8 ensemble). We focused our analysis on Australia's short-term (inter-annual) and long-term (decadal to centennial) terrestrial carbon dynamics. The TRENDY models simulated differing magnitudes of NBP on inter-annual timescales, and these differences resulted in significant differences in long-term vegetation carbon accumulation (–4.7 to 9.5 Pg C). We compared the TRENDY ensemble to several satellite-derived datasets and showed that the spread in the models' simulated carbon storage resulted from varying changes in carbon residence time rather than differences in net carbon uptake. Differences in simulated long-term accumulated NBP between models were mostly due to model responses to land-use change. The DGVMs also simulated different sensitivities to atmospheric carbon dioxide (CO 2 ) concentration, although notably, the models with nutrient cycles did not simulate the smallest NBP response to CO 2 . Our results suggest that a change in the climate forcing did not have a large impact on the carbon cycle on long timescales. However, the inter-annual variability in precipitation drives the year-to-year variability in NBP. We analysed the impact of key modes of climate variability, including the El Niño–Southern Oscillation (ENSO) and the Indian Ocean Dipole (IOD), on NBP. While the DGVMs agreed on sign of the response of NBP to El Niño and La Niña and to positive and negative IOD events, the magnitude of inter-annual variability in NBP differed strongly between models. In addition, we find that differences in the timing of simulated phenology and fire dynamics are associated with differences in simulated or prescribed vegetation cover and process representation. We further find model disagreement in simulated vegetation carbon, phenology, and apparent carbon residence time, indicating that the models have different types and coverage of vegetation across Australia (whether prescribed or emergent). Our study highlights the need to evaluate parameter assumptions and the key processes that drive vegetation dynamics, such as phenology, mortality, and fire, in an Australian context to reduce uncertainty across models.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2023

Probabilistic Forecasting of Ground Magnetic Perturbation Spikes at Mid-Latitude Stations

The prediction of large fluctuations in the ground magnetic field (dB/dt) is essential for preventing damage from Geomagnetically Induced Currents. Directly forecasting these fluctuations has proven difficult, but accurately determining the risk of extreme events can allow for the worst of the damage to be prevented. Here we trained Convolutional Neural Network models for eight mid-latitude magnetometers to predict the probability that dB/dt will exceed the 99th percentile threshold 30–60 min in the future. Two model frameworks were compared, a model trained using solar wind data from the Advanced Composition Explorer (ACE) satellite, and another model trained on both ACE and SuperMAG ground magnetometer data. The models were compared to examine if the addition of current ground magnetometer data significantly improved the forecasts of dB/dt in the future prediction window. A bootstrapping method was employed using a random split of the training and validation data to provide a measure of uncertainty in model predictions. The models were evaluated on the ground truth data during eight geomagnetic storms and a suite of evaluation metrics are presented. The models were also compared to a persistence model to ensure that the model using both datasets did not over-rely on dB/dt values in making its predictions. Overall, we find that the models using both the solar wind and ground magnetometer data had better metric scores than the solar wind only and persistence models, and was able to capture more spatially localized variations in the dB/dt threshold crossings.

Michael Coughlan↗
DOE OSTI2025

Incorporating geological structure into sensitivity analysis of subsurface contaminant transport

Simulating subsurface contaminant transport at the kilometer-scale often entails modeling reactive flow and transport within and through complex geologic structures. These structures are typically meshed by hand and as a result geologic structure is usually represented by one or a few deterministically generated geological models for uncertainty studies of flow and transport in the subsurface. Uncertainty in geologic structure can have a significant impact on contaminant transport. In this study, the impact of geologic structure on contaminant tracer transport in a shale formation is investigated for a simplified generic deep geologic repository for permanent disposal of spent nuclear fuel. An open-source modeling framework is used to perform a sensitivity analysis study on transport of two tracers from a generic spent nuclear fuel repository with uncertain location of the interfaces between the stratum of the geologic structure. The automated workflow uses sampled realizations of the geological structural model in addition to uncertain flow parameters in a nested sensitivity analysis. Concentration of the tracers at observation points within, in line with, and downstream of the repository are used as the quantities of interest for determining model sensitivity to input parameters and geological realization. Finally, the results of the study indicate that the location of strata interfaces in the geological structure has a first-order impact on tracer transport in the example shale formation, and that this impact may be greater than that of the uncertain flow parameters.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

Evaluating integrated surface/subsurface permafrost thermal hydrology models in ATS (v0.88) against observations from a polygonal tundra site

Numerical simulations are essential tools for understanding the complex hydrologic response of Arctic regions to a warming climate. However, strong coupling among thermal and hydrological processes on the surface and in the subsurface and the significant role that subtle variations in surface topography have in regulating flow direction and surface storage lead to significant uncertainties. Careful model evaluation against field observations is thus important to build confidence.We evaluate the integrated surface/subsurface permafrost thermal hydrology models in the Advanced Terrestrial Simulator (ATS) against field observations from polygonal tundra at the Barrow Environmental Observatory. ATS couples a multiphase, 3D representation of subsurface thermal hydrology with representations of overland nonisothermal flows, snow processes, and surface energy balance. We simulated thermal hydrology of a 3D ice-wedge polygon with geometry that is abstracted but broadly consistent with the surface microtopography at our study site. The simulations were forced by meteorological data and observed water table elevations in ice-wedge polygon troughs. With limited calibration of parameters appearing in the soil evaporation model, the 3-year simulations agreed reasonably well with snow depth, summer water table elevations in the polygon center, and high-frequency soil temperature measurements at several depths in the trough, rim, and center of the polygon. Upscaled evaporation is in good agreement with flux tower observations. The simulations were found to be sensitive to parameters in the bare soil evaporation model, snowpack, and the lateral saturated hydraulic conductivity.Timing of fall freeze-up was found to be sensitive to initial snow density, illustrating the importance of including snow aging effects. The study provides new support for an emerging class of integrated surface/subsurface permafrost simulators.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Ground Heat Flux Reconstruction Using Bayesian Uncertainty Quantification Machinery and Surrogate Modeling

Ground heat flux (G 0 ) is a key component of the land-surface energy balance of high-latitude regions. Despite its crucial role in controlling permafrost degradation due to global warming, G 0 is sparsely measured and not well represented in the outputs of global scale model simulation. In this study, an analytical heat transfer model is tested to reconstruct G 0 across seasons using soil temperature series from field measurements, Global Climate Model, and climate reanalysis outputs. The probability density functions of ground heat flux and of model parameters are inferred using available G 0 data (measured or modeled) for snow-free period as a reference. When observed G 0 is not available, a numerical model is applied using estimates of surface heat flux (dependent on parameters) as the top boundary condition. These estimates (and thus the corresponding parameters) are verified by comparing the distributions of simulated and measured soil temperature at several depths. Aided by state-of-the-art uncertainty quantification methods, the developed G 0 reconstruction approach provides novel means for assessing the probabilistic structure of the ground heat flux for regional permafrost change studies.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Black hole spin in X-ray binaries: giving uncertainties an $\mathcal{f}$

The two established techniques for measuring black hole spin in X-ray binaries often yield conflicting results, which must be resolved before either method may be deemed robust. In practice, black hole spin measurements based on fitting the accretion disc continuum effectively do not marginalize over the colour-correction factor f col . This factor parametrizes spectral hardening of the disc continuum by the disc atmosphere, whose true properties are poorly constrained. We incorporate reasonable systematic uncertainties in f col into the eight (non-maximal) black hole spin measurements vetted by the disc continuum fitting community. In most cases, an f col uncertainty of ±0.2–0.3 dominates the black hole spin error budget. We go on to demonstrate that plausible departures in f col values from those adopted by the disc continuum fitting practitioners can bring the discrepant black hole spins into agreement with those from iron line modelling. Systematic uncertainties in f col , such as the effects of strong magnetization, should be better understood before dismissing their potentially dominant impact on the black hole spin error budget.

4U 1543–47↗
NASA NTRS1976

Probabilistic and deterministic aspects of linear estimation in geodesy

Recent advances in observational techniques related to geodetic work (VLBI, laser ranging) make it imperative that more consideration should be given to modeling problems. Uncertainties in the effect of atmospheric refraction, polar motion and precession-nutation parameters, cannot be dispensed with in the context of centimeter level geodesy. Even physical processes that have generally been previously altogether neglected (station motions) must now be taken into consideration. The problem of modeling functions of time or space, or at least their values at observation points (epochs) is explored. When the nature of the function to be modeled is unknown. The need to include a limited number of terms and to a priori decide upon a specific form may result in a representation which fails to sufficiently approximate the unknown function. An alternative approach of increasing application is the modeling of unknown functions as stochastic processes.

Dermanis, A.↗
NASA NTRS1988

Precise orbit determination for NASA's earth observing system using GPS (Global Positioning System)

An application of a precision orbit determination technique for NASA's Earth Observing System (EOS) using the Global Positioning System (GPS) is described. This technique allows the geometric information from measurements of GPS carrier phase and P-code pseudo-range to be exploited while minimizing requirements for precision dynamical modeling. The method combines geometric and dynamic information to determine the spacecraft trajectory; the weight on the dynamic information is controlled by adjusting fictitious spacecraft accelerations in three dimensions which are treated as first order exponentially time correlated stochastic processes. By varying the time correlation and uncertainty of the stochastic accelerations, the technique can range from purely geometric to purely dynamic. Performance estimates for this technique as applied to the orbit geometry planned for the EOS platforms indicate that decimeter accuracies for EOS orbit position may be obtainable. The sensitivity of the predicted orbit uncertainties to model errors for station locations, nongravitational platform accelerations, and Earth gravity is also presented.

Williams, B. G.↗
NASA NTRS2013

Human and Natural Influences on the Changing Thermal Structure of the Atmosphere

Since the late 1970s, satellite-based instruments have monitored global changes in atmospheric temperature. These measurements reveal multidecadal tropospheric warming and stratospheric cooling, punctuated by short-term volcanic signals of reverse sign. Similar long- and short-term temperature signals occur in model simulations driven by human-caused changes in atmospheric composition and natural variations in volcanic aerosols. Most previous comparisons of modeled and observed atmospheric temperature changes have used results from individual models and individual observational records. In contrast, we rely on a large multimodel archive and multiple observational datasets. We show that a human-caused latitude/altitude pattern of atmospheric temperature change can be identified with high statistical confidence in satellite data. Results are robust to current uncertainties in models and observations. Virtually all previous research in this area has attempted to discriminate an anthropogenic signal from internal variability. Here, we present evidence that a human-caused signal can also be identified relative to the larger "total" natural variability arising from sources internal to the climate system, solar irradiance changes, and volcanic forcing. Consistent signal identification occurs because both internal and total natural variability (as simulated by state-of-the-art models) cannot produce sustained global-scale tropospheric warming and stratospheric cooling. Our results provide clear evidence for a discernible human influence on the thermal structure of the atmosphere.

Santer, Benjamin D.↗
NASA NTRS1984

In-flight thrust determination on a real-time basis

A real time computer program was implemented on a F-15 jet fighter to monitor in-flight engine performance of a Digital Electronic Engine Controlled (DEES) F-100 engine. The application of two gas generator methods to calculate in-flight thrust real time is described. A comparison was made between the actual results and those predicted by an engine model simulation. The percent difference between the two methods was compared to the predicted uncertainty based on instrumentation and model uncertainty and agreed closely with the results found during altitude facility testing. Data was obtained from acceleration runs of various altitudes at maximum power settings with and without afterburner. Real time in-flight thrust measurement was a major advancement to flight test productivity and was accomplished with no loss in accuracy over previous post flight methods.

Ray, R. J.↗
NASA NTRS1979

The probability distribution of the predicted CFM-induced ozone depletion

It is argued from the central limit theorem that the uncertainty in model predicted changes of the ozone column density is best represented by a normal probability density distribution. This conclusion is validated by comparison with a probability distribution generated by a Monte Carlo technique. In the case of the CFM-induced ozone depletion, and based on the estimated uncertainties in the reaction rate coefficients alone the relative mean standard deviation of this normal distribution is estimated to be 0.29.

Ehhalt, D. H.↗
DOE OSTI2023

Clarifying trust of materials property predictions using neural networks with distribution-specific uncertainty quantification

It is critical that machine learning (ML) model predictions be trustworthy for high-throughput catalyst discovery approaches. Uncertainty quantification (UQ) methods allow estimation of the trustworthiness of an ML model, but these methods have not been well explored in the field of heterogeneous catalysis. Herein, we investigate different UQ methods applied to a crystal graph convolutional neural network to predict adsorption energies of molecules on alloys from the Open Catalyst 2020 dataset, the largest existing heterogeneous catalyst dataset. We apply three UQ methods to the adsorption energy predictions, namely k-fold ensembling, Monte Carlo dropout, and evidential regression. The effectiveness of each UQ method is assessed based on accuracy, sharpness, dispersion, calibration, and tightness. Evidential regression is demonstrated to be a powerful approach for rapidly obtaining tunable, competitively trustworthy UQ estimates for heterogeneous catalysis applications when using neural networks. Recalibration of model uncertainties is shown to be essential in practical screening applications of catalysts using uncertainties.

36 MATERIALS SCIENCE↗
NASA NTRS2020

MODIS Reflective Solar Bands Lunar Calibration Update and Improvements

The MODerate resolution Imaging Spectroradiometer (MODIS) instruments on-board the Terra and Aqua spacecrafts were launched on December 18, 1999 and May 4, 2002, respectively. Each instrument has been scheduled to view the Moon through its Space View (SV) port approximately once per month in order to monitor the long-term radiometric stability of their reflective solar bands (RSB). The lunar calibration has played a key role in tracking the change in the response versus scan angle (RVS) of the instrument’s scan mirror, which is an essential part of the RSB calibration. The lunar irradiance observed by MODIS depends on the view geometry and it is crucial to remove the geometric effects in order to derive accurate RSB calibration coefficients. With a scheduled roll maneuver, the lunar phase angles are kept in a narrow range of 1° for each MODIS instrument such that the impact of these geometric effects is minimized. Nevertheless, it remains a challenge to completely remove the impact of geometric effects in the calibration coefficients derived from the lunar observations, considering the high expectations for the accuracy and quality of the results. In the current MODIS lunar calibration, the geometric effects on the lunar irradiance are corrected by the RObotic Lunar Observatory (ROLO) model. The overall relative uncertainty of the ROLO model for the MODIS calibration has been assessed to be about 1% in the selected lunar phase angle range of 1°. It could be as large as 4% beyond this small phase angle range, especially for the shortest wavelengths. This uncertainty induces noticeable oscillations in the calibration coefficients derived from the lunar observations. We developed a simple lunar model, which is based on MODIS lunar measurements, derived from scheduled lunar observations, as well as those obtained from intrusion of the Moon in the SV, referred to as unscheduled lunar observations, in the time period from 2005 to 2012. Here, the scheduled lunar observations for the entire mission are reprocessed with the new lunar model applied to correct the view geometry effects for each MODIS instrument. New calibration coefficients for the RSBs are calculated. We show that oscillations and noise in the derived lunar calibration coefficients are significantly reduced for both MODIS instruments.

Terra↗
NASA NTRS2010

Understanding Aerosol Direct Radiative Forcing - How Satellites Fit In

The aerosol data products from the NASA Earth Observing System's MISR and MODIS instruments provide significant advances in regional and global aerosol optical depth (AOD) mapping, aerosol type measurement, and source plume characterization from space. Although these products have been and are being used for many applications, ranging from regional air quality assessment, to aerosol air mass type identification and evolution, to aerosol injection height and aerosol transport model validation, uncertainties still limit the quantitative constraints these satellite data place on global-scale direct aerosol radiative forcing. Some further refinement of the current aerosol products is possible, but a major advance in this area seems to require a different paradigm, involving the integration of satellite and suborbital data with models. This presentation will briefly summarize where we stand, and what incremental advances we can expect, with the current aerosol products, and will then elaborate on some initial steps aimed at the necessary integration of data from multiple sources and models.

Kahn, R. A.↗
DOE OSTI2025

Subsurface hydrological controls on the short-term effects of hurricanes on nitrate–nitrogen runoff loading: a case study of Hurricane Ida using the Energy Exascale Earth System Model (E3SM) Land Model (v2.1)

When the nutrient level in the soil surpasses vegetation demand, nutrient losses due to surface runoff and subsurface leaching are the major reasons for the deterioration of water quality. The lower Mississippi River basin (LMRB) is one of the sub-basins that deliver the highest nitrogen loads to the Gulf of Mexico. Potential changes in episodic events induced by hurricanes may exacerbate water quality issue in the future. However, uncertainties in modeling the hydrologic response to hurricanes may limit the modeling of nutrient losses during such events. Using a machine learning approach, we calibrated the land component of the Energy Exascale Earth System Model (E3SM), or ELM, version 2.1, based on the water table depth (WTD) of a calibrated 3D subsurface hydrology model. While the overall performance of the calibrated ELM is satisfactory, some discrepancies in WTD remain in slope areas with low precipitation due to the missing lateral flow process in ELM. Simulations including biogeochemistry performed using ELM with and without model calibration showed important influences of soil hydrology, precipitation intensity, and runoff parameterization on the magnitude of nitrogen runoff loss and the leaching pathway. Despite such sensitivities, both ELM simulations produced reduced WTD and increased runoff and accelerated nitrate–nitrogen runoff loading during Hurricane Ida in August 2021, consistent with the observations. With observations suggesting more pronounced effects of Hurricane Ida on nitrogen runoff than the simulations, we identified factors for model improvement to provide a useful tool for studying hurricane-induced nutrient losses in the LMRB region.

54 ENVIRONMENTAL SCIENCES↗