Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model diagnostics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

Unique Observational Constraints on the Seasonal and Longitudinal Variability of the Earth’s Planetary Albedo and Cloud Distribution Inferred From EPIC Measurements

Thorough comparison to observations is key to developing a credible climate model forecasting capability. Deep Space Climate Observatory (DSCOVR) measurements of Earth’s reflected solar and emitted thermal radiation provide a unique observational perspective that permits a more reliable model/data comparison than is possible with the otherwise available satellite data. The uniqueness is in the DSCOVR satellite’s viewing geometry, which enables continuous viewing of the Earth’s sunlit hemisphere from its Lissajous orbit around the Lagrangian L1 point. The key instrument is the Earth Polychromatic Imaging Camera (EPIC), which views the Earth’s sunlit hemisphere with 1024-by-1024-pixel imagery in 10 narrow spectral bands from 317 to 780 nm, acquiring up to 22 high spatial resolution images per day. The additional feature is that the frequency of EPIC image acquisition is nearly identical to that of the climate GCM data generation scheme where climate data for the entire globe are ‘instantaneously’ calculated at 1-h radiation time-step intervals. Implementation of the SHS (Sunlit Hemisphere Sampling) EPIC-view geometry for the in-line GCM output data sampling establishes a precise self-consistency in the space-time data sampling between EPIC observational and GCM output data generation and sampling. The remaining problem is that the GCM generated data are radiative fluxes, while the EPIC measurements are backscatter-dependent radiances. Radiance to flux conversion is a complex problem with no simple way to convert GCM radiative fluxes into spectral radiances. The more expedient approach is to convert the EPIC spectral radiances into broadband radiances by MODIS/CERES-based regression relationships and then into solar radiative fluxes using the CERES angular distribution models. Averaging over the sunlit hemisphere suppresses the meteorological weather noise, but preserves the intra-seasonal larger scale variability. Longitudinal slicing by the Earth’s rotation permits a self-consistent model/data comparison of the longitudinal model/data differences in the variability of the reflected solar radiation. Ancillary EPIC Composite data provide additional cloud property information for climate model diagnostics. Comparison of EPIC-derived seasonal and longitudinal variability of the Earth’s planetary albedo with the GISS ModelE2 results shows systematic overestimate of cloud reflectivity over the Pacific Ocean with corresponding underestimates over continental land areas.

DSCOVR↗

Earth System Model Aerosol–Cloud Diagnostics (ESMAC Diags) package, version 1: assessing E3SM aerosol predictions using aircraft, ship, and surface measurements

Abstract. An Earth system model (ESM) aerosol–cloud diagnostics package is developed to facilitate the routine evaluation of aerosols, clouds, and aerosol–cloud interactions simulated by the Energy Exascale Earth System Model (E3SM) from the US Department of Energy (DOE). The first version focuses on comparing simulated aerosol properties with aircraft, ship, and surface measurements, which are mostly measured in situ. The diagnostics currently cover six field campaigns in four geographical regions: eastern North Atlantic (ENA), central US (CUS), northeastern Pacific (NEP), and Southern Ocean (SO). These regions produce frequent liquid- or mixed-phase clouds, with extensive measurements available from the Atmospheric Radiation Measurement (ARM) program and other agencies. Various types of diagnostics and metrics are performed for aerosol number, size distribution, chemical composition, cloud condensation nuclei (CCN) concentration, and various meteorological quantities to assess how well E3SM represents observed aerosol properties across spatial scales. Overall, E3SM qualitatively reproduces the observed aerosol number concentration, size distribution, and chemical composition reasonably well, but it overestimates Aitken-mode aerosols and underestimates accumulation-mode aerosols over the CUS and ENA regions, suggesting that processes related to particle growth or coagulation might be too weak in the model. The current version of E3SM struggles to reproduce the new particle formation events frequently observed over both the CUS and ENA regions, indicating missing processes in current parameterizations. The diagnostics package is coded and organized in a way that can be extended to other field campaign datasets and adapted to higher-resolution model simulations.

54 ENVIRONMENTAL SCIENCES↗

Demonstrating Assurance of Model-Based Fault Diagnosis Systems on an Operational Mission

Developers of robotic scientific and commercial spacecraft are trending towards use of onboard autonomous capabilities for responding quickly to dynamic environments and rapidly changing situations. These capabilities need to know the state of the spacecraft’s health. Model-based fault diagnosis (MBFD) is an approach to estimating health by continuously verifying accurate behavior and diagnosing off-nominal behavior. Proper functioning of MBFD depends on 1) the quality of the diagnostic system model that is analyzed and compared to commands and onboard measurements to estimate a system’s health state, and 2) the correct functionality of the diagnosis engine interrogating the model and comparing its analyses to observed system behavior. Our goal is to develop Verification and Validation (V&V) techniques for MBFD to provide future missions sufficient confidence in its functionality and performance to deploy it on the systems they develop. Our work has been focused on infusing the techniques we developed earlier to an operational mission. First, we are constructing diagnostic models of a spacecraft attitude control system and updating our diagnostic engine so they can be demonstrated aboard the Arcsecond Space Telescope Enabling Research in Astrophysics (ASTERIA) mission, an operational spacecraft for which experiments in autonomy are being planned and executed, using the V&V techniques we have previously developed to assure they are both correct and complete. Since it is nearing the end of its life, ASTERIA provides a unique opportunity to demonstrate MBFD since the monitored components are expected to fail. Our demonstration will give system developers additional confidence to make timely, informed MBFD deployment decisions. Second, we will be completing performance assessments of the diagnostic engine/diagnostic model ensemble both on the flight system and ground-based testbeds to gain confidence in MBFD’s ability to run successfully in a spacecraft’s resource-constrained environment without adversely affecting other on-board activities. Finally, we are capturing our experience in preparing this demonstration in a set of checklists and guidance documents. Current practice includes high-level institutional guidance documents and standards, but at a high level of abstraction that does not necessarily address specific MBFD concerns. The purpose of the new checklists is to provide future mission developers clear, unambiguous, procedure-oriented guidance on assuring MBFD. This paper describes our work in these areas. For the first area, we describe the diagnostic models and updated diagnostic engine that will be used for the on-board demonstration. We describe how the V&V techniques we developed earlier are used to assure model and engine correctness and completeness. For the second area, we identify the performance measurement and assessment techniques used to characterize the diagnostic engine and diagnostic models, and discuss the effect of measured performance on overall mission operation. Finally, we present the checklist and guidance documents and describe how they meet the goals of providing system developers with clear, unambiguous, procedure-oriented guidance on MBFD assurance. We show how the techniques we have developed map into those artifacts.

Nikora, Allen↗

Research in Integrated Assessment Inter-Model Development, Testing and Diagnostics (Final Technical Report)

This is the final report on the second phase (spanning 2013-2018) of a project focused on further establishing an integrated assessment modeling (IAM) community research program on IAM model development, diagnostic testing, and inter-comparisons (PIAMDDI), including multi-model scenario ensemble analyses. The program was dedicated to improving the science of integrated assessment by doing cutting edge research in critical scientific areas of IAM development, and integrating that research with research programs on improved methods for model diagnostics, inter-comparison and ensemble-like analyses. We continued to link this program closely to other climate change research programs in the U.S. and abroad, and to make the work highly transparent, accessible and credible. As anticipated, progress in the scientific research areas informed the model comparison methods development and scenario tasks, and the comparisons and ensemble activities helped set priorities for the research areas.

54 ENVIRONMENTAL SCIENCES↗

Diagnostic indicators for integrated assessment models of climate policy

Integrated assessments of how climate policy interacts with energy-economic systems can be performed by a variety of models with different functional structures. This article proposes a diagnostic scheme that can be applied to a wide range of integrated assessment models to classify differences among models based on their carbon price responses. Model diagnostics can uncover patterns and provide insights into why, under a given scenario, certain types of models behave in observed ways. Such insights are informative since model behavior can have a significant impact on projections of climate change mitigation costs and other policy-relevant information. The authors propose diagnostic indicators to characterize model responses to carbon price signals and test these in a diagnostic study with 11 global models. Indicators describe the magnitude of emission abatement and the associated costs relative to a harmonized baseline, the relative changes in carbon intensity and energy intensity and the extent of transformation in the energy system. This study shows a correlation among indicators suggesting that models can be classified into groups based on common patterns of behavior in response to carbon pricing. Such a classification can help to more easily explain variations among policy-relevant model results.

Climate change economics↗

Earth System Model Aerosol–Cloud Diagnostics (ESMAC Diags) package, version 2: assessing aerosols, clouds, and aerosol–cloud interactions via field campaign and long-term observations

Abstract. Poor representations of aerosols, clouds, and aerosol–cloud interactions (ACIs) in Earth system models (ESMs) have long been the largest uncertainties in predicting global climate change. Huge efforts have been made to improve the representation of these processes in ESMs, and the key to these efforts is the evaluation of ESM simulations with observations. Most well-established ESM diagnostics packages focus on the climatological features; however, they lack process-level understanding and representations of aerosols, clouds, and ACIs. In this study, we developed the Earth System Model Aerosol–Cloud Diagnostics (ESMAC Diags) package to facilitate the routine evaluation of aerosols, clouds, and ACIs simulated the Energy Exascale Earth System Model (E3SM) from the US Department of Energy (DOE). This paper documents its version 2 functionality (ESMAC Diags v2), which has substantial updates compared with version 1 (Tang et al., 2022a). The simulated aerosol and cloud properties have been extensively compared with in situ and remote-sensing measurements from aircraft, ship, surface, and satellite platforms in ESMAC Diags v2. It currently includes six field campaigns and two permanent sites covering four geographical regions: the eastern North Atlantic, the central US, the northeastern Pacific, and the Southern Ocean. These regions produce frequent liquid- or mixed-phase clouds, with extensive measurements available from the DOE Atmospheric Radiation Measurement user facility and other agencies. ESMAC Diags v2 generates various types of single-variable and multivariable diagnostics, including percentiles, histograms, joint histograms, and heatmaps, to evaluate the model representation of aerosols, clouds, and ACIs. Select examples highlighting the capabilities of ESMAC Diags are shown using E3SM version 2 (E3SMv2). In general, E3SMv2 can reasonably reproduce many observed aerosol and cloud properties, with biases in some variables such as aerosol particle and cloud droplet sizes and number concentrations. The coupling of aerosol and cloud number concentrations may be too strong in E3SMv2, possibly indicating a bias in processes that control aerosol activation. Furthermore, the liquid water path response to a perturbed cloud droplet number concentration behaves differently in E3SMv2 and observations, which warrants further study to improve the cloud microphysics parameterizations in E3SMv2.

54 ENVIRONMENTAL SCIENCES↗

The Cloud Feedback Model Intercomparison Project (CFMIP) Diagnostic Codes Catalogue – metrics, diagnostics and methodologies to evaluate, understand and improve the representation of clouds and cloud feedbacks in climate models

The CFMIP Diagnostic Codes Catalogue assembles cloud metrics, diagnostics and methodologies, together with programs to diagnose them from general circulation model (GCM) outputs written by various members of the CFMIP community. This aims to facilitate use of the diagnostics by the wider community studying climate and climate change. Here, this paper describes the diagnostics and metrics which are currently in the catalogue, together with examples of their application to model evaluation studies and a summary of some of the insights these diagnostics have provided into the main shortcomings in current GCMs. Analysis of outputs from CFMIP and CMIP6 experiments will also be facilitated by the sharing of diagnostic codes via this catalogue. Any code which implements diagnostics relevant to analysing clouds – including cloud–circulation interactions and the contribution of clouds to estimates of climate sensitivity in models – and which is documented in peer-reviewed studies, can be included in the catalogue. We very much welcome additional contributions to further support community analysis of CMIP6 outputs.

58 GEOSCIENCES↗

Toward model-free temperature diagnostics of warm dense matter from multiple scattering angles

Warm dense matter plays an important role in astrophysical objects and technological applications, but the rigorous diagnostics of corresponding experiments is notoriously difficult. In this work, we present a model-free analysis of x-ray Thomson scattering (XRTS) measurements on isochorically heated graphite obtained at the Linac Coherent Light Source at multiple scattering angles. We demonstrate that the recent imaginary-time thermometry technique works for scattering data that have been measured in both forward and backward scattering geometry. This opens up the way toward a rigorous quantification of nonequilibrium effects in future experiments, where XRTS measurements are being obtained from multiple scattering angles from the same sample.

Equations of state↗

Integrating diagnostics and model-based optimization

Energy Management and Information Systems are a family of analytics technologies that include energy information systems, fault detection and diagnostics (FDD), and automated system optimization tools. Such systems have the potential to enable buildings to meet energy management goals of reducing total energy consumption and cost. Most current market offerings use data-driven and rule-based analytics. However, the use of physics-based models in the analytics offers potential improvements by providing an accurate estimation of outputs based on representation of the physical principles governing the building system behaviors. This also permits the use of design stage models to inform commissioning and operation. This paper describes the development and testing of a hybrid data-driven and physics model-based operational tool for energy efficiency in central cooling plants. The tool offers FDD functionality, setpoint optimization, and visualization of key performance parameters. It was demonstrated at a university campus in the mixed-humid ASHRAE Climate Zone 4A. Key performance metrics that were analyzed include plant electricity use reduction, plant model calibration, and system economics. Annual simulations indicate the tool can provide electricity savings of greater than 10% for approximately six months of the year, mainly during the winter season when wet bulb temperatures are low, though only 1.38% savings for the entire year. Additionally, over a 4-day period in April, recommended optimal setpoints were implemented, resulting in 17% savings versus metered baseline consumption. With respect to model calibration, the difference between model-predicted and measured parameters was less than 10% for 90% of data points acquired for three of six chillers, and for each ten cooling towers. Finally, the tool users reported that satisfaction with the capabilities was equal to or better than that with the preexisting BAS system.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

The Atmospheric Chemistry and Climate Model Intercomparison Project (ACCMIP): Overview and Description of Models, Simulations and Climate Diagnostics

The Atmospheric Chemistry and Climate Model Intercomparison Project (ACCMIP) consists of a series of time slice experiments targeting the long-term changes in atmospheric composition between 1850 and 2100, with the goal of documenting composition changes and the associated radiative forcing. In this overview paper, we introduce the ACCMIP activity, the various simulations performed (with a requested set of 14) and the associated model output. The 16 ACCMIP models have a wide range of horizontal and vertical resolutions, vertical extent, chemistry schemes and interaction with radiation and clouds. While anthropogenic and biomass burning emissions were specified for all time slices in the ACCMIP protocol, it is found that the natural emissions are responsible for a significant range across models, mostly in the case of ozone precursors. The analysis of selected present-day climate diagnostics (precipitation, temperature, specific humidity and zonal wind) reveals biases consistent with state-of-the-art climate models. The model-to- model comparison of changes in temperature, specific humidity and zonal wind between 1850 and 2000 and between 2000 and 2100 indicates mostly consistent results. However, models that are clear outliers are different enough from the other models to significantly affect their simulation of atmospheric chemistry.

humidity↗

Diagnostic and Prognostic Models for Generator Step-Up Transformers

In 2014, the online monitoring (OLM) of active components project under the Light Water Reactor Sustainability program at Idaho National Laboratory (INL) focused on diagnostic and prognostic capabilities for generator step-up transformers. INL worked with subject matter experts from the Electric Power Research Institute (EPRI) to augment and revise the GSU fault signatures previously implemented in the Electric Power Research Institute’s (EPRI’s) Fleet-Wide Prognostic and Health Management (FW-PHM) Suite software. Two prognostic models were identified and implemented for GSUs in the FW-PHM Suite software. INL and EPRI demonstrated the use of prognostic capabilities for GSUs. The complete set of fault signatures developed for GSUs in the Asset Fault Signature Database of the FW-PHM Suite for GSUs is presented in this report. Two prognostic models are described for paper insulation: the Chendong model for degree of polymerization, and an IEEE model that uses a loading profile to calculates life consumption based on hot spot winding temperatures. Both models are life consumption models, which are examples of type II prognostic models. Use of the models in the FW-PHM Suite was successfully demonstrated at the 2014 August Utility Working Group Meeting, Idaho Falls, Idaho, to representatives from different utilities, EPRI, and the Halden Research Project.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Some Eulerian and Lagrangian diagnostics for a model stratospheric warming

The Eliassen-Palm (EP) cross sections for the model are discussed. EP cross sections conveniently display the principal eddy fluxes of heat, momentum, and quasi-geostrophic potential vorticity on one diagram. The EP diagnostics are used to follow the evolution of the mean flow through into the second stage of the model warming, which includes the reversal of the polar westerlies and which appears to have more in common with observed warmings. The transformed Eulerian-mean equations presented by Andrews and McIntyre (1976, 1978) provide the basis for an investigation in which an especially clear picture of the warming dynamics is obtained. In the second stage of the model warming, the decelerating torque per unit mass is large throughout a deep layer in the middle atmosphere, spanning several scale heights.

Dunkerton, T.↗

Improving Reliability of Large Language Models for Nuclear Power Plant Diagnostics [Poster]

Large Language Models (LLMs) struggle out of the box when answering factually about detailed questions, especially in domains that are sparsely represented in their training data. This causes hallucinations and reduces reliability making it difficult for them to be used in practice. This work shows that using RAG techniques can improve factual accuracy and reliability, allowing for the application of LLMs in specialized areas, even when those areas that aren’t extensively covered in their initial training.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

The E3SM Diagnostics Package (E3SM Diags v2.7): a Python-based diagnostics package for Earth system model evaluation

Abstract. The E3SM Diagnostics Package (E3SM Diags) is a modern, Python-based Earth system model (ESM) evaluation tool (with Python module name e3sm_diags), developed to support the Department of Energy (DOE) Energy Exascale Earth System Model (E3SM). E3SM Diags provides a wide suite of tools for evaluating native E3SM output, as well as ESM data on regular latitude–longitude grids, including output from Coupled Model Intercomparison Project (CMIP) class models. E3SM Diags is modeled after the National Center for Atmospheric Research (NCAR) Atmosphere Model Working Group (AMWG, 2022) diagnostics package. In its version 1 release, E3SM Diags included a set of core essential diagnostics to evaluate the mean physical climate from model simulations. As of version 2.7, more process-oriented and phenomenon-based evaluation diagnostics have been implemented, such as analysis of the quasi-biennial oscillation (QBO), the El Niño–Southern Oscillation (ENSO), streamflow, the diurnal cycle of precipitation, tropical cyclones, ozone and aerosol properties. An in situ dataset from DOE's Atmospheric Radiation Measurement (ARM) program has been integrated into the package for evaluating the representation of simulated cloud and precipitation processes. This tool is designed with enough flexibility to allow for the addition of new observational datasets and new diagnostic algorithms. Additional features include customizable figures; streamlined installation, configuration and execution; and multiprocessing for fast computation. The package uses an up-to-date observational data repository maintained by its developers, where recent datasets are added to the repository as they become available. Finally, several applications for the E3SM Diags module were introduced to fit a diverse set of use cases from the scientific community.

54 ENVIRONMENTAL SCIENCES↗

SU-E-I-06: A Dose Calculation Algorithm for KV Diagnostic Imaging Beams by Empirical Modeling

Purpose: To develop accurate three-dimensional (3D) empirical dose calculation model for kV diagnostic beams for different radiographic and CT imaging techniques. Methods: Dose was modeled using photon attenuation measured using depth dose (DD), scatter radiation of the source and medium, and off-axis ratio (OAR) profiles. Measurements were performed using single-diode in water and a diode-array detector (MapCHECK2) with kV on-board imagers (OBI) integrated with Varian TrueBeam and Trilogy linacs. The dose parameters were measured for three energies: 80, 100, and 125 kVp with and without bowtie filters using field sizes 1×1–40×40 cm2 and depths 0–20 cm in water tank. Results: The measured DD decreased with depth in water because of photon attenuation, while it increased with field size due to increased scatter radiation from medium. DD curves varied with energy and filters where they increased with higher energies and beam hardening from half-fan and full-fan bowtie filters. Scatter radiation factors increased with field sizes and higher energies. The OAR was with 3% for beam profiles within the flat dose regions. The heal effect of this kV OBI system was within 6% from the central axis value at different depths. The presence of bowtie filters attenuated measured dose off-axis by as much as 80% at the edges of large beams. The model dose predictions were verified with measured doses using single point diode and ionization chamber or two-dimensional diode-array detectors inserted in solid water phantoms. Conclusion: This empirical model enables fast and accurate 3D dose calculation in water within 5% in regions with near charge-particle equilibrium conditions outside buildup region and penumbra. It considers accurately scatter radiation contribution in water which is superior to air-kerma or CTDI dose measurements used usually in dose calculation for diagnostic imaging beams. Considering heterogeneity corrections in this model will enable patient specific dose calculation.

60 APPLIED LIFE SCIENCES↗

ARM Data-Oriented Metrics and Diagnostics Package for Climate Model Evaluation

A Python-based metrics and diagnostics package is currently being developed by the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Infrastructure Team at Lawrence Livermore National Laboratory (LLNL) to facilitate the use of long-term, high-frequency measurements from the ARM Facility in evaluating the regional climate simulation of clouds, radiation, and precipitation. This metrics and diagnostics package computes climatological means of targeted climate model simulation and generates tables and plots for comparing the model simulation with ARM observational data. The Coupled Model Intercomparison Project (CMIP) model data sets are also included in the package to enable model intercomparison as demonstrated in Zhang et al. (2017). The mean of the CMIP model can serve as a reference for individual models. Basic performance metrics are computed to measure the accuracy of mean state and variability of climate models. The evaluated physical quantities include cloud fraction, temperature, relative humidity, cloud liquid water path, total column water vapor, precipitation, sensible and latent heat fluxes, and radiative fluxes, with plan to extend to more fields, such as aerosol and microphysics properties. Process-oriented diagnostics focusing on individual cloud- and precipitation-related phenomena are also being developed for the evaluation and development of specific model physical parameterizations. The version 1.0 package is designed based on data collected at ARM’s Southern Great Plains (SGP) Research Facility, with the plan to extend to other ARM sites. The metrics and diagnostics package is currently built upon standard Python libraries and additional Python packages developed by DOE (such as CDMS and CDAT). The ARM metrics and diagnostic package is available publicly with the hope that it can serve as an easy entry point for climate modelers to compare their models with ARM data. In this report, we first present the input data, which constitutes the core content of the metrics and diagnostics package in section 2, and a user's guide documenting the workflow/structure of the version 1.0 codes, and including step-by-step instruction for running the package in section 3.

54 ENVIRONMENTAL SCIENCES↗

Multiscale modeling and laser diagnostics to reveal non-equilibrium reaction chemistry at a plasma-liquid interface

Low-temperature, atmospheric-pressure plasmas in contact with liquid are at the core of a wide range of applications including water treatment, medicine, materials synthesis, and chemical transformation. In general, plasma-liquid processes are scientifically compelling because reactivity can be produced without a catalyst, in relatively inert molecules, such as air or nitrogen and liquid water, in their native states at ambient conditions. However, reactions at a plasma-liquid interface are extremely complex, occurring at a multiphase, gas-liquid boundary where excited or dissociate gaseous species dissolve and react with solution-phase species, and unique reaction pathways are induced by non-equilibrium chemistry. In particular, detailed knowledge of the physical and chemical processes, including what species are produced in the gas phase and how these species are subsequently transported across and react near the interfacial region, remains largely unanswered. In this project, modeling of the non-equilibrium reaction chemistry at the interface of low-temperature, atmospheric-pressure plasmas and liquid water is developed, supported by advanced laser diagnostics.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

ARM Data-Oriented Metrics and Diagnostics Package for Climate Model Evaluation

A Python-based metrics and diagnostics package is currently being developed by the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) user facility Infrastructure Team at Lawrence Livermore National Laboratory to facilitate the use of long-term, high-frequency measurements from ARM in evaluating the regional climate simulation of clouds, radiation, and precipitation. This metrics and diagnostics package computes climatological means of targeted climate model simulation and generates tables and plots for comparing the model simulation with ARM observational data.

54 ENVIRONMENTAL SCIENCES↗