WS&H Metrics Summit 2024 Environment, Safety, Health & Quality Metrics Overview
Explore the source record for details and available documents.
Barraza, Elizabeth Ann↗
SEARCH · Engineering Papers
Results for “METRICS”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Exploratory Precipitation Metrics: Spatiotemporal Characteristics, Process-Oriented, and Phenomena-Based
Abstract Precipitation sustains life and supports human activities, making its prediction one of the most societally relevant challenges in weather and climate modeling. Limitations in modeling precipitation underscore the need for diagnostics and metrics to evaluate precipitation in simulations and predictions. While routine use of basic metrics is important for documenting model skill, more sophisticated diagnostics and metrics aimed at connecting model biases to their sources and revealing precipitation characteristics relevant to how model precipitation is used are critical for improving models and their uses. This paper illustrates examples of exploratory diagnostics and metrics including 1) spatiotemporal characteristics metrics such as diurnal variability, probability of extremes, duration of dry spells, spectral characteristics, and spatiotemporal coherence of precipitation; 2) process-oriented metrics based on the rainfall–moisture coupling and temperature–water vapor environments of precipitation; and 3) phenomena-based metrics focusing on precipitation associated with weather phenomena including low pressure systems, mesoscale convective systems, frontal systems, and atmospheric rivers. Together, these diagnostics and metrics delineate the multifaceted and multiscale nature of precipitation, its relations with the environments, and its generation mechanisms. The metrics are applied to historical simulations from phases 5 and 6 of the Coupled Model Intercomparison Project. Models exhibit diverse skill as measured by the suite of metrics, with very few models consistently ranked as top or bottom performers compared to other models in multiple metrics. Analysis of model skill across metrics and models suggests possible relationships among subsets of metrics, motivating the need for more systematic analysis to understand model biases for informing model development.
54 ENVIRONMENTAL SCIENCES↗
Toward Human-Centric Transportation and Energy Metrics: Influence of Mode, Vehicle Occupancy, Trip Distance, and Fuel Economy
Traditional metrics measuring transportation and energy outcomes can be augmented to better represent impacts on people's lives and systems-level performance. In this context, this study introduces two novel metrics: road capacity (as number of people traveling and accessing services) and energy intensity (as energy use for people traveling and accessing services). Current national-level distributions of available data in the United States for factors contributing to the two new integrated metrics are used as context to evaluate potential outcomes. These factors include vehicle occupancy, mode share, fuel economy, and trip distance. Variations in input values provide insights on how these factors shape efficiencies in road capacity and energy intensity. Parametric sensitivity analysis indicates that the impact of each input depends upon the metric being evaluated. For the human-centered road capacity mobility metric, increasing vehicle occupancy has the largest effect – twice that of increasing mode share for bike, walk, and transit. For the energy intensity mobility metric, the effect of improving fuel economy is the largest. However, when the focus is on accessibility (instead of mobility), for both metrics the effect of lowering average trip distance is the largest. Additionally, a novel interactive tool to visualize the results for various parameter combinations makes the metrics practitioner ready. The findings suggest that the diffusion of new human-centric metrics that benchmark outcomes associated with road capacity and energy may be significant in motivating new sustainable transportation investments and efficient utilization of infrastructure, mobility assets, and services.
ADVANCED PROPULSION SYSTEMS↗
Competitiveness Metrics for Electricity System Technologies
The relative economic competitiveness of power generation technologies is a topic of much interest to diverse electric industry participants. However, assessing competitiveness can be challenging as it requires considering both total costs and total system value of each technology, which are complicated by the (1) numerous and diverse grid services needed to operate a reliable power system; (2) variations in the economic value of the grid services with system state and location, and over multiple timescales, due to the challenges of transporting and storing electricity; and (3) the unique characteristics of different electric system assets. Ideally, metrics designed or used to convey technology competitiveness must consider these complexities, but existing metrics often fall short. For example, the levelized cost of energy does not consider the system economic value of the various technologies nor does it consider services beyond electricity production. Various other metrics have been designed with the purpose of more-accurately communicating the economic viability of electric system technologies. In this report, we summarize the primary sources and components of costs and value and review the known competitiveness metrics by presenting their definitions, applications, advantages, and disadvantages. We also introduce a new set of competitiveness metrics, which we refer to as System Profitability metrics, that more-directly applies the economic principles of return-on-investment to electric system technologies. We use conceptual examples to show how the System Profitability metrics better reflect economic viability and relative technology competitiveness compared with existing metrics. We also describe how competitiveness metrics can be quantified using optimization-based models and demonstrate this capability using a U.S. electric sector capacity expansion model.
29 ENERGY PLANNING, POLICY, AND ECONOMY↗
Metric Type in the Target-matrix Mesh Optimization Paradigm
The Target Matrix Optimization Paradigm (TMOP) is a method for improving the accuracy, efficiency, and robustness of numerical solutions to partial differential equations by improving the geometric quality of the computational mesh, primarily through node movement. TMOP has been successfully applied to a number applications even though the paradigm was not fully understood at the time. With this work, TMOP can be seen to be a tightly woven fabric of interconnecting ideas and concepts that provides a powerful approach to mesh optimization. The central unifying concepts in TMOP are the concept of a Target Matrix and the concept of Metric Type. Target matrices are motivated by the desire to make mesh quality improvement application-specific and, when needed, solution-adaptive. Metric type plays an essential role because it provides, through the use of typed metrics, the bridge between application-specific quality and target construction. It is shown that there are eight theoretical metric types, including the shape and shape+size types used informally in the past. It is shown further that there exist well-posed metrics corresponding to six of the eight metric types. A well-posed metric is a metric that is typed and convex, polyconvex, or invex, and further, it is a metric that simplifies target construction.
97 MATHEMATICS AND COMPUTING↗
A biology-informed similarity metric for simulated patches of human cell membrane
Complex scientific inquiries rely increasingly upon large and autonomous multiscale simulation campaigns, which fundamentally require similarity metrics to quantify ‘sufficient’ changes among data and/or configurations. However, subject matter experts are often unable to articulate similarity precisely or in terms of well-formulated definitions, especially when new hypotheses are to be explored, making it challenging to design a meaningful metric. Furthermore, the key to practical usefulness of such metrics to enable autonomous simulations lies in in situ inference, which requires generalization to possibly substantial distributional shifts in unseen, future data. Here, we address these challenges in a cancer biology application and develop a meaningful similarity metric for ‘patches’—regions of simulated human cell membrane that express interactions between certain proteins of interest and relevant lipids. In the absence of well-defined conditions for similarity, we leverage several biology-informed notions about data and the underlying simulations to impose inductive biases on our metric learning framework, resulting in a suitable similarity metric that also generalizes well to significant distributional shifts encountered during the deployment. We combine these intuitions to organize the learned embedding space in a multiscale manner, which makes the metric robust to incomplete and even contradictory intuitions. Our approach delivers a metric that not only performs well on the conditions used for its development and other relevant criteria, but also learns key spatiotemporal relationships without ever being exposed to any such information during training.
97 MATHEMATICS AND COMPUTING↗
Metrics as tools for bridging climate science and applications
In climate science and applications, the term “metric” is used to describe the distillation of complex, multifaceted evaluations to summarize the overall quality of a model simulation, or other data product, and/or as a means to quantify some response to climate change. Metrics provide insights into the fidelity of processes and outcomes from climate models and can assist with both differentiating models' representation of variables or processes and informing whether models are “fit for purpose.” Metrics can also provide a valuable reference point for co-production of knowledge between climate scientists and climate impact practitioners. Although continued metric developments enable model developers to better understand the impacts of decisions made in the model design process, metrics also have implications for the characterization of uncertainty and facilitating analyses of underlying physical processes. As a result, comprehensive evaluation with multiple metrics enhances usability of climate information by both scientific and stakeholder communities. Here, this paper presents examples of insights gained from the development and appropriate use of metrics, and provides examples of how metrics can be used to engage with stakeholders and inform decision-making.
54 ENVIRONMENTAL SCIENCES↗
A spectral metric for collider geometry
By quantifying the distance between two collider events, one can triangulate a metric space and reframe collider data analysis as computational geometry. One popular geometric approach is to first represent events as an energy flow on an idealized celestial sphere and then define the metric in terms of optimal transport in two dimensions. In this paper, we advocate for representing events in terms of a spectral function that encodes pairwise particle angles and products of particle energies, which enables a metric distance defined in terms of one-dimensional optimal transport. This approach has the advantage of automatically incorporating obvious isometries of the data, like rotations about the colliding beam axis. It also facilitates first-principles calculations, since there are simple closed-form expressions for optimal transport in one dimension. Up to isometries and event sets of measure zero, the spectral representation is unique, so the metric on the space of spectral functions is a metric on the space of events. At lowest order in perturbation theory in electron-positron collisions, our metric is simply the summed squared invariant masses of the two event hemispheres. Going to higher orders, we present predictions for the distribution of metric distances between jets in fixed-order and resummed perturbation theory as well as in parton-shower generators. Finally, we speculate on whether the spectral approach could furnish a useful metric on the space of quantum field theories.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
Are light curve classification metrics good proxies for SN Ia cosmological constraining power?
Context. When selecting a light curve classifier for use as part of a photometric supernova Ia (SN Ia) cosmological analysis, it is common to make decisions based on metrics of classification performance, such as the contamination within the photometrically classified SN Ia sample, rather than a measure of cosmological constraining power. If the former is an appropriate proxy for the latter, this practice would eliminate the computational expense of a full cosmology forecast in the analysis pipeline design process. Aims. This study tests the assumption that light curve classification metrics are an appropriate proxy for cosmology metrics. Methods. We emulated photometric SN Ia cosmology light curve samples with controlled contamination rates of individual contaminant classes and evaluated each of them under a set of classification metrics. We then derived cosmological parameter constraints from all samples under two common analysis approaches and quantified the impact of contamination by each contaminant class on the resulting cosmological parameter estimates. Results. We observe that cosmology metrics are sensitive to both the contamination rate and the class of the contaminating population, whereas the classification metrics are shown to be insensitive to the latter. Conclusions. Based on these findings, we discourage any exclusive reliance on light curve classification-based metrics for analysis design decisions, which (counterintuitively) include but are not limited to the classifier choice. Instead, we recommend optimising science analysis pipeline design choices using a metric of the information gained about the physical parameters of interest.
79 ASTRONOMY AND ASTROPHYSICS↗
The Making of a Metric: Co-Producing Decision-Relevant Climate Science
Developing decision-relevant science for adaptation requires the identification of climatic parameters that are both actionable for practitioners as well as tractable for modelers. In many sectors, these decision-relevant climatic metrics and the approaches that enable their identification remain largely unknown. “Co-production” of science with scientists and decision-makers is one potential way to identify these metrics, but there is little research describing specific and successful co-production approaches. This paper examines the negotiations and outcomes from Project Hyperion, wherein scientists and water managers jointly developed decision-relevant climatic metrics for adaptive water management. We identify successful co-production strategies by analyzing the project’s numerous back-and-forth engagements and tracing the evolution of the science during these engagements. We found that effective mediation between scientists and managers needed dedicated “boundary spanners” with significant modeling expertise. Translating practitioners’ information needs into tractable climatic metrics required direct and indirect methods of eliciting knowledge. We identified four indirect methods that were particularly salient for extracting tacitly held knowledge and enabling shared learning: developing a hierarchical framework linking management issues with metrics, starting discussions from the planning challenges, collaboratively exploring the planning relevance of new scientific capabilities, and using analogies of other “good” metrics. The decision-relevant metrics we developed provide insights into advancing adaptation-relevant climate science in the water sector. The co-production strategies we identified can be used to design and implement productive scientist–decision-maker interactions. Overall, the approaches and metrics we developed can help climate science to expand in new and more use-inspired directions.
Planning↗
Reliability metrics and their management implications for open pond algae cultivation
The prevalence of contaminating organisms in outdoor algae cultivation, with the often-associated dramatic crop failures, necessitates the need for metrics describing production system reliability. Standard metrics for algae cultivation reliability are critically needed to be able to map improvements in operational parameters, but do not currently exist. In this work, we present a set of standard metrics including mean time to failure (MTTF) and mean time between failures (MTBF) as the basis for the calculation of pond failure rate (FR) and reliability coefficient (RC). Metrics associated with the numbers of contaminating organisms such as abundance ratio (AR), prevalence of infection (PI), and mean intensity of infection (MII) are also relevant. These metrics, based on measured experimental values of outdoor pond performance during the Algae Testbed Public-Private Partnership (ATP3) Unified Field Studies (UFS), provide a basis for quantifying pond failure and provide insight into potential pond management and contaminant mitigation strategies. From these reliability metrics applied to this dataset, we are able to link operational parameters, such as harvest frequency and inoculum source, to pond reliability for different algae strains and seasons, and from an assessment of AR, provide contamination thresholds beyond which a culture may be unrecoverable. Ultimately, the implementation of widely-practiced and simple-to-calculate algae pond reliability metrics calculated from rapid and easy to collect data or observations will reduce risk and uncertainty in large-scale algae deployment and aid in the development of integrated pest management (IPM) strategies.
09 BIOMASS FUELS↗
Ecologically inspired metrics for transitioning to a sustainable and resilient circular economy with application to multilayer plastic films
Current Circular Economy (CE) frameworks applied to product chains exhibit notable shortcomings. These include neglecting the resilience and robustness of design, requiring detailed economic and environmental data for impact assessment, and relying on qualitative rather than quantitative metrics capturing certain CE design aspects. In the current contribution, we addressed these shortcomings by developing an Ecologically inspired (Eco-inspired) Framework using the mathematical foundations of Ecological Network Analysis (ENA). While ENA metrics have previously found application in designing circular economies, particularly in Industrial Symbiosis (IS) networks, our adaptation tailors these metrics for use in product-level CE, recognizing the inherent distinctions between product-level CE and IS. Our Eco-inspired Framework comprises three key categories to provide holistic and granular-level metrics for designing product-level CE. The first set of metrics assesses circularity and resource efficiency. The second set gauges network intensity and robustness as complementary indicators ensuring a CE is both sustainable and resilient. The third group of metrics evaluates enhancement potential of CE strategies through introducing quantitative metrics for measuring the degree of closed-loop strategies and average circularity level of a CE design. The three comprehensive set of indicators within the Eco-inspired Framework uniquely captures various facets of circular design, whether originating from technological innovations and recovery improvement at the end of life (EoL), shifts in human consumption patterns, alterations in product design, or changes in business models. The framework’s application is tested in designing a CE for multilayer Polyethylene-Polyamide (PE-PA) films. Using the Eco-inspired Framework, we identified the best strategy for designing a resilient and sustainable CE for PE-PA films. A diverse set of EoL strategies along with a reduction in product consumption can improve circularity and resilience by 650% and 255%, respectively, and mitigate greenhouse gas emissions by 90%. The framework minimizes trade-offs between sustainability and circularity goals and offers insights on how to enhance each strategy for achieving a resilient and sustainable CE for products.
54 ENVIRONMENTAL SCIENCES↗
A New Evaluation Metric for Demand Response-Driven Real-Time Price Prediction Towards Sustainable Manufacturing
Abstract The increasing industry energy demand highlights the urgency of demand response management, while the emerging smart manufacturing technologies pave the way for the implementation of real-time price (RTP)-based demand response management towards sustainable manufacturing. The demand response management requires scheduling of manufacturing systems based on RTP predictions, and thus the prediction quality can directly alter the effectiveness of demand response. However, since the general price prediction algorithms and prediction evaluation metrics are not specifically designed for RTP in demand response problems, a good RTP prediction obtained and evaluated by these algorithms and metrics may not be suitable for demand response scheduling. Therefore, in this study, the relationships between the effectiveness of demand response for manufacturing systems and evaluation results from six commonly used metrics are investigated. Meanwhile, a new metric called k-peak distance (KPD), considering the characteristics of the demand response problem, is proposed and compared with the other six metrics. Furthermore, an encoder-decoder long short-term memory recurrent neural network with KPD is proposed to provide better RTP prediction for manufacturing demand response problems. The case studies indicate that the proposed KPD metric shows a 1.8–3.6 times higher correlation with the demand response effectiveness compared to the other metrics. In addition, the production schedule based on the RTP prediction obtained from the proposed algorithm can improve the effectiveness of demand response by 23.4% on average.
Engineering↗
ARM data-oriented metrics and diagnostics package for climate model evaluation (ARM-DIAGS-V3) version 3
A Python-based metrics and diagnostics package is currently being developed by the ARM Infrastructure Team at Lawrence Livermore National Laboratory to facilitate the use of long-term high frequency measurements from the ARM program in evaluating the regional climate simulation of clouds, radiation, precipitation, and aerosols. This metrics and diagnostics package computes climatological means of targeted climate model simulation and generates tables and plots for comparing the model simulation with ARM observational data. The CMIP model data sets are also included in the package to enable model inter-comparison as demonstrated in Zhang et al. (2018) and Zhang et al. (2020). The mean of the CMIP model can be served as a reference for individual models. Basic performance metrics are computed to measure the accuracy of mean state and variability of climate models. The evaluated physical quantities include cloud fraction, temperature, relative humidity, cloud liquid water path, total column water vapor, precipitation, sensible and latent heat fluxes, aerosol optical depth, and radiative fluxes, with plan to extend to more fields, such as the evaluation of model simulated aerosol physicochemical properties and cloud microphysics properties. Process-oriented diagnostics focusing on aerosol, cloud, and precipitation-related phenomena are also being developed for the evaluation and development of specific model physical parameterizations. In addition to the Southern Great Plains (SGP), North Slope of Alaska (NSA) and Tropical Western Pacific (TWP) atmospheric observatories in the ARMDIAGS version 2.0, the version 3.0 package have extended to the data collected at the ARM Eastern North Atlantic (ENA) atmospheric observatory and the Observation and Modeling of the Green Ocean Amazon (GOAMAZON) field campaign. The metrics and diagnostics package are currently built upon standard Python libraries and additional Python packages developed by DOE (CDAT). The ARM metrics and diagnostic package is available publicly with the hope that it can serve as an easy entry point for climate modelers to compare their models with ARM data. In this report, we first provide an overview of major metrics in section 2. The input data, which constitutes the core content of the metrics and diagnostics package, is summarized in section 3. A user's guide documenting the workflow/structure of the version 3.0 codes and including step-by-step instruction for running the package is described in section 4.
54 ENVIRONMENTAL SCIENCES↗
NLR HPC Eagle GPU Node Metrics
Ganglia node metrics and iLO (Integrated Lights Out) power data captured from six representative Eagle GPU nodes The Eagle HPC operated at NLR from 2019 through 2024. Eagle was a 2,000-node, 8-petaflop system. This dataset is a representative sample of metrics for 6 of the GPU nodes. Each GPU node contained 2 CPUs and 2 GPUs. Data provided in compressed CSV format. Ganglia and iLO Power Time Series Fields ts: Timestamp dv: Device / Node - Rack and Unit - r103u17 == r(ack)103u(nit)17 mt: Metric (only present for Ganglia) vl: Value - Value in watts for iLO power (instantaneous value at sampling time) or specified Ganglia metric below Ganglia Metrics Metric name -- Metric description -- Unit cpu_aidle -- Percent of time since boot idle CPU -- Percent cpu_idle -- Percent CPU idle -- Percent cpu_nice -- Percent CPU nice -- Percent cpu_speed -- Speed in MHz of CPU -- MHz cpu_user -- Percent CPU user -- Percent cpu_wio -- The percentage of CPU Wait I/O -- Percent gpu0_bar1_memory -- Used GPU bar1 memory -- MB gpu0_decoder_util -- GPU decoder utilization -- Percent gpu0_ecc_db_error -- Total ECC error counts for the GPU -- Number gpu0_encoder_util -- GPU encoder utilization -- Percent gpu0_fan -- Fan speed -- RPM gpu0_fb_memory -- Used GPU framebuffer memory -- MB gpu0_graphics_clock_report -- Current clock speeds for the device -- MHz gpu0_mem_total -- Memory total -- MB gpu0_mem_util -- Memory utilization -- Percent gpu0_power_usage_report -- Power usage report -- Watts gpu0_temp -- GPU 1 temperature -- Celsius gpu1_bar1_memory -- Used GPU bar1 memory -- MB gpu1_decoder_util -- GPU decoder utilization -- Percent gpu1_ecc_db_error -- Total ECC error counts for the GPU -- Number gpu1_encoder_util -- GPU encoder utilization -- Percent gpu1_fan -- Fan speed -- RPM gpu1_fb_memory -- Used GPU framebuffer memory -- MB gpu1_graphics_clock_report -- Current clock speeds for the GPU -- MHz gpu1_mem_total -- Memory total -- MB gpu1_mem_util -- Memory utilization -- MB gpu1_power_usage_report -- Power usage report -- Watts gpu1_temp -- GPU 1 temperature -- Celsius ipmi_cpu1_temp -- CPU 1 temperature -- Celsius ipmi_cpu2_temp -- CPU 2 temperature -- Celsius ipmi_inlet_ambient_temp -- Temperature measured at intake -- Celsius ipmi_vr_p1_temp -- CPU 1 voltage regulator temperature -- Celsius ipmi_vr_p2_temp -- CPU 2 voltage regulator temperature -- Celsius mem_buffers -- Amount of buffered memory -- Bytes mem_cached -- Amount of cached memory -- Bytes mem_free -- Amount of available memory -- Bytes mem_shared -- Amount of shared memory -- Bytes mem_total -- Amount of available memory -- Bytes
97 MATHEMATICS AND COMPUTING↗
Velocity- and pressure-based metrics for estimating strike injuries during fish passage through hydro turbines
Hydropower is a crucial source of clean and reliable energy worldwide, and its importance will continue to grow. To enhance the environmental sustainability of hydropower development and operations, it is essential to predict the strike injury rates of fish passing through turbines accurately and cost-effectively. However, conventional experiments involving a large number of live fish are still commonly conducted in practice, and previous attempts mainly focused on the dose-response relationships at an individual level or relative comparisons of biological characterization between different conditions. Thus, this study proposes two novel strike metrics based on velocity and pressure (M V and M P ) measured by the cutting-edge Sensor Fish (SF) technology, designed to quantify the biological effects of strikes and collisions between fish and rigid hydraulic structures. SF measurements and 48-hour survival rates of live fish from a study conducted at the Ice Harbor Dam on the Snake River were used to establish thresholds for the two proposed metrics. These metrics and thresholds were then applied to three additional SF studies conducted at hydropower facilities featuring Kaplan turbines within the Columbia River basin. Results from the application of the strike metrics indicate that the estimated survival rates derived from both metrics agree well with previously reported live fish survival rates. Furthermore, the study identified specific passage regions with elevated risks of fish mortality due to strikes and collisions. Overall, the proposed strike metrics present a reliable and cost-effective approach to reducing reliance on live fish and lowering costs in fish passage studies.
Biological characterization↗
A Customizable Metric to Provide a Comprehensive Picture of the Mobility Potential of a Location
Accessibility in a geo-spatial context refers to the ease of reaching a variety of opportunities from a given location. Accessibility theories (and resulting metrics) have traditionally focused on quantifying access to specific opportunities (such as jobs), or focused on specific modes (such as car, bike, etc.). Such approaches often fall short of providing a comprehensive picture of the true accessibility potential of a location as a combination of multiple modes to multiple types of destinations. Addressing this drawback, a novel metric labeled the ‘Mobility Energy Productivity (MEP) Metric’ was developed at the National Renewable Energy Laboratory to quantify the mobility potential of a location to connect people to goods, services, and employment using a variety of modes, while accounting for time, energy, and affordability. The MEP metric has been integrated with advance travel behavior models to compute the changes in mobility potential for various future scenarios, such as introduction of automated vehicles, and/or electric vehicles—but does so at the aggregate, or average-citizen level. The MEP in its initial iteration is not customized to the particular socio-economic contingents, or even to an individual whose modal availability or pattern of trip making may substantially differ from the average. Addressing this gap, this research effort extends the MEP framework from a static state to a more tailored and dynamic state, one in which an individual, or group can customize the metric for their unique characteristics, such as modes, activity patterns, and time-of-day preferences. The extended MEP metric framework can now be integrated to assess the customized mobility energy productivity of an individual.
ADVANCED PROPULSION SYSTEMS,ENERGY PLANNING, POLIC↗
ARM Data-Oriented Metrics and Diagnostics Package for Climate Model Evaluation
A Python-based metrics and diagnostics package is currently being developed by the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Infrastructure Team at Lawrence Livermore National Laboratory (LLNL) to facilitate the use of long-term, high-frequency measurements from the ARM Facility in evaluating the regional climate simulation of clouds, radiation, and precipitation. This metrics and diagnostics package computes climatological means of targeted climate model simulation and generates tables and plots for comparing the model simulation with ARM observational data. The Coupled Model Intercomparison Project (CMIP) model data sets are also included in the package to enable model intercomparison as demonstrated in Zhang et al. (2017). The mean of the CMIP model can serve as a reference for individual models. Basic performance metrics are computed to measure the accuracy of mean state and variability of climate models. The evaluated physical quantities include cloud fraction, temperature, relative humidity, cloud liquid water path, total column water vapor, precipitation, sensible and latent heat fluxes, and radiative fluxes, with plan to extend to more fields, such as aerosol and microphysics properties. Process-oriented diagnostics focusing on individual cloud- and precipitation-related phenomena are also being developed for the evaluation and development of specific model physical parameterizations. The version 1.0 package is designed based on data collected at ARM’s Southern Great Plains (SGP) Research Facility, with the plan to extend to other ARM sites. The metrics and diagnostics package is currently built upon standard Python libraries and additional Python packages developed by DOE (such as CDMS and CDAT). The ARM metrics and diagnostic package is available publicly with the hope that it can serve as an easy entry point for climate modelers to compare their models with ARM data. In this report, we first present the input data, which constitutes the core content of the metrics and diagnostics package in section 2, and a user's guide documenting the workflow/structure of the version 1.0 codes, and including step-by-step instruction for running the package in section 3.
54 ENVIRONMENTAL SCIENCES↗