SEARCH · Engineering Papers

Results for “METRICS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,189 records · Page 66

All sources NASA NTRS DOE OSTI

DOE OSTI2025

EVI-EnSitePy (Electric Vehicle Infrastructure – Energy Estimation and Site Optimization Tool in Python) [EVI-X Modeling Suite] [SWR-25-07]

EVI-EnSitePy is a comprehensive agent-based tool designed for the analysis and design of high-power charging sites, encompassing a wide array of site agents including Electric Vehicles (EVs), chargers, energy storage units (ESS), renewable energy resources (DER), and loads. This versatile tool offers diverse functionalities and a modular modeling approach, allowing detailed configuration of agents based on power ratings, port numbers, energy capacities, demand requirements, charger interfaces, and flexibility to customize the tool for project specific goals. By simulating agent interactions and employing various metrics, EVI-EnSitePy enables the assessment of site performance, exploration of energy management systems (EMS), and implementation of innovative EV charging policies. Utilizing EV charge schedules and arrival states, the tool performs thorough charging site simulations, with outputs consisting of agent and site-level power profiles and statistical metrics. Employing a tree graph structure, EVI-EnSitePy supports nested site structures and power distribution modeling. The tool's ability to generate charging schedules deterministically or via stochastic analysis further enhances its versatility. Through its features and capabilities, EVI-EnSitePy offers a powerful platform for informed decision-making in the realm of high-power charging site design and operation.

Jackson, Derek [National Renewable Energy Laborato↗

DOE OSTI2026

PYDICE

A new Python program has been created that calculates similarity metrics (E, ck), available as a web server. Users can post one or more Sensitivity Data Files (SDFs) to either calculate similarity metrics (returned as a JSON response packet) or to be converted into a different SDF format (returned as a zip file).

Holcomb, Andrew [Oak Ridge National Laboratory (OR↗

DOE OSTI2025

Intercomparison of Deep Learning Model Architectures for Atmospheric River Prediction

With a rapid surge in the application of machine learning (ML) for a diverse range of tasks in climate science, the present study addresses a challenge for climate scientists when selecting the optimal ML or deep learning (DL) architecture for a given application. In particular, a DL intercomparison study was performed with a focus on forecasting the position of atmospheric rivers (ARs) on short-range time scales (up to 5-day lead times). AR predictions from multiple DL architectures, including various types of convolutional autoencoders and a vision transformer (ViT), were compared against ECMWF ERA5 reanalysis and hindcasts from a global climate model. DL models with similar trainable parameters were trained on ERA5 reanalysis data and AR positions derived from a thresholding algorithm to ensure a fair comparison among the DL models. Each model’s performance and accuracy in forecasting AR location and key input fields within a 5-day window were assessed using metrics of root-mean-square error, anomaly correlation, and mean intersection over union. The ViT architecture outperformed other autoencoder models in most of the metrics. Incorporating additional meteorological fields only yielded slight improvements in forecasting certain fields at longer lead times. The results also suggest that a smaller number of input time steps or smaller number of autoregressive steps can achieve better prediction skills, while also improving the overall computational efficiency. This research offers valuable insights into the strengths and weaknesses of different DL techniques for AR forecasting, hopefully guiding the development of improved models for forecasting this phenomenon.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Process-Oriented Diagnostics: Principles, Practice, Community Development, and Common Standards

Process-oriented diagnostics (PODs) aim to provide feedback for model developers through model analysis based on physical hypotheses. However, the step from a diagnostic based on relationships among variables, even when hypothesis driven, to specific guidance for revising model formulation or parameterizations can be substantial. The POD may provide more information than a purely performance-based metric, but a gap between POD principles and providing actionable information for specific model revisions can remain. Furthermore, in coordinating diagnostics development, there is a trade-off between freedom for the developer, aiming to capture innovation, and near-term utility to the modeling center. Best practices that allow for the former, while conforming to specifications that aid the latter, are important for community diagnostics development that leads to tangible model improvements. Promising directions to close the gap between principles and practice include the interaction of PODs with perturbed physics experiments and with more quantitative process models as well as the inclusion of personnel from modeling centers in diagnostics development groups for immediate feedback during climate model revisions. Examples are provided, along with best-practice recommendations, based on practical experience from the NOAA Model Diagnostics Task Force (MDTF). Common standards for metrics and diagnostics that have arisen from a collaboration between the MDTF and the Department of Energy’s Coordinated Model Evaluation Capability are advocated as a means of uniting community diagnostics efforts.

58 GEOSCIENCES↗

DOE OSTI2026

Object-Based Evaluation of Dynamical and Statistical Downscaled Precipitation Products over CONUS

High-resolution precipitation data, generated through dynamical downscaling (DD) or statistical downscaling (SD) of global climate model output, provide critical information for regional climate assessment and adaptation planning. Most downscaling development and validation have focused on accurate gridscale precipitation construction and ignored the spatial structure of precipitation across model grids and at the event scale. However, many applications, e.g., hydrologic modeling and the analysis using the downscaled precipitation, require a reasonable representation of the spatial structure of precipitation within watersheds. Therefore, a set of standard metrics to evaluate the representation of the spatial structure of individual storms across diverse downscaled precipitation products is desired. To address this need, we conducted an object-based evaluation of precipitation in decades-long DD and SD products over the contiguous United States (CONUS). Specifically, we evaluate their ability to reproduce various features of precipitation objects in the observations: total volume, precipitation area, peak intensity, and spatial structure. Multiple metrics (bias, Perkins score, and nonparametric statistical tests) are used to quantify model performance. Our evaluation reveals notable variations in performance among individual products across different climate zones and seasons, as well as between extreme and nonextreme events. In general, most DD products exhibit balanced performance across the four precipitation object features, while SD products vary more significantly in their performance across products. Based on this comprehensive evaluation, we provide guidance on choosing downscaled products for specific regions, seasons, and precipitation object features. These findings and recommendations can inform precipitation-relevant modeling and analysis over CONUS, guide future downscaling technique developments, and provide actionable information for climate impact assessment and adaptation.

Downscaling↗

DOE OSTI2026

Object-Based Evaluation of Dynamical and Statistical Downscaled Precipitation Products over CONUS

Environmental sciences↗

DOE OSTI2025

An Observational Evaluation of RKW Theory over the U.S. Southern Great Plains

The theory of Rotunno et al. (“RKW” theory) addresses the behavior of squall-line cold pools in vertically sheared flows. It predicts that, within a given thermodynamic environment, a balance between baroclinic vorticity generation by the cold pool and low-level environmental vertical wind shear induces an upright updraft along the gust front that maximizes the initiation of new convective cells. Although this theory has been evaluated numerically, its applicability to observed systems remains unclear and is limited by a lack of critical measurements, including high-frequency thermodynamic and wind profiles across the gust front. Herein, observations from the Atmospheric Radiation Measurement Southern Great Plains (ARM-SGP) observatory near Lamont, Oklahoma, are used to evaluate RKW theory for 10 well-observed squall lines over a 11-yr period. For this evaluation, RKW parameters including cold-pool intensity (c), low-level ambient, line-normal vertical shear (ΔV n ), subcloud and cloud-layer updraft tilts, and multiple measures of system intensity are estimated. Furthermore, the c estimates rely on thermodynamic retrievals from the Atmosphere Emitted Radiance Interferometer (AERI), which are uncertain but verify reasonably well against independent observations. As predicted by the theory, for c/ΔV n ≥ 1, c/ΔV n correlates positively with updraft tilt and negatively with system intensity, but these results are not always statistically significant and are also sensitive to the method by which ΔV n is evaluated. Specifically, ΔV n evaluations that extend above the cold-pool top yield greater consistency with RKW predictions. Also, some measures of intensity correlate more strongly with standard moist instability metrics than with RKW parameters.

Cold pools↗

DOE OSTI2025

The Effect of Updraft Entrainment on Convective Cell Deepening in Realistic Large-Eddy Simulations

Entrainment of surrounding cooler and drier air into convective updrafts is one of the key processes that influence deep convection initiation and growth. Numerous studies have investigated the effect of entrainment on isolated convective cloud growth in idealized simulations, but the importance of this effect in realistic conditions with many interacting convective clouds remains uncertain. We examine the impact of entrainment on the depth reached by convective clouds in realistic large-eddy simulations (LES) over central Argentina during the Cloud, Aerosol, and Complex Terrain Interactions (CACTI) field campaign. Cloudy updrafts and their associated properties are assigned to convective cells tracked with radar reflectivity signatures. Several thousand convective cells are tracked over two high convective available potential energy (CAPE) and two low CAPE cases that support cells of varying depths and intensities. Entrainment is calculated explicitly as the fluxes of air into the outer surface of each cloudy updraft. Single-predictor logistic regression models are used to determine the relative importance of updraft, near-updraft, and preconvective initiation atmospheric conditions in predicting whether convective cells become deep. We then build a multiple-predictor regression model pairing important updraft and meteorological metrics with fractional entrainment rate. The probability of cells transitioning to deep convection is most sensitive to ambient 600-hPa relative humidity (42% of total metric contribution to cloud depth predictability), followed by low-level CAPE (28%), cloud-base updraft width (19%), and fractional entrainment (11%). Thus, the initial width of the updraft along with potential buoyancy and its dilution through the midtroposphere collectively determine whether deep convection will result from shallower clouds.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2020

MJO Teleconnections over the PNA Region in Climate Models. Part II: Impacts of the MJO and Basic State

In an assessment of 29 global climate models (GCMs), Part I of this study identified biases in boreal winter MJO teleconnections in anomalous 500-hPa geopotential height over the Pacific–North America (PNA) region that are common to many models: an eastward shift, a longer persistence, and a larger amplitude. In Part II, we explore the relationships of the teleconnection metrics developed in Part I with several existing and newly developed MJO and basic state (the mean subtropical westerly jet) metrics. The MJO and basic state diagnostics indicate that the MJO is generally weaker and less coherent and propagates faster in models compared to observations. The mean subtropical jet also exhibits notable biases such as too strong amplitude, excessive eastward extension, or southward shift. The following relationships are found to be robust among the models: 1) models with a faster MJO propagation tend to produce weaker teleconnections; 2) models with a less coherent eastward MJO propagation tend to simulate more persistent MJO teleconnections; 3) models with a stronger westerly jet produce stronger and eastward shifted MJO teleconnections; 4) models with an eastward extended jet produce an eastward shift in MJO teleconnections; and 5) models with a southward shifted jet produce stronger MJO teleconnections. The results are supported by linear baroclinic model experiments. Our results suggest that the larger amplitude and eastward shift biases in GCM MJO teleconnections can be attributed to the biases in the westerly jet, and that the longer persistence bias is likely due to the lack of coherent eastward MJO propagation.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2022

Benchmarking Simulated Precipitation Variability Amplitude across Time Scales

Here, objective performance metrics that measure precipitation variability across time scales from subdaily to interannual are presented and applied to Historical simulations of Coupled Model Intercomparison Project phase 5 and 6 (CMIP5 and CMIP6) models. Three satellite-based precipitation estimates (IMERG, TRMM, and CMORPH) are used as reference data. We apply two independent methods to estimate temporal variability of precipitation and compare the consistency in their results. The first method is derived from power spectra analysis of 3-hourly precipitation, measuring forced variability by solar insolation (diurnal and annual cycles) and internal variability at different time scales (subdaily, synoptic, subseasonal, seasonal, and interannual). The second method is based on time averaging and facilitates estimating the seasonality of subdaily variability. Supporting the robustness of our metric, we find a near equivalence between the results obtained from the two methods when examining simulated-to-observed ratios over large domains (global, tropics, extratropics, land, or ocean). Additionally, we demonstrate that our model evaluation is not very sensitive to the discrepancies between observations. Our results reveal that CMIP5 and CMIP6 models in general overestimate the forced variability while they underestimate the internal variability, especially in the tropical ocean and higher-frequency variability. The underestimation of subdaily variability is consistent across different seasons. The internal variability is overall improved in CMIP6, but remains underestimated, and there is little evidence of improvement in forced variability. Increased horizontal resolution results in some improvement of internal variability at subdaily and synoptic time scales, but not at longer time scales.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

ENSO-Induced Teleconnection: Process-Oriented Diagnostics to Assess Rossby Wave Sources and Ambient Flow Properties in Climate Models

Climate model fidelity in representing ENSO-induced teleconnection is assessed with process-oriented diagnostics that examine a chain of processes, from equatorial Pacific precipitation to the midlatitude circulation pattern over the Pacific–North American regions. Such processes are rarely addressed during model development. Using an upper-tropospheric divergent level, local vorticity gradient of the ambient zonal flow (∂ 2 U¯/∂y 2 ) and a restoring force for Rossby waves (β * ) are estimated, the equivalent barotropic vorticity equation is solved, and an anomalous Rossby wave source (RWS') quantified. The analysis is applied to AMIP5 and AMIP6 simulations. For a realistic circulation response representation, the hypothesis that models accurately represent the strength and location of RWS', and spatial variations in β * is tested. Compared to AMIP5, in AMIP6 there are clear improvements in representing RWS' and β * . To validate the hypothesis, the analysis identifies two metrics: spatially coherent RWS' in the subtropical North Pacific, and longitudes of negative β * over the western-central North Pacific. By projecting these metrics in two and three-dimensional views, improvements or degradations in model versions are apparent. If a model’s fidelity in representing ∂ 2 U¯/∂y 2 and RWS' are compromised, then radiated Rossby waves are reflected more equatorward, resulting in zonally elongated circulation anomalies over the central North Pacific. Thus, during climate model development, applying this analysis frequently will keep a regular check on the fidelity of the modeled response to anomalous El Niño convection in conjunction with changing model ambient flow dependencies. As a result, this analysis is intended to form a process-oriented diagnostics package, a community contribution to the NOAA Model Diagnostics Task Force.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Sea Ice and Cloud Processes Mediating Compensation between Atmospheric and Oceanic Meridional Heat Transports across the CMIP6 Preindustrial Control Experiment

Abstract Bjerknes compensation (BJC) refers to the anticorrelation observed between atmospheric and oceanic heat transport (AHT/OHT) variability, particularly on decadal to longer time scales that may be important to the predictability of the climate system. This study investigates the spread in BJC across fully coupled simulations of phase 6 of the Coupled Model Intercomparison Project (CMIP6) and critical processes (particularly related to sea ice and clouds) that may contribute to that spread. BJC on decadal to longer time scales is confirmed across all the simulations evaluated, and it is strongest in the Northern Hemisphere (NH) between 60° and 70°N. At these latitudes, BJC appears to be primarily driven by the exchange of turbulent fluxes (sensible and latent) in the Greenland, Iceland, and Barents Seas. Metrics to break down how sea ice and clouds uniquely modify the radiative balance of the polar atmosphere during anomalous OHT events are presented. These metrics quantify the impacts of sea ice and clouds on surface and top of atmosphere (latent, sensible, longwave, and shortwave radiative) energy fluxes. Cloud responses tend to counter the clear sky impacts over the Marginal Ice Zone (MIZ). It is further shown that the degree of BJC present in a simulation at high latitudes is heavily influenced by the sensitivity of the sea ice to OHT, which is most influential over the MIZ. These results are qualitatively robust across models and explain the intermodel spread in NH BJC in the preindustrial control experiment.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Near-Cloud Atmospheric Ingredients for Deep Convection Initiation

A lack of routine environmental observations located near deepening cumulus congestus clouds limits verification of important theorized and simulated updraft–environment interaction processes occurring during deep convection initiation (CI). We analyze radiosonde profiles collected during several hundred CI events near a mountain range in central Argentina during the CACTI field campaign. Statistical analyses illustrate environmental conditions supporting radar-observed CI outcomes that span a spectrum of convective cell depths, widths, and durations, as well as events lacking precipitating convection. Tested environmental factors include a large variety of sounding-derived measurements of CAPE, CIN, moisture, terrain-relative winds, vertical shear, and lifted parcel properties, with supplemental model reanalysis of background larger-scale vertical motion. CAPE and CIN metrics do not consistently differentiate CI success from failure. Only a few environmental factors contain consistent monotonic relationships among the spectrum of cloud depths achieved during CI: (i) the depth and strength of background ascent, and (ii) the component of low-level flow oriented parallel to the ridgeline. These metrics suggest that the ability of the surrounding flow to lift parcels to their LFC and terrain-modified flow are consistently relevant processes for CI. Low- to midlevel relative humidity strongly discriminated between CI and non-CI events, likely reflecting entrainment-driven dilution processes. However, we could not confidently conclude that relative humidity similarly discriminated robust from marginal CI events. Circumstantial evidence was found linking cell width, an important cloud property governing the probability of CI, to LCL height, boundary layer depth, depth and magnitude of the CIN layer, and ambient wind shear.

Storm environments↗

DOE OSTI2022

Time-Constrained Capacitated Vehicle Routing Problem in Urban E-Commerce Delivery

Electric vehicle routing problems can be particularly complex when recharging must be performed mid-route. In some applications, such as e-commerce parcel delivery truck routing, however, mid-route recharging may not be necessary because of constraints on vehicle capacities and the maximum allowed time for delivery. In this study, we develop a mixed-integer optimization model that exactly solves such a time-constrained capacitated vehicle routing problem, especially of interest for e-commerce parcel delivery vehicles. We compare our solution method with an existing metaheuristic and carry out exhaustive case studies considering four U.S. cities—Austin, TX; Bloomington, IL; Chicago, IL; and Detroit, MI—and two vehicle types: conventional vehicles and battery electric vehicles (BEVs). In these studies we examine the impact of vehicle capacity, maximum allowed travel time, service time (dwelling time to physically deliver the parcel), and BEV range on system-level performance metrics, including vehicle miles traveled (VMT). We find that the service time followed by the vehicle capacity plays a key role in the performance of our approach. We assume an 80-mi BEV range as a baseline without mid-route recharging. Our results show that the BEV range has a minimal impact on performance metrics because the VMT per vehicle averages around 72 mi. In a case study for shared-economy parcel deliveries, we observe that VMT could be reduced by 38.8% in Austin if service providers were to operate their distribution centers jointly.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2021

A better future

We report in some ways, lighting is dramatically different than it was at the turn of the century: LEDs are now ubiquitous and there is growing awareness of the important role of lighting beyond facilitating visual tasks. In other ways, lighting science and practice have not changed in decades. Many core lighting metrics with known limitations—such as V λ (97 years old), the 1931 CIE 2° standard colorimetric observer (90 years old), and CRI (47 years old)—remain in use despite the availability of alternatives based on more recent research. There are also the practices of specifying the quantity of illumination based on average horizontal illuminance, managing contrast with maximum to minimum or similar ratios, and addressing energy use with lighting power density. Never mind the traditional lamp and luminaire form factors that still predominate despite the very different emission characteristics of LEDs. As LEDs mature in the next decades, outdated metrics and practices could become an even greater impediment to optimizing the value generated per Watt—for performance, productivity, wellbeing, or satisfaction.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2023

HLA-Clus: HLA class I clustering based on 3D structure

In a previous paper, we classified populated HLA class I alleles into supertypes and subtypes based on the similarity of 3D landscape of peptide binding grooves, using newly defined structure distance metric and hierarchical clustering approach. Compared to other approaches, our method achieves higher correlation with peptide binding specificity, intra-cluster similarity (cohesion), and robustness. Here we introduce HLA-Clus, a Python package for clustering HLA Class I alleles using the method we developed recently and describe additional features including a new nearest neighbor clustering method that facilitates clustering based on user-defined criteria. The HLA-Clus pipeline includes three stages: First, HLA Class I structural models are coarse grained and transformed into clouds of labeled points. Second, similarities between alleles are determined using a newly defined structure distance metric that accounts for spatial and physicochemical similarities. Finally, alleles are clustered via hierarchical or nearest-neighbor approaches. We also interfaced HLA-Clus with the peptide:HLA affinity predictor MHCnuggets. By using the nearest neighbor clustering method to select optimal allele-specific deep learning models in MHCnuggets, the average accuracy of peptide binding prediction of rare alleles was improved. The HLA-Clus package offers a solution for characterizing the peptide binding specificities of a large number of HLA alleles. This method can be applied in HLA functional studies, such as the development of peptide affinity predictors, disease association studies, and HLA matching for grafting. HLA-Clus is freely available at our GitHub repository (https://github.com/yshen25/HLA-Clus).

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2020

Generation and evaluation of synthetic patient data

Background: Machine learning (ML) has made a significant impact in medicine and cancer research; however, its impact in these areas has been undeniably slower and more limited than in other application domains. A major reason for this has been the lack of availability of patient data to the broader ML research community, in large part due to patient privacy protection concerns. High-quality, realistic, synthetic datasets can be leveraged to accelerate methodological developments in medicine. By and large, medical data is high dimensional and often categorical. These characteristics pose multiple modeling challenges. Methods: In this paper, we evaluate three classes of synthetic data generation approaches; probabilistic models, classification-based imputation models, and generative adversarial neural networks. Metrics for evaluating the quality of the generated synthetic datasets are presented and discussed. Results: While the results and discussions are broadly applicable to medical data, for demonstration purposes we generate synthetic datasets for cancer based on the publicly available cancer registry data from the Surveillance Epidemiology and End Results (SEER) program. Specifically, our cohort consists of breast, respiratory, and non-solid cancer cases diagnosed between 2010 and 2015, which includes over 360,000 individual cases. Conclusions: We discuss the trade-offs of the different methods and metrics, providing guidance on considerations for the generation and usage of medical synthetic data.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2020

Performance and community structure dynamics of microbial electrolysis cells operated on multiple complex feedstocks

Microbial electrolysis is a promising technology for converting aqueous wastes into hydrogen. However, substrate adaptability is an important feature, seldom documented in microbial electrolysis cells (MECs). In addition, the correlation between substrate composition and community structure has not been well established. This study used an MEC capable of producing over 10 L/L-day of hydrogen from a switchgrass-derived bio-oil aqueous phase and investigated four additional substrates, tested in sequence on a mature biofilm. The additional substrates included a red oak-derived bio-oil aqueous phase, a corn stover fermentation product, a mixture of phenol and acetate, and acetate alone. The MECs fed with the corn stover fermentation product resulted in the highest performance among the complex feedstocks, producing an average current density of 7.3 ± 0.51 A/m 2 , although the acetate fed MECs outperformed complex substrates, producing 12.3 ± 0.01 A/m 2 . 16S rRNA gene sequencing showed that community structure and community diversity were not predictive of performance, and replicate community structures diverged despite identical inoculum and enrichment procedure. The trends in each replicate, however, were indicative of the influence of the substrates. Geobacter was the most dominant genus across most of the samples tested, but its abundance did not correlate strongly to current density. High-performance liquid chromatography (HPLC) showed that acetic acid accumulated during open circuit conditions when MECs were fed with complex feedstocks and was quickly degraded once closed circuit conditions were applied. The largest net acetic acid removal rate occurred when MECs were fed with red oak bio-oil aqueous phase, consuming 2.93 ± 0.00 g/L-day. Principal component analysis found that MEC performance metrics such as current density, hydrogen productivity, and chemical oxygen demand removal were closely correlated. Net acetic acid removal was also found to correlate with performance. However, no bacterial genus appeared to correlated to these performance metrics strongly, and the analysis suggested that less than 70% of the variance was accounted for by the two components. This study demonstrates the robustness of microbial communities to adapt to a range of feedstocks and conditions without relying on specific species, delivering high hydrogen productivities despite differences in community structure. The results indicate that functional adaptation may play a larger role in performance than community composition. Further investigation of the roles each microbe plays in these communities will help MECs to become integral in the 21st-century bioeconomy to produce zero-emission fuels.

09 BIOMASS FUELS↗