Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Error Metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Uncertainty in Thermal Modeling of Spent Nuclear Fuel Casks

Uncertainty is a key metric in computational modeling that must be evaluated for results to have wide ranging applicability. A well characterized uncertainty range is ideal with clear error bars on results that can be presented to stakeholders. In the field of spent fuel cask modeling, this ideal has been historically difficult to achieve in practice because of the computationally intensive nature of the models used and the difficulty assigning reasonable uncertainties to quantities in as-built systems. The work in this report has been conducted to evaluate the overall state of uncertainty and sensitivity in spent fuel cask models and develop methodologies for evaluating these uncertainties. These methodologies must be practical for engineering applications. They should not require excessive computational resources or calendar time to achieve results. In engineering, the model must be on a scale such that it can be changed and adapted throughout a project as new information is discovered and project goals evolve. This report covers three major modeling task areas that provide an overview of the types of sensitivity and uncertainty present in a spent fuel storage and transportation system. Section 3 discusses sensitivity and uncertainty analysis in the effective thermal conductivity model for the fuel region and applies these results to a single assembly model. Section 4 shows sensitivity analysis of a full cask model in the TN-32B and Section 5 demonstrates the overall uncertainty workflow using Coolant Boiling in Rod Arrays – Spent Fuel Storage and STAR-CCM+ developed from the sensitivity work in the preceding sections.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

Validation of the IRI-2016 model with Indian NavIC data for future navigation applications

The position accuracy of Navigation with Indian Constellation (NavIC) system is affected by several sources of errors. Among them, the Ionospheric Time Delay (ITD) error is the most predominant one which depends upon the total electron content (TEC) present in the ionosphere. The ITD variations are more intense over low latitude regions due to the equatorial anomaly effects. Hence, modelling of ITD error is necessary. The International Reference Ionosphere (IRI)-2016 model is one of the standard global ionospheric models to estimate the Vertical TEC (VTEC). This paper discusses about the VTEC deviations due to the IRI-2016 model over low latitude Hyderabad station, Indian region using NavIC, Global Positioning System and Global Navigation Satellite System signals at corresponding Ionospheric Pierce Point latitude and longitudes for all months and various Kp indices during the low solar activity year 2017. In this work, cross correlation coefficient, the metric norm (L2N), Symmetric Kullbacke Leibler Distance metrics are used to evaluate the performance of IRI-2016 model TEC with NavIC, GPS and GLONASS data. From the results, it is found that TEC predicted by the IRI-2016 model produced smaller estimation errors with NavIC data over Indian region. The obtained results will be helpful for future updates of IRI model.

42 ENGINEERING↗
DOE OSTI2020

Replicating Machine Learning Experiments in Materials Science

Transparency and reproducibility are important aspects of validation for Machine Learning (ML) models that are not fully understood and applies independently of the application domain.We offer a case study of reproducibility that highlights the challenges encountered when attempting to reproduce analyzes obtained with Machine Learning methods in materials informatics. Our study explores prediction results obtained with ML models and issues in training data serving as input. We discuss challenges related to theory-driven and numerical errors in training data, lack of reproducibility across platforms and versions, and effects of randomness when varying hyperparameters. In addition to model accuracy, a main metric of interest in the ML community, our results show that model sensitivity may be equally important for applying ML in domain applications such a materials science.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Improved Prediction of Cold-Air Pools in the Weather Research and Forecasting Model Using a Truly Horizontal Diffusion Scheme for Potential Temperature

The terrain-following vertical coordinate system used by many atmospheric models, including the Weather Research and Forecasting (WRF) Model, is prone to errors in regions of complex terrain. These errors stem, in part, from the calculation of horizontal gradients within the diffusion term of the momentum or scalar evolution equations. In WRF, such gradients can be calculated along coordinate surfaces, or using metric terms that help account for grid skewness. However, neither of these options ensures a truly horizontal gradient calculation, especially if a grid cell is skewed enough that the heights of the neighboring grid points used in the calculation fall outside the vertical range of the cell. In this work, an improved scheme that uses Taylor series approximations to vertically interpolate variables to the level necessary for a truly horizontal gradient calculation is implemented in WRF for the diffusion of potential temperature. The scheme is validated using an atmosphere-at-rest configuration, in which spurious flows develop only as a result of numerical errors and can thus be used as a proxy for model performance. Following validation, the method is applied to the simulation of cold-air pools (CAPs), which occur in regions of complex terrain and are characterized by strong near-surface temperature gradients. Using the truly horizontal scheme, idealized simulations demonstrate reduced numerical mixing in a quiescent CAP, and a realistic case study in the Columbia River basin shows a reduction in positive wind speed bias by up to roughly 20% compared to observations from the Second Wind Forecast Improvement Project.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Quantitative assessment of eddy viscosity rans models for turbulent mixed convection in a differentially heated plane channel

Turbulent mixed convection between two vertical, infinite parallel plates at different temperatures is studied using various two-equation turbulence models. The numerical simulations are performed at a turbulent Reynolds number of Re τ = 150 and a Grashof number of Gr = 9.6 × 10 5 . Comparisons are made against the highly trusted DNS results. Consistent with the DNS approach, the current simulations are performed using constant properties and the Boussinesq approximation to predict the influence of buoyancy. Previous studies have provided assessments of two-equation turbulence models for various scenarios, but often rely on a qualitative “eye” test in order to determine the most appropriate model to predict a given flow. This study aims to provide a new form of quantitative assessment that accounts for both the physics captured by the turbulence model as well as the magnitude of the system response quantities (SRQ) using a modified symmetric mean absolute percent error (SMAPE) method. This method is designed to be approachable to researchers at any level and can be applied to system response quantities from multiple research fields. Uncertainty quantification is also performed to determine the discretization error for each turbulence model. Recommendations are made as to which turbulence models best capture the physics – hydrodynamically and thermally – using both local and global validation metrics. Lastly, a sensitivity analysis is performed on the damping functions used in the most accurate models. This underpins the potential of model developments and adjustments most worth pursuing for buoyant flows. Finally, this framework provides a more physics-based comparative analysis of the selected turbulence models.

42 ENGINEERING↗
DOE OSTI2025

CuXASNet: Rapid and accurate prediction of copper L-edge x-ray absorption spectra using machine learning

In this work, we have developed CuXASNet, a dense neural network that predicts simulated Cu -edge x-ray absorption spectra (XAS) from atomic structures. Featurization of the Cu local environment is performed using a component of M3GNet, a graph neural network developed for predicting the potential energy surface. CuXASNet is trained on simulated spectra from FEFF9 at the multiple scattering level of theory, and can predict the and edges for Cu sites to quantitative accuracy. To validate our approach, we compare 14 experimental spectra extracted from the literature with the predictions of CuXASNet. The agreement of CuXASNet with experiments is shown by an average mean absolute error of 0.125 and an average Spearman's correlation coefficient of 0.891, which is comparable to FEFF9's values of 0.131 and 0.898 for the same metrics. As such, CuXASNet can rapidly predict a large number of -edge XAS spectra at the same accuracy as FEFF9 simulations. This can be used as a drop-in replacement for multiple scattering codes for fast screening of candidate atomic structure models of a measured system. This model establishes a general framework for Cu XAS prediction, and can be extended to more computationally expensive levels of theory and to other transition metal edges.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Understanding the Biases in Global Monsoon Simulations from the Perspective of Atmospheric Energy Transport

Understanding global monsoon (GM) variability and projecting its future changes rely heavily on climate models. However, climate models generally show pronounced biases in GM simulations, and the reasons for this remain unclear. Here, in this study, we evaluate the performance of 20 pairs of climate models that participated in both phase 5 of the Coupled Model Intercomparison Project (CMIP5) and phase 6 of CMIP (CMIP6) and identify the sources of their GM simulation biases from an energy transport perspective. The multimodel mean improvement in CMIP6 compared to CMIP5 is demonstrated by the increasing skill scores for various GM metrics from 0.20–0.79 to 0.48–0.83. More specifically, the dry biases in the Northern Hemisphere Summer Monsoon (NHSM) precipitation in CMIP5 [root-mean-square error (RMSE): 1.85 mm day −1 ] are reduced in CMIP6 (RMSE: 1.66 mm day −1 ). This higher simulation skill is associated with higher skill in simulating the precipitation-solstitial mode, monsoon intensity, and monsoon domains. The improvement in the NHSM precipitation simulation results from that in the meridional transport of atmospheric energy. Atmospheric energy budget analysis shows that the negative biases in downward surface longwave radiation and northward energy transport are smaller in CMIP6 than in CMIP5 in the boreal summer, resulting in a more realistic interhemispheric thermal contrast and meridional gradient of moist static energy. However, a major weakness of the CMIP6 models is found in the Southern Hemisphere Summer Monsoon precipitation simulation due to the positive bias in the top-of-the-atmosphere downward longwave radiation. This study shows that reasonably reproducing the meridional global atmospheric energy transportation is necessary for skillful GM simulation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

OptZConfig: Efficient Parallel Optimization of Lossy Compression Configuration

Lossless compressors have very low compression ratios that do not meet the needs of today's large-scale scientific applications that produce vast volumes of data. Error-bounded lossy compression (EBLC) is considered a critical technique for the success of scientific research. Although EBLC allows users to set an error bound for the compression, users have been unable to specify the requirements on the compression quality, limiting practical use. Our contributions are: (1) We formulate the problem of configuring EBLC to preserve a user-defined metric as an optimization problem. This allows many classes of new metrics to be preserved, which improves over current practices. (2) We present a framework, OptZConfig, that can adapt to improvements in the search algorithm, compressor, and metrics with minimal changes, enabling future advancements in this area. (3) We demonstrate the advantages of our approach against the leading methods to configure compressors to preserve specific metrics. Here, our approach improves compression ratios against a specialized compressor by up to 3 x, has a 56x speedup over FRaZ, 1000x speedup over MGARD-QOI post tuning, and 110x speedup over systematic approaches which had not been bounded by compressors before.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

International collaboration framework for the calculation of performance loss rates: Data quality, benchmarks, and trends (towards a uniform methodology)

Abstract The IEA PVPS Task 13 group, experts who focus on photovoltaic performance, operation, and reliability from several leading R&D centers, universities, and industrial companies, is developing a framework for the calculation of performance loss rates of a large number of commercial and research photovoltaic (PV) power plants and their related weather data coming across various climatic zones. The general steps to calculate the performance loss rate are (i) input data cleaning and grading; (ii) data filtering; (iii) performance metric selection, corrections, and aggregation; and finally, (iv) application of a statistical modeling method to determine the performance loss rate value. In this study, several high‐quality power and irradiance datasets have been shared, and the participants of the study were asked to calculate the performance loss rate of each individual system using their preferred methodologies. The data are used for benchmarking activities and to define capabilities and uncertainties of all the various methods. The combination of data filtering, metrics (performance ratio or power based), and statistical modeling methods are benchmarked in terms of (i) their deviation from the average value and (ii) their uncertainty, standard error, and confidence intervals. It was observed that careful data filtering is an essential foundation for reliable performance loss rate calculations. Furthermore, the selection of the calculation steps filter/metric/statistical method is highly dependent on one another, and the steps should not be assessed individually.

14 SOLAR ENERGY↗
DOE OSTI2024

Core Design Optimization of the Westinghouse Lead Fast Reactor

Westinghouse is pursuing an advanced Nuclear Power Plant design based on Lead Fast Reactor (LFR) technology for global commercialization. To achieve an optimal combination of key attributes, such as safety, sustainability, and economic competitiveness, Westinghouse and ANL partnered in developing and applying a formalized core design optimization strategy. An LFR analysis workflow was developed to automate a suite of reactor physics, fuels performance, safety, and economics simulations on a selected LFR concept. The workflow streamlines analysis of a wide range of LFR designs with different dimensions and fuel types to assess their viability and economic performance, significantly reducing human processing time and risks of processing errors. The LFR optimization exercise was defined, resulting in selection of the design constraints (geometric, neutronics, thermo-mechanical, safety, thermal-hydraulics, and economics) and performance metrics researched (minimization of both the fuels LCOE and the first core inventory cost). A total of 14 varied design parameters were considered, including assembly dimensions, coolant temperature, and enrichment distribution throughout the core. The LFR analysis workflow was connected to DAKOTA for sensitivity and optimization analyses. Due to the extremely large size of the potential LFR optimization solution space relative to the computing time required to characterize one LFR solution, a multi-stage optimization approach was proposed to breakdown the problem into several stages with more reasonable sizes. This optimization approach enabled finding various viable core solutions with different cost tradeoffs that were considered by Westinghouse and justify selection of a smaller core with multi-batch 2-year cycle length.

Stauff, Nicolas E.↗
DOE OSTI2020

Single Bimodular Sensor for Differentiated Detection of Multiple Oxidative Gases

Semiconductive metal-oxide sensors suffer from cross-sensitivities under mixed chemical condition, specifically upon mixture of multiple oxidative or reductive gases. Herein, a single bimodular sensor is demonstrated for smart differentiation of multiple oxidative analytes by relating the resistance-metric mode to impedance-metric mode. The sensor construct based on ZnO nanorods readily outputs three response datasets upon exposure of oxidative-gas mixture including O 2 , SO 2 , and NO 2 , the resistance, real part impedance, and imaginary part impedance. The differentiative and correlated nature between these response signals allows such a single sensor platform to differentiate these oxidative gases accurately and robustly. Linear and non-linear decision boundaries are established over a large gas-concentration range from 2 ppm to 3% through a combination of principal component analysis and artificial neural network training. A facile user interface is demonstrated for recognition and measurement of unknown gas analytes, with the error of the predicted analyte-concentration as low as 2%.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Geometric Interpretation of the Cluster Location Problem Part I: Theory

We present a new framing of the seismic location problem using principles drawn from differential geometry. Our interpretation relies upon the common assumption that travel times observed across a network are continuous, differentiable functions of source location. In consequence, travel‐time functions constitute a differentiable map between the source region and a Riemannian manifold. The manifold is said to be the image of the source region embedded in a generally high‐dimension travel‐time vector space. A cluster of events in the source region has an image of discrete points on the manifold, that, except in the simplest cases, cannot be viewed directly. However, it is possible to project the image of a cluster into a tangent space of the manifold for direct visualization. The projection operator can be computed directly from the data without a velocity model, but produces a distorted rendering of the cluster geometry. With a model we can predict the distortions and correct them to estimate cluster geometry. We develop these points with the simplest possible example, one for which direct visualization of the manifold is possible, using the example as an introduction to the relevant concepts from differential geometry in a familiar setting. The tangent space, a local linearization of the manifold, plays a key role. We develop a metric to estimate the limits of linearization, that is, to determine when the curvature of the manifold invalidates the linear assumption. We also examine the interplay of model error, inadequate network geometry, and pick error. We then generalize our results from the simple case to the general case of 3D source regions observed by general networks. Although we do suggest a new “project and correct” method for location, we do not develop it into a practical algorithm. In conclusion, our intention rather is to highlight new analytical methods grounded in differential geometry.

East Pacific Ocean Islands↗
DOE OSTI2020

Relation Inference among Sensor Time Series in Smart Buildings with Metric Learning

Smart Building Technologies hold promise for better livability for residents and lower energy footprints. Yet, the rollout of these technologies, from demand response controls to fault detection and diagnosis, significantly lags behind and is impeded by the current practice of manual identification of sensing point relationships, e.g., how equipment is connected or which sensors are co-located in the same space. This manual process is still error-prone, albeit costly and laborious.We study relation inference among sensor time series. Our key insight is that, as equipment is connected or sensors co-locate in the same physical environment, they are affected by the same real-world events, e.g., a fan turning on or a person entering the room, thus exhibiting correlated changes in their time series data. To this end, we develop a deep metric learning solution that first converts the primitive sensor time series to the frequency domain, and then optimizes a representation of sensors that encodes their relations. Built upon the learned representation, our solution pinpoints the relationships among sensors via solving a combinatorial optimization problem. Extensive experiments on real-world buildings demonstrate the effectiveness of our solution.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2026

Hourly PM 2.5 Estimates across California from 2018 to 2023

This study presents a new data set of hourly PM 2.5 concentrations across California from 2018 to 2023 at a three-kilometer resolution. This data set was developed by assimilating observations from PurpleAir and the U.S. EPA Air Quality System monitors into wildfire smoke forecasts from the High-Resolution Rapid Refresh Smoke (HRRR-Smoke) model using the Gridpoint Statistical Interpolation (GSI) three-dimensional variational data assimilation framework. Archived forecasts of modeled wildfire smoke PM 2.5 from HRRR-Smoke create the background field for assimilation, which is then corrected using surface observations of total PM 2.5 . The resulting reanalysis from GSI provides an estimate of total PM 2.5 that minimizes error from both the observational and the model data. Validation results indicate strong performance, with monthly R 2 values ranging from 0.73 to 0.91 across the six-year data set, comparable to other PM 2.5 data sets. Case studies are presented for three major fire events, the 2018 Camp Fire, 2019 Kincade Fire, and 2020 Lightning Complex Fires to demonstrate the data set’s fidelity in resolving plume dynamics and local exposure patterns. Root-mean-squared error averaged over each month scales with average PM 2.5 concentrations, resulting in a low error under typical conditions but higher absolute errors during extreme smoke events. This is the first long-term, hourly PM 2.5 data set of its kind for California and enables the generation of subdaily exposure metrics, such as peak hourly concentrations, exceedance durations, and time-of-day exposure peaks. The novelty and strong validation of this data set make it a compelling resource for future studies on the impact and significance of subdaily PM 2.5 exposure.

PM2.5↗
DOE OSTI2020

Relating CMIP5 Model Biases to Seasonal Forecast Skill in the Tropical Pacific

Abstract We examine links between tropical Pacific mean state biases and El Niño/Southern Oscillation forecast skill, using model‐analog hindcasts of sea surface temperature (SST; 1961–2015) and precipitation (1979–2015) at leads of 0–12 months, generated by 28 different models from the fifth phase of the Coupled Model Intercomparison Project (CMIP5). Model‐analog forecast skill has been demonstrated to match or even exceed traditional assimilation‐initialized forecast skill in a given model. Models with the most realistic mean states and interannual variability for SST, precipitation, and 10‐m zonal winds in the equatorial Pacific also generate the most skillful precipitation forecasts in the central equatorial Pacific and the best SST forecasts at 6‐month or longer leads. These results show direct links between model climatological biases and seasonal forecast errors, demonstrating that model‐analog hindcast skill—that is, how well a model can capture the observed evolution of tropical Pacific anomalies—is an informative El Niño/Southern Oscillation metric for climate simulations.

Ding, Hui↗
DOE OSTI2021

Evaluation of extreme sub-daily precipitation in high-resolution global climate model simulations

We examine the resolution dependence of errors in extreme sub-daily precipitation in available high-resolution climate models. We find that simulated extreme precipitation increases as horizontal resolution increases but that appropriately constructed model skill metrics do not significantly change. We find little evidence that simulated extreme winter or summer storm processes significantly improve with the resolution because the model performance changes identified are consistent with expectations from scale dependence arguments alone. We also discuss the implications of these scale-dependent limitations on the interpretation of simulated extreme precipitation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

HOLISTIC ENERGY EFFICIENCY ANALYSIS OF ELECTRIFIED OFF-HIGHWAY MATERIAL HANDLER: FROM DRIVE CYCLE CHARACTERIZATION TO POWERTRAIN, HYDRAULIC, AND THERMAL SYSTEM PERFORMANCE

Three complexities surrounding the operation and testing of hybrid electric, heavy-duty nonroad machines have been addressed experimentally and using 1D simulation. Their resolutions have been intertwined with the development of a prototype machine that was proven to reduce fuel consumption in excess of 20%. A real-world drive cycle that leveraged hydraulic cylinder position was developed and utilized to ensure accurate reproduction of hydraulic work between the baseline and hybrid machines, while simultaneously maintaining less than 5% RMS error in position for main load handling functions. The newly developed, machine-specific drive cycle also contributed towards making equivalent comparisons in energy consumption between machine types through composite performance metrics that were extrapolated over a typical shift duration. Lastly, this work addressed thermal management energy consumption, a topic of increasing popularity when discussing electrified vehicles, by proposing a 1.4% energy savings through special mechanization and control of cooling system components.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Haar-Like Wavelets on Hierarchical Trees

Here, discrete wavelet methods, originally formulated in the setting of regularly sampled signals, can be adapted to data defined on a point cloud if some multiresolution structure is imposed on the cloud. A wide variety of hierarchical clustering algorithms can be used for this purpose, and the multiresolution structure obtained can be encoded by a hierarchical tree of subsets of the cloud. Prior work introduced the use of Haar-like bases defined with respect to such trees for approximation and learning tasks on unstructured data. This paper builds on that work in two directions. First, we present an algorithm for constructing Haar-like bases on general discrete hierarchical trees. Second, with an eye towards data compression, we present thresholding techniques for data defined on a point cloud with error controlled in the $L$ $\infty$ norm and in a Hölder-type norm. In a concluding trio of numerical examples, we apply our methods to compress a point cloud dataset, study the tightness of the $L$ $\infty$ error bound, and use thresholding to identify MNIST classifiers with good generalizability.

97 MATHEMATICS AND COMPUTING↗