Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Error Metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Surrogate Model of Flexible Research Platform EnergyPlus Models to Enable Sensitivity Analysis

This letter report describes the surrogate models developed from the EnergyPlus model of Oak Ridge National Laboratory’s Flexible Research Platform. Two data-driven black-box models were developed, and the outputs of the surrogate models were compared with the EnergyPlus model. The two models developed are a multilayer perceptron deep learning model, and a long short-term memory (LSTM) neural network model. The three factors for selecting the black-box models are scalability, computation time, and accuracy. A total of 107 input variables were the dominant variables in determining the outputs of building energy consumptions and thermal comfort. A total of 54 output variables were identified as the prediction targets, including the system- and zone-level outputs. The large set of the simulation cases were generated by integrating sensor errors into an emulator based on EnergyPlus and Python EMS, which includes advanced control sequences from ASHRAE Guideline 36-2018: High-Performance Sequences of Operation. The surrogate models were developed based on a set of large-scale simulation runs (i.e., 4,000 runs) on a cloud platform. The comparison analysis shows that the two black-box models had good accuracy for predicting new outputs for sensitivity analysis using the root mean square error metric. As a next step, the developed surrogate models will be used to perform sensitivity analysis for different sensor impacts (e.g., sensor types, sensor locations).

42 ENGINEERING↗

Intelligent Experiments through Real-Time AI: Fast Data Processing and Autonomous Detector Control for High-Energy Nuclear Experiments

The aim of this project is to develop software and hardware for fast real-time data processing and autonomous detector control and calibration for the sPHENIX and the future EIC experiments. Below summarizes Georgia Tech team efforts in the past year: 1. We developed a real-time clustering algorithm and FPGA-based pipeline architecture for processing fired pixel data from ALPIDE sensors in sPHENIX experiments. Our Columnar Clustering Co-Design introduces a hardware-aware, stream-friendly approach that segments pixel data by column pairs using a Column Pair Clustering (CPC) strategy, followed by Cluster Stitching to merge adjacent subclusters. Implemented in Vitis HLS, the pipeline comprises five stages—read-in, subclustering, stitching, analysis, and write-out—connected by tagged HLS streams with custom end-of-event signaling for robust synchronization. We designed a pipelined dataflow model optimized for throughput, low latency, and minimal buffering, enabling scalable clustering across events of arbitrary size. Our system maintains spatial precision via center-of-mass and shape key extraction and efficiently handles edge cases such as fragmented or nested clusters. Compared against DBSCAN in both software and hardware, our approach demonstrates competitive performance under FPGA constraints. 2. We also conducted a comprehensive algorithm-to-hardware co-design of connected component analysis tailored for sPHENIX experiments, focusing on real-time, low-latency processing using FPGAs and High-Level Synthesis (HLS). Starting from a Python-based particle tracking pipeline, the team translated the core logic—graph traversal via DFS and Union-Find—into an HLS-compatible C++ model, replacing dynamic memory and recursion with static arrays and pipelined control flow. The final design includes a fully streamed and dataflow-compatible Union-Find kernel optimized across five iterations, incorporating loop pipelining, array partitioning, AXI/FIFO interface tuning, and function flattening. Experimental results show up to 14.8× speedup over the CPU baseline, reducing per-graph latency to 1.58 μs and demonstrating strong resource efficiency with only ~7k LUTs and zero BRAM usage. The design maintains functional correctness against the Python reference using a Python-based C-simulation framework and Mean Squared Error metrics. This work validates the potential of HLS-driven FPGA designs for edge-level HEP data acquisition, laying a scalable foundation for future integration with real-time detector pipelines and multi-graph processing systems.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

A Fusion of Geothermal and InSAR Data with Machine Learning for Enhanced Deformation Forecasting at the Geysers

The Geysers geothermal field in California is experiencing land subsidence due to the seismic and geothermal activities taking place. This poses a risk not only to the underlying infrastructure but also to the groundwater level which would reduce the water availability for the local community. Because of this, it is crucial to monitor and assess the surface deformation occurring and adjust geothermal operations accordingly. In this study, we examine the correlation between the geothermal injection and production rates as well as the seismic activity in the area, and we show the high correlation between the injection rate and the number of earthquakes. This motivates the use of this data in a machine learning model that would predict future deformation maps. First, we build a model that uses interferometric synthetic aperture radar (InSAR) images that have been processed and turned into a deformation time series using LiCSBAS, an open-source InSAR time series package, and evaluate the performance against a linear baseline model. The model includes both convolutional neural network (CNN) layers as well as long short-term memory (LSTM) layers and is able to improve upon the baseline model based on a mean squared error metric. Then, after getting preprocessed, we incorporate the geothermal data by adding them as additional inputs to the model. This new model was able to outperform both the baseline and the previous version of the model that uses only InSAR data, motivating the use of machine learning models as well as geothermal data in assessing and predicting future deformation at The Geysers as part of hazard mitigation models which would then be used as fundamental tools for informed decision making when it comes to adjusting geothermal operations.

Yazbeck, Joe (ORCID:0000000302235260)↗

A Comparison of Infectious Disease Forecasting Methods across Locations, Diseases, and Time

Accurate infectious disease forecasting can inform efforts to prevent outbreaks and mitigate adverse impacts. This study compares the performance of statistical, machine learning (ML), and deep learning (DL) approaches in forecasting infectious disease incidences across different countries and time intervals. We forecasted three diverse diseases: campylobacteriosis, typhoid, and Q-fever, using a wide variety of features (n = 46) from public datasets, e.g., landscape, climate, and socioeconomic factors. We compared autoregressive statistical models to two tree-based ML models (extreme gradient boosted trees [XGB] and random forest [RF]) and two DL models (multi-layer perceptron and encoder–decoder model). The disease models were trained on data from seven different countries at the region-level between 2009–2017. Forecasting performance of all models was assessed using mean absolute error, root mean square error, and Poisson deviance across Australia, Israel, and the United States for the months of January through August of 2018. The overall model results were compared across diseases as well as various data splits, including country, regions with highest and lowest cases, and the forecasted months out (i.e., nowcasting, short-term, and long-term forecasting). Overall, the XGB models performed the best for all diseases and, in general, tree-based ML models performed the best when looking at data splits. There were a few instances where the statistical or DL models had minutely smaller error metrics for specific subsets of typhoid, which is a disease with very low case counts. Feature importance per disease was measured by using four tree-based ML models (i.e., XGB and RF with and without region name as a feature). The most important feature groups included previous case counts, region name, population counts and density, mortality causes of neonatal to under 5 years of age, sanitation factors, and elevation. This study demonstrates the power of ML approaches to incorporate a wide range of factors to forecast various diseases, regardless of location, more accurately than traditional statistical approaches.

59 BASIC BIOLOGICAL SCIENCES↗

Predicting Short-Term Deformation in the Central Valley Using Machine Learning

Land subsidence caused by excessive groundwater pumping in Central Valley, California, is a major issue that has several negative impacts such as reduced aquifer storage and damaged infrastructures which, in turn, produce an economic loss due to the high reliance on crop production. This is why it is of utmost importance to routinely monitor and assess the surface deformation occurring. Two main goals that this paper attempts to accomplish are deformation characterization and deformation prediction. The first goal is realized through the use of Principal Component Analysis (PCA) applied to a series of Interferomtric Synthetic Aperture Radar (InSAR) images that produces eigenimages displaying the key characteristics of the subsidence. Water storage changes are also directly analyzed by the use of data from the Gravity Recovery and Climate Experiment (GRACE) twin satellites and the Global Land Data Assimilation System (GLDAS). The second goal is accomplished by building a Long Short-Term Memory (LSTM) model to predict short-term deformation after developing an InSAR time series using LiCSBAS, an open-source InSAR time series package. The model is applied to the city of Madera and produces better results than a baseline averaging model and a one dimensional convolutional neural network (CNN) based on a mean squared error metric showing the effectiveness of machine learning in deformation prediction as well as the potential for incorporation in hazard mitigation models. The model results can directly aid policy makers in determining the appropriate rate of groundwater withdrawal while maintaining the safety and well-being of the population as well as the aquifers’ integrity.

58 GEOSCIENCES↗

Motion tomography via occupation kernels

The goal of motion tomography is to recover a description of a vector flow field using measurements along the trajectory of a sensing unit. In this paper, we develop a predictor corrector algorithm designed to recover vector flow fields from trajectory data with the use of occupation kernels developed by Rosenfeld et al. [9,10]. Specifically, we use the occupation kernels as an adaptive basis; that is, the trajectories defining our occupation kernels are iteratively updated to improve the estimation in the next stage. Initial estimates are established, then under mild assumptions, such as relatively straight trajectories, convergence is proven using the Contraction Mapping Theorem. We then compare the developed method with the established method by Chang et al. [5] by defining a set of error metrics. Here, we found that for simulated data, where a ground truth is available, our method offers a marked improvement over [5]. For a real-world example, where ground truth is not available, our results are similar results to the established method.

97 MATHEMATICS AND COMPUTING↗

Using a surrogate-assisted Bayesian framework to calibrate the runoff-generation scheme in the Energy Exascale Earth System Model (E3SM) v1

Abstract. Runoff is a critical component of the terrestrial water cycle, and Earth system models (ESMs) are essential tools to study its spatiotemporal variability. Runoff schemes in ESMs typically include many parameters so that model calibration is necessary to improve the accuracy of simulated runoff. However, runoff calibration at a global scale is challenging because of the high computational cost and the lack of reliable observational datasets. In this study, we calibrated 11 runoff relevant parameters in the Energy Exascale Earth System Model (E3SM) Land Model (ELM) using a surrogate-assisted Bayesian framework. First, the polynomial chaos expansion machinery with Bayesian compressed sensing is used to construct computationally inexpensive surrogate models for ELM-simulated runoff at 0.5∘ × 0.5∘ for 1991–2010. The error metric between the ELM simulations and the benchmark data is selected to construct the surrogates, which facilitates efficient calibration and avoids the more conventional, but challenging, construction of high-dimensional surrogates for the ELM simulated runoff. Second, the Sobol' index sensitivity analysis is performed using the surrogate models to identify the most sensitive parameters, and our results show that, in most regions, ELM-simulated runoff is strongly sensitive to 3 of the 11 uncertain parameters. Third, a Bayesian method is used to infer the optimal values of the most sensitive parameters using an observation-based global runoff dataset as the benchmark. Our results show that model performance is significantly improved with the inferred parameter values. Although the parametric uncertainty of simulated runoff is reduced after the parameter inference, it remains comparable to the multimodel ensemble uncertainty represented by the global hydrological models in ISMIP2a. Additionally, the annual global runoff trend during the simulation period is not well constrained by the inferred parameter values, suggesting the importance of including parametric uncertainty in future runoff projections.

58 GEOSCIENCES↗

Efficient flexible characterization of quantum processors with nested error models

We present a simple and powerful technique for finding a good error model for a quantum processor. The technique iteratively tests a nested sequence of models against data obtained from the processor, and keeps track of the best-fit model and its wildcard error (a metric of the amount of unmodeled error) at each step. Each best-fit model, along with a quantification of its unmodeled error, constitutes a characterization of the processor. We explain how quantum processor models can be compared with experimental data and to each other. We demonstrate the technique by using it to characterize a simulated noisy two-qubit processor.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Non-Parametric Statistical Analysis of Current Waveforms through Power System Sensors

The protection, control, and monitoring of the power grid is not possible without accurate measurement devices. As the percentage of renewable energy sources penetrating the existing grid infrastructure increases, so do uncertainties surrounding their effects on the everyday operation of the power system. Many of these devices are sources of high-frequency transients. These transients may be useful for identifying certain events or behaviors otherwise not seen in traditional analysis techniques. Therefore, the ability of sensors to accurately capture these phenomena is paramount. In this work, two commercial-grade power system distribution sensors are investigated in terms of their ability to replicate high-frequency phenomena by studying their responses to three events: a current inrush, a microgrid “close-in”, and a fault on the terminals of a wind turbine. Kernel density estimation is used to derive the non-parametric probability density functions of these error distributions and their adequateness is quantified utilizing the commonly used root mean square error (RMSE) metric. It is demonstrated that both sensors exhibit characteristics in the high harmonic range that go against the assumption that measurement error is normally distributed.

47 OTHER INSTRUMENTATION↗

Test Points for Online Monitoring of Quantum Circuits

Noisy Intermediate-Scale Quantum (NISQ) computers consisting of tens of inherently noisy quantum bits (qubits) suffer from reliability problems. Qubits and their gates are susceptible to various types of errors. Due to limited numbers of qubits and high error rates, quantum error correction cannot be applied. Physical constraints of quantum hardware including the error rates are used to guide the design and the layout of quantum circuits. The error rates determine the selection of qubits and their operations. The resulting circuit is executed on the quantum computer. This study explores the risk of unexpected changes in the error rates of NISQ computers post-calibration. We show that unexpected changes in error rates can alter the output state of a quantum circuit. To detect these changes, we propose the insertion of test points into the quantum circuit to enable online monitoring of the physical qubit behavior. We utilize classical, superposition, and uncompute test points. Furthermore, we use a gate error coverage metric to assess the quality of the tests. We verify the effectiveness of the proposed scheme on different IBM quantum computers (IBM Q), in addition to a noisy simulation that shows the scalability of the proposed approach.

97 MATHEMATICS AND COMPUTING↗

Comparative Performance of Gaussian Plume and Backward Lagrangian Stochastic Models for Near-Field Methane Emission Estimation Using a Single Controlled Release Experiment

Methane (CH 4 ) is a major component of natural gas and a potent greenhouse gas. Increasing atmospheric methane concentrations are attributed to emissive anthropogenic activities by an average of 13 ppb per yr since 2020 and are linked to a changing global climate. Mitigating CH 4 emissions from oil and gas production sites has recently become a target to reduce overall greenhouse gas emissions; however, monitoring the efficacy of mitigation strategies depends on accurate quantification of CH 4 emissions at the facility-level. Near-field quantification of methane (CH 4 ) emissions from oil and gas (O&G) facilities remains challenging due to the effects of atmospheric variability and sensor configuration on atmospheric dispersion models. This study evaluates the performance of two atmospheric dispersion models, the Gaussian plume (GP) and backward Lagrangian stochastic (bLS), by comparing calculated CH 4 emissions to controlled single-point emissions between 0.4 and 5.2 kg CH 4 h −1 . Emissions were calculated by both models using 121 individual sets of measurements comprising five-minute averaged downwind methane mixing ratios and matching meteorological data. The comparison shows that the bLS approach achieved a higher proportion of emission estimates within a factor of two (FAC2) of the known emission rates compared to the GP approach. The emissions calculated by the bLS model also had a lower multiplicative error and reduced bias relative to GP. Other error-based metrics further confirmed the bLS model performed better, as it yielded lower RMSE and MAE than GP. Statistical analysis of the emission data shows that the lateral and vertical alignment of the source and the sensor plays a critical role in emission estimations, as measurements made closer to the plume centerline and at a distance between 40 and 80 m downwind yielded the best FAC2 agreement. High wind meander degraded the ability of both approaches to generate representative emissions, particularly with the GP approach, as it violates the modeling approach’s assumption of steady-state emissions. Data suggest emissions calculated by the bLS model are comprehensively in better agreement, but the computational demands of the modeling approach and integration into fenceline systems limit real-time applicability. While these results provide insight into model performance under controlled near-field conditions, their applicability to more complex or heterogeneous oil and gas production environments (e.g., the regions Marcellus or Unita Basins) remains limited and uncertain.

gaussian plume↗

Predicting Initial Trans-Membrane Pressure for Optimized Operations in UF Unit Using Random Forest

With the growing scarcity of freshwater, innovative process design mechanisms like Reverse Osmosis (RO) are increasingly gaining attention among water treatment utilities to address the rising demand. Ensuring reliable water production necessitates efficient resource utilization, minimizing downtime in (ultra-filtration) UF systems. Recent advancements in machine learning (ML) have enabled the development of accurate data-driven models for Model Predictive Control (MPC), often requiring minimal prior knowledge of underlying physical processes. In this study, we present predictive regression models based on Random Forest (RF) and Auto-Regressive (AR) approaches to forecast the initial Trans-Membrane Pressure (TMP) for each filtration cycle in data generated by Direct Potable Reuse (DPR) systems. The proposed RF-based model demonstrates superior performance compared to baseline methods, including historical mean, Last Observation Carried Forward (LOCF), and naïve AR models, across various forecasting horizons in terms of root mean square error (RMSE) metric. To evaluate how different classes of process variables contribute to TMP dynamics over time, we examine the feature importance of independent covariates across multiple forecast horizons. This analysis provides insight into the temporal relevance of operational and sensor-derived features, guiding control and monitoring strategies. Additionally, the impact of hyperparameter tuning on TMP prediction performance is studied for both direct and recursive RF modelling approaches across increasing forecast horizons. Accurate prediction of initial TMP is critical for optimizing RO operations, as it enables the development of robust modelling frameworks by accurately estimating membrane fouling trends, thereby enhancing process efficiency and long-term reliability. The demonstrated efficacy of the RF-based approach highlights its potential as a tool for real-time decision-making in water treatment systems, paving the way for advanced process optimization and sustainable water resource management.

Mukherjee, Subrata [ORNL] (ORCID:0000000309930338)↗

Exploring Hilbert space on a budget: Novel benchmark set and performance metric for testing electronic structure methods in the regime of strong correlation

This work explores the ability of classical electronic structure methods to efficiently represent (compress) the information content of full configuration interaction (FCI) wave functions. We introduce a benchmark set of four hydrogen model systems of different dimensionalities and distinctive electronic structures: a 1D chain, a 1D ring, a 2D triangular lattice, and a 3D close-packed pyramid. To assess the ability of a computational method to produce accurate and compact wave functions, we introduce the accuracy volume, a metric that measures the number of variational parameters necessary to achieve a target energy error. Using this metric and the hydrogen models, we examine the performance of three classical deterministic methods: (i) selected configuration interaction (sCI) realized both via an a posteriori (ap-sCI) and variational selection of the most important determinants, (ii) an a posteriori singular value decomposition (SVD) of the FCI tensor (SVD-FCI), and (iii) the matrix product state representation obtained via the density matrix renormalization group (DMRG). We find that the DMRG generally gives the most efficient wave function representation for all systems, particularly in the 1D chain with a localized basis. For the 2D and 3D systems, all methods (except DMRG) perform best with a delocalized basis, and the efficiency of sCI and SVD-FCI is closer to that of DMRG. For larger analogs of the models, the DMRG consistently requires the fewest parameters but still scales exponentially in 2D and 3D systems, and the performance of SVD-FCI is essentially equivalent to that of ap-sCI.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A semblance measure for model comparison

Algorithmic and computational advances have made it possible that geophysical survey and earth model design can be aided by many systematic trial inverse-modelling runs with synthetic data. Such may, for example, come up in machine-learning approaches. Automated image appraisal pertaining to such applications will involve common statistical tests for goodness-of-data fit as a primary evaluation method. However, solution non-uniqueness may render multiple images equivalent in terms of their data fit, requiring secondary categorizers. A logical choice for classifying synthetic-imaging results quantifies the goodness of model fit where a known reference model replaces the observational input. The task of model intercomparison in terms of measuring the resemblance to the reference model poses challenges to common distance-based metrics like root mean square error and mean absolute error. First, distance-based metrics can introduce spurious contributions when smooth models with fuzzy target contours are to be compared against a sharp reference. Second, large differences due to parameter-estimation overshoots can dominate distance metrics. Here, we propose a remedy that is referred to as semblance and is based on the idea of logistic functions, where a binary-dependent variable adds non-zero or zero accumulation terms for the, respectively, passing or failing of preset target thresholds. This classifying approach is amenable to an objective where model feature recognition is primary. Numerical comparisons to distance-based metrics provide evidence for the advantages of the semblance in view of this objective. Geophysical imaging in conjunction with machine-learning is seen as a benefitting upcoming application area.

58 GEOSCIENCES↗

ReMU: regional minimal updating for model-based derivative-free optimization

Derivative-free optimization (DFO) problems are optimization problems where derivative information is unavailable or extremely difficult to obtain. Model-based DFO solvers have been applied extensively in scientific computing. Powell's NEWUOA (2004) [Powell, The NEWUOA software for unconstrained optimization without derivatives, in Large-Scale Nonlinear Optimization, Nonconvex Optimization and its Applications Vol. 83, G. Di Pillo and M. Roma, eds., Springer, 2006, pp. 255–297] and Wild's POUNDerS (2014) [Wild, Solving derivative-free nonlinear least squares problems with POUNDERS, in Advances and Trends in Optimization with Engineering Applications, T. Terlaky, M.F. Anjos, and S. Ahmed, eds., SIAM, 2017, pp. 529–540] explore the numerical power of the minimal norm Hessian (MNH) model for DFO and contributed to the open discussion on building better models with fewer data to achieve faster numerical convergence. Another decade later, we propose the regional minimal updating (ReMU) models, and extend the previous models into a broader class, including the H 2 norm models [Xie and Yuan, Least H 2 norm updating of quadratic interpolation models for derivative-free trust-region algorithms, IMA J. Numer. Anal. 46 (2025), pp. 21–50]. This paper shows motivation behind ReMU models, computational details, theoretical and numerical results on particular extreme points and the barycentre of ReMU's weight coefficient region, and the associated KKT matrix error and distance. Novel metrics, such as the truncated Newton step error, are proposed to numerically understand the new models' properties. A new algorithmic strategy, based on iteratively adjusting the ReMU model type, is also proposed, and shows numerical advantages by combining and switching between the barycentric model and the classic least Frobenius norm model in an online fashion.

derivative-free trust-region methods↗

Increasing the Measured Effective Quantum Volume with Zero Noise Extrapolation

Quantum volume is a full-stack benchmark for near-term quantum computers. It quantifies the largest size of a square circuit which can be executed on the target device with reasonable fidelity. Error mitigation is a set of techniques intended to remove the effects of noise present in the computation of noisy quantum computers when computing an expectation value of interest. Effective quantum volume is a proposed metric that applies error mitigation to the quantum volume protocol to evaluate the effectiveness not only of the target device but also of the error mitigation algorithm. Digital zero-noise extrapolation is an error mitigation technique that estimates the noiseless expectation value using circuit folding to amplify errors by known scale factors and then extrapolating computed expectation values to the zero-noise limit. Here we demonstrate that zero-noise extrapolation, with global and local unitary folding with fractional scale factors, in conjunction with dynamical decoupling, can increase the effective quantum volume over the vendor-measured quantum volume. Specifically, we measure the effective quantum volume of four IBM Quantum superconducting processor units, obtaining values that are larger than the vendor-measured quantum volume on each device. This is the first such increase reported.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Modeling land use and land cover change: using a hindcast to estimate economic parameters in gcamland v2.0

Abstract. Future changes in land use and cover have important implications for agriculture, energy, water use, and climate. Estimates of future land use and land cover differ significantly across economic models as a result of differences in drivers, model structure, and model parameters; however, these models often rely on heuristics to determine model parameters. In this study, we demonstrate a more systematic and empirically based approach to estimating a few key parameters for an economic model of land use and land cover change, gcamland. Specifically, we generate a large set of model parameter perturbations for the selected parameters and run gcamland simulations with these parameter sets over the historical period in the United States to quantify land use and land cover, determine how well the model reproduces observations, and identify parameter combinations that best replicate observations, assuming other model parameters are fixed. We also test alternate methods for forming expectations about uncertain crop yields and prices, including adaptive, perfect, linear, and hybrid approaches. In particular, we estimate parameters for six parameters used in the formation of expectations and three of seven logit exponents for the USA only. We find that an adaptive expectation approach minimizes the error between simulated outputs and observations, with parameters that suggest that for most crops, landowners put a significant weight on previous information. Interestingly, for corn, where ethanol policies have led to a rapid growth in demand, the resulting parameters show that a larger weight is placed on more recent information. We examine the change in model parameters as the metric of model error changes, finding that the measure of model fitness affects the choice of parameter sets. Finally, we discuss how the methodology and results used in this study could be used for other regions or economic models to improve projections of future land use and land cover change.

54 ENVIRONMENTAL SCIENCES↗