Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “error metric”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Understanding and reducing mid-spatial frequency ripples during hemispherical sub-aperture tool glass polishing

We report during sub-aperture tool polishing of glass optics, mid-spatial surface ripples are generated because of material removal non-uniformities during tool linear translation (resulting in feed ripples) and tool pathway step overlaps (resulting in pitch ripples). A variety of tool influence function (TIF) spots, trenches, and patches were created to understand and minimize such ripples on fused silica workpieces after polishing with cerium oxide slurry using a rotating hemispherical pad-foam tool. The feed ripple amplitude can be decreased by reducing the non-uniformities in the pad texture and/or by minimizing a derived feed ripple metric ($r_f = V^{0.5}_{max}V_f/R_t$) via adjustments in processing parameters. Pitch ripples can be minimized by reducing relative step distance to spot radius ratio ($x_s/a_t$) and by achieving a flat bottom trench shape cross section or by reducing the material removal per pass. Using the combined methods, an overall ripple error of ~1.2nm rms has been achieved.

47 OTHER INSTRUMENTATION↗
NASA NTRSDate not supplied

Calculator for Agricultural Water Use Fractions in California

Improved quantification of agricultural water use efficiency is a primary focus of California's Water Conservation Act. The California Department of Water Resources (CDWR), after extensive stakeholder consultation, has published methods and indicators to help identify opportunities for improved efficiency of irrigation management at various spatial scales. Additional 2017 legislation directs water suppliers to quantify water use efficiency using the CDWR metrics to support preparation of agricultural water management plans. The IrriQuest calculator derives the Crop Consumptive Use Fraction (CCUF), a foundational metric relating seasonal crop evapotranspiration (ET) to total applied water. Satellite normalized difference vegetation index (NDVI) data are used to develop fractional canopy cover timeseries. Additional key inputs include daily reference ET, irrigation volume, irrigation method, precipitation, soil texture and crop type. IrriQuest estimates of seasonal crop ET show less than 10% mean absolute error compared to ground-based measurements of several annual and perennial crops monitored by sensor deployment in commercial fields. Two additional indicators build on CCUF. The Agronomic Water Use Fraction adds consideration of water applied for beneficial purposes such as pre-plant soil preparation, soil salinity management, and climate control. Total Water Use Fraction additionally accounts for water applied for environmental purposes such as maintenance of wetland and riparian habitat. A convenient on-line portal is under development for CCUF estimation and a standalone spreadsheet is available for computation of all three use fractions.

Calculator↗
NASA NTRSDate not supplied

Assessing Utility of Different Orbits for a Lunar PNT Constellation

"As the Artemis program establishes a long-term presence on the lunar surface, lunar surface users will need an accurate source of Position, Navigation, and Timing (PNT). Under the LunaNet framework, NASA has proposed the Lunar Navigation System (LNS), a constellation of satellites in lunar orbit which will broadcast LunaNet’s Augmented Forward Signal (AFS) [1] to the lunar surface. LNS will have the objective of providing user position accuracies of 10 m and timing accuracies of 10 microseconds on the lunar surface and up to 200 km above the surface. Several classes of orbits are possibilities for the LNS constellation, including circular orbits, eccentric orbits, and various families of 3-body orbits. Each of these orbit classes have both advantages and disadvantages in their utility as PNT constellations. In this study we examine a wide variety of potential LNS constellations and evaluate their performance against several metrics, including: a. Dilution of Precision (DOP) performance over the lunar service volume and the South Pole region, during both the constellation buildup and after the constellation is completed. b. Number of satellites required to achieve required DOP performance. c. Relative dynamics of LNS satellites with respect to surface users. d. Signal in space error (SISE) performance, assuming orbit determination that includes the Global Positioning System (GPS), crosslinks, and monitor sites on the surface of the moon. e. Constellation insertion, station keeping, and disposal costs. f. Specific Concept of Operations (CONOPS) and engineering challenges associated with establishing and operating the constellations. (E.g., frequent station keeping, variable signal strength, divergent CONOPS for different planes of the constellation.) The system’s performance is considered both independently and in combination with other proposed systems, such as NASA’s Lunar Communication Relay and Navigation System (LCRNS) and ESA’s Moonlight constellation. The study concludes that a 3-plane Walker constellation at 6 lunar radii (slightly higher than GPS’s 4.2 earth radii from a geometry perspective, but significantly closer in an absolute sense) is a good candidate for the LNS constellation, providing the user with good SISE and DOP performance, while not requiring excessive station keeping or complex CONOPS. However, other viable concepts exist and could be chosen if early South Pole coverage is a driving factor in the system design."

navigation↗
DOE OSTI2021

Benchmarking performance changes in the simulation of extratropical modes of variability across CMIP generations

In this work, we evaluate extratropical modes of variability in the three most recent phases of the Coupled Model Intercomparison Project (CMIP3, 5, and 6) to gauge improvement of climate models over time. A suite of high-level metrics is employed to objectively evaluate how well climate models simulate the qobserved Northern Annular Mode (NAM), North Atlantic Oscillation (NAO), Pacific North America pattern (PNA), Southern Annular Mode (SAM), Pacific Decadal Oscillation (PDO), North Pacific Oscillation (NPO), and North Pacific Gyre Oscillation (NPGO). We apply a Common Basis Function (CBF) approach that projects model anomalies onto observed empirical orthogonal functions (EOFs), together with the traditional EOF approach, to CMIP Historical and AMIP models. We find simulated spatial patterns of those modes have been significantly improved in the newer models, although the skill improvement is sensitive to the mode and season considered. We identify some potential contributions to the pattern improvement of certain modes (e.g., the Southern Hemisphere jet and High-top vertical coordinate), however the performance changes are likely attributed to gradual improvement of the base climate and multiple relevant processes. Less performance improvement is evident in the mode amplitude of these modes and systematic overestimation of the mode amplitude in spring remains in the newer climate models. We find that the post-dominant season amplitude errors in atmospheric modes are not limited to coupled runs but are often already evident in AMIP simulations. This suggests that rectifying the egregious post-dominant season amplitude errors found in many models can be addressed in an atmospheric-only framework making it more tractable to address in the model development process.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

The DESC stellarator code suite. Part 1. Quick and accurate equilibria computations

Three-dimensional equilibrium codes are vital for stellarator design and operation, and high-accuracy equilibria are also necessary for stability studies. This paper details comparisons of two three-dimensional equilibrium codes: VMEC, which uses a steepest-descent algorithm to reach a minimum-energy plasma state, and DESC, which minimizes the magnetohydrodynamic (MHD) force error in real space directly. Accuracy as measured by satisfaction of MHD force balance is presented for each code, along with the computation time. It is shown that DESC is able to achieve more accurate solutions, especially near axis. The importance of higher-accuracy equilibria is shown in DESC's better agreement of stability metrics with asymptotic formulae. DESC's global Fourier–Zernike basis also yields solutions with analytic derivatives explicitly everywhere in the plasma volume, provides improved accuracy in the radial direction versus conventional finite differences and allows for exponential convergence. Further, DESC can compute a solution with the same accuracy as VMEC in order-of-magnitude less time.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Earth Mover’s Distance as a Metric to Evaluate the Extent of Charge Transfer in Excitations Using Discretized Real-Space Densities

This paper presents a novel theoretical measure, μ EMD , based on the earth mover's distance (EMD), for quantifying the density shift caused by electronic excitations in molecules. As input, the EMD metric uses only the discretized ground- and excited-state electron densities in real space, rendering it compatible with almost all electronic structure methods used to calculate excited states. The EMD metric is compared against other popular theoretical metrics for describing the extent of electron-hole separation in a wide range of excited states (valence, Rydberg, charge transfer, etc.). Further, the results showcase the EMD metric's effectiveness across all excitation types and suggest that it is useful as an additional tool to characterize electronic excitations. The study also reveals that μ EMD can function as a promising diagnostic tool for predicting the failure of pure exchange-correlation functionals. Specifically, we show statistical relationships among the functional-driven errors, the exact exchange content within the functional, and the magnitude of μ EMD values.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

A deep learning approach for semantic segmentation of unbalanced data in electron tomography of catalytic materials

In computed TEM tomography, image segmentation represents one of the most basic tasks with implications not only for 3D volume visualization, but more importantly for quantitative 3D analysis. In case of large and complex 3D data sets, segmentation can be an extremely difficult and laborious task, and thus has been one of the biggest hurdles for comprehensive 3D analysis. Heterogeneous catalysts have complex surface and bulk structures, and often sparse distribution of catalytic particles with relatively poor intrinsic contrast, which possess a unique challenge for image segmentation, including the current state-of-the-art deep learning methods. To tackle this problem, we apply a deep learning-based approach for the multi-class semantic segmentation of a γ-Alumina/Pt catalytic material in a class imbalance situation. Specifically, we used the weighted focal loss as a loss function and attached it to the U-Net’s fully convolutional network architecture. We assessed the accuracy of our results using Dice similarity coefficient (DSC), recall, precision, and Hausdorff distance (HD) metrics on the overlap between the ground-truth and predicted segmentations. Our adopted U-Net model with the weighted focal loss function achieved an average DSC score of 0.96 ± 0.003 in the γ-Alumina support material and 0.84 ± 0.03 in the Pt NPs segmentation tasks. We report an average boundary-overlap error of less than 2 nm at the 90th percentile of HD for γ-Alumina and Pt NPs segmentations. The complex surface morphology of γ-Alumina and its relation to the Pt NPs were visualized in 3D by the deep learning-assisted automatic segmentation of a large data set of high-angle annular dark-field (HAADF) scanning transmission electron microscopy (STEM) tomography reconstructions.

36 MATERIALS SCIENCE↗
NASA NTRS1974

Performance analysis of a frame sync algorithm for uncoded PSK telemetry

The optimum procedure for locating a frame sync word periodically inserted in uncoded binary data received over a binary symmetric channel is based on the Hamming distance metric. In this paper, a practical frame sync acquisition and maintenance algorithm is described, and its performance is analyzed. Specifically, with respect to this algorithm, an upper bound on the probability of false sync acquisition, the mean time to sync acquisition, and the subsequent mean time to loss of sync are computed for arbitrary bit error rates, frame lengths, sync word lengths, and algorithm parameters.

Levitt, B. K.↗
NASA NTRS1991

Visual direction as a metric of virtual space

Two experiments examine the abilities of ten subjects to visualize directions shown on a perspective display. Subjects indicated their perceived directions by adjusting a head-mounted cursor to correspond to the direction depicted on the display. This task is required of telerobotic operators who use map-like pictures of their workspace to determine the direction of objects seen by direct view. Results show significant open loop judgment biases that may be composed of errors arising from misinterpretation of the map geometry and overestimation of gaze direction.

Ellis, Stephen R.↗
DOE OSTI2021

Validation Metrics for Fixed Effects and Mixed-Effects Calibration

The modern scientific process often involves the development of a predictive computational model. To improve its accuracy, a computational model can be calibrated to a set of experimental data. A variety of validation metrics can be used to quantify this process. Some of these metrics have direct physical interpretations and a history of use, while others, especially those for probabilistic data, are more difficult to interpret. In this work, a variety of validation metrics are used to quantify the accuracy of different calibration methods. Frequentist and Bayesian perspectives are used with both fixed effects and mixed-effects statistical models. Through a quantitative comparison of the resulting distributions, the most accurate calibration method can be selected. Two examples are included which compare the results of various validation metrics for different calibration methods. It is quantitatively shown that, in the presence of significant laboratory biases, a fixed effects calibration is significantly less accurate than a mixed-effects calibration. This is because the mixed-effects statistical model better characterizes the underlying parameter distributions than the fixed effects model. The results suggest that validation metrics can be used to select the most accurate calibration model for a particular empirical model with corresponding experimental data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Exploring Hydrologic Model Process Connectivity at the Continental Scale Through an Information Theory Approach

Exploring water fluxes between hydrological model (HM) components is essential to assess and improve model realism. Many classical metrics for HM diagnosis rely solely on streamflow and hence provide limited insights into model performance across processes. This study applies an information theory measure known as “transfer entropy” (TE) to systematically quantify the transfer of information among major HM components. To test and demonstrate the benefits of TE, we use the Framework for Understanding Structural Errors (FUSE) model to mimic and compare four commonly used HM structures, VIC, PRMS, SACRAMENTO, and TOPMODEL, across 671 catchments spanning a variety of hydrologic regimes in the conterminous United States. We explore connections between HM components and catchment landscape characteristics (e.g., climate, topography, soil, and vegetation) and characterize their nonlinear associations using distance correlation and Spearman correlation coefficients. Our results indicate that while the information transferred from precipitation to runoff is similar across model structures (likely as a result of calibration), the information transferred among other components can vary significantly from a FUSE structure to another. We find that aridity, precipitation duration and frequency, snow fraction, mean elevation, forest area, and leaf area index are often significantly associated with TE between the main HM components. Here, we propose that the presence of meaningful nonlinear associations can be used to diagnose process representation in HMs. Our results highlight the necessity to enhance the conventional streamflow-only calibration approach for a more realistic representation of water dynamics in the models.

large-sample hydrology↗
NASA NTRS1989

Visual direction as a metric of virtual space

Two experiments examine the abilities of ten subjects to visualize directions shown on a perspective display. Subjects indicated their perceived directions by adjusting a head-mounted cursor to correspond to the direction depicted on the display. This task is required of telerobotic operators who use map-like pictures of their workspace to determine the direction of objects seen by direct view. Results show significant open-loop judgment biases that may be composed of errors arising from misinterpretation of the map geometry and overestimation of gaze direction.

Ellis, Stephen R.↗
NASA NTRS2015

A Comparison of Three Algorithms for Orion Drogue Parachute Release

The Orion Multi-Purpose Crew Vehicle is susceptible to ipping apex forward between drogue parachute release and main parachute in ation. A smart drogue release algorithm is required to select a drogue release condition that will not result in an apex forward main parachute deployment. The baseline algorithm is simple and elegant, but does not perform as well as desired in drogue failure cases. A simple modi cation to the baseline algorithm can improve performance, but can also sometimes fail to identify a good release condition. A new algorithm employing simpli ed rotational dynamics and a numeric predictor to minimize a rotational energy metric is proposed. A Monte Carlo analysis of a drogue failure scenario is used to compare the performance of the algorithms. The numeric predictor prevents more of the cases from ipping apex forward, and also results in an improvement in the capsule attitude at main bag extraction. The sensitivity of the numeric predictor to aerodynamic dispersions, errors in the navigated state, and execution rate is investigated, showing little degradation in performance.

Matz, Daniel A.↗
NASA NTRS2016

Assimilation of SMOS Brightness Temperatures or Soil Moisture Retrievals into a Land Surface Model

Three different data products from the Soil Moisture Ocean Salinity (SMOS) mission are assimilated separately into the Goddard Earth Observing System Model, version 5 (GEOS-5) to improve estimates of surface and root-zone soil moisture. The first product consists of multi-angle, dual-polarization brightness temperature (Tb) observations at the bottom of the atmosphere extracted from Level 1 data. The second product is a derived SMOS Tb product that mimics the data at a 40 degree incidence angle from the Soil Moisture Active Passive (SMAP) mission. The third product is the operational SMOS Level 2 surface soil moisture (SM) retrieval product. The assimilation system uses a spatially distributed ensemble Kalman filter (EnKF) with seasonally varying climatological bias mitigation for Tb assimilation, whereas a time-invariant cumulative density function matching is used for SM retrieval assimilation. All assimilation experiments improve the soil moisture estimates compared to model-only simulations in terms of unbiased root-mean-square differences and anomaly correlations during the period from 1 July 2010 to 1 May 2015 and for 187 sites across the US. Especially in areas where the satellite data are most sensitive to surface soil moisture, large skill improvements (e.g., an increase in the anomaly correlation by 0.1) are found in the surface soil moisture. The domain-average surface and root-zone skill metrics are similar among the various assimilation experiments, but large differences in skill are found locally. The observation-minus-forecast residuals and analysis increments reveal large differences in how the observations add value in the Tb and SM retrieval assimilation systems. The distinct patterns of these diagnostics in the two systems reflect observation and model errors patterns that are not well captured in the assigned EnKF error parameters. Consequently, a localized optimization of the EnKF error parameters is needed to further improve Tb or SM retrieval assimilation.

SMOS↗
NASA NTRS2017

Annoyance to Noise Produced by a Distributed Electric Propulsion High-Lift System

A psychoacoustic test was performed using simulated sounds from a distributed electric propulsion aircraft concept to help understand factors associated with human annoyance. A design space spanning the number of high-lift leading edge propellers and their relative operating speeds, inclusive of time varying effects associated with motor controller error and atmospheric turbulence, was considered. It was found that the mean annoyance response varies in a statistically significant manner with the number of propellers and with the inclusion of time varying effects, but does not differ significantly with the relative RPM between propellers. An annoyance model was developed, inclusive of confidence intervals, using the noise metrics of loudness, roughness, and tonality as predictors.

Rizzi, Stephen A.↗
DOE OSTI2024

Enriching OpenStreetMap network data for transportation applications: Insights into the impact of urban congestion on accessibility

OpenStreetMap (OSM) data is a valuable open-source resource for various transportation, traffic, and planning applications. However, OSM network data lack operating traffic speed information, which is critical for transport planning and operations. Addressing this shortcoming, this study leverages commercial vendor data (to serve as ground truth) with exogenous, open-source variables characterizing local transport infrastructure, land use, and demographic information to predict average congested traffic speeds on OSM networks. Three machine-learning models were tested and estimated for OSM links with and without speed limit information in the Denver metropolitan region. Among these, XGBoost performed best, with mean absolute errors of 3.27 and 3.62 mph for links with and without speed limits, respectively. The developed models accurately predicted traffic speeds for different hours and days of the week compared to ground truth data. Using these predicted speeds, drive accessibility scores were computed for the Denver region for different time periods using the Mobility Energy Productivity (MEP) metric to understand the impact of congestion on energy-efficient accessibility. Results show that congestion-adjusted drive accessibility can be significantly lower compared to accessibility calculated using free flow speeds. Specifically, weekday evening hours saw a 42 % drop in accessibility due to reduced speeds, particularly around downtown Denver. Across the Denver metro region, approximately half as many opportunities and jobs are accessible in under 20 min by car during the evening peak period relative to free flow conditions. These findings underscore the importance of using congestion-adjusted operating speeds rather than speed limits in accessibility calculations, as reliance on speed limits can substantially overestimate energy-efficient drive accessibility in large, car-centric cities susceptible to significant congestion. In conclusion, the methodology presented here could further enrich OSM network data, making them useful for an even broader range of transportation applications.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2025

Rapid Adaptation of Chemical Named Entity Recognition Using Few-Shot Learning and LLM Distillation

Named entity recognition (NER) has been widely used in chemical text mining for the automatic identification and extraction of chemical entities. However, existing chemical NER systems primarily focus on scenarios with abundant training data, requiring significant human effort on annotations. This poses challenges for applications in the chemical field, such as catalysis, where many advancements have traditionally relied on trial-and-error investigations and incremental adjustment of variables. This hinders catalysis science and technology progress in addressing emerging energy and environmental crises. In this work, we propose a few-shot NER model that can quickly adapt to extract new types of chemical entities by using only a limited number of annotated examples. Our model employs a metric-learning approach to transfer entity similarity knowledge from high-resource chemical domains (with abundant annotations) to enable effective entity recognition in low-resource specialized domains (limited annotation). We validate the effectiveness of our model on a few-shot chemical NER benchmark built based on six existing chemical NER data sets. Experiments show that the proposed few-shot NER model can achieve reasonable performance with only 5 examples per entity type and shows consistent improvement as the number of examples increases. Furthermore, we demonstrate how the proposed model can be trained with large language model (LLM) annotated data, opening a new pathway for rapid adaptation of NER systems. Furthermore, our approach leverages the knowledge broadness of large language models for chemistry while distilling this knowledge into a lightweight model suitable for efficient and in-house use.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Simulating organic aerosol in Delhi with WRF-Chem using the volatility-basis-set approach: exploring model uncertainty with a Gaussian process emulator

The nature and origin of organic aerosol in the atmosphere remain unclear. The gas–particle partitioning of semi-volatile organic compounds (SVOCs) that constitute primary organic aerosols (POAs) and the multigenerational chemical aging of SVOCs are particularly poorly understood. The volatility basis set (VBS) approach, implemented in air quality models such as WRF-Chem (Weather Research and Forecasting model with Chemistry), can be a useful tool to describe emissions of POA and its chemical evolution. However, the evaluation of model uncertainty and the optimal model parameterization may be expensive to probe using only WRF-Chem simulations. Gaussian process emulators, trained on simulations from relatively few WRF-Chem simulations, are capable of reproducing model results and estimating the sources of model uncertainty within a defined range of model parameters. In this study, a WRF-Chem VBS parameterization is proposed; we then generate a perturbed parameter ensemble of 111 model runs, perturbing 10 parameters of the WRF-Chem model relating to organic aerosol emissions and the VBS oxidation reactions. This allowed us to cover the model's uncertainty space and to compare outputs from each run to aerosol mass spectrometer observations of organic aerosol concentrations and O:C ratios measured in New Delhi, India. The simulations spanned the organic aerosol concentrations measured with the aerosol mass spectrometer (AMS). However, they also highlighted potential structural errors in the model that may be related to unsuitable diurnal cycles in the emissions and/or failure to adequately represent the dynamics of the planetary boundary layer. While the structural errors prevented us from clearly identifying an optimized VBS approach in WRF-Chem, we were able to apply the emulator in the following two periods: the full period (1–29 May) and a subperiod period of 14:00–16:00 h LT (local time) on 1–29 May. The combination of emulator analysis and model evaluation metrics allowed us to identify plausible parameter combinations for the analyzed periods. We demonstrate that the methodology presented in this study can be used to determine the model uncertainty and to identify the appropriate parameter combination for the VBS approach and hence to provide valuable information to improve our understanding of OA production.

54 ENVIRONMENTAL SCIENCES↗