Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “error metric”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
NASA NTRS2019

Fast Steering Mirror Disturbance Effects on Overall System Optical Performance for the Large Ultraviolet/optical/infrared Surveyor (LUVOIR) Concept Using a Non-Contact Vibration Isolation and Precision Pointing System

As the optical performance requirements of space telescopes get more stringent, the need to analyze all possible error sources early in the mission design becomes critical. One large telescope with tight performance requirements is the Large Ultraviolet / Optical / Infrared Surveyor (LUVOIR) concept. The LUVOIR concept includes a 15-meter-diameter segmented-aperture telescope with a suite of serviceable instruments operating over a range of wavelengths between 100nm to 2.5um. Using an isolation architecture that involves no mechanical contact between the telescope and the host spacecraft structure allows for tighter performance metrics than current space-based telescopes being flown. Because of this separation, the spacecraft disturbances can be greatly reduced and disturbances on the telescope payload contribute more to the optical performance error. A portion of the optical performance error comes from the disturbances generated from the motion of the Fast Steering Mirror (FSM) on the payload. Characterizing the effects of this disturbance gives insight into the specifications on the FSM needed to achieve the tight optical performance requirements of the overall system. Through analysis of the LUVOIR finite element model and linear optical model given a range of input disturbances at the FSM, the optical performance of the telescope and recommendations for FSM specifications can be determined. The LUVOIR observatory control strategy consists of a multi-loop control architecture including the spacecraft Attitude Control System (ACS), Vibration Isolation and Precision Pointing System (VIPPS), and FSM. This paper focuses on the control loop containing the FSM disturbances and their effects on the telescope optical performance.

Sacks, Lia W.↗
DOE OSTI2023

Tree-Based Ensemble Learning Models for Wall Temperature Predictions in Post-Critical Heat Flux Flow Regimes at Subcooled and Low-Quality Conditions

Accurately predicting post-critical heat flux (CHF) heat transfer is an important but challenging task in water-cooled reactor design and safety analysis. Although numerous heat transfer correlations have been developed to predict post-CHF heat transfer, these correlations are only applicable to relatively narrow ranges of flow conditions due to the complex physical nature of the post-CHF heat transfer regimes. In this paper, a large quantity of experimental data is collected and summarized from the literature for steady-state subcooled and low-quality film boiling regimes with water as the working fluid in vertical tubular test sections. In addition, a low-quality water film boiling (LWFB) database is consolidated with a total of 22,813 experimental data points, which cover a wide flow range of the system pressure from 0.1 to 9.0 MPa, mass flux from 25 to 2750 kg/m 2 s, and inlet subcooling from 1 to 70 °C. Two machine learning (ML) models, based on random forest (RF) and gradient boosted decision tree (GBDT), are trained and validated to predict wall temperatures in post-CHF flow regimes. The trained ML models demonstrate significantly improved accuracies compared to conventional empirical correlations. To further evaluate the performance of these two ML models from a statistical perspective, three criteria are investigated and three metrics are calculated to quantitatively assess the accuracy of these two ML models. For the full LWFB database, the root-mean-square errors between the measured and predicted wall temperatures by the GBDT and RF models are 5.7% and 6.2%, respectively, confirming the accuracy of the two ML models.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Scalable deep learning for watershed model calibration

Watershed models such as the Soil and Water Assessment Tool (SWAT) consist of high-dimensional physical and empirical parameters. These parameters often need to be estimated/calibrated through inverse modeling to produce reliable predictions on hydrological fluxes and states. Existing parameter estimation methods can be time consuming, inefficient, and computationally expensive for high-dimensional problems. In this paper, we present an accurate and robust method to calibrate the SWAT model (i.e., 20 parameters) using scalable deep learning (DL). We developed inverse models based on convolutional neural networks (CNN) to assimilate observed streamflow data and estimate the SWAT model parameters. Scalable hyperparameter tuning is performed using high-performance computing resources to identify the top 50 optimal neural network architectures. We used ensemble SWAT simulations to train, validate, and test the CNN models. We estimated the parameters of the SWAT model using observed streamflow data and assessed the impact of measurement errors on SWAT model calibration. We tested and validated the proposed scalable DL methodology on the American River Watershed, located in the Pacific Northwest-based Yakima River basin. Our results show that the CNN-based calibration is better than two popular parameter estimation methods (i.e., the generalized likelihood uncertainty estimation [GLUE] and the dynamically dimensioned search [DDS], which is a global optimization algorithm). For the set of parameters that are sensitive to the observations, our proposed method yields narrower ranges than the GLUE method but broader ranges than values produced using the DDS method within the sampling range even under high relative observational errors. The SWAT model calibration performance using the CNNs, GLUE, and DDS methods are compared using R 2 and a set of efficiency metrics, including Nash-Sutcliffe, logarithmic Nash-Sutcliffe, Kling-Gupta, modified Kling-Gupta, and non-parametric Kling-Gupta scores, computed on the observed and simulated watershed responses. The best CNN-based calibrated set has scores of 0.71, 0.75, 0.85, 0.85, 0.86, and 0.91. The best DDS-based calibrated set has scores of 0.62, 0.69, 0.8, 0.77, 0.79, and 0.82. The best GLUE-based calibrated set has scores of 0.56, 0.58, 0.71, 0.7, 0.71, and 0.8. The scores above show that the CNN-based calibration leads to more accurate low and high streamflow predictions than the GLUE and DDS sets. Our research demonstrates that the proposed method has high potential to improve our current practice in calibrating large-scale integrated hydrologic models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Inpainting radar missing data regions with deep learning

Abstract. Missing and low-quality data regions are a frequent problem for weather radars. They stem from a variety of sources: beam blockage, instrument failure, near-ground blind zones, and many others. Filling in missing data regions is often useful for estimating local atmospheric properties and the application of high-level data processing schemes without the need for preprocessing and error-handling steps – feature detection and tracking, for instance. Interpolation schemes are typically used for this task, though they tend to produce unrealistically spatially smoothed results that are not representative of the atmospheric turbulence and variability that are usually resolved by weather radars. Recently, generative adversarial networks (GANs) have achieved impressive results in the area of photo inpainting. Here, they are demonstrated as a tool for infilling radar missing data regions. These neural networks are capable of extending large-scale cloud and precipitation features that border missing data regions into the regions while hallucinating plausible small-scale variability. In other words, they can inpaint missing data with accurate large-scale features and plausible local small-scale features. This method is demonstrated on a scanning C-band and vertically pointing Ka-band radar that were deployed as part of the Cloud Aerosol and Complex Terrain Interactions (CACTI) field campaign. Three missing data scenarios are explored: infilling low-level blind zones and short outage periods for the Ka-band radar and infilling beam blockage areas for the C-band radar. Two deep-learning-based approaches are tested, a convolutional neural network (CNN) and a GAN that optimize pixel-level error or combined pixel-level error and adversarial loss respectively. Both deep-learning approaches significantly outperform traditional inpainting schemes under several pixel-level and perceptual quality metrics.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1997

Shape Optimization by Bayesian-Validated Computer-Simulation Surrogates

A nonparametric-validated, surrogate approach to optimization has been applied to the computational optimization of eddy-promoter heat exchangers and to the experimental optimization of a multielement airfoil. In addition to the baseline surrogate framework, a surrogate-Pareto framework has been applied to the two-criteria, eddy-promoter design problem. The Pareto analysis improves the predictability of the surrogate results, preserves generality, and provides a means to rapidly determine design trade-offs. Significant contributions have been made in the geometric description used for the eddy-promoter inclusions as well as to the surrogate framework itself. A level-set based, geometric description has been developed to define the shape of the eddy-promoter inclusions. The level-set technique allows for topology changes (from single-body,eddy-promoter configurations to two-body configurations) without requiring any additional logic. The continuity of the output responses for input variations that cross the boundary between topologies has been demonstrated. Input-output continuity is required for the straightforward application of surrogate techniques in which simplified, interpolative models are fitted through a construction set of data. The surrogate framework developed previously has been extended in a number of ways. First, the formulation for a general, two-output, two-performance metric problem is presented. Surrogates are constructed and validated for the outputs. The performance metrics can be functions of both outputs, as well as explicitly of the inputs, and serve to characterize the design preferences. By segregating the outputs and the performance metrics, an additional level of flexibility is provided to the designer. The validated outputs can be used in future design studies and the error estimates provided by the output validation step still apply, and require no additional appeals to the expensive analysis. Second, a candidate-based a posteriori error analysis capability has been developed which provides probabilistic error estimates on the true performance for a design randomly selected near the surrogate-predicted optimal design.

Patera, Anthony T.↗
NASA NTRS2021

Validation of Satellite-Based Precipitation Products from TRMM to GPM

The global precipitation measurement mission (GPM) has been in operation for seven years and continues to provide a vast quantity of global precipitation data at finer temporospatial resolutions with improved accuracy and coverage. GPM’s signature algorithm, the integrated multisatellite retrievals for GPM (IMERG) is a next-generation of precipitation product expected for wide variety of research and operational applications. This study evaluates the latest version (V06B) of IMERG and its predecessor, the tropical rainfall measuring mission (TRMM) multisatellite precipitation (TMPA) 3B42 (V7) using ground-based and gauge-corrected multiradar multisensor system (MRMS) precipitation products over the conterminous United States (CONUS). The spatial distributions of all products are analyzed. The error characteristics are further examined for 3B42 and IMERG in winter and summer by an error decomposition approach, which partitions total bias into hit bias, biases due to missed precipitation and false precipitation. The volumetric and categorical statistical metrics are used to quantitatively evaluate the performance of the two satellite-based products. All products show a similar precipitation climatology with some regional differences. The two satellite-based products perform better in the eastern CONUS than in the mountainous Western CONUS. The evaluation demonstrates the clear improvement in IMERG precipitation product in comparison with its predecessor 3B42, especially in reducing missed precipitation in winter and summer, and hit bias in winter, resulting in better performance in capturing lighter and heavier precipitation.

MRMS↗
NASA NTRS2022

Integrated Topographic Corrections Improve Forest Mapping Using Landsat Imagery

In mountainous environments, topography strongly affects the reflectance due to illumination effects and cast shadows, which introduce errors in land cover classifications. However, topographic correction is not routinely implemented in standard data pre-processing chains (e.g., Landsat Analysis Ready Data), and there is a lack of consensus whether topographic correction is necessary, and if so, how to conduct it. Furthermore, methods that correct simultaneously for atmospheric and topographic effects are becoming available, but they have not been compared directly. Our objects were to investigate (1) the effectiveness of two topographic correction approaches that integrate atmospheric and topographic correction, (2) improvements in classification accuracy when analyzing topographically corrected single-date imagery (14 July 2016 and 2 October 2016), versus a full Landsat time series from 2014 to 2016, and 3) improvements in classification accuracy when including additional terrain information (i.e., topographic slope, elevation, and aspect). We developed a physical based model and compared it with an enhanced C-correction, both of which integrate atmospheric and topographic correction. We compared classification accuracies with and without topographic correction using combinations of single-date imagery, image composites and spectral-temporal metrics generated from the full Landsat time series, and additional terrain information in the Caucasus Mountains. We found that both the enhanced C-correction and the physical model performed very well and largely eliminated the correlation (Pearson’s correlation coefficient r ranges from 0.06 to 0.24) between surface reflectance and illumination condition, but the physical model performed best (r ranges from 0.05 to 0.11). Both image composites, and spectral-temporal metrics generated from corrected imagery, resulted in significantly (p ≤ 0.05) higher classification accuracies and better forest classifications, especially for the mixed forests. Adding terrain information reduced classification error significantly, but not as much as topographic correction. In summary, topographic correction remains necessary, even when analyzing a full Landsat time series and including a digital elevation model in the classification. We recommend that topographic correction should be applied when analyzing Landsat satellite imagery in mountainous region for forest cover classification.

Atmospheric correction↗
DOE OSTI2022

Physics-Informed and Data-Driven Prediction of Residual Stress in Three-Dimensional Machining

Efficient and reliable prediction of machining-induced residual stress (RS) is a key requirement for truly integrated computational materials engineering (ICME). Currently available process modeling approaches, including empirical, analytical, and numerical methodologies lack predictive power and require substantial calibration and validation data. Moreover, most model-based approaches consider only two-dimensional (2D) (i.e., orthogonal), cutting processes. Meanwhile, industrial processes such as milling, turning, and drilling are inherently three-dimensional (3D). The present work attempts to bridge the gap between 2D and 3D through careful consideration of the process physics, including geometric, kinematic, and size-effect constraints to realize robust prediction of how RS develops in 3D machining. Using a novel in-situ experimental technique and digital image correlation (DIC) to determine equivalent Hertzian contact widths, contact pressures, and friction coefficients, the proposed methodology leverages a discretized conversion algorithm that includes multi-pass shakedown effects. This paper presents a semi-analytical model to predict machining-induced RS in 3D turning operations, which are used representatively for 3D processes more generally. Rather than follow a ‘brute force’ 3D FEM approach or conduct countless experiments to train a purely data-driven machine learning algorithm, the proposed approach builds on previous 2D modeling work. Through careful consideration of the process physics, including complex geometry/kinematic considerations of 3D turning, the authors demonstrated an experimentally calibrated approach, as well as validation based on published RS data. Model predictions and previously published measurement data of RS depth profiles for turning of Inconel 718 were compared for a range of process parameters. Correlation between the proposed 3D model and validation data was found to be within the margin of experimental error for most conditions. The proposed model appears to capture the overall behavior of 3D RS depth profiles with acceptable accuracy, particularly the key metrics of near-surface stress, peak stress magnitude and location, as well as overall stress profile depth. This report presents a physics-informed, data-driven approach for efficient calibration of a 2D model for machining-induced RS through DIC analysis of in-situ characterized subsurface displacement fields.

42 ENGINEERING↗
DOE OSTI2022

Predicting the Cetane Number, Sooting Tendency, and Energy Density of Terpene Fuel Additives

Abstract Discovering renewable fuels and fuel additives is paramount in reducing carbon emissions from internal combustion engines. Terpenes, a group of compounds that can be synthesized from plant matter and microorganisms, have gained significant interest in recent years as promising candidates for fuels/additives. Terpenes are a diverse class of compounds that contain rings and methyl branches, resulting in high energy densities and optimal cold weather behavior. Their variation in bond order, carbon chains, and functional groups lead to varying degrees of soot formation and performance in existing engines. The present work leverages predictive models, namely artificial neural networks, to predict the cetane number (CN), sooting tendency (quantified with yield sooting index, YSI), and energy density (quantified with lower heating value, LHV) of terpenes and hydrogenated terpenes whose sooting propensities were previously determined through experimental means. Predicted sooting propensities of these terpenes are compared with experimental values, and predicted cetane numbers and energy densities are used to comment on the compounds’ ability to act as fuels/additives. Expected prediction errors for CN, YSI, and LHV, defined by blind test set median absolute error, are within 5.56 cetane units, 3.63 yield sooting index units, and 0.77 MJ/kg respectively. Additionally, the present work investigates a variety of correlation/dependence metrics for property-property relationships, furthering our understanding of how combustion-relevant properties are related.

09 BIOMASS FUELS↗
DOE OSTI2022

Uncertainty in Thermal Modeling of Spent Nuclear Fuel Casks

Uncertainty is a key metric in computational modeling that must be evaluated for results to have wide ranging applicability. A well characterized uncertainty range is ideal with clear error bars on results that can be presented to stakeholders. In the field of spent fuel cask modeling, this ideal has been historically difficult to achieve in practice because of the computationally intensive nature of the models used and the difficulty assigning reasonable uncertainties to quantities in as-built systems. The work in this report has been conducted to evaluate the overall state of uncertainty and sensitivity in spent fuel cask models and develop methodologies for evaluating these uncertainties. These methodologies must be practical for engineering applications. They should not require excessive computational resources or calendar time to achieve results. In engineering, the model must be on a scale such that it can be changed and adapted throughout a project as new information is discovered and project goals evolve. This report covers three major modeling task areas that provide an overview of the types of sensitivity and uncertainty present in a spent fuel storage and transportation system. Section 3 discusses sensitivity and uncertainty analysis in the effective thermal conductivity model for the fuel region and applies these results to a single assembly model. Section 4 shows sensitivity analysis of a full cask model in the TN-32B and Section 5 demonstrates the overall uncertainty workflow using Coolant Boiling in Rod Arrays – Spent Fuel Storage and STAR-CCM+ developed from the sensitivity work in the preceding sections.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
NASA NTRS2003

An Evaluation of Recent Gravity Models wrt. Altimeter Satellite Missions

With the launch of CHAMP and GRACE, we have entered a new phase in the history of satellite geodesy. For the first time, geopotential models are now available based almost exclusively on satellite-satellite tracking either with GPS in the case of the CHAMP-based geopotential models, or co-orbital intersatellite ultra-precise ranging in the case of GRACE. Different groups have analyzed these data, and produced a series of geopotential models (e.g., EIGENlS, EIGEN2, GGM0lS, GGMOlC) that incorporate the new data. We will compare the performance of these "newer" geopotential models with the standard models now used for computations, (e.g., JGM-3, BGM-96, PGS7727, and GRIMS-C1) for TOPEX, JASON, Geosat-Follow-On (GFO), and Envisat using standard metrics such as SLR RMS of fit, altimeter crossovers, and orbit overlaps. Where covariances are available we can evaluate the predicted geographically correlated orbit error. These predicted results can be compared with the Earth-fixed differences between dynamic and reduced-dynamic orbits to test the predictive accuracy of the covariances, as well as to calibrate the error of the solutions.

Lemoine, Frank G.↗
NASA NTRS2010

Transfer of Satellite Rainfall Uncertainty from Gauged to Ungauged Regions at Regional and Seasonal Timescales

Hydrologists and other users need to know the uncertainty of the satellite rainfall data sets across the range of time/space scales over the whole domain of the data set. Here, uncertainty' refers to the general concept of the deviation' of an estimate from the reference (or ground truth) where the deviation may be defined in multiple ways. This uncertainty information can provide insight to the user on the realistic limits of utility, such as hydrologic predictability, that can be achieved with these satellite rainfall data sets. However, satellite rainfall uncertainty estimation requires ground validation (GV) precipitation data. On the other hand, satellite data will be most useful over regions that lack GV data, for example developing countries. This paper addresses the open issues for developing an appropriate uncertainty transfer scheme that can routinely estimate various uncertainty metrics across the globe by leveraging a combination of spatially-dense GV data and temporally sparse surrogate (or proxy) GV data, such as the Tropical Rainfall Measuring Mission (TRMM) Precipitation Radar and the Global Precipitation Measurement (GPM) mission Dual-Frequency Precipitation Radar. The TRMM Multi-satellite Precipitation Analysis (TMPA) products over the US spanning a record of 6 years are used as a representative example of satellite rainfall. It is shown that there exists a quantifiable spatial structure in the uncertainty of satellite data for spatial interpolation. Probabilistic analysis of sampling offered by the existing constellation of passive microwave sensors indicate that transfer of uncertainty for hydrologic applications may be effective at daily time scales or higher during the GPM era. Finally, a commonly used spatial interpolation technique (kriging), that leverages the spatial correlation of estimation uncertainty, is assessed at climatologic, seasonal, monthly and weekly timescales. It is found that the effectiveness of kriging is sensitive to the type of uncertainty metric, time scale of transfer and the density of GV data within the transfer domain. Transfer accuracy is lowest at weekly timescales with the error doubling from monthly to weekly.However, at very low GV data density (<20% of the domain), the transfer accuracy is too low to show any distinction as a function of the timescale of transfer.

Tang, Ling↗
DOE OSTI2020

Validation of the IRI-2016 model with Indian NavIC data for future navigation applications

The position accuracy of Navigation with Indian Constellation (NavIC) system is affected by several sources of errors. Among them, the Ionospheric Time Delay (ITD) error is the most predominant one which depends upon the total electron content (TEC) present in the ionosphere. The ITD variations are more intense over low latitude regions due to the equatorial anomaly effects. Hence, modelling of ITD error is necessary. The International Reference Ionosphere (IRI)-2016 model is one of the standard global ionospheric models to estimate the Vertical TEC (VTEC). This paper discusses about the VTEC deviations due to the IRI-2016 model over low latitude Hyderabad station, Indian region using NavIC, Global Positioning System and Global Navigation Satellite System signals at corresponding Ionospheric Pierce Point latitude and longitudes for all months and various Kp indices during the low solar activity year 2017. In this work, cross correlation coefficient, the metric norm (L2N), Symmetric Kullbacke Leibler Distance metrics are used to evaluate the performance of IRI-2016 model TEC with NavIC, GPS and GLONASS data. From the results, it is found that TEC predicted by the IRI-2016 model produced smaller estimation errors with NavIC data over Indian region. The obtained results will be helpful for future updates of IRI model.

42 ENGINEERING↗
DOE OSTI2020

Replicating Machine Learning Experiments in Materials Science

Transparency and reproducibility are important aspects of validation for Machine Learning (ML) models that are not fully understood and applies independently of the application domain.We offer a case study of reproducibility that highlights the challenges encountered when attempting to reproduce analyzes obtained with Machine Learning methods in materials informatics. Our study explores prediction results obtained with ML models and issues in training data serving as input. We discuss challenges related to theory-driven and numerical errors in training data, lack of reproducibility across platforms and versions, and effects of randomness when varying hyperparameters. In addition to model accuracy, a main metric of interest in the ML community, our results show that model sensitivity may be equally important for applying ML in domain applications such a materials science.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1973

Effect of photogrammetric reading error on slope-frequency distributions

The effect of reading error on two hypothetical slope frequency distributions and two slope frequency distributions from actual lunar data in order to ensure that these errors do not cause excessive overestimates of algebraic standard deviations for the slope frequency distributions. The errors introduced are insignificant when the reading error is small and the slope length is large. A method for correcting the errors in slope frequency distributions is presented and applied to 11 distributions obtained from Apollo 15, 16, and 17 panoramic camera photographs and Apollo 16 metric camera photographs.

Moore, H. J.↗
NASA NTRS1994

Trellis Coding of Non-coherent Multiple Symbol Full Response M-ary CPFSK with Modulation Index 1/M

This paper introduces a trellis coded modulation (TCM) scheme for non-coherent multiple full response M-ary CPFSK with modulation index 1/M. A proper branch metric for the trellis decoder is obtained by employing a simple approximation of the modified Bessel function for large signal to noise ratio (SNR). Pairwise error probability of coded sequences is evaluated by applying a linear approximation to the Rician random variable.

trellis coded modulation TCM↗
NASA NTRS2003

Interplanetary optical navigation 101

The JPL Navigation and Mission Design Section has two related tasks: Mission Design: calculate a nominal trajectory which meets the diverse needs of the scientists on a particular mission. Locate deterministic maneuvers required to fly the nominal trajectory, while locating statistical maneuvers which correct errors in the actual trajectory. Navigation estimates the flight path of a spacecraft (orbit determination) using three fundamental data sets: ground based astrometric data; radio metric tracking data (Doppler, range).

mission design↗
NASA NTRS2008

Radiometric tracking for deep space navigation

Metric information derived from radio communication links between Earth and spacecraft are used for navigation of interplanetary probes. This paper summarizes radiometric techniques and their error budgets.

Shin, Dong K.↗