Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Statistical accuracy”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
NASA NTRS2005

Apperception of Clouds in AIRS Data

Our capacity to simulate the radiative characteristics of the Earth system has advanced greatly over the past decade. However, new space based measurements show that idealized simulations might not adequately represent the complexity of nature. For example, AIRS simulated multi-layer cloud clearing research provides an excellent groundwork for early Atmospheric Infra-Red Sounder (AIRS) operational cloud clearing and atmospheric profile retrieval. However, it doesn't reflect the complicated reality of clouds over land and coastal areas. Thus far, operational AIRS/AMSU (Advanced Microwave Sounding Unit) cloud clearing is not only of low yield but also of unsatisfying quality. This is not an argument for avoiding this challenging task, rather a powerful argument for exploring other synergistic approaches, and for adapting these strategies toward improving both indirect and direct use of cloudy infrared sounding data. Ample evidence is shown in this paper that the indirect use of cloudy sounding data by way of cloud clearing is sub-optimal for data assimilation. Improvements are needed in quality control, retrieval yield, and overall cloud clearing retrieval performance. For example, cloud clearing over land, especially over the desert surface, has led to much degraded retrieval quality and often a very low yield of quality controlled cloud cleared radiances. If these indirect cloud cleared radiances are instead to be directly assimilated into NWP models, great caution must be used. Our limited and preliminary cloud clearing results from AIRS/AMSU (with the use of MODIS data) and an AIRS/MODIS synergistic approach have, however, shown that higher spatial resolution multispectral imagery data can provide much needed quality control of the AIRS/AMSU cloud clearing retrieval. When AIRS and Moderate Resolution Imaging Spectroradiometer (MODIS) are used synergistically, a higher spatial resolution over difficult terrain (especially desert areas) can be achieved and with a much improved accuracy. Preliminary statistical analyses of cloud cleared radiances derived from (1) operational AIRS/AMSU, (2) operational AIRS/AMSU plus the use of MODIS data as quality control, and (3) AIRS/MODIS synergistic single channel and two field of views cloud clearing are Our capacity to simulate the radiative characteristics of the Earth system has

Huang, Hung-Lung↗
NASA NTRS2011

Multi-Axis Identifiability Using Single-Surface Parameter Estimation Maneuvers on the X-48B Blended Wing Body

The problem of parameter estimation on hybrid-wing-body type aircraft is complicated by the fact that many design candidates for such aircraft involve a large number of aero- dynamic control effectors that act in coplanar motion. This fact adds to the complexity already present in the parameter estimation problem for any aircraft with a closed-loop control system. Decorrelation of system inputs must be performed in order to ascertain individual surface derivatives with any sort of mathematical confidence. Non-standard control surface configurations, such as clamshell surfaces and drag-rudder modes, further complicate the modeling task. In this paper, asymmetric, single-surface maneuvers are used to excite multiple axes of aircraft motion simultaneously. Time history reconstructions of the moment coefficients computed by the solved regression models are then compared to each other in order to assess relative model accuracy. The reduced flight-test time required for inner surface parameter estimation using multi-axis methods was found to come at the cost of slightly reduced accuracy and statistical confidence for linear regression methods. Since the multi-axis maneuvers captured parameter estimates similar to both longitudinal and lateral-directional maneuvers combined, the number of test points required for the inner, aileron-like surfaces could in theory have been reduced by 50%. While trends were similar, however, individual parameters as estimated by a multi-axis model were typically different by an average absolute difference of roughly 15-20%, with decreased statistical significance, than those estimated by a single-axis model. The multi-axis model exhibited an increase in overall fit error of roughly 1-5% for the linear regression estimates with respect to the single-axis model, when applied to flight data designed for each, respectively.

Ratnayake, Nalin A.↗
NASA NTRS2020

Hawai'i Water Resources: Utilizing NASA Earth Observations to Assess Ocean Conditions Leading to the Spread of the Nuisance Red Algae (Chondria tumulosa) in Papahānaumokuākea Marine National Monument, Hawai’i

Chondria tumulosa, a newly discovered red alga, was observed in low abundance in 2016 but has since proliferated and is now smothering and decimating vast expanse of coral reefs in Manawai, located in Papahānaumokuākea Marine National Monument (PMNM). If the spread persists, the outbreak of this cryptogenic species could potentially cause region-wide ecosystem degradation. In coordination with the U.S. Fish and Wildlife Service, Marine National Monuments of the Pacific and the National Oceanographic and Atmospheric Administration (NOAA) Office of National Marine Sanctuaries’ Papahānaumokuākea Marine National Monument, this project created a tool to analyze oceanographic conditions (sea surface temperature (SST), chlorophyll-a, water velocity, salinity, turbidity) across the Monument that could potentially be driving the algal spread. The Google Earth Engine tool enabled the partners to visualize oceanographic conditions and gather time-series graphs utilizing Aqua and Terra Moderate Resolution Imaging Spectroradiometer (MODIS), Suomi National Polar-orbiting Partnership (NPP) Visible Infrared Imaging Radiometer Suite (VIIRS), Sentinel-3 Ocean and Land Colour Instrument (OLCI), Hybrid Coordinate Ocean Model (HYCOM) and NOAA's Climate Data Record in a user-friendly interface. The team used in situ SST data from subsurface temperature recorders provided by the partners to validate the tool's accuracy. Preliminary statistical analysis of MODIS data found warming trends in SST in Manawai as well as increased chlorophyll-a levels during the summer months in contrast to the control (non-infected) Lalo atoll. The tool did not aim to classify algal presence due to limited availability of higher resolution satellite imagery but instead enabled PMNM managers to monitor conditions that may be conducive to algal growth around the monument to make informed decisions and mitigation practices.

DEVELOP Project Summary↗
DOE OSTI2024

Elimination LArTPC Simulation Uncertainty

Liquid Argon Time Projection Chambers (LArTPC) are crucial for measuring muons and neutrinos by capturing the paths of fast-moving particles through argon gas. However, these detectors face challenges such as electron-ion recombination, diffusion, and attenuation, which introduce uncertainties in simulation models. This study, conducted by Ka ren Mkrtchyan at FERMILAB, aims to reduce these uncertainties by adjusting the amplitude and width of signals detected by the TPC wires. Initial findings indicate that the current modification algorithm requires further refinement to better align simulations with observed data. Ongoing work focuses on correcting computational bugs and enhancing the simulation model for improved accuracy and statistical confidence.

Mkrtchyan, Ka'ren↗
NASA NTRS1976

Evaluation criteria for software classification inventories, accuracies, and maps

Statistical criteria are presented for modifying the contingency table used to evaluate tabular classification results obtained from remote sensing and ground truth maps. This classification technique contains information on the spatial complexity of the test site, on the relative location of classification errors, on agreement of the classification maps with ground truth maps, and reduces back to the original information normally found in a contingency table.

Jayroe, R. R., Jr.↗
DOE OSTI2025

Trust Your Gut: Comparing Human and Machine Inference from Noisy Visualizations

People commonly utilize visualizations not only to examine a given dataset, but also to draw generalizable conclusions about the underlying models or phenomena. Prior research has compared human visual inference to that of an optimal Bayesian agent, with deviations from rational analysis viewed as problematic. However, human reliance on non-normative heuristics may prove advantageous in certain circumstances. We investigate scenarios where human intuition might surpass idealized statistical rationality. In two experiments, we examine individuals’ accuracy in characterizing the parameters of known data-generating models from bivariate visualizations. Our findings indicate that, although participants generally exhibited lower accuracy compared to statistical models, they frequently outperformed Bayesian agents, particularly when faced with extreme samples. Participants appeared to rely on their internal models to filter out noisy visualizations, thus improving their resilience against spurious data. However, participants displayed overconfidence and struggled with uncertainty estimation. They also exhibited higher variance than statistical machines. Our findings suggest that analyst gut reactions to visualizations may provide an advantage, even when departing from rationality. These results carry implications for designing visual analytics tools, offering new perspectives on how to integrate statistical models and analyst intuition for improved inference and decision-making. The data and materials for this paper are available at https://osf.io/qmfv6

human-machine collaboration↗
NASA NTRS1994

Determining the accuracy of maximum likelihood parameter estimates with colored residuals

An important part of building high fidelity mathematical models based on measured data is calculating the accuracy associated with statistical estimates of the model parameters. Indeed, without some idea of the accuracy of parameter estimates, the estimates themselves have limited value. In this work, an expression based on theoretical analysis was developed to properly compute parameter accuracy measures for maximum likelihood estimates with colored residuals. This result is important because experience from the analysis of measured data reveals that the residuals from maximum likelihood estimation are almost always colored. The calculations involved can be appended to conventional maximum likelihood estimation algorithms. Simulated data runs were used to show that the parameter accuracy measures computed with this technique accurately reflect the quality of the parameter estimates from maximum likelihood estimation without the need for analysis of the output residuals in the frequency domain or heuristically determined multiplication factors. The result is general, although the application studied here is maximum likelihood estimation of aerodynamic model parameters from flight test data.

Morelli, Eugene A.↗
NASA NTRS1995

Determining the Accuracy of Aerodynamic Model Parameters Estimated from Flight Test Data

An important part of building mathematical models based on measured data is calculating the accuracy associated with statistical estimates of the model parameters. Indeed, without some idea of this accuracy, the parameter estimates themselves have limited value. In this work, an expression for computing quantitatively correct parameter accuracy measures for maximum likelihood parameter estimates with colored residuals is developed and validated. This result is important because experience in analyzing flight test data reveals that the output residuals from maximum likelihood estimation are almost always colored. The calculations involved can be appended to conventional maximum likelihood estimation algorithms. Monte Carlo simulation runs were used to show that parameter accuracy measures from the new technique accurately reflect the quality of the parameter estimates from maximum likelihood estimation without the need for correction factors or frequency domain analysis of the output residuals. The technique was applied to flight test data from repeated maneuvers flown on the F-18 High Alpha Research Vehicle (HARV). As in the simulated cases, parameter accuracy measures from the new technique were in agreement with the scatter in the parameter estimates from repeated maneuvers, while conventional parameter accuracy measures were optimistic.

Morelli, Eugene A.↗
NASA NTRS1997

Accuracy of Aerodynamic Model Parameters Estimated from Flight Test Data

An important put of building mathematical models based on measured date is calculating the accuracy associated with statistical estimates of the model parameters. Indeed, without some idea of this accuracy, the parameter estimates themselves have limited value. An expression is developed for computing quantitatively correct parameter accuracy measures for maximum likelihood parameter estimates when the output residuals are colored. This result is important because experience in analyzing flight test data reveals that the output residuals from maximum likelihood estimation are almost always colored. The calculations involved can be appended to conventional maximum likelihood estimation algorithms. Monte Carlo simulation runs were used to show that parameter accuracy measures from the new technique accurately reflect the quality of the parameter estimates from maximum likelihood estimation without the need for correction factors or frequency domain analysis of the output residuals. The technique was applied to flight test data from repeated maneuvers flown on the F-18 High Alpha Research Vehicle. As in the simulated cases, parameter accuracy measures from the new technique were in agreement with the scatter in the parameter estimates from repeated maneuvers, whereas conventional parameter accuracy measures were optimistic.

Morelli, Eugene A.↗
NASA NTRS1994

Statistical uncertainties in temperature diagnostics for hot coronal plasma using the ASCA SIS

Statistical uncertainties in determining the temperatures of hot (0.5-10 keV) coronal plasmas are investigated. The statistical presicion of various spectral temperature diagnostics is established by analyzing synthetic ASCA solid-state imaging spectrometer (SIS) CCD spectra. The diagnostics considered are the ratio of hydrogen-like to helium-like line complexes of Z greater than or = 14 elements, line-free portions of the continuum, and the entire spectrum. While fits to the entire spectrum yield the highest statistical precision, it is argued that fits to the line-free continuum are less susceptible to atomic data uncertainties but lead to a modest increase in statistical uncertainty over full spectral fits. Temperatures deduced from line ratios can have similar accuracy, but only over a narrow range of temperatures. Convenient estimates of statistical accuracies for the various temperature diagnostics are provided which may be used in planning ASCA SIS observations.

Swartz, Douglas A.↗
NASA NTRS2013

Achieving Climate Change Absolute Accuracy in Orbit

The Climate Absolute Radiance and Refractivity Observatory (CLARREO) mission will provide a calibration laboratory in orbit for the purpose of accurately measuring and attributing climate change. CLARREO measurements establish new climate change benchmarks with high absolute radiometric accuracy and high statistical confidence across a wide range of essential climate variables. CLARREO's inherently high absolute accuracy will be verified and traceable on orbit to Système Internationale (SI) units. The benchmarks established by CLARREO will be critical for assessing changes in the Earth system and climate model predictive capabilities for decades into the future as society works to meet the challenge of optimizing strategies for mitigating and adapting to climate change. The CLARREO benchmarks are derived from measurements of the Earth's thermal infrared spectrum (5-50 micron), the spectrum of solar radiation reflected by the Earth and its atmosphere (320-2300 nm), and radio occultation refractivity from which accurate temperature profiles are derived. The mission has the ability to provide new spectral fingerprints of climate change, as well as to provide the first orbiting radiometer with accuracy sufficient to serve as the reference transfer standard for other space sensors, in essence serving as a "NIST [National Institute of Standards and Technology] in orbit." CLARREO will greatly improve the accuracy and relevance of a wide range of space-borne instruments for decadal climate change. Finally, CLARREO has developed new metrics and methods for determining the accuracy requirements of climate observations for a wide range of climate variables and uncertainty sources. These methods should be useful for improving our understanding of observing requirements for most climate change observations.

Wielicki, Bruce A.↗
NASA NTRS1972

Statistical versus nonstatistical temperature inversion methods

Vertical temperature profiles are derived from radiation measurements by inverting the integral equation of radiative transfer. Because of the nonuniqueness of the solution, the particular temperature profile obtained depends on the numerical inversion technique used and the type of auxiliary information incorporated in the solution. The choice of an inversion algorithm depends on many factors; including the speed and size of computer, the availability of representative statistics, and the accuracy of initial data. Results are presented for a numerical study comparing two contrasting inversion methods: the statistical-matrix inversion method and the nonstatistical-iterative method. These were found to be the most applicable to the problem of determining atmospheric temperature profiles. Tradeoffs between the two methods are discussed.

Smith, W. L.↗
NASA NTRS1986

Turbulent compressible convection in a deep atmosphere. III - Tests on the validity and limitation of the numerical approach

The effects of various approximations on the results of three-dimensional computation of turbulent compressible convection are examined and related to the phenomenon of bubble breaking in stellar convection. The problem of viscosity and the numerical issue of efficiency versus accuracy in such computations are discussed, and a model is formulated and tested with regard to turbulent viscosity and accuracy. Results on the convergence to a statistically stationary state, the effects of time accuracy on the statistics, the effects of the adjustable constant parameter c(mu), and the effects of aspect ratio, horizontal grid size, and vertical grid distribution are discussed. A preliminary look at the three-dimensional flow structure is given.

Chan, K. L.↗
NASA NTRS1991

Systematic methods for knowledge acquisition and expert system development

Nine cooperating rule-based systems, collectively called AUTOCREW which were designed to automate functions and decisions associated with a combat aircraft's subsystems, are discussed. The organization of tasks within each system is described; performance metrics were developed to evaluate the workload of each rule base and to assess the cooperation between the rule bases. Simulation and comparative workload results for two mission scenarios are given. The scenarios are inbound surface-to-air-missile attack on the aircraft and pilot incapacitation. The methodology used to develop the AUTOCREW knowledge bases is summarized. Issues involved in designing the navigation sensor selection expert in AUTOCREW's NAVIGATOR knowledge base are discussed in detail. The performance of seven navigation systems aiding a medium-accuracy INS was investigated using Kalman filter covariance analyses. A navigation sensor management (NSM) expert system was formulated from covariance simulation data using the analysis of variance (ANOVA) method and the ID3 algorithm. ANOVA results show that statistically different position accuracies are obtained when different navaids are used, the number of navaids aiding the INS is varied, the aircraft's trajectory is varied, and the performance history is varied. The ID3 algorithm determines the NSM expert's classification rules in the form of decision trees. The performance of these decision trees was assessed on two arbitrary trajectories, and the results demonstrate that the NSM expert adapts to new situations and provides reasonable estimates of the expected hybrid performance.

Belkin, Brenda L.↗
DOE OSTI2024

Statistically-driven Experimental Design to Improve Reference-free Quantification of Small Molecules by Liquid Chromatography-Mass Spectrometry

Non-targeted analysis of small molecules and metabolites in unknown, complex samples using liquid chromatography-tandem mass spectrometry remains challenging. One of the main bottlenecks is the extensive unannotated regions of metabolomics mass spectrometry data, resulting in knowledge gaps. Small molecule annotation in mass spectrometry data has conventionally relied on reference standards and libraries for compound identification and confirmation, which can constrain compound identification to those molecules already known, thus limiting the ability to discover new knowledge and new markers. Retention time prediction can facilitate and expedite unknown compound identification in non-targeted analysis of complex metabolomics samples. Additionally, accurate retention time predictions can also inform sample mixture design for LC-MS/MS analyses. However, current machine learning-based methods for retention time prediction are typically developed for specific chromatographic platforms and are not generalizable across scales. And while technologies and methods to improve reference-free metabolite identification for more comprehensive annotation of unknowns has received much attention, development of the same for quantitation without reference standards has been much more limited, despite its importance in toxicological, environmental, food safety, forensics, and clinical applications. We believe that a reference-free quantitation strategy that exploits mass spectrometry data already collected for reference-free identification can provide much more insight on unknowns, and move the metabolomics field for more complete unknowns characterization. As such, we pursue two efforts to improve upon current state-of-the-art methods in non-targeted analysis: (1) machine learning-based retention time prediction and (2) statistical design of experiments framework for reference-free quantitation. In this work, we develop and demonstrate (1) a generalizable retention time prediction capability across chromatographic conditions and scales, and (2) a statistical design-based framework for response factor contribution elucidation and reference-free quantitation. Evaluation of our retention time prediction model, PrediToR, showed approximately 24% improvement over current models, and we observed approximately 10X improvement in concentration estimation accuracy from our statistical design-based response factor model over a primarily ionization efficiency-based model. We expect that future efforts to improve upon these new capabilities will further advance non-targeted analysis of small molecules towards truly reference-free metabolomics.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2017

Nominal 30-M Cropland Extent Map of Continental Africa by Integrating Pixel-Based and Object-Based Algorithms Using Sentinel-2 and Landsat-8 Data on Google Earth Engine

A satellite-derived cropland extent map at high spatial resolution (30-m or better) is a must for food and water security analysis. Precise and accurate global cropland extent maps, indicating cropland and non-cropland areas, is a starting point to develop high-level products such as crop watering methods (irrigated or rainfed), cropping intensities (e.g., single, double, or continuous cropping), crop types, cropland fallows, as well as assessment of cropland productivity (productivity per unit of land), and crop water productivity (productivity per unit of water). Uncertainties associated with the cropland extent map have cascading effects on all higher-level cropland products. However, precise and accurate cropland extent maps at high spatial resolution over large areas (e.g., continents or the globe) are challenging to produce due to the small-holder dominant agricultural systems like those found in most of Africa and Asia. Cloud-based Geospatial computing platforms and multi-date, multi-sensor satellite image inventories on Google Earth Engine offer opportunities for mapping croplands with precision and accuracy over large areas that satisfy the requirements of broad range of applications. Such maps are expected to provide highly significant improvements compared to existing products, which tend to be coarser in resolution, and often fail to capture fragmented small-holder farms especially in regions with high dynamic change within and across years. To overcome these limitations, in this research we present an approach for cropland extent mapping at high spatial resolution (30-m or better) using the 10-day, 10 to 20-m, Sentinel-2 data in combination with 16-day, 30-m, Landsat-8 data on Google Earth Engine (GEE). First, nominal 30-m resolution satellite imagery composites were created from 36,924 scenes of Sentinel-2 and Landsat-8 images for the entire African continent in 2015-2016. These composites were generated using a median-mosaic of five bands (blue, green, red, near-infrared, NDVI) during each of the two periods (period 1: January-June 2016 and period 2: July-December 2015) plus a 30-m slope layer derived from the Shuttle Radar Topographic Mission (SRTM) elevation dataset. Second, we selected Cropland/Non-cropland training samples (sample size 9791) from various sources in GEE to create pixel-based classifications. As supervised classification algorithm, Random Forest (RF) was used as the primary classifier because of its efficiency, and when over-fitting issues of RF happened due to the noise of input training data, Support Vector Machine (SVM) was applied to compensate for such defects in specific areas. Third, the Recursive Hierarchical Segmentation (RHSeg) algorithm was employed to generate an object-oriented segmentation layer based on spectral and spatial properties from the same input data. This layer was merged with the pixel-based classification to improve segmentation accuracy. Accuracies of the merged 30-m crop extent product were computed using an error matrix approach in which 1754 independent validation samples were used. In addition, a comparison was performed with other available cropland maps as well as with LULC maps to show spatial similarity. Finally, the cropland area results derived from the map were compared with UN FAO statistics. The independent accuracy assessment showed a weighted overall accuracy of 94, with a producers accuracy of 85.9 (or omission error of 14.1), and users accuracy of 68.5 (commission error of 31.5) for the cropland class. The total net cropland area (TNCA) of Africa was estimated as 313 Mha for the nominal year 2015.

Cropland mapping; cropland areas; 30-m; Landsat-8;↗
DOE OSTI2022

Inadequacy of current approaches for characterizing membrane transport properties at high salinities

Cost optimal design of osmotic membrane processes requires an accurate estimate of membrane transport parameters across their full operational range. However, standard approaches for estimating these parameters rely on empirical methods, the accuracy of which remains unquantified as a function of temperature, salinity, and measurement error. Herein, we present a systematic accuracy analysis of previously developed methods for estimation of membrane transport properties in reverse osmosis, high-pressure reverse osmosis, forward osmosis, pressure retarded reverse osmosis, and osmotically assisted reverse osmosis. We use a Monte Carlo approach to sample the full range of feasible membrane water permeabilities, salt permeabilities, structural parameters, and operating conditions for these processes. These material and process parameters are then incorporated into a physical transport model for each process. Our analysis shows that the statistical uncertainty of current empirical methods for estimating membrane parameters increases by 5 times from low-salinity to high-salinity conditions. Here, the result of this work demonstrates that empirical methods are inadequate for precisely estimating membrane transport properties at high salinity and highlight a critical need for the development of statistically validated higher accuracy methods.

42 ENGINEERING↗