Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Classification bias”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS1999

TRMM Precipitation Radar Reflectivity Profiles Compared to High-Resolution Airborne and Ground-Based Radar Measurements

In this paper, TRMM (Tropical Rainfall Measuring Mission Satellite) Precipitation Radar (PR) products are evaluated by means of simultaneous comparisons with data from the high-altitude ER-2 Doppler Radar (EDOP), as well as ground-based radars. The comparison is aimed primarily at the vertical reflectivity structure, which is of key importance in TRMM rain type classification and latent heating estimation. The radars used in this study have considerably different viewing geometries and resolutions, demanding non-trivial mapping procedures in common earth-relative coordinates. Mapped vertical cross sections and mean profiles of reflectivity from the PR, EDOP, and ground-based radars are compared for six cases. These cases cover a stratiform frontal rainband, convective cells of various sizes and stages, and a hurricane. For precipitating systems that are large relative to the PR footprint size, PR reflectivity profiles compare very well to high-resolution measurements thresholded to the PR minimum reflectivity, and derived variables such as bright band height and rain types are accurate, even at high PR incidence angles. It was found that for, the PR reflectivity of convective cells small relative to the PR footprint is weaker than in reality. Some of these differences can be explained by non-uniform beam filling. For other cases where strong reflectivity gradients occur within a PR footprint, the reflectivity distribution is spread out due to filtering by the PR antenna illumination pattern. In these cases, rain type classification may err and be biased towards the stratiform type, and the average reflectivity tends to be underestimated. The limited sensitivity of the PR implies that the upper regions of precipitation systems remain undetected and that the PR storm top height estimate is unreliable, usually underestimating the actual storm top height. This applies to all cases but the discrepancy is larger for smaller cells where limited sensitivity is compounded by incomplete beam filling. Users of level three TRMM PR products should be aware of this scale dependency.

Heymsfield, G. M.↗
NASA NTRS1990

Bias and design in software specifications

Implementation bias in a specification is an arbitrary constraint in the solution space. Presented here is a model of bias in software specifications. Bias is defined in terms of the specification process and a classification of the attributes of the software product. Our definition of bias provides insight into both the origin and the consequences of bias. It also shows that bias is relative and essentially unavoidable. Finally, we describe current work on defining a measure of bias, formalizing our model, and relating bias to software defects.

Straub, Pablo A.↗
NASA NTRS2018

CALIPSO Lidar Level 3 Aerosol Profile Product: Version 3 Algorithm Design

The CALIPSO (Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations) level 3 aerosol profile product reports globally gridded, quality-screened, monthly mean aerosol extinction profiles retrieved by CALIOP (the Cloud-Aerosol Lidar with Orthogonal Polarization). This paper describes the quality screening and averaging methods used to generate the version 3 product. The fundamental input data are CALIOP level 2 aerosol extinction profiles and layer classification information (aerosol, cloud, and clear-air). Prior to aggregation, the extinction profiles are quality-screened by a series of filters to reduce the impact of layer detection errors, layer classification errors, extinction retrieval errors, and biases due to an intermittent signal anomaly at the surface. The relative influence of these filters are compared in terms of sample rejection frequency, mean extinction, and mean aerosol optical depth (AOD). The “extinction QC flag” filter is the most influential in preventing high-biases in level 3 mean extinction, while the “misclassified cirrus fringe” filter is most aggressive at rejecting cirrus misclassified as aerosol. The impact of quality screening on monthly mean aerosol extinction is investigated globally and regionally. After applying quality filters, the level 3 algorithm calculates monthly mean AOD by vertically integrating the monthly mean quality-screened aerosol extinction profile. Calculating monthly mean AOD by integrating the monthly mean extinction profile prevents a low bias that would result from alternately integrating the set of extinction profiles first and then averaging the resultant AOD values together. Ultimately, the quality filters reduce level 3 mean AOD by −24 and −31 % for global ocean and global land, respectively, indicating the importance of quality screening.

Tackett, Jason L.↗
NASA NTRS1979

Sampling for area estimation - A comparison of full-frame sampling with the sample segment approach

The objective of this investigation was to evaluate the effect of sampling on the accuracy (precision and bias) of crop area estimates made from classifications of Landsat MSS data. Full-frame classifications of wheat and non-wheat for eighty counties in Kansas were repetitively sampled to simulate alternative sampling plans. Four sampling schemes involving different numbers of samples and different size sampling units were evaluated. The precision of the wheat area estimates increased as the segment size decreased and the number of segments was increased. Although the average bias associated with the various sampling schemes was not significantly different, the maximum absolute bias was directly related to sampling unit size.

Hixson, M. M.↗
NASA NTRS1979

Sampling for area estimation: A comparison of full-frame sampling with the sample segment approach

The effect of sampling on the accuracy (precision and bias) of crop area estimates made from classifications of LANDSAT MSS data was investigated. Full-frame classifications of wheat and non-wheat for eighty counties in Kansas were repetitively sampled to simulate alternative sampling plants. Four sampling schemes involving different numbers of samples and different size sampling units were evaluated. The precision of the wheat area estimates increased as the segment size decreased and the number of segments was increased. Although the average bias associated with the various sampling schemes was not significantly different, the maximum absolute bias was directly related to sampling unit size.

Hixson, M. M.↗
DOE OSTI2024

Addressing bias in bagging and boosting regression models

As artificial intelligence (AI) becomes widespread, there is increasing attention on investigating bias in machine learning (ML) models. Previous research concentrated on classification problems, with little emphasis on regression models. This paper presents an easy-to-apply and effective methodology for mitigating bias in bagging and boosting regression models, that is also applicable to any model trained through minimizing a differentiable loss function. Our methodology measures bias rigorously and extends the ML model's loss function with a regularization term to penalize high correlations between model errors and protected attributes. We applied our approach to three popular tree-based ensemble models: a random forest model (RF), a gradient-boosted model (GBT), and an extreme gradient boosting model (XGBoost). We implemented our methodology on a case study for predicting road-level traffic volume, where RF, GBT, and XGBoost models were shown to have high accuracy. Despite high accuracy, the ML models were shown to perform poorly on roads in minority-populated areas. Our bias mitigation approach reduced minority-related bias by over 50%.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1974

An automated and repeatable data analysis procedure for remote sensing applications

A new multispectral data analysis procedure, based on LARSYS, has been developed which substantially reduces the influence of the analyst. The analysis is automated, including the interpretation of clustering results. The classification results obtained are repeatable and not biased by analyst subjectivity during the analysis.

Davis, B. J.↗
NASA NTRS2018

NASA Guidelines for Promoting Scientific and Research Integrity

The National Aeronautics and Space Administration (NASA) seeks to explore and expand human knowledge of the Earth, solar system, and universe and to enable the development of aeronautical and space exploration systems. NASA's ability to achieve these purposes depends on the integrity of the research and technology activities the agency conducts and supports. The NASA workforce - as well as all external entities who review proposals for or receive NASA funding - must maintain the highest standards of scientific integrity. These responsibilities include selecting the most meritorious research activities through open and fair competition, peer review and other appropriate merit review processes, and avoidance of actual and perceived conflicts of interest; avoiding fabrication, falsification, or plagiarism in proposing, performing, or reviewing research, or in reporting research results; openly sharing results and methods not subject to classification or privacy standards; disclosing assumptions and biases in sharing and applying scientific information and data; and acting honestly and transparently in using and serving on advisory committees and in engaging in professional development activities.

Guidelines↗
DOE OSTI2024

Evaluation of data collection bias of third molar stages of mineralisation for age estimation in the living

Abstract Age assessment of the living is a fundamental procedure in the process of human identification, in order to guarantee fair treatment of individuals, which has ethical, civil, legal, and medical repercussions. The careful selection of the appropriate methods requires evaluation of several parameters: accuracy, precision of the method, as well as its reproducibility. The approach proposed by Mincer et al. adapted from Demirjian et al. exploring third molar mineralisation, is one of the most frequently considered for age estimation of the living. Thus, this work aims to assess potential bias in the data collection when applying the classification stages for dental mineralisation adapted by Mincer et al. A total of 102 orthopantomographs, of clinical origin, belonging to individuals aged between 12 and 25 years ($ \bar{\textit x} $ = 20.12 years, SD = 3.49 years; 65 females, 37 males, all of Portuguese nationality) were included and a retrospective analysis performed by five observers with different levels of experience (high, average, and basic). The performance and agreement between five observers were evaluated using Weighted Cohen’s Kappa and the Intraclass Correlation Coefficient. To access the influence of impaction on third molar classification, variables were tested using ordinal logistic regression Generalised Linear Model. It was observed that there were variations in the number of teeth identified among the observers, but the agreement levels ranged from moderate to substantial (0.4–0.8). Upon closer examination of the results, it was observed that although there were discernible differences between highly experienced observers and those with less experience, the gap was not as significant as initially hypothesised, and a greater disparity between the classifications of the upper (0.24–0.49) and lower third molars (>0.55) was observed. When bone superimposition is present, the classification process is not significantly influenced; however, variation in teeth angulation affects the assessment. The results suggest that with an efficient preparation, the level of experience as a factor can be overcome. Mincer and colleague's classification system can be replicated with ease and consistency, even though the classification of upper and lower third molars presents distinct challenges.

de Oliveira Santos, Inês (ORCID:0000000267324347)↗
DOE OSTI2023

Characteristics and selection of near-fault simulated earthquake ground motions for nonlinear analysis of buildings

Earthquake-induced ground shaking near rupturing faults is highly sensitive to the rupture characteristics, seismic wave propagation patterns and site conditions, and field recordings of near-fault shaking are relatively sparse. These challenges complicate the assessment of the seismic performance of near-fault structures. A common approach to representing near-fault ground motion in engineering analysis is to explicitly consider and select records with strong directivity pulses (pulse records). We use three-dimensional high-resolution physics-based earthquake simulations to test this approach in the context of scenario-based ground motion record selection, and to study the important characteristics of near-fault ground shaking. We highlight the deficiencies associated with classifying near-fault simulated records as “pulse” or “non-pulse,” based on the presence of a single dominating pulse in the velocity time history. We show that this approach is inadequate for characterizing near-fault shaking on soft soils which can be dominated by both forward rupture directivity and basin amplification effects. We conduct ground motion selection experiments for the analysis of near-fault structures with and without explicit classification of the pulse features in the records, and evaluate the bias in the predicted structural demands. We find that the maximum interstory drift demands on building structures imposed by unscaled site-specific simulated ground motion records selected based on relevant spectral shape features are not sensitive to the classification of records as pulse/non-pulse. Therefore, with regard to predicting the maximum interstory drifts in near-fault buildings, we do not find justification for the binary pulse classification of near-fault records.

42 ENGINEERING↗
NASA NTRS1976

Large Area Crop Inventory Experiment (LACIE). Review of LACIE methodology, a project evaluation of technical acceptability

The author has identified the following significant results. Results indicated that the LANDSAT data and the classification technology can estimate the small grains area within a sample segment accurately and reliably enough to meet the LACIE goals. Overall, the LACIE estimates in a 9 x 11 kilometer segment agree well with ground and aircraft determined area within these segments. The estimated c.v. of the random classification error was acceptably small. These analyses confirmed that bias introduced by various factors, such as LANDSAT spatial resolution, lack of spectral resolution, classifier bias, and repeatability, was not excessive in terms of the required performance criterion. Results of these tests did indicate a difficulty in differentiating wheat from other closely related small grains. However, satisfactory wheat area estimates were obtained through the reduction of the small grain area estimates in accordance with relative amounts of these crops as determined from historic data; these procedures are being further refined.

Source record↗
DOE OSTI2023

Galaxy Spin Classification. I. Z-wise versus S-wise Spirals with the Chirality Equivariant Residual Network

The angular momentum of galaxies (galaxy spin) contains rich information about the initial condition of the universe, yet it is challenging to efficiently measure the spin direction for the tremendous amount of galaxies that are being mapped by ongoing and forthcoming cosmological surveys. We present a machine-learning-based classifier for the Z-wise versus S-wise spirals, which can help to break the degeneracy in the galaxy spin direction measurement. The proposed chirality equivariant residual network (CE-ResNet) is manifestly equivariant under a reflection of the input image, which guarantees that there is no inherent asymmetry between the Z-wise and S-wise probability estimators. We train the model with Sloan Digital Sky Survey images, with the training labels given by the Galaxy Zoo 1 project. A combination of data augmentation techniques is used during the training, making the model more robust to be applied to other surveys. We find an ~30% increase in both types of spirals when Dark Energy Spectroscopic Instrument (DESI) images are used for classification, due to the better imaging quality of DESI. We verify that the ~7σ difference between the numbers of Z-wise and S-wise spirals is due to human bias, since the discrepancy drops to <1.8σ with our CE-ResNet classification results. We discuss the potential systematics relevant to future cosmological applications.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Jensen–Shannon divergence based novel loss functions for Bayesian neural networks

Bayesian neural networks (BNNs) are state-of-the-art machine learning methods that can naturally regularize and systematically quantify uncertainties using their stochastic parameters. Kullback–Leibler (KL) divergence-based variational inference used in BNNs suffer from unstable optimization and challenges in approximating light-tailed posteriors due to the unbounded nature of the KL divergence. To resolve these issues, we formulate a novel loss function for BNNs based on a new modification to the generalized Jensen–Shannon (JS) divergence, which is bounded. In addition, we propose a Geometric JS divergence-based loss, which is computationally efficient since it can be evaluated analytically. We found that the JS divergence-based variational inference is intractable, and hence employed a constrained optimization framework to formulate these losses. Our theoretical analysis and empirical experiments on multiple regression and classification data sets suggest that the proposed losses perform better than the KL divergence-based loss, especially when the data sets are noisy or biased. Specifically, there are approximately 5% and 8% improvements in accuracy for a noise-added CIFAR-10 dataset and a regression dataset, respectively. There is about 13% reduction in false negative predictions of a biased histopathology dataset. Additionally, we quantify and compare the uncertainty metrics for the regression and classification tasks.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2010

Satellite Sampling and Retrieval Errors in Regional Monthly Rain Estimates from TMI AMSR-E, SSM/I, AMSU-B and the TRMM PR

Passive and active microwave rain sensors onboard earth-orbiting satellites estimate monthly rainfall from the instantaneous rain statistics collected during satellite overpasses. It is well known that climate-scale rain estimates from meteorological satellites incur sampling errors resulting from the process of discrete temporal sampling and statistical averaging. Sampling and retrieval errors ultimately become entangled in the estimation of the mean monthly rain rate. The sampling component of the error budget effectively introduces statistical noise into climate-scale rain estimates that obscure the error component associated with the instantaneous rain retrieval. Estimating the accuracy of the retrievals on monthly scales therefore necessitates a decomposition of the total error budget into sampling and retrieval error quantities. This paper presents results from a statistical evaluation of the sampling and retrieval errors for five different space-borne rain sensors on board nine orbiting satellites. Using an error decomposition methodology developed by one of the authors, sampling and retrieval errors were estimated at 0.25 resolution within 150 km of ground-based weather radars located at Kwajalein, Marshall Islands and Melbourne, Florida. Error and bias statistics were calculated according to the land, ocean and coast classifications of the surface terrain mask developed for the Goddard Profiling (GPROF) rain algorithm. Variations in the comparative error statistics are attributed to various factors related to differences in the swath geometry of each rain sensor, the orbital and instrument characteristics of the satellite and the regional climatology. The most significant result from this study found that each of the satellites incurred negative longterm oceanic retrieval biases of 10 to 30%.

Fisher, Brad↗
DOE OSTI2023

Heterogeneous Graph Neural Network for identifying hadronically decayed tau leptons at the High Luminosity LHC

Here, we present a new algorithm that identifies reconstructed jets originating from hadronic decays of tau leptons against those from quarks or gluons. No tau lepton reconstruction algorithm is used. Instead, the algorithm represents jets as heterogeneous graphs with tracks and energy clusters as nodes and trains a Graph Neural Network to identify tau jets from other jets. Different attributed graph representations and different GNN architectures are explored. We propose to use differential track and energy cluster information as node features and a heterogeneous sequentially-biased encoding for the inputs to final graph-level classification.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Creating ground truth for nanocrystal morphology: a fully automated pipeline for unbiased transmission electron microscopy analysis

Control over colloidal nanocrystal morphology (size, size distribution, and shape) is important for tailoring the functionality of individual nanocrystals and their ensemble behavior. Despite this, traditional methods to quantify nanocrystal morphology are laborious. New developments in automated morphology classification will accelerate these analyses but the assessment of machine learning models is limited by human accuracy for ground truth, causing even unsupervised machine learning models to have inherent bias. Herein, we introduce synthetic image rendering to solve the ground truth problem of nanocrystal morphology classification. By simulating 2D images of nanocrystal shapes via a function of high-dimensional parameter space, we trained a convolutional neural network to link unique morphologies to their simulated parameters, defining nanocrystal morphology quantitatively rather than qualitatively. An automated pipeline then processes, quantitatively defines, and classifies nanocrystal morphology from experimental transmission electron microscopy (TEM) images. Using improved computer vision techniques, 42,650 nanocrystals were identified, assessed, and labeled with quantitative parameters, offering a 600-fold improvement in efficiency over best-practice manual measurements. Further, a classification algorithm was trained with a prediction accuracy of 99.5%, which can successfully analyze a range of concave, convex, and irregular nanocrystal shapes. The resulting pipeline was applied to differentiating two syntheses of nominally cuboidal CsPbBr 3 nanocrystals and uniquely classifying binary nickel sulfide nanocrystal phase based on morphology. This pipeline provides a simple, efficient, and unbiased method to quantify nanocrystal morphology and represents a practical route to construct large datasets with an absolute ground truth for training unbiased morphology-based machine learning algorithms.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
NASA NTRS1984

A comparison of Landsat point and rectangular field training sets for land-use classification

Rectangular training fields of homogeneous spectroreflectance are commonly used in supervised pattern recognition efforts. Trial image classification with manually selected training sets gives irregular and misleading results due to statistical bias. A self-verifying, grid-sampled training point approach is proposed as a more statistically valid feature extraction technique. A systematic pixel sampling network of every ninth row and ninth column efficiently replaced the full image scene with smaller statistical vectors which preserved the necessary characteristics for classification. The composite second- and third-order average classification accuracy of 50.1 percent for 331,776 pixels in the full image substantially agreed with the 51 percent value predicted by the grid-sampled, 4,100-point training set.

Tom, C. H.↗
NASA NTRS2020

Automated Cardiovascular Pathology Assessment using Semantic Segmentation and Ensemble Learning

Cardiac magnetic resonance imaging provides high spatial resolution, enabling improved extraction of important functional and morphological features for cardiovascular disease staging. Segmentation of ventricular cavities and myocardium in cardiac cine sequencing provides a basis to quantify cardiac measures such as ejection fraction. A method is presented that curtails the expense and observer bias of manual cardiac evaluation by combining semantic segmentation and disease classification into a fully automatic processing pipeline. The initial processing element consists of a robust dilated convolutional neural network architecture for voxel-wise segmentation of the myocardium and ventricular cavities. The resulting comprehensive volumetric feature matrix captures diagnostic clinical procedure data and is utilized by the final processing element to model a cardiac pathology classifier. Our approach evaluated anonymized cardiac images from a training data set of 100 patients (4 pathology groups, 1 healthy group, 20 patients per group) examined at the University Hospital of Dijon. The top average Dice index scores achieved were 0.940, 0.886, 0.849 for structure segmentation of the left ventricle (LV), myocardium and right ventricle (RV) respectively. A 5-ary pathology classification accuracy of 90% was recorded on an independent test set using the trained model. Performance results demonstrate potential for advanced machine learning methods to deliver accurate, efficient and reproducible cardiac pathological assessment.

Semantic Segmentation↗