Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “missing values”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2004

Clustering with Missing Values: No Imputation Required

Clustering algorithms can identify groups in large data sets, such as star catalogs and hyperspectral images. In general, clustering methods cannot analyze items that have missing data values. Common solutions either fill in the missing values (imputation) or ignore the missing data (marginalization). Imputed values are treated as just as reliable as the truly observed data, but they are only as good as the assumptions used to create them. In contrast, we present a method for encoding partially observed features as a set of supplemental soft constraints and introduce the KSC algorithm, which incorporates constraints into the clustering process. In experiments on artificial data and data from the Sloan Digital Sky Survey, we show that soft constraints are an effective way to enable clustering with missing values.

constraints↗
NASA NTRS2004

Making the most of missing values : object clustering with partial data in astronomy

We demonstrate a clustering analysis algorithm, KSC, that a) uses all observed values and b) does not discard the partially observed objects. KSC uses soft constraints defined by the fully observed objects to assist in the grouping of objects with missing values. We present an analysis of objects taken from the Sloan Digital Sky Survey to demonstrate how imputing the values can be misleading and why the KSC approach can produce more appropriate results.

clustering↗
NASA NTRS1996

Packaging and distributing ecological data from multisite studies

Studies of global change and other regional issues depend on ecological data collected at multiple study areas or sites. An information system model is proposed for compiling diverse data from dispersed sources so that the data are consistent, complete, and readily available. The model includes investigators who collect and analyze field measurements, science teams that synthesize data, a project information system that collates data, a data archive center that distributes data to secondary users, and a master data directory that provides broader searching opportunities. Special attention to format consistency is required, such as units of measure, spatial coordinates, dates, and notation for missing values. Often data may need to be enhanced by estimating missing values, aggregating to common temporal units, or adding other related data such as climatic and soils data. Full documentation, an efficient data distribution mechanism, and an equitable way to acknowledge the original source of data are also required.

Information Systems↗
NASA NTRS2015

Cara Status and Upcoming Enhancements

RIC Miss Values in Summary TableTabular presentation of miss vector in Summary Section RIC Uncertainty Values in Details SectionNumerical presentation of miss component uncertainty values in Details SectionGreen Events with Potentially Maneuverable Secondary ObjectsAll potentially maneuverable secondary objects will be reported out to 7-days prior to TCA for LEO events and 10-days for NONLEO events, regardless of risk (relates to MOWG Action Item 1309-11) All green events with potentially active secondary objects included in Summary ReportsAllows more time for contacting other OOBlack Box FixSometimes a black square appeared in the summary report where the ASW RIC time history plot should beAppendix Orbit RegimeMission Name MismatchPc 0 Plotting BugAll Pc points less than 1e-10 (zero) are now plotted as 1e-10 (instead of not at all)Maneuver Indication FixManeuver indicator now present even if maneuver was in the past.

Operations↗
NASA NTRS1988

Two dimensional convolute integers for machine vision and image recognition

Machine vision and image recognition require sophisticated image processing prior to the application of Artificial Intelligence. Two Dimensional Convolute Integer Technology is an innovative mathematical approach for addressing machine vision and image recognition. This new technology generates a family of digital operators for addressing optical images and related two dimensional data sets. The operators are regression generated, integer valued, zero phase shifting, convoluting, frequency sensitive, two dimensional low pass, high pass and band pass filters that are mathematically equivalent to surface fitted partial derivatives. These operators are applied non-recursively either as classical convolutions (replacement point values), interstitial point generators (bandwidth broadening or resolution enhancement), or as missing value calculators (compensation for dead array element values). These operators show frequency sensitive feature selection scale invariant properties. Such tasks as boundary/edge enhancement and noise or small size pixel disturbance removal can readily be accomplished. For feature selection tight band pass operators are essential. Results from test cases are given.

Edwards, Thomas R.↗
NASA NTRS2019

Predictive Modeling for Differential Diagnosis and Mortality Risk Assessment

The prevalence of electronic health record (EHR) systems has brought prodigious biomedical informatics opportunity. Automated machine learning methods can effectively utilize such data and have become common tools for healthcare predictive modeling. Researches in medical informatics have explored the potential of deep learning and classical models in emergent care scenarios. In particular, predicting differential diagnoses for admissions have proven useful in decreasing unnecessary lab tests and improving inpatient triage decision-making. Moreover, identification of high-risk patients for in-hospital mortality is vitally important to maximize allocation of medical resources.The Medical Information Mart for Intensive Care (MIMIC-III) database, containing de-identified critical care inpatient was used in our study. This data set captures hospital patient laboratory measurements, pharmacologic prescriptions, diagnostic data and procedure event recordings. When considering adult patients and discounting admissions with ICU length of stay less than 24 hours, there were 37,787 unique admissions and 30,414 total patients. We examined the top 25 most prevalent ICD-9 group-level disease specificities in MIMIC-III using a multi-label classification model. In-hospital mortality was modeled as binary classification with 4,155 (13%) adult patients that expired, of which 3,138 (75.5%) were in the ICU setting. The metrics AUC, F1 score, sensitivity and specificity values calculated for each disease label measured prediction performance.The usage of ICD-9 group codes reduced feature dimension from 14,567 to 942 and greatly improved distribution of patient diagnostic categories. Disease temporal patterns were captured by considering the most frequently sampled 6 vital signs and 13 laboratory values. Missing data were imputed at each time-stamp. Time-series raw hourly average values were converted into 5 summary features (mean, standard deviation, number of observations, min & max values). Patient demographic variables such as age, gender, marital status and ethnicity were also factored into the modeling. Choi et al showed that contextual embedding of medical data, diagnostic and procedural codes alone can predict future diagnoses with sensitivity as high as 0.79. We utilized an embedding technique called word2vec which allowed sparse representations of medical history to be transformed into dense word vectors. The mappings captured contextual information by treating each admission as a sentence and learning the most likely neighboring words in a sliding window fashion. Binary and multi-label classification was achieved via collapse models, which do not consider temporal information, as well as recurrent neural networks with regularization, Softmax output layer activation together with categorical cross-entropy as the loss function.

US Army collaboration↗
NASA NTRS1992

Incomplete fuzzy data processing systems using artificial neural network

In this paper, the implementation of a fuzzy data processing system using an artificial neural network (ANN) is discussed. The binary representation of fuzzy data is assumed, where the universe of discourse is decartelized into n equal intervals. The value of a membership function is represented by a binary number. It is proposed that incomplete fuzzy data processing be performed in two stages. The first stage performs the 'retrieval' of incomplete fuzzy data, and the second stage performs the desired operation on the retrieval data. The method of incomplete fuzzy data retrieval is proposed based on the linear approximation of missing values of the membership function. The ANN implementation of the proposed system is presented. The system was computationally verified and showed a relatively small total error.

Patyra, Marek J.↗
NASA NTRS2010

Confidence-Based Feature Acquisition

Confidence-based Feature Acquisition (CFA) is a novel, supervised learning method for acquiring missing feature values when there is missing data at both training (learning) and test (deployment) time. To train a machine learning classifier, data is encoded with a series of input features describing each item. In some applications, the training data may have missing values for some of the features, which can be acquired at a given cost. A relevant JPL example is that of the Mars rover exploration in which the features are obtained from a variety of different instruments, with different power consumption and integration time costs. The challenge is to decide which features will lead to increased classification performance and are therefore worth acquiring (paying the cost). To solve this problem, CFA, which is made up of two algorithms (CFA-train and CFA-predict), has been designed to greedily minimize total acquisition cost (during training and testing) while aiming for a specific accuracy level (specified as a confidence threshold). With this method, it is assumed that there is a nonempty subset of features that are free; that is, every instance in the data set includes these features initially for zero cost. It is also assumed that the feature acquisition (FA) cost associated with each feature is known in advance, and that the FA cost for a given feature is the same for all instances. Finally, CFA requires that the base-level classifiers produce not only a classification, but also a confidence (or posterior probability).

Wagstaff, Kiri L.↗
NASA NTRS2010

Land Surface Phenology from MODIS: Characterization of the Collection 5 Global Land Cover Dynamics Product

Information related to land surface phenology is important for a variety of applications. For example, phenology is widely used as a diagnostic of ecosystem response to global change. In addition, phenology influences seasonal scale fluxes of water, energy, and carbon between the land surface and atmosphere. Increasingly, the importance of phenology for studies of habitat and biodiversity is also being recognized. While many data sets related to plant phenology have been collected at specific sites or in networks focused on individual plants or plant species, remote sensing provides the only way to observe and monitor phenology over large scales and at regular intervals. The MODIS Global Land Cover Dynamics Product was developed to support investigations that require regional to global scale information related to spatiotemporal dynamics in land surface phenology. Here we describe the Collection 5 version of this product, which represents a substantial refinement relative to the Collection 4 product. This new version provides information related to land surface phenology at higher spatial resolution than Collection 4 (500-m vs. 1-km), and is based on 8-day instead of 16-day input data. The paper presents a brief overview of the algorithm, followed by an assessment of the product. To this end, we present (1) a comparison of results from Collection 5 versus Collection 4 for selected MODIS tiles that span a range of climate and ecological conditions, (2) a characterization of interannual variation in Collections 4 and 5 data for North America from 2001 to 2006, and (3) a comparison of Collection 5 results against ground observations for two forest sites in the northeastern United States. Results show that the Collection 5 product is qualitatively similar to Collection 4. However, Collection 5 has fewer missing values outside of regions with persistent cloud cover and atmospheric aerosols. Interannual variability in Collection 5 is consistent with expected ranges of variance suggesting that the algorithm is reliable and robust, except in the tropics where some systematic differences are observed. Finally, comparisons with ground data suggest that the algorithm is performing well, but that end of season metrics associated with vegetation senescence and dormancy have higher uncertainties than start of season metrics.

Land cover dynamics↗
NASA NTRS2018

MODIS Collection 6 MAIAC Algorithm

This paper describes the latest version of the algorithm MAIAC (Multi-Angle Implementation of Atmospheric Correction) used for processing the MODIS (Moderate-resolution Imaging Spectroradiometer) Collection6 data record. Since initial publication in 2011-2012, MAIAC has changed considerably to adapt to global processing and improve cloud/snow detection, aerosol retrievals and atmospheric correction of MODIS data. The main changes include (1) transition from a 25 to 1 km scale for retrieval of the spectral regression coefficient (SRC) which helped to remove occasional blockiness at 25 km scale in the aerosol optical depth (AOD) and in the surface reflectance, (2) continuous improvements of cloud detection, (3) introduction of smoke and dust tests to discriminate absorbing fine- and coarse mode aerosols, (4) adding over-water processing, (5) general optimization of the LUT (LookUp-Table)-based radiative transfer for the global processing, and others. MAIAC provides an interdisciplinary suite of atmospheric and land products, including cloud mask (CM), column water vapor (CWV), AOD at 0.47 and 0.55 m, aerosol type (background, smoke or dust) and fine-mode fraction over water; spectral bidirectional reflectance factors (BRF), parameters of Ross-thick Lisparse (RTLS) bidirectional reflectance distribution function (BRDF) model and instantaneous albedo. For snow-covered surfaces, we provide subpixel snow fraction and snow grain size. All products come in standard HDF4 (software library) format at 1 km resolution, except for BRF, which is also provided at 500 m resolution on a sinusoidal grid adopted by the MODIS Land team. All products are provided on per-observation basis in daily files except for the BRDF/Albedo product, which is reported every 8 days. Because MAIAC uses a time series approach, BRDF/Albedo is naturally gap-filled over land where missing values are filled-in with results from the previous retrieval. While the BRDF model is reported for MODIS Land bands 1-7 and ocean band 8, BRF is reported for both land and ocean bands 1-12. This paper focuses on MAIAC cloud detection, aerosol retrievals and atmospheric correction and describes MCD19 data products and quality assurance (QA) flags.

MAIAC Algorithm↗
NASA NTRS2022

MLtool: Universal Supervised Machine Learning Tool to Model Tabulated Data

Machine Learning (ML) is a subfield of Artificial Intelligence that gives computers the ability to learn from past data without being explicitly programmed. The predictive capabilities of ML models have already been used to facilitate several scientific breakthroughs. However, the practical application of ML is often limited due to the gaps in technical knowledge of its users. The common issue faced by many scientific researchers is the inability to choose the appropriate ML pipelines that are needed to treat real-world data, which is often sparse and noisy. To solve this problem, we have developed an automated Machine Learning tool (MLtool) that includes a set of ML algorithms and approaches to aid scientific researchers. The current version of MLtool is implemented as an object-oriented Python code that is easily extensible. It includes 44 different regression algorithms used to model data. MLtool helps users select the best model for their data, based on the scoring metrics used. Besides regression algorithms, MLtool also includes a suite of pre- and post-processing techniques such as missing value imputation, categorical variable encoding, input feature normalization, uncertainty quantification, exploratory data analysis (EDA), etc. MLtool was tested on several publicly available multi-dimensional data sets and was found capable of making accurate predictions.

Machine learning↗
NASA NTRS2022

MLtool Python Code

Machine Learning (ML) is a subfield of Artificial Intelligence that gives computers the ability to learn from past data without being explicitly programmed. The predictive capabilities of ML models have already been used to facilitate several scientific breakthroughs. However, the practical application of ML is often limited due to the gaps in technical knowledge of its users. The common issue faced by many scientific researchers is the inability to choose the appropriate ML pipelines that are needed to treat real-world data, which is often sparse and noisy. To solve this problem, we have developed an automated Machine Learning tool (MLtool) that includes a set of ML algorithms and approaches to aid scientific researchers. The current version of MLtool is implemented as an object-oriented Python code that is easily extensible. It includes 44 different regression algorithms used to model data. MLtool helps users select the best model for their data, based on the scoring metrics used. Besides regression algorithms, MLtool also includes a suite of pre- and post-processing techniques such as missing value imputation, categorical variable encoding, input feature normalization, uncertainty quantification, exploratory data analysis (EDA), etc. MLtool was tested on several publicly available multi-dimensional data sets and was found capable of making accurate predictions.

Machine Learning↗
NASA NTRS2014

Merging of OMI and AIRS Ozone Data

The OMI Instrument measures ozone using the backscattered light in the UV part of the spectrum. In polar night there are no OMI measurements so we hope to incorporate the AIRS ozone data to fill in these missing regions. AIRS is on the Aqua platform and has been operating since May 2002. AIRS is a multi-detector array grating spectrometer containing 2378 IR channels between 650 per centimeter and 2760 per centimeter which measures atmospheric temperature, precipitable water, water vapor, CO, CH4, CO2 and ozone profiles and column amount. It can also measure effective cloud fraction and cloud top pressure for up to two cloud layers and sea-land skin temperature. Since 2008, OMI has had part of its aperture occulted with a piece of the thermal blanket resulting in several scan positions being unusable. We hope to use the AIRS data to fill in the missing ozone values for those missing scan positions.

OMI↗
NASA NTRS1987

A measurement-based study of concurrency in a multiprocessor

A systematic measurement-based methodology for characterizing the amount of concurrency present in a workload, and the effect of concurrency on system performance indices such as cache miss rate and bus activity are developed. Hardware and software instrumentation of an Alliant FX/8 was used to obtain data from a real workload environment. Results show that 35% of the workload is concurrent, with the concurrent periods typically using all available processors. Measurements of periods of change in concurrency show uneven usage of processors during these times. Other system measures, including cache miss rate and processor bus activity, are analyzed with respect to the concurrency measures. Probability of a cache miss is seen to increase with concurrency. The change in cache miss rate is much more sensitive to the fraction of concurrent code in the worklaod than the number of processors active during concurrency. Regression models are developed to quantify the relationships between cache miss rate, bus activity, and the concurrency measures. The model for cache miss rate predicts an increase in the median miss rate value as much as 300% for a 100% increase in concurrency in the workload.

Mcguire, Patrick John↗
NASA NTRS1994

The VIS-AD data model: Integrating metadata and polymorphic display with a scientific programming language

The VIS-AD data model integrates metadata about the precision of values, including missing data indicators and the way that arrays sample continuous functions, with the data objects of a scientific programming language. The data objects of this data model form a lattice, ordered by the precision with which they approximate mathematical objects. We define a similar lattice of displays and study visualization processes as functions from data lattices to display lattices. Such functions can be applied to visualize data objects of all data types and are thus polymorphic.

Hibbard, William L.↗
NASA NTRS1999

Dreams, Hopes, Realities: NASA's Goddard Space Flight Center, the First Forty Years

Throughout history, the great achievements of civilizations and cultures have been recorded in lists of dates and events. But to look only at the machinery, discoveries, or milestones is to miss the value of these achievements. Each goal achieved or discovery or made represents a supreme effort on the part of individual people who came and worked together for a purpose greater than themselves. Driven by an innate curiosity of the spirit, we have built civilizations and discovered new worlds, always reaching out beyond what we knew or thought was possible. These efforts may have used ships or machinery, but the achievement was that of the humans who made those machines possible- remarkable people willing to endure discomfort, frustration, fatigue, and the risk of failure in the hope of finding out something new. This is the case with the history of the Goddard Space Flight Center. This publication traces the legacy of successes, risks, disappointments and internationally recognized triumphs of the Center's first 40 years. It is a story of technological achievement and scientific discovery; of reaching back to the dawn of time and opening up a new set of eyes on our own planet Earth. In the end, it is not a story about machinery or discoveries, but a story about ourselves. If we were able to step off our planet, and if we continue to discover new mysteries and better technology, it is because the people who work at Goddard always had a passion for exploration and the dedication to make it happen. The text that follows is a testimony to the challenges people at the Goddard Space Flight Center have faced and overcome over almost half a century. Today, we stand on the threshold of a new and equally challenging era. It will once again test our ingenuity, skills, and flexibility as we find new ways of working with our colleagues in industry, government, and academia. Doing more with less is every bit as ambitious as designing the first science instrument to study the heavens. But if we are to continue exploring our world and our universe, it is every bit as important. Robert H. Goddard once said, "The dream of yesterday is the hope of today and the reality of tomorrow." This is our heritage. Our challenge is to keep our spirit of dedication, vision, and innovative thinking alive, so we can turn today's dreams into a new century of possibility and progress. Our journeys into space are the greatest ongoing adventure the human race has ever undertaken, and everyone here has played an important role in that endeavor. This book is about everyone who has worked at the Goddard Space Flight Center.

Wallace, Lane E.↗