Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Missing data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Comparing Individualized Survival Predictions From Random Survival Forests and Multistate Models in the Presence of Missing Data: A Case Study of Patients With Oropharyngeal Cancer

Background: In recent years, interest in prognostic calculators for predicting patient health outcomes has grown with the popularity of personalized medicine. These calculators, which can inform treatment decisions, employ many different methods, each of which has advantages and disadvantages. Methods: We present a comparison of a multistate model (MSM) and a random survival forest (RSF) through a case study of prognostic predictions for patients with oropharyngeal squamous cell carcinoma. The MSM is highly structured and takes into account some aspects of the clinical context and knowledge about oropharyngeal cancer, while the RSF can be thought of as a black-box non-parametric approach. Key in this comparison are the high rate of missing values within these data and the different approaches used by the MSM and RSF to handle missingness. Results: We compare the accuracy (discrimination and calibration) of survival probabilities predicted by both approaches and use simulation studies to better understand how predictive accuracy is influenced by the approach to (1) handling missing data and (2) modeling structural/disease progression information present in the data. We conclude that both approaches have similar predictive accuracy, with a slight advantage going to the MSM. Conclusions: Although the MSM shows slightly better predictive ability than the RSF, consideration of other differences are key when selecting the best approach for addressing a specific research question. These key differences include the methods’ ability to incorporate domain knowledge, and their ability to handle missing data as well as their interpretability, and ease of implementation. Ultimately, selecting the statistical method that has the most potential to aid in clinical decisions requires thoughtful consideration of the specific goals.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Multitaper Magnitude‐Squared Coherence for Time Series With Missing Data: Understanding Oscillatory Processes Traced by Multiple Observables

To explore the hypothesis of a common source of variability in two time series, observers may estimate the magnitude-squared coherence (MSC), which is a frequency-domain view of the cross correlation. For time series that do not have uniform observing cadence, MSC can be estimated using Welch's overlapping segment averaging. However, multitaper has superior statistical properties to Welch's method in terms of the tradeoff between bias, variance, and bandwidth. The classical multitaper technique has recently been extended to accommodate time series with underlying uniform observing cadence from which some observations are missing. This situation is common for solar and geomagnetic data sets, which may have gaps due to breaks in satellite coverage, instrument downtime, or poor observing conditions. We demonstrate the scientific use of missing-data multitaper magnitude-squared coherence by detecting known solar mid-term oscillations in simultaneous, missing-data time series of solar Lyman α flux and geomagnetic Disturbance Storm Time index. Due to their superior statistical properties, we recommend that multitaper methods be used for all heliospheric time series with underlying uniform observing cadence.

Astro-statistics techniques (1886)↗
DOE OSTI2021

Explaining Missing Data in Graphs: A Constraint-based Approach

Abstract: This paper introduces a constraint-based approach to clarify missing values in graphs. Our method capitalizes on a set S of graph data constraints. An explanation is a sequence of operational enforcement of S towards the recovery of interested yet missing data (e.g., attribute values, edges). We show that constraint-based approach helps us to understand not only why a value is missing, but also how to recover the missing value. We study S-explanation problem, which is to compute the optimal explanations with guarantees on the informativeness and conciseness. We show the problem is in ?P^2 for established graph data constraints such as graph keys and graph association rules. We develop an efficient bidirectional algorithm to compute optimal explanations, without enforcing S on the entire graph. We also show our algorithm can be easily extended to support graph refinement within limited time, and to explain missing answers. Using real-world graphs, we experimentally verify the effectiveness and efficiency of our algorithms.

Data Analytics↗
DOE OSTI2020

Reliable machine prognostic health management in the presence of missing data

Prognostics and health management enables the prediction of future degradation and remaining useful life (RUL) for in-service systems based on historical and contemporary data, showing promise for many practical applications. One major challenge for prognostics is the common occurrence of missing values in time-series data, often caused by disruptions in sensor communication or hardware/software failures. Another major concern is that the sufficient prior knowledge of critical component degradation with a clear failure threshold is often not readily available in practice. These issues can significantly hinder the application of advanced signal and data analysis methods and consequently degrade the health management performance. In this article, we propose a novel data-driven framework that is capable of providing accurate and reliable predictions of degradation and RUL. In this approach, one-hot health state indicators are appended to the historical time series so that the model learns end-of-life automatically. A modified gate recurrent unit based variational autoencoder is employed in generative adversarial networks to model the temporal irregularity of the incomplete time series. Furthermore, experiments on multivariate time-series datasets collected from real-world aeroengines verify that significant performance improvement can be achieved using the proposed model for robust long-term prognostics.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
NASA NTRS2020

Continued Discussion of Failure Mode Modeling and Overall Component Reliability: Are the Data Missing or Censored?

This paper is the continuation of a paper presented at the 13th Probabilistic Safety Assessment and Management Conference, in which a methodology of modeling failure modes of complex components was presented; see Paulos and Smith (2016). This methodology is not particularly helpful in the space industry where there is a lack of failure data, but is more helpful in industries that see a lot of component repairs and improvements, such as in the aircraft or automotive industries. The previous paper demonstrated how the typical approach of treating failure modes as being exponential in nature may yield optimistic predictions when estimating how improvements to components will perform in the future. It is more accurate to model the failure modes as a race in time; unfortunately, this does not give a closed-form solution. This paper uses simulation to solve for the model of the world, and the results compared to the standard methodology of treating the failure modes as being exponential random failures. The standard method is shown to have optimistic predictions, which will lead to prediction errors when failure modes are removed or “fixed.” The failure mode methodology presented in the first paper treated the data as being censored when the test stopped. In this paper, we will compare the results from treating the data as both censored and missing.

Smith, Curtis↗
DOE OSTI2020

Continued Discussion of Failure Mode Modeling and Overall Component Reliability: Is the Data Missing or Censored?

This paper is a continuation of a paper presented at the 13th Probabilistic Safety Assessment and Management Conference, in which a discussion of modeling failure modes of complex components was presented; see Paulos and Smith (2016). The paper demonstrated how the typical method of treating failure modes as being exponential in nature may yield optimistic predictions when estimating how improvements to components will perform in the real world. Instead of relying on textbook methods, it is more accurate to model the failure modes as a race in time; unfortunately, this does not give a closed-form solution. This work uses a simulation to solve for the model of the world, and is compared to the standard methodology of treating the failure modes as exponential random failures. The standard method is shown to have optimistic predictions, which will lead to prediction errors when failure modes were removed or “fixed.” The failure mode methodology presented in the first paper treated the data as being censored when the test stopped. In this paper, we will compare the results from treating the data as both censored and missing, and comparisons will be shown showing which method is more accurate and under what conditions.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1980

On the existence, uniqueness, and asymptotic normality of a consistent solution of the likelihood equations for nonidentically distributed observations: Applications to missing data problems

A general theorem is given which establishes the existence and uniqueness of a consistent solution of the likelihood equations given a sequence of independent random vectors whose distributions are not identical but have the same parameter set. In addition, it is shown that the consistent solution is a MLE and that it is asymptotically normal and efficient. Two applications are discussed: one in which independent observations of a normal random vector have missing components, and the other in which the parameters in a mixture from an exponential family are estimated using independent homogeneous sample blocks of different sizes.

Peters, C.↗
NASA NTRS2007

Reducing a Knowledge-Base Search Space When Data Are Missing

This software addresses the problem of how to efficiently execute a knowledge base in the presence of missing data. Computationally, this is an exponentially expensive operation that without heuristics generates a search space of 1 + 2n possible scenarios, where n is the number of rules in the knowledge base. Even for a knowledge base of the most modest size, say 16 rules, it would produce 65,537 possible scenarios. The purpose of this software is to reduce the complexity of this operation to a more manageable size. The problem that this system solves is to develop an automated approach that can reason in the presence of missing data. This is a meta-reasoning capability that repeatedly calls a diagnostic engine/model to provide prognoses and prognosis tracking. In the big picture, the scenario generator takes as its input the current state of a system, including probabilistic information from Data Forecasting. Using model-based reasoning techniques, it returns an ordered list of fault scenarios that could be generated from the current state, i.e., the plausible future failure modes of the system as it presently stands. The scenario generator models a Potential Fault Scenario (PFS) as a black box, the input of which is a set of states tagged with priorities and the output of which is one or more potential fault scenarios tagged by a confidence factor. The results from the system are used by a model-based diagnostician to predict the future health of the monitored system.

James, Mark↗
DOE OSTI2017

Driving missing data at the LHC: NNLO predictions for the ratio of γ + j and Z + j

In this paper we present a calculation of the $\gamma+j$ process at next-to-next-to-leading order (NNLO) in QCD and compare the resulting predictions to 8 TeV CMS data. We find good agreement with the shape of the photon $p_T$ spectrum, particularly after the inclusion of additional electroweak corrections, but there is a tension between the overall normalization of the theoretical prediction and the measurement. We use our results to compute the ratio of $Z(\to \ell^+\ell^-)+j$ to $\gamma+j$ events as a function of the vector boson transverse momentum at NNLO, a quantity that is used to normalize $Z(\rightarrow\nu\overline{\nu}) +j$ backgrounds in searches for dark matter and supersymmetry. Our NNLO calculation significantly reduces the theoretical uncertainty on this ratio, thus boosting its power for future searches of new physics.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2018

Bayesian Hierarchical Models for Parameter Inference with Missing Data: Supernova Cosmology Case Study [Slides]

Large scale astronomical surveys are going wider and deeper than ever before. However, astronomers, cosmologists and theorists continue to face the perennial issue that their data sets are often incomplete in magnitude space and must be carefully treated in order to avoid Malmquist bias, especially in the field of supernova cosmology. Historically, cosmological parameter inference in supernova cosmology was done using χ2 methodology; however, recent years have seen a rise in the use of Bayesian Hierarchical Models. In this presentation we develop a Bayesian Hierarchical methodology to account for magnitude limited surveys and present a specific application to cosmological parameter inference and model selection in supernova cosmology.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS2008

Using an Informative Missing Data Model to Predict the Ability to Assess Recovery of Balance Control after Spaceflight

Astronauts show degraded balance control immediately after spaceflight. To assess this change, astronauts' ability to maintain a fixed stance under several challenging stimuli on a movable platform is quantified by "equilibrium" scores (EQs) on a scale of 0 to 100, where 100 represents perfect control (sway angle of 0) and 0 represents data loss where no sway angle is observed because the subject has to be restrained from falling. By comparing post- to pre-flight EQs for actual astronauts vs. controls, we built a classifier for deciding when an astronaut has recovered. Future diagnostic performance depends both on the sampling distribution of the classifier as well as the distribution of its input data. Taking this into consideration, we constructed a predictive ROC by simulation after modeling P(EQ = 0) in terms of a latent EQ-like beta-distributed random variable with random effects.

Feiveson, Alan H.↗