Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Missing data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
NASA NTRS2009

Assessment of the Short-Term Radiometric Stability between Terra MODIS and Landsat 7 ETM+ Sensors

The Landsat 7 (L7) Enhanced Thematic Mapper (ETM+) sensor was launched on April 15th, 1999 and has been in operation for over nine years. It has six reflective solar spectral bands located in the visible and shortwave infrared part of the electromagnetic spectrum (0.5 - 2.5 micron) at a spatial resolution of 30 m. The on-board calibrators are used to monitor the on-orbit sensor system changes. The ETM+ performs solar calibrations using on-board Full Aperture Solar Calibrator (FASC) and the Partial Aperture Solar Calibrator (PASC). The Internal Calibrator Lamp (IC) lamps, a blackbody and shutter optics constitute the on-orbit calibration mechanism for ETM+. On 31 May 2003, a malfunction of the scan-line corrector (SLC) mirror assembly resulted in the loss of approximately 22% of the normal scene area. The missing data affects most of the image with scan gaps varying in width from one pixel or less near the centre of the image to 14 pixels along the east and west edges of the image, creating a wedge-shaped pattern. However, the SLC failure has no impacts on the radiometric performance of the valid pixels. On December 18, 1999, the Moderate Resolution Imaging Spectroradiometer (MODIS) Proto-Flight Model (PFM) was launched on-board the NASA's EOS Terra spacecraft. Terra MODIS has 36 spectral bands with wavelengths ranging from 0.41 to 14.5 micron and collects data over a wide field of view angle (+/-55 deg) at three nadir spatial resolutions of 250 m, 500 in 1 km for bands 1 to 2, 3 to 7, and 8 to 36, respectively. It has 20 reflective solar bands (RSB) with spectral wavelengths from 0.41 to 2.1 micron. The RSB radiometric calibration is performed by using on-board solar diffuser (SD), solar diffuser stability monitor (SDSM), space-view (SV), and spectro-radiometric calibration assembly (SRCA). Through the SV port, periodic lunar observations are used to track radiometric response changes at different angles of incidence (AOI) of the scan mirror. As a part of the AM Constellation satellites, Terra MODIS flies approximately 30 minutes behind L7 ETM+ in the same orbit. The orbit of L7 is repetitive, circular, sunsynchronous, and near polar at a nominal altitude of 705 km (438 miles) at the Equator. The spacecraft crosses the Equator from north to south on a descending node between 10:00 AM and 10:15 AM. Circling the Earth at 7.5 km/sec, each orbit takes nearly 99 minutes. The spacecraft completes just over 14 orbits per day, covering the entire Earth between 81 degrees north and south latitude every 16 days. The longest continuous imaging swath that L7 sensor can collect is for a 14-minute subinterval contact period which is equivalent to 35 full WRS-2 scenes. On the other hand, Terra can provide the entire corresponding orbit with wider swath at any given ETM+ collection without contact time limitation. There are six spectral matching band pairs between MODIS (bands 3, 4, 1, 2, 6, 7) and ETM+ (bands 1, 2, 3, 4, 5, 7) sensor. MODIS has narrower spectral responses than ETM+ in all the bands. A short-term radiometric stability was evaluated using continuous ETM+ scenes within the contact period and the corresponding half orbit MODIS scenes. The near simultaneous earth observations (SNO) were limited by the smaller swath size of ETM+ (187 km) as compared to MODIS (2330 km). Two sets of continuous granules for MODIS and ETM+ were selected and mosaiced based on pixel geolocation information for non cloudy pixels over the North American continent. The Top-of- Atmosphere (TOA) reflectances were computed for the spectrally matching bands between ETM+ and MODIS over the regions of interest (ROI). The matching pixel pairs were aggregated from a finer to a coarser pixel resolution and the TOA reflectance values covering a wide dynamic range of the sensors were compared and analyzed. Considering the uncertainties of the absolute calibration of the both sensors, radiometric stability was verified for the band pairs. The Railroad Valley Playa, Nada (RVPN) was included in the path of this continuous orbit, which served as a verification point between the shortterm and the long-term trending results from previous studies. This work focuses on monitoring the short-term on-orbit stability of MODIS and the ETM+ RSB. It also provides an assessment of the absolute calibration differences between the two sensors over their wide dynamic ranges.

Choi, Taeyoung↗
NASA NTRS2017

Relationship Between Carbon Dioxide Levels and Reported Congestion and Headaches on the International Space Station

Congestion is commonly reported during spaceflight, and most crewmembers have reported using medications for congestion during International Space Station (ISS) missions. Although congestion has been attributed to fluid shifts during spaceflight, fluid status reaches equilibrium during the first week after launch while congestion continues to be reported throughout long duration missions. Congestion complaints have anecdotally been reported in relation to ISS CO2 levels; this evaluation was undertaken to determine whether or not an association exists. METHODS: Reported headaches, congestion symptoms, and CO2 levels were obtained for ISS expeditions 2-31, and time-weighted means and single-point maxima were determined for 24-hour (24hr) and 7-day (7d) periods prior to each weekly private medical conference. Multiple imputation addressed missing data, and logistic regression modeled the relationship between probability of reported event of congestion or headache and CO2 levels, adjusted for possible confounding covariates. The first seven days of spaceflight were not included to control for fluid shifts. Data were evaluated to determine the concentration of CO2 required to maintain the risk of congestion below 1% to allow for direct comparison with a previously published evaluation of CO2 concentrations and headache. RESULTS: This study confirmed a previously identified significant association between CO2 and headache and also found a significant association between CO2 and congestion. For each 1-mm Hg increase in CO2, the odds of a crew member reporting congestion doubled. The average 7-day CO2 would need to be maintained below 1.5 mmHg to keep the risk of congestion below 1%. The predicted probability curves of ISS headache and congestion curves appear parallel when plotted against ppCO2 levels with congestion occurring at approximately 1mmHg lower than a headache would be reported. DISCUSSION: While the cause of congestion is multifactorial, this study showed congestion is associated with CO2 levels on ISS. Data from additional expeditions could be incorporated to further assess this finding. CO2 levels are also associated with reports of headaches on ISS. While it may be expected for astronauts with congestion to also complain of headaches, these two symptoms are commonly mutually exclusive. Furthermore, it is unknown if a temporal CO2 relationship exists between congestion and headache on ISS. CO2 levels were time-weighted for 24hr and 7d, and thus the time course of congestion leading to headache was not assessed; however, congestion could be an early CO2-related symptom when compared to headache. Future studies evaluating the association of CO2-related congestion leading to headache would be difficult due to the relatively stable daily CO2 levels on ISS currently, but a systematic study could be implemented on-orbit if desired.

Cole, Robert↗
NASA NTRS2019

Computational Bayesian Methods Applied to Complex Problems in Bio and Astro Statistics

In this dissertation we apply computational Bayesian methods to three distinct problems. In the first chapter, we address the issue of unrealistic covariance matrices used to estimate collision probabilities. We model covariance matrices with a Bayesian Normal-Inverse-Wishart model, which we fit with Gibbs sampling. In the second chapter, we are interested in determining the sample sizes necessary to achieve a particular interval width and establish non-inferiority in the analysis of prevalences using two fallible tests. To this end, we use a third order asymptotic approximation. In the third chapter, we wish to synthesize evidence across multiple domains in measurements taken longitudinally across time, featuring a substantial amount of structurally missing data, and fit the model with Hamiltonian Monte Carlo in a simulation to analyze how estimates of a parameter of interest change across sample sizes.

Elrod, Chris↗
NASA NTRS2021

Satellite Remote Sensing to Assess Cyanobacterial Bloom Frequency Across the United States at Multiple Spatial Scales

Cyanobacterial blooms can have negative effects on human health and local ecosystems. Field monitoring of cyanobacterial blooms can be costly, but satellite remote sensing has shown utility for more efficient spatial and temporal monitoring across the United States. Here, satellite imagery was used to assess the annual frequency of surface cyanobacterial blooms, defined for each satellite pixel as the percentage of images for that pixel throughout the year exhibiting detectable cyanobacteria. Cyanobacterial frequency was assessed across 2,196 large lakes in 46 states across the continental United States (CONUS) using imagery from the European Space Agency’s Ocean and Land Colour Imager for the years 2017 through 2019. In 2019, across all satellite pixels considered, annual bloom frequency had a median value of 4% and a maximum value of 100%, the latter indicating that for those satellite pixels, a cyanobacterial bloom was detected by the satellite sensor for every satellite image considered. In addition to annual pixel-scale cyanobacterial frequency, results were summarized at the lake- and state-scales by averaging annual pixel-scale results across each lake and state. For 2019, average annual lake-scale frequencies also had a maximum value of 100%, and Oregon and Ohio had the highest average annual state-scale frequencies at 65% and 52%. Pixel-scale frequency results can assist in identifying portions of a lake that are more prone to cyanobacterial blooms, while lake- and state-scale frequency results can assist in the prioritization of sampling resources and mitigation efforts. Satellite imagery is limited by the presence of snow and ice, as imagery collected in these conditions are quality flagged and discarded. Thus, annual bloom frequencies within nine climate regions were investigated to determine whether missing data biased results in climate regions more prone to snow and ice, given that their annual summaries would be weighted toward the summer months when cyanobacterial blooms tend to occur. Results were unbiased by the time period selected in most climate regions, but a large bias was observed for the Northwest Rockies and Plains climate region. Moderate biases were observed for the Ohio Valley and the Southeast climate regions. Finally, a clustering analysis was used to identify areas of high and low cyanobacterial frequency across CONUS based on average annual lake-scale cyanobacterial frequencies for 2019. Several clusters were identified that transcended state, watershed, and eco-regional boundaries. Combined with additional data, results from the clustering analysis may offer insight regarding large-scale drivers of cyanobacterial blooms.

remote sensing↗
NASA NTRS2021

Understanding Pilot Breathing – A Case Study in Systems Engineering

This position paper assesses the Pilot Breathing Assessment’s driving requirement to collect missing data on pilot breathing pressure, flow, rate, oxygen and CO2, simultaneously with aircraft data. Repeat measures data were collected over a 2-year period from 5 NASA test-pilots flying 8 flight profiles on NASA F-15 and F/A-18 (A, B) aircraft. This Pilot Breathing Assessment Systems Engineering (PiBASE) position paper serves as a case-study of systems engineering that accommodates the human as a distinct part of the whole. It acknowledges that a successful machine design intended for human use must include human systems biology/physiology as an integral part of planning, design, and implementation.

Pilot Breathing Assessment Systems Engineering↗
DOE OSTI2015

Nuclear Forensics Analysis with Missing and Uncertain Data

We have applied a new imputation-based method for analyzing incomplete data, called Monte Carlo Bayesian Database Generation (MCBDG), to the Spent Fuel Isotopic Composition (SFCOMPO) database. About 60% of the entries are absent for SFCOMPO. The method estimates missing values of a property from a probability distribution created from the existing data for the property, and then generates multiple instances of the completed database for training a machine learning algorithm. Uncertainty in the data is represented by an empirical or an assumed error distribution. The method makes few assumptions about the underlying data, and compares favorably against results obtained by replacing missing information with constant values.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Imputing data that are missing at high rates using a boosting algorithm

Traditional multiple imputation approaches may perform poorly for datasets with high rates of missingness unless many m imputations are used. This paper implements an alternative machine learning-based approach to imputing data that are missing at high rates. Here, we use boosting to create a strong learner from a weak learner fitted to a dataset missing many observations. This approach may be applied to a variety of types of learners (models). The approach is demonstrated by application to a spatiotemporal dataset for predicting dengue outbreaks in India from meteorological covariates. A Bayesian spatiotemporal CAR model is boosted to produce imputations, and the overall RMSE from a k-fold cross-validation is used to assess imputation accuracy.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Statistical Performance of Forced Oscillation Detectors in the Presence of Missing Measurements

In bulk power systems, measurement-based monitoring for large oscillations can help maintain system reliability. One of the challenges encountered in a recent field demonstration was the unavailability of measurements due to underlying measurement quality or communication problems. During the demonstration, the oscillation detector ignored a measurement location if even 10 seconds of data was missing. To extend the detector's ability to operate in these conditions, this paper evaluates the impact of three methods for addressing missing data. The strengths and weaknesses of each approach are evaluated using theoretical expressions for the probability of detection along with results from simulated data and publicly available field measurements. Based on these results, a suitable approach is identified that can extend the oscillation detector's performance when large segments of data are missing.

Follum, James D.↗
DOE OSTI2026

Satellite Embedding-Based Population Imputation for Areas with Missing Building Footprint Data: A Computer Vision-Based Approach

High-resolution population modeling is important for supporting effective decision-making across diverse sectors. LandScan Mosaic generates population estimates at the level of individual buildings and aggregates them to 3 arc-second grids, and this approach performs well in regions where building footprint data are comprehensive and reliable. However, large portions of the globe still suffer from incomplete, sparse, or entirely missing building stock datasets, creating a structural limitation for strictly building-based population models. To address this research gap, this study proposes a computer vision-based framework that employs Google Earth Engine satellite embeddings and UNet, which allows us to directly impute grid-level population estimates in building-data-deficient areas. Applied to Taiwan as a case study, the framework achieved strong predictive performance with R$^{2}$ of 0.89, RMSE of 18.70, and MAE of 8.41, outperforming traditional machine learning approaches. Notably, the proposed framework effectively addressed building false-positive errors inherent in Global Human Settlement Layer (GHSL) data, correctly identifying uninhabited areas that were erroneously classified as populated. The framework also offers significant advantages for global population mapping, particularly in terms of scalability and temporal consistency, thereby extending the coverage and accuracy of high-resolution population products in data-scarce regions worldwide. Urban planners, decision makers, and related stakeholders can obtain granular population distributions to support more accurate and targeted infrastructure investment, service delivery, resource allocation, and risk assessment decisions.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2019

Atomic Oxygen Erosion Data from the MISSE 2-8 Missions

Polymers and other oxidizable materials on the exterior of spacecraft in the low Earth orbit (LEO) space environment can be eroded from reaction with atomic oxygen (AO). Therefore, in order to design durable spacecraft it is important to know the extent of erosion that will occur during a mission. This can be determined by knowing the LEO AO erosion yield, E(sub y) (volume loss per incident oxygen atom), of materials susceptible to AO reaction. In addition, recent flight experiments have shown that the AO E(sub y) can vary with the AO fluence and/or solar exposure. Therefore obtaining AO E(sub y) data for materials flown on various spaceflight missions is important. NASA Glenn Research Center has flown numerous experiments as part of the Materials International Space Station Experiment (MISSE) missions on the exterior of the International Space Station to characterize the LEO E(sub y) of polymers, composites, protective coatings, and other spacecraft materials. This report provides a summary of the erosion data for ram samples from six Glenn polymer experiments flown as part of MISSE 2, 4, 6, 7, and 8. A total of 71 types of materials with 111 E(sub y) values are provided. The E(sub y) values for uncoated polymers range from 3.81×10(exp –27) cu cm/atom for DC 93-500 silicone exposed to an AO fluence of 4.62×10(exp 21) atoms/sq cm on MISSE 8 to 9.14×10(exp –24) cu cm/atom for polyoxymethylene (POM) exposed to an AO fluence of 8.43×10(exp 21) atoms/sq cm on MISSE 2. One polymer, Triton oxygen resistant, low modulus (TOR(TM) LM), experienced mass gain when exposed to an AO fluence of 2.15×10(exp 21) atoms/sq cm on MISSE 4. In many cases the same material was flown on numerous missions so that trends for E(sub y) versus AO fluence and/or solar exposure can be determined, along with temperature effects.

Erosion↗
DOE OSTI2019

Evaluating the Effects of Missing Values and Mixed Data Types on Social Sequence Clustering Using t-SNE Visualization

© 2019 Association for Computing Machinery. The goal of this work is to investigate the impact of missing values in clustering joint categorical social sequences. Identifying patterns in sociodemographic longitudinal data is important in a number of social science settings. However, performing analytical operations, such as clustering on life course trajectories, is challenging due to the categorical and multidimensional nature of the data, their mixed data types, and corruption by missing and inconsistent values. Data quality issues were investigated previously on single variable sequences. To understand their effects on multivariate sequence analysis, we employ a dataset of mixed data types and missing values, a dissimilarity measure designed for joint categorical sequence data, together with dimensionality reduction methodologies in a systematic design of sequence clustering experiments. Given the categorical nature of our data, we employ an “edit” distance using optimal matching. Because each data record has multiple variables of different types, we investigate the impact of mixing these variables in a single dissimilarity measure. Between variables with binary values and those with multiple nominal values, we find that the ability to overcome missing data problems is more difficult in the nominal domain than in the binary domain. Additionally, alignment of leading missing values can result in systematic biases in dissimilarity matrices and subsequently introduce both artificial clusters and unrealistic interpretations of associated data domains. We demonstrate the usage of t-distributed stochastic neighborhood embedding to visually guide mitigation of such biases by tuning the missing value substitution cost parameter or determining an optimal sequence span.

Lazar, Alina↗
NASA NTRS2016

Erosion Data from the MISSE 8 Polymers Experiment After 2 Years of Space Exposure on the International Space Station

The Polymers Experiment was exposed to the low Earth orbit (LEO) space environment for 2.14 and 2.0 years as part of the Materials International Space Station Experiment 8 (MISSE 8) and the Optical Reflector Materials Experiment-III (ORMatE-III), respectively. The experiment contained 42 samples, which were flown in either ram, wake, or zenith orientations. The primary objective was to determine the effect of solar exposure on the atomic oxygen erosion yield (Ey) of fluoropolymers. This paper provides an overview of the experiment with details on the polymers flown, the characterization techniques used, the atomic oxygen fluence for each exposure orientation, and the LEO Ey results. The Ey values for the fluoropolymers range from 1.45 x 10(exp -25) cm(exp 3)/atom for white Tedlar Registered Trademark (polyvinyl fluoride with white titanium dioxide pigment) flown in the ram orientation to 6.32 x 10(exp -24) cm(exp 3)/atom for aluminized-Teflon Registered Trademark fluorinated ethylene propylene (Al-FEP) flown in the zenith orientation. Erosion yield data for FEP flown in ram, wake and zenith orientations are compared, and the Ey was found to be highly dependent on orientation, hence environmental exposure. Teflon FEP had an order of magnitude higher Ey when flown in the zenith direction (6.32 x10(exp -24) cm(exp3)/atom) as compared to the ram direction (2.37 x 10(exp -25) cm(exp 3)/atom). The Ey of FEP was found to increase with a direct correlation to the solar exposure/AO fluence ratio showing the effect of solar radiation and/or heating due to solar exposure on FEP erosion. In addition, back-surface carbon painted FEP (C-FEP) flown in the zenith orientation had a significantly higher Ey than clear FEP or Al-FEP further indicating that heating has a significant impact on the erosion of FEP, particularly in the zenith orientation.

Solar radiation↗
NASA NTRS2001

XTE Observations of the Intraday Variable BL LAC S5 0716+714 in a coordinated Multiwavelength Study

S5 0716+714 was observed with the Rossi X-ray Timing Explorer (RXTE) between 6 and 22 April 1996 as part of a multifrequency monitoring campaign for this source observed to show extreme intra-day variability (IDV) at a number of frequencies. There were 24 distinct epochs of observation ('views') with the RXTE Proportional Counter Array (PCA), each lasting on average 3-4 ks. There were a number of technical problems with the data. During view 10 no Proportional Counter Unit (PCU) was on. A gain change (from gain epoch 2 to gain epoch 3) took place during view 11, on 15 April 1996. No data were received for view 18. Views 10, 11 and 18 have been omitted from our analysis. There were two distinct pointings of the telescope in views 4 and 17 (all the other observations have only one pointing), and in many of the pointings the data are broken by periods where the high voltage was turned off during South Atlantic Anomaly (SAA) passages and Earth occultations. Originally no filter file was available for views 21 and 22, and for the second pointing of view 17, because housekeeping data were missing, but these data were supplied when the observations were reprocessed in early 1998. For most of the remaining observations only three PCUs were on; the exceptions are views 23, 24 (four PCUs) and 17, 19, 20, 21 and 22 (five PCUs). All working PCUs have been analyzed. The third part of the first pointing of view 17 was found to be very short and is discarded from our analysis. The second part of observation 2 is pointed far off-axis and is discarded.

Worrall, D. M.↗
DOE OSTI2025

Data and scripts from: “Denoising autoencoder for reconstructing sensor observation data and predicting evapotranspiration: noisy and missing values repair and uncertainty quantification”

This data package includes data and scripts from the manuscript “Denoising autoencoder for reconstructing sensor observation data and predicting evapotranspiration: noisy and missing values repair and uncertainty quantification”.The study addressed common challenges faced in environmental sensing and modeling, including uncertain input data, missing sensor observations, and high-dimensional datasets with interrelated but redundant variables. Point-scaled meteorological and soil sensor observations were perturbed with noises and missing values, and denoising autoencoder (DAE) neural networks were developed to reconstruct the perturbed data and further predict evapotranspiration. This study concluded that (1) the reconstruction quality of each variable depends on its cross-correlation and alignment to the underlying data structure, (2) uncertainties from the models were overall stronger than those from the data corruption, and (3) there was a tradeoff between reducing bias and reducing variance when evaluating the uncertainty of the machine learning models.This package includes:(1) Four ipython scripts (.ipynb): “DAE_train.ipynb” trains and evaluates DAE neural networks, “DAE_predict.ipynb” makes predictions from the trained DAE models, “ET_train.ipynb” trains and evaluates ET prediction neural networks, and “ET_predict.ipynb” makes predictions from trained ET models.(2) One python file (.py): “methods.py” includes all user-defined functions and python codes used in the ipython scripts.(3) A “sub_models” folder that includes five trained DAE neural networks (in pytorch format, .pt), which could be used to ingest input data before being fed to the downstream ET models in ‘ET_train.ipynb” or ‘ET_predict.ipynb’.(4) Two data files (.csv). Daily meteorological, vegetation, and soil data is in “df_data.csv”, where “df_meta.csv” contains the location and time information of “df_data.csv”. Each row (index) in “df_meta.csv” corresponds to each row in “df_data.csv”. These data files are formatted to follow the data structure requirements and be directly used in the ipython scripts, and they have been shuffled chronologically to train machine learning models. The meteorological and soil data was collected using point sensors between 2019-2023 at(4.a) Three shrub-dominated field sites in East River, Colorado (named “ph1”, “ph2” and “sg5” in “df_meta.csv”, where “ph1” and “ph2” were located at PumpHouse Hillslopes, and “sg5” was at Snodgrass Mountain meadow) and(4.b) One outdoor, mesoscale, and herbaceous-dominated experiment in Berkeley, California (named “tb” in “df_meta.csv”, short for Smartsoils Testbed at Lawrence Berkeley National Lab).- See "df_data_dd.csv" and "df_meta_dd.csv" for variable descriptions and the Methods section for additional data processing steps. See "flmd.csv" and "README.txt" for brief file descriptions.- All ipython scripts and python files are written in and require PYTHON language software.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2014

Measurement of the 12C(e,e'p)11B Two-Body Breakup Reaction at High Missing Momentum Values

The five-fold differential cross section for the 12C(e,e'p)11B reaction was determined over a missing momentum range of 200-400 MeV/c, in a kinematics regime with Bjorken x > 1 and Q2 = 2.0 (GeV/c)2. A comparison of the results and theoretical models and previous lower missing momentum data is shown. The theoretical calculations agree well with the data up to a missing momentum value of 325 MeV/c and then diverge for larger missing momenta. The extracted distorted momentum distribution is shown to be consistent with previous data and extends the range of available data up to 400 MeV/c.

Monaghan, P↗