Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Microstructure Segmentation With Deep Learning Encoders Pre-Trained on a Large Microscopy Dataset

This study examined the improvement of microscopy segmentation intersection over union accuracy by transfer learning from a large dataset of microscopy images called MicroNet. Many neural network encoder architectures were trained on over 100,000 labeled microscopy images from 54 material classes. These pre-trained encoders were then embedded into multiple segmentation architectures including UNet and DeepLabV3+ to evaluate segmentation performance on created benchmark microscopy datasets. Compared to ImageNet pre-training, models pre-trained on MicroNet generalized better to out-of-distribution micrographs taken under different imaging and sample conditions and were more accurate with less training data. When training with only a single Ni-superalloy image, pre-training on MicroNet produced a 72.2% reduction in relative intersection over union error. These results suggest that transfer learning from large in-domain datasets generate models with learned feature representations that are more useful for downstream tasks and will likely improve any microscopy image analysis technique that can leverage pre-trained encoders.

machine learning↗
NASA NTRS2000

A Catchment-Based Approach to Modeling Land Surface Processes in a GCM: Parameter Estimation and Model Demonstration - Part 2

The viability of a new catchment-based land surface model (LSM) developed for use with general circulation models is demonstrated. First, simple empirical functions -- tractable enough for operational use in the LSM -- are established that faithfully capture the control of topography on the subgrid variability of soil moisture and the surface water budget, as predicted by theory. Next, the full LSM is evaluated offline. Using forcing and validation datasets developed for PILPS Phase 2c, the minimally calibrated model is shown to reproduce observed evaporation and runoff fluxes successfully in the Red-Arkansas River Basin. A complementary idealized study that employs the range of topographic variability seen over North America demonstrates that the simulated surface water budget does vary strongly with topography, which can, by itself, induce variations in annual evaporation as high as 20%.

Ducharne, Agnes↗
NASA NTRS2011

A Data Mining Project to Identify Cardiovascular Related Factors That May Contribute to Changes in Visual Acuity Within the US Astronaut Corps

Many of the cardiovascular-related adaptations that occur in the microgravity environment are due, in part, to a well-characterized cephalad-fluid shift that is evidenced by facial edema and decreased lower limb circumference. It is believed that most of these alterations occur as a compensatory response necessary to maintain a "normal" blood pressure and cardiac output while in space. However, data from both flight and analog research suggest that in some instances these microgravity-induced alterations may contribute to cardiovascular-related pathologies. Most concerning is the potential relation between the vision disturbances experienced by some long duration crewmembers and changes in cerebral blood flow and intra-ocular pressure. The purpose of this project was to identify cardiovascular measures that may potentially distinguish individuals at risk for visual disturbances after long duration space flight. Toward this goal, we constructed a dataset from Medical Operation tilt/stand test evaluations pre- (days L-15-L-5) and immediate post-flight (day R+0) on 20 (3 females, 17 males). We restricted our evaluation to only crewmembers who participated in both shuttle and space station missions. Data analysis was performed using both descriptive and analytical methods (Stata 11.2, College Station, TX) and are presented as means +/- 95% CI. Crewmembers averaged 5207 (3447 - 8934) flight hours across both long (MIR-23 through Expedition16) and short (STS-27 through STS-101) duration missions between 1988 and 2008. The mean age of the crew at the time of their most recent shuttle flight was 41 (34-44) compared to 47 (40-54) years during their time on station. In order to focus our analysis (we did not have codes to separate out subjects by symptomotology) , we performed a visual inspection of each cardiovascular measures captured during testing and plotted them against stand time, pre- to post-flight, and between mission duration. It was found that pulse pressure most clearly differentiated the two mission types. Statistical analysis confirmed that pulse pressure was significantly higher before [45.6; (42.1 to 49.1)] and after [50.7; (46.9 to 54.6)] time on station compared with their most recent shuttle flight [31.6 (27.8 to 35.4), and 32.2 (28.3 to 36.0) respectively] even after correcting differences in age and cumulative number of mission hours. Without knowing the identity of which long duration crewmembers demonstrated visual changes, we were limited to examining whether certain crew regulate components of pulse pressure, systolic and diastolic blood pressure, differently due to microgravity exposure. To that end, we stratified crew into tertiles based on either their pre-flight measure of systolic or diastolic blood pressure. Those crew in the highest tertile for both systolic (lower tertile (n=8; 103-111), middle tertile (n=7; 113-121), and upper tertile (n=5; 125-136) and diastolic blood pressure (lower tertile (n=8; 58-64), middle tertile (n=7; 67-73), and upper tertile (n=5; 75-81) demonstrated less variability in pulse pressure between R+0 and L-10 (Figure 2). Interestingly, those crewmembers with the highest resting systolic blood pressure demonstrated either no change or in some instances an increase in total peripheral resistance, where those in the lower tertiles had lower values of total peripheral resistance compared to pre-flight levels. In this study, it was found that crewmembers in the highest tertile for both systolic and diastolic blood pressure demonstrated less variability in pulse pressure and that the decrease in variability was due in part to lower levels of compliance as indicated by similar or higher levels of total peripheral resistance after compared with before flight levels. Whether there is a relation between blood pressure regulation and total peripheral resistance in crew presenting with negative changes in visual acuity remains unknown.

Westby, Christian M.↗
NASA NTRS2014

Application of Suomi-NPP Green Vegetation Fraction and NUCAPS for Improving Regional Numerical Weather Prediction

The NASA SPoRT Center is working to incorporate Suomi‐NPP products into its research and transition activities to improve regional numerical weather prediction (NWP). Specifically, SPoRT seeks to utilize two data products from NOAA/NESDIS: (1) daily global VIIRS green vegetation fraction (GVF), and (2) NOAA Unique CrIS and ATMS Processing System (NUCAPS) temperature and moisture retrieved profiles. The goal of (1) is to improve the representation of vegetation in the Noah land surface model (LSM) over existing climatological GVF datasets in order to improve the land‐atmosphere energy exchanges in NWP models and produce better temperature, moisture, and precipitation forecasts. The goal of (2) is to assimilate NUCAPS retrieved profiles into the Gridpoint Statistical Interpolation (GSI) data assimilation system to assess the impact on a summer pre‐frontal convection case. Most regional NWP applications make use of a monthly GVF climatology for use in the Noah LSM within the Weather Research and Forecasting (WRF) model. The GVF partitions incoming energy into direct surface heating/evaporation over bare soil versus evapotranspiration processes over vegetated surfaces. Misrepresentations of the fractional coverage of vegetation during anomalous weather/climate regimes (e.g., early/late bloom or freeze; drought) can lead to poor NWP model results when land‐atmosphere feedback is important. SPoRT has been producing a daily MODIS GVF product based on the University of Wisconsin Direct Broadcast swaths of Normalized Difference Vegetation Index (NDVI). While positive impacts have been demonstrated in the WRF model for some cases, the reflectances composing these NDVI do not correct for atmospheric aerosols nor satellite view angle, resulting in temporal noisiness at certain locations (especially heavy vegetation). The method behind the NESDIS VIIRS GVF is expected to alleviate the issues seen in the MODIS GVF real‐time product, thereby offering a higher‐quality dataset for modeling applications. SPoRT is evaluating the VIIRS GVF data against the MODIS real‐time and climatology GVF in both WRF and the NASA Land Information System. SPoRT has a history of assimilating hyperspectral infrared retrieved profiles

Numerical Weather Prediction↗
NASA NTRS2016

NASA AVOSS Fast-Time Models for Aircraft Wake Prediction: User's Guide (APA3.8 and TDP2.1)

NASA's current distribution of fast-time wake vortex decay and transport models includes APA (Version 3.8) and TDP (Version 2.1). This User's Guide provides detailed information on the model inputs, file formats, and model outputs. A brief description of the Memphis 1995, Dallas/Fort Worth 1997, and the Denver 2003 wake vortex datasets is given along with the evaluation of models. A detailed bibliography is provided which includes publications on model development, wake field experiment descriptions, and applications of the fast-time wake vortex models.

Ahmad, Nash'at N.↗
NASA NTRS2018

Identification of Security related Bug Reports via Text Mining using Supervised and Unsupervised Classification

This paper is focused on automated classification of software bug reports to security and non-security related, using both supervised and unsupervised approaches. For both approaches, three types of feature vectors are used. For supervised learning, we experiment with multiple learning algorithms and training sets with different sizes. Furthermore, we propose a novel unsupervised approach based on anomaly detection. The evaluated is based on three NASA datasets. The results show that supervised classification is affected more by the learning algorithms than by feature vectors and using only 25% of the data for training provides as good results as if 90% of data are used for training. Both supervised and unsupervised learning can be used for identification of security bug reports; the former slightly outperforms the latter at the expense of labeling the testing set. In general, the performance differs across datasets, mainly due to the different amounts of security related information.

Goseva-Popstojanova, Katerina↗
NASA NTRS2022

Development of a “Nature Run” for Observing System Simulation Experiments (OSSEs) for Snow Mission Development

Snow is a fundamental component of global and regional water budgets, particularly in mountainous areas and regions downstream that rely on snowmelt for water resources. Land surface models (LSMs) are commonly used to develop spatially distributed estimates of snow water equivalent (SWE) and runoff. However, LSMs are limited by uncertainties in model physics and parameters, among other factors. In this study, we describe the use of model calibration tools to improve snow simulations within the Noah-MP LSM as the first step in an observing system simulation experiment (OSSE). Noah-MP is calibrated against the University of Arizona (UA) SWE product over a western Colorado domain. With spatially varying calibrated parameters, we run calibrated and default Noah-MP simulations for water years 2010–20. By evaluating both simulations against the UA dataset, we show that calibration decreases domain averaged temporal RMSE and bias for snow depth from 0.15 to 0.13 m and from −0.036 to −0.0023 m, respectively, and improves the timing of snow ablation. Increased snow simulation performance also improves estimates of model-simulated runoff in four of six study basins, though only one has statistically significant improvement. Spatially distributed Noah-MP snow parameters perform better than default uniform values. We demonstrate that calibrating variables related to snow albedo calculations and rain–snow partitioning, among other processes, is a necessary step for creating a nature run that reasonably approximates true snow conditions for the OSSEs. Additionally, the inclusion of a snowfall scaling term can address biases in precipitation from meteorological forcing datasets, further improving the utility of LSMs for generating reliable spatiotemporal estimates of snow.

Melissa L. Wrzesien↗
NASA NTRS2022

Cloud Mask Intercomparison eXercise (CMIX): An evaluation of cloud masking algorithms for Landsat 8 and Sentinel-2

Cloud cover is a major limiting factor in exploiting time-series data acquired by optical spaceborne remote sensing sensors. Multiple methods have been developed to address the problem of cloud detection in satellite imagery and a number of cloud masking algorithms have been developed for optical sensors but very few studies have carried out quantitative intercomparison of state-of-the-art methods in this domain. This paper summarizes results of the first Cloud Masking Intercomparison eXercise (CMIX) conducted within the Committee Earth Observation Satellites (CEOS) Working Group on Calibration & Validation (WGCV). CEOS is the forum for space agency coordination and cooperation on Earth observations, with activities organized under working groups. CMIX, as one such activity, is an international collaborative effort aimed at intercomparing cloud detection algorithms for moderate-spatial resolution (10–30 m) spaceborne optical sensors. The focus of CMIX is on open and free imagery acquired by the Landsat 8 (NASA/USGS) and Sentinel-2 (ESA) missions. Ten algorithms developed by nine teams from fourteen different organizations representing universities, research centers and industry, as well as space agencies (CNES, ESA, DLR, and NASA), are evaluated within the CMIX. Those algorithms vary in their approach and concepts utilized which were based on various spectral properties, spatial and temporal features, as well as machine learning methods. Algorithm outputs are evaluated against existing reference cloud mask datasets. Those datasets vary in sampling methods, geographical distribution, sample unit (points, polygons, full image labels), and generation approaches (experts, machine learning, sky images). Overall, the performance of algorithms varied depending on the reference dataset, which can be attributed to differences in how the reference datasets were produced. The algorithms were in good agreement for thick cloud detection, which were opaque and had lower uncertainties in their identification, in contrast to thin/semi-transparent clouds detection. Not only did CMIX allow identification of strengths and weaknesses of existing algorithms and potential areas of improvements, but also the problems associated with the existing reference datasets. The paper concludes with recommendations on generating new reference datasets, metrics, and an analysis framework to be further exploited and additional input datasets to be considered by future CMIX activities.

Sergii Skakun↗
NASA NTRS2022

Design of Materials with Alchemite

Machine learning models that establish the relationships between materials processing and properties can enable inverse design of materials through active learning. Alchemite is a commercial software that can perform inverse materials design on sparse data. Here we evaluate Alchemite’s performance on a dataset of shape memory alloys and a dataset of heat exchangers compared to baseline random forest models. Alchemite had higher accuracy when making predictions on sparse data and was more accurate or nearly as accurate as random forests on complete datasets while also quantifying uncertainty. The software was also used to suggest processing steps and design parameters to optimize properties and performance; however, physical validation of the suggested design parameters was beyond the scope of this work. Several useful design insights were gained about the impact of the design parameters on properties and performance including the importance of dopant choice and amount for shape memory alloys and the importance of height and weight on the thermal resistance of heat exchangers.

Machine learning↗
NASA NTRSDate not supplied

Classification of Notices to Airmen using Natural Language Processing

This paper establishes the feasibility of using Natural Language Processing (NLP) to classify NOTAMs or Notices to Airmen – a pilot messaging framework to gather real-time situational awareness. Present day air mobility operations heavily rely on NOTAMs. However, pilots often have difficulty interpreting NOTAMs due to the sheer volume of inapplicable messages and unclear abbreviations. Using NLP, the presented study analyzes the accuracy of classifying NOTAMs and, thereby, the efficiency of generating actionable interpretations in real time. To this effect, efficacies of four NLP neural network architectures were analyzed, including three Recurrent Neural Networks (RNNs) with GloVe, Word2Vec, and FastText word embeddings, and one trained Bi-Directional Encoder Representations from Transformers (BERT) model. The four neural networks were trained and evaluated on three open-source datasets of varying text lengths, vocabularies, and grammars, taken from e-commerce product descriptions, social media tweets, and unstructured descriptions for data and analytics services on open data marketplaces such as NASA’s Data and Reasoning Fabric (DRF) platform. This provided cross-analysis of each neural network architecture’s performance per text type. The best performing architecture, BERT, was then fine-tuned on a collection of open-source NOTAM data. Post-training, a real-time NOTAM classification service was implemented to draw inference on new NOTAMs using the trained model, which demonstrated close to 99% accuracy in classification. This modular classification service is envisioned to be integrated with a data and analytics delivery platform, such as the DRF, thus availing real-time contextualization of NOTAMs to air mobility clients, humans, and machines for enhanced decision making.

Aiden C. Szeto↗
NASA NTRSDate not supplied

Classification of Notices to Airmen using Natural Language Processing

This paper establishes the feasibility of using Natural Language Processing (NLP) to classify NOTAMs or Notices to Airmen – a pilot messaging framework to gather real-time situational awareness. Present day air mobility operations heavily rely on NOTAMs. However, pilots often have difficulty interpreting NOTAMs due to the sheer volume of inapplicable messages and unclear abbreviations. Using NLP, the presented study analyzes the accuracy of classifying NOTAMs and, thereby, the efficiency of generating actionable interpretations in real time. To this effect, efficacies of four NLP neural network architectures were analyzed, including three Recurrent Neural Networks (RNNs) with GloVe, Word2Vec, and FastText word embeddings, and one trained Bi-Directional Encoder Representations from Transformers (BERT) model. The four neural networks were trained and evaluated on three open-source datasets of varying text lengths, vocabularies, and grammars, taken from e-commerce product descriptions, social media tweets, and unstructured descriptions for data and analytics services on open data marketplaces such as NASA’s Data and Reasoning Fabric (DRF) platform. This provided cross-analysis of each neural network architecture’s performance per text type. The best performing architecture, BERT, was then fine-tuned on a collection of open-source NOTAM data. Post-training, a real-time NOTAM classification service was implemented to draw inference on new NOTAMs using the trained model, which demonstrated close to 99% accuracy in classification. This modular classification service is envisioned to be integrated with a data and analytics delivery platform, such as the DRF, thus availing real-time contextualization of NOTAMs to air mobility clients, humans, and machines for enhanced decision making.

Aiden Szeto↗
NASA NTRS2018

The SUMup Dataset: Compiled Measurements of Surface Mass Balance Components over Ice Sheets and Sea Ice with Analysis over Greenland

Increasing atmospheric temperatures over ice cover affect surface processes, including melt, snowfall, and snow density. Here, we present the Surface Mass Balance and Snow on Sea Ice Working Group (SUMup) dataset, a standardized dataset of Arctic and Antarctic observations of surface mass balance components. The July 2018 SUMup dataset consists of three subdatasets, snow/firn density (https://doi.org/10.18739/A2JH3D23R), at least near-annually resolved snow accumulation on land ice (https://doi.org/10.18739/A2DR2P790), and snow depth on sea ice (https://doi.org/10.18739/A2WS8HK6X), to monitor change and improve estimates of surface mass balance. The measurements in this dataset were compiled from field notes, papers, technical reports, and digital files. SUMup is a compiled, community-based dataset that can be and has been used to evaluate modeling efforts and remote sensing retrievals. Active submission of new or past measurements is encouraged. Analysis of the dataset shows that Greenland Ice Sheet density measurements in the top 1m do not show a strong relationship with annual temperature. At Summit Station, Greenland, accumulation and surface density measurements vary seasonally with lower values during summer months. The SUMup dataset is a dynamic, living dataset that will be updated and expanded for community use as new measurements are taken and new processes are discovered and quantified.

Montgomery, Lynn↗
NASA NTRS2011

Evaluation of the Event Driven Phenology Model Coupled with the VegET Evapotranspiration Model Through Comparisons with Reference Datasets in a Spatially Explicit Manner

A new model coupling scheme with remote sensing data assimilation was developed for estimation of daily actual evapotranspiration (ET). The scheme represents a mix of the VegET, a physically based model to estimate ET from a water balance, and an event driven phenology model (EDPM), where the EDPM is an empirically derived crop specific model capable of producing seasonal trajectories of canopy attributes. In this experiment, the scheme was deployed in a spatially explicit manner within the croplands of the Northern Great Plains. The evaluation was carried out using 2007-2009 land surface forcing data from the North American Land Data Assimilation System (NLDAS) and crop maps derived from remotely sensed data of NASA's Moderate Resolution Imaging Spectroradiometer (MODIS). We compared the canopy parameters produced by the phenology model with normalized difference vegetation index (NDVI) data derived from the MODIS nadir bi-directional reflectance distribution function (BRDF) adjusted reflectance (NBAR) product. The expectations of the EDPM performance in prognostic mode were met, producing determination coefficient (r2) of 0.8 +/-.0.15. Model estimates of NDVI yielded root mean square error (RMSE) of 0.1 +/-.0.035 for the entire study area. Retrospective correction of canopy dynamics with MODIS NDVI brought the errors down to just below 10% of observed data range. The ET estimates produced by the coupled scheme were compared with ones from the MODIS land product suite. The expected r2=0.7 +/-.15 and RMSE = 11.2 +/-.4 mm per 8 days were met and even exceeded by the coupling scheme0 functioning in both prognostic and retrospective modes. Minor setbacks of the EDPM and VegET performance (r2 about 0.5 and additional 30 % of RMSR) were found on the peripheries of the study area and attributed to the insufficient EDPM training and to spatially varying accuracy of crop maps. Overall the experiment provided sufficient evidence of soundness and robustness of the EDPM and VegET coupling scheme, assuring its potential for spatially explicit applications.

Kovalskyy, V.↗
NASA NTRS2000

Visual detection of spatial contrast patterns: evaluation of five simple models

The ModelFest Phase One dataset is a collection of luminance contrast thresholds for 43 two-dimensional monochromatic spatial patterns confined to an area of approximately two by two degrees. These data were collected by a collaboration among twelve laboratories, and were designed to provide a common database for calibration and testing of spatial vision models. Here I report fits of the ModelFest data with five models: Peak Contrast, Contrast Energy, Generalized Energy, a Gabor Channels model, and a Discrete Cosine Transform model. The Gabor Channels model provides the best fit, though the other, simpler models, with the exception of Peak Contrast, provide remarkably good fits as well. Though there are clear individual differences, regularities in the data suggest the possibility of constructing a standard observer for spatial vision. c2000 Optical Society of America.

NASA Discipline Space Human Factors↗
NASA NTRS2020

Evaluation of Modeled Precipitation in Oceanic Extratropical Cyclones Using IMERG

Using the high spatial and temporal resolution precipitation dataset Integrated Multi-satellitE Retrievals for GPM (IMERG), extratropical cyclone precipitation is evaluated in two reanalyses and two climate models. Based on cyclone-centered composites, all four models overestimate precipitation in the western subsiding and dry side of the cyclones, and underestimate the precipitation in the eastern ascending and moist side. By decomposing the composites into frequency of occurrence and intensity (mean precipitation rate when precipitating), the analysis reveals a tendency for all four models to overestimate frequency and underestimate intensity, with the former issue dominating in the western half and the latter in the eastern half of the cyclones. Differences in frequency are strongly dependent on cyclone environmental moisture, while the differences in intensity are strongly impacted by the strength of ascent within the cyclone. There are some uncertainties associated with the observations: IMERG might under-report frozen precipitation and possibly exaggerate rates in vigorously ascending regions. Nevertheless, the analysis suggests that all models produce extratropical cyclone precipitation too often and too lightly. These biases have consequences when evaluating the changes in precipitation characteristics with changes in cyclone properties: the models disagree on the magnitude of the change in precipitation intensity with a change in environmental moisture and in precipitation frequency with a change in cyclone strength. This complicates accurate predictions of precipitation changes in a changing climate.

Catherine M Naud↗
NASA NTRSDate not supplied

AstroAmpSeq: Microbial Bioinformatics Education with NASA GeneLab’s Amplicon Pipeline

The prevalence and importance of large sequencing datasets in microbiology has led to a movement to share microbial ecology experimental data through open-access databases. This is particularly true of experiments that are difficult to replicate, such as those conducted in the spaceflight environment and shared via NASA GeneLab. It is now possible and indeed valuable for students to access and re-analyze these shared datasets for educational and research purposes. To provide students with experience utilizing microbial bioinformatics tools, GeneLab for Colleges and Universities (GL4U) has designed AstroAmpSeq, a week-long, virtually implemented project-based learning (PBL) minicourse to instruct undergraduate students on 16S amplicon sequencing. AstroAmpSeq was created to be accessible to students without prior bioinformatics or microbial ecology experience. During the minicourse students work in teams to process, analyze, and visualize a subsample of GeneLab dataset GLDS-280 using GeneLab’s standard amplicon processing pipeline, which is based in R. Students develop a hypothesis related to the dataset then generate and analyze figures to evaluate their hypothesis. Formative assessment of student learning is determined via pre- and post-evaluations, peer feedback, and self-reflection. Project and presentation rubrics serve as a summative assessment of student learning. GL4U AstroAmpSeq not only meets American Society for Microbiology Curriculum Guidelines, but also incites student interest in research by an inquiry-based approach and can be made part of a larger semester-long curriculum. GL4U AstroAmpSeq raises awareness of space microbiology and bioinformatics as a field and career path among undergraduates. Further, by using a GeneLab dataset and nesting microbiology techniques into the real-world application of space biology, AstroAmpSeq enforces deeper and longer-lasting student learning.

microbiology↗