Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS2017

Oxygen Depletion on the Surface of Mercury: Evidence of Silicon Smelting?

The MErcury Surface, Space ENvironment, GEochemistry, and Ranging (MESSENGER) spacecraft collected data that provided important insights into the structure, chemical makeup, and compositional diversity of Mercury. The X-Ray Spectrometer (XRS) and Gamma-Ray Spectrometer (GRS) onboard MESSENGER provided the first detailed chemical analyses of Mercury's surface. Among the many discoveries included several surprising characteristics about the surface of Mercury, including elevated S abundances (up to 4 percent by weight), low Fe abundances (less than 4 percent by weight), and relatively low O abundances (O/Si ratio of 1.40 plus or minus 0.03). The surface chemistry as determined by MESSENGER has been used to identify up to nine distinct geochemical terranes on Mercury. Numerous modeling and experimental efforts have been undertaken to infer the mineralogy and petrology of mercurian lavas and surface materials. However, all of these efforts have presumed valence states for each of the elements according to the following: Si4+, Ti4+, Al3+, Cr2+, Fe2+, Mn2+, Mg2+, Ca2+, Na+, K+, S2-, Cl-. Based on these valence assignments, cations are charged balanced with the anions O2-, S2-, and Cl- and the compositions are recast in terms of oxides, sulfides, and chlorides. Based on these assumptions, the geochemical terranes that have been identified on Mercury yield O/Si wt. ratios ranging from 1.61 to 1.84, which is substantially higher than the preliminary O/Si ratio of 1.40 plus or minus 0.03 determined by the MESSENGER GRS]. We have re-evaluated the O/Si ratio using the entire MESSENGER dataset to reassess its implications for the geochemistry of Mercury.

McCubbin, F. M.↗
NASA NTRS2022

Assessment of Two Stochastic Cloud Subcolumn Generators Using Observed Fields of Vertically Resolved Cloud Extinction

We evaluate two stochastic subcolumn generators used in GCMs to emulate subgrid cloud variability enabling comparisons with satellite observations and simulations of certain physical processes. Our evaluation necessitated the creation of a reference observational dataset that resolves horizontal and vertical cloud variability. The dataset combines two CloudSat cloud products that resolve two-dimensional cloud optical depth variability of liquid, ice, and mixed phase clouds when blended at ~200 m vertical and ~ 2 km horizontal scales. Upon segmenting the dataset to individual “scenes”, mean profiles of the cloud fields are passed as input to generators that produce scene-level cloud subgrid variability. The assessment of generator performance at the scale of individual scenes and in a mean sense is largely based on inferred joint histograms that partition cloud fraction within predetermined combinations of cloud top pressure –cloud optical thickness ranges. Our main finding is that both generators tend to underestimate optically thin clouds, while one of them also tends to overestimate some cloud types of moderate and high optical thickness. Associated radiative flux errors are also calculated by applying a simple transformation to the cloud fraction histogram errors, and are found to approach values almost as high as 3 Wm-2 for the cloud radiative effect in the shortwave part of the spectrum.

cloud variability↗
NASA NTRS2020

Assessment of Extremes in Global Precipitation Products: How Reliable Are They?

Global gridded precipitation products have proven essential for many applications ranging from hydrological modeling and climate model validation to natural hazard risk assessment. They provide a global picture of how precipitation varies across time and space, specifically in regions where ground-based observations are scarce. While the application of global precipitation products has become widespread, there is limited knowledge on how well these products represent the magnitude and frequency of extreme precipitation—the key features in triggering flood hazards. Here, five global precipitation datasets (MSWEP, CFSR, CPC, PERSIANN-CDR, and WFDEI) are compared to each other and to surface observations. The spatial variability of relatively high precipitation events (tail heaviness) and the resulting discrepancy among datasets in the predicted precipitation return levels were evaluated for the time period 1979–2017. The analysis shows that 1) these products do not provide a consistent representation of the behavior of extremes as quantified by the tail heaviness, 2) there is strong spatial variability in the tail index, 3) the spatial patterns of the tail heaviness generally match the Köppen–Geiger climate classification, and 4) the predicted return levels for 100 and 1000 years differ significantly among the gridded products. More generally, our findings reveal shortcomings of global precipitation products in representing extremes and highlight that there is no single global product that performs best for all regions and climates.

Risk assessment↗
NASA NTRSDate not supplied

Laser Beam Welding Benchmark Experiments Performed in Reduced Gravity and Vacuum

Laser beam welding (LBW) is affected by the extreme temperatures, reduced pressure, and reduced gravity present in space environments. Gravity and pressure especially influence its melt pool and solidification dynamics. A compact, modular vacuum chamber adaptable to flight platforms from parabolic to orbital currently hosts an experiment to investigate the combined influence of reduced gravity and pressure on LBW. A swappable cartridge contains a rotating platen on which customizable workpieces can be welded under vacuum, greatly increasing experimental throughput. Instrumentation includes weld and thermal cameras observing the process, thermocouples placed on workpieces, accelerometers, and vacuum sensors. Experimental data gathered during the welding process will be combined with post-flight nondestructive evaluation, metallography, and mechanical testing to provide validation datasets for computational modeling. Phase I of this effort involves a parabolic flight campaign in low gravity while an anticipated Phase II would proceed to in-space demonstration to access extended duration microgravity.

in-space welding↗
NASA NTRS2020

Harnessing Big Data to Support the Conservation and Rehabilitation of Mangrove Forests Globally

Mangrove forests are found on sheltered coastlines in tropical, subtropical, and some warm temperate regions. These forests support unique biodiversity and provide a range of benefits to coastal communities, but as a result of large-scale conversion for aquaculture, agriculture, and urbanization, mangroves are considered increasingly threatened ecosystems. Scientific advances have led to accurate and comprehensive global datasets on mangrove extent, structure, and condition, and these can support evaluation of ecosystem services and stimulate greater conservation and rehabilitation efforts. To increase the utility and uptake of these products, in this Perspective we provide an overview of these recent and forthcoming global datasets and explore the challenges of translating these new analyses into policy action and on-the-ground conservation. We describe a new platform for visualizing and disseminating these datasets to the global science community, non-governmental organizations, government officials, and rehabilitation practitioners and highlight future directions and collaborations to increase the uptake and impact of large-scale mangrove research.

mangrove↗
NASA NTRS2019

Evaluation of Classifier Complexity for Delay Tolerant Network Routing

The growing popularity of small cost effective satellites (SmallSats, CubeSats, etc.) creates the potential for a variety of new science applications involving multiple nodes functioning together or independently to achieve a task, such as swarms and constellations. As this technology develops and is deployed for missions in Low Earth Orbit and beyond, the use of delay tolerant networking (DTN) techniques may improve communication capabilities within the network. In this paper, a network hierarchy is developed from heterogeneous networks of SmallSats, surface vehicles, relay satellites and ground stations which form an integrated network. There is a tradeoff between complexity, flexibility, and scalability of user defined schedules versus autonomous routing as the number of nodes in the network increases. To address these issues, this work proposes a machine learning classifier based on DTN routing metrics. A framework is developed which will allow for the use of several categories of machine learning algorithms (decision tree, random forest and deep learning) to be applied to a dataset of historical network statistics, which allows for the evaluation of algorithm complexity versus performance to be explored. We develop the emulation of a hierarchical network, consisting of tens of nodes which form a cognitive network architecture. CORE (Common Open Research Emulator) is used to emulate the network using bundle protocol and DTN IP neighbor discovery.

Dudukovich, Rachel↗
NASA NTRS2007

Evaluation of Radiation Belt Space Weather Forecasts for Internal Charging Analyses

A variety of static electron radiation belt models, space weather prediction tools, and energetic electron datasets are used by spacecraft designers and operations support personnel as internal charging code inputs to evaluate electrostatic discharge risks in space systems due to exposure to relativistic electron environments. Evaluating the environment inputs is often accomplished by comparing whether the data set or forecast tool reliability predicts measured electron flux (or fluence over a given period) for some chosen period. While this technique is useful as a model metric, it does not provide the information necessary to evaluate whether short term deviances of the predicted flux is important in the charging evaluations. In this paper, we use a 1-D internal charging model to compute electric fields generated in insulating materials as a function of time when exposed to relativistic electrons in the Earth's magnetosphere. The resulting fields are assumed to represent the "true" electric fields and are compared with electric field values computed from relativistic electron environments derived from a variety of space environment and forecast tools. Deviances in predicted fields compared to the "true" fields which depend on insulator charging time constants will be evaluated as a potential metric for determining the importance of predicted and measured relativistic electron flux deviations over a range of time scales.

Minow, Joseph I.↗
NASA NTRS2019

Approach for Propagating Radiometric Data Uncertainties Through NASA Ocean Color Algorithms

Spectroradiometric satellite observations of the ocean are commonly referred to as “ocean color” remote sensing. NASA has continuously collected, processed, and distributed ocean color datasets since the launch of the Sea-viewing Wide-field-of-view Sensor (SeaWiFS) in 1997. While numerous ocean color algorithms have been developed in the past two decades that derive geophysical data products from sensor-observed radiometry, few papers have clearly demonstrated how to estimate measurement uncertainty in derived data products. As the uptake of ocean color data products continues to grow with the launch of new and advanced sensors, it is critical that pixel-by-pixel data product uncertainties are estimated during routine data processing. Knowledge of uncertainties can be used when studying long-term climate records, or to assist in the development and performance appraisal of bio-optical algorithms. In this method paper we provide a comprehensive overview of how to formulate first-order first-moment (FOFM) calculus for propagating radiometric uncertainties through a selection of bio-optical models. We demonstrate FOFM uncertainty formulations for the following NASA ocean color data products: chlorophyll-a pigment concentration (Chl), the diffuse attenuation coefficient at 490 nm (K(sub d,490)), particulate organic carbon (POC), normalized fluorescent line height (nflh), and inherent optical properties (IOPs). Using a quality-controlled in situ hyperspectral remote sensing reflectance (R(sub rs,i)) dataset, we show how computationally inexpensive, yet algebraically complex, FOFM calculations may be evaluated for correctness using the more computationally expensive Monte Carlo approach. We compare bio-optical product uncertainties derived using our test R(sub rs) dataset assuming spectrally-flat, uncorrelated relative uncertainties of 1, 5, and 10%. We also consider spectrally dependent, uncorrelated relative uncertainties in R(sub rs). The importance of considering spectral covariances in R(sub rs), where practicable, in the FOFM methodology is highlighted with an example SeaWiFS image. We also present a brief case study of two POC algorithms to illustrate how FOFM formulations may be used to construct measurement uncertainty budgets for ecologically-relevant data products. Such knowledge, even if rudimentary, may provide useful information to end-users when selecting data products or when developing their own algorithms.

Bio-optics↗
NASA NTRSDate not supplied

A Comprehensive Forward Model for Spaceborne Radar Instruments

We present the development and validation of a comprehensive forward model designed to enhance remote sensing capabilities of spaceborne radar instruments. To overcome limitations in existing models, we integrated a Discrete Dipole Approximation (DDA) cloud scattering database into our Radiative Transfer Model (RTM), focusing on microwave frequencies. By simulating the optical properties of non-spherical frozen hydrometeors, the DDA technique effectively reduced discrepancies between simulated and observed values, surpassing traditional Mie tables. The evaluation of DDA lookup tables involved comparisons with a collocated dataset comprising short-term forecasts and satellite microwave data, providing evidence of their superiority. Additionally, we address the challenges of assimilating active radar measurements, which offer vertically resolved insights into clouds and precipitation. We explored the assimilation of spaceborne radar measurements in Numerical Weather Prediction (NWP) models by integrating a forward radar model, along with its adjoint and tangent linear, into the data assimilation system. Evaluation using CloudSat measurements demonstrated promising agreement between simulations and observations, particularly when the input hydrometeor profiles aligned with the measured reflectivity profiles, showcasing the potential of the developed forward radar model. Moreover, we discuss other challenges in radar measurement assimilation within NWP models, including potential observation errors and biases.

Isaac Moradi↗
NASA NTRSDate not supplied

An Intercomparison of High Spectral Resolution Lidar and Satellite Ocean Color Backscatter Retrievals

We present an intercomparison of satellite ocean color and high spectral resolution lidar (HSRL) retrievals of upper ocean particulate backscattering (bbp) and diffuse attenuation coefficients (Kd). Ocean retrievals of bbp and Kd have been performed over a wide variety of optical and ecological domains using the NASA Langley Research Center HSRL-1 (532 nm) and the recently upgraded HSRL-2 (532 & 355 nm). These datasets provide a unique opportunity to perform a critical assessment of the HSRL technique and to explore the utility of HSRL measurements for evaluating the quality of ocean color remote sensing retrievals on regional scales. A matchup dataset of co-located and high-quality HSRL and ocean color remote sensing measurements was created using data from research flights conducted over the western North Atlantic Ocean during the NASA sponsored SABOR, NAAMES, and ACTIVATE campaigns. Overall, comparisons showed a strong agreement between HSRL and ocean color retrievals of bbp and Kd, providing confidence in our ability to retrieve upper ocean optical properties using oceanographic lidar. However, comparisons from individual flights can exhibit deviations that were not attributed to shifts in water column optical domains. In one of these cases, the simultaneous HSRL atmosphere-ocean retrievals from two consecutive days and within the same region were used to identify ocean color atmospheric correction errors resulting from the presence of absorbing aerosols. These results highlight the utility of airborne HSRL as a standalone ocean observing technology and as an independent and calibrated technique for assessing the quality of ocean color remote sensing retrievals.

Brian Collister↗
NASA NTRS2022

Optimizing Landsat Next Shortwave Infrared Bands for Crop Residue Characterization

This study focused on optimizing the placement of shortwave infrared (SWIR) bands for pixel-level estimation of fractional crop residue cover (f R ) for the upcoming Landsat Next mission. We applied an iterative wavelength shift approach to a database of crop residue field spectra collected in Beltsville, Maryland, USA (n = 916) and computed generalized two- and three-band spectral indices for all wavelength combinations between 2000 and 2350 nm, then used these indices to model field-measured f R . A subset of the full dataset with a Normalized Difference Vegetation Index (NDVI) < 0.3 threshold (n = 643) was generated to evaluate green vegetation impacts on f R estimation. For the two-band wavelength shift analyses applied to the NDVI < 0.3 dataset, a generalized normalized difference using 2226 nm and 2263 nm bands produced the top f R estimation performance (R 2 = 0.8222; RMSE = 0.1296). These findings were similar to the established two-band Shortwave Infrared Normalized Difference Residue Index (SINDRI) (R 2 = 0.8145; RMSE = 0.1324). Performance of the two-band generalized normalized difference and SINDRI decreased for the full-NDVI dataset (R 2 = 0.5865 and 0.4144, respectively). For the three-band wavelength shift analyses applied to the NDVI < 0.3 dataset, a generalized ratio-based index with a 2031–2085–2216 nm band combination, closely matching established Cellulose Absorption Index (CAI) bands, was top performing (R 2 = 0.8397; RMSE = 0.1231). Three-band indices with CAI-type wavelengths maintained top f R estimation performance for the full-NDVI dataset with a 2036–2111–2217 nm band combination (R 2 = 0.7581; RMSE = 0.1548). The 2036–2111–2217 nm band combination was also top performing in f R estimation (R 2 = 0.8690; RMSE = 0.0970) for an additional analysis assessing combined green vegetation cover and surface moisture effects. Our results indicate that a three-band configuration with band centers and wavelength tolerances of 2036 nm (±5 nm), 2097 nm (±14 nm), and 2214 (±11 nm) would optimize Landsat Next SWIR bands for f R estimation.

Landsat Next↗
NASA NTRSDate not supplied

Creating Benchmark Data for Artificial Intelligence and Machine Learning Space Biology Research

To identify an appropriate AI/ML approach for a specific problem, the best practice is to measure algorithm performance through the benchmarking process. A scientific benchmark consists of an AI-ready dataset and a reference implementation on a specific scientific question. The NASA Science Mission Directorate (SMD) has started the “Benchmark Initiative for AI/ML to create scientific benchmark datasets in three applications: 1) scientific benchmarking, which finds the best algorithm for a specific problem; 2) application benchmarking, which measures algorithm performance against a set of parameters; and 3) system benchmarking, which evaluates performance of hardware and software architecture. Currently, there are no standardized datasets available to benchmark AI/ML algorithms in the domain of space biology. In this work, we constructed two AI/ML-ready biological datasets from experiments in space-flown mice: cellular imaging and RNA-seq. First, radiation-exposed immune cells harbor DNA damage foci that can be fluorescently marked to visualize the amount of damage following exposure to ionizing radiation. However, such large datasets are difficult to analyze visually, due to imaging inconsistencies and human bias, and classical image processing approaches can fail on imaging artifacts. AI/ML are therefore exciting alternative, providing the speed of machines and the accuracy of humans. We have made this dataset available at https://registry.opendata.aws/bps_microscopy/. Second, high-throughput nucleic acid sequencing (DNA-seq, RNA-seq) has become widespread in biomedical research due to the growing availability and affordability of these assays. However, most sequencing datasets suffer from high dimensionality and low sample count. In this work, we used a generative adversarial network to synthesize a standardized, AI-ready, publicly available benchmark dataset for space biology RNA-seq data with sufficient space-flown and ground control mouse liver samples from NASA GeneLab. This dataset is available at https://registry.opendata.aws/bps_rnaseq/. These datasets are now fully open the Space Biology community to test their favorite AI/ML approaches.

James Casaletto↗
NASA NTRS2017

Experimental Evaluation of Acoustic Engine Liner Models Developed with COMSOL Multiphysics

Accurate modeling tools are needed to design new engine liners capable of reducing aircraft noise. The purpose of this study is to determine if a commercially-available finite element package, COMSOL Multiphysics, can be used to accurately model a range of different acoustic engine liner designs, and in the process, collect and document a benchmark dataset that can be used in both current and future code evaluation activities. To achieve these goals, a variety of liner samples, ranging from conventional perforate-over-honeycomb to extended-reaction designs, were installed in one wall of the grazing flow impedance tube at the NASA Langley Research Center. The liners were exposed to high sound pressure levels and grazing flow, and the effect of the liner on the sound field in the flow duct was measured. These measurements were then compared with predictions. While this report only includes comparisons for a subset of the configurations, the full database of all measurements and predictions is available in electronic format upon request. The results demonstrate that both conventional perforate-over-honeycomb and extended-reaction liners can be accurately modeled using COMSOL. Therefore, this modeling tool can be used with confidence to supplement the current suite of acoustic propagation codes, and ultimately develop new acoustic engine liners designed to reduce aircraft noise.

Schiller, Noah H.↗
NASA NTRS1995

Comparison of simulated cloud cover with satellite obsrvations over the Western United States

Satellite imagery datasets and regional climate model results are intercompared for evaluation of model accuracy in the simulation of cloud cover. Both monthly average individual simulation times are analyzed. To provide a consistent comparison, satellite data are first mapped into the model's geographic projection, grid domain, and resolution. It is found that September 1988 monthly average cloud fraction results from the modeled simulations correspond to observations, in both spatial pattern and magnitude, with bias less than +/- 20% cloud fraction over the entire inland West. Agreement in the pattern of cloud fraction also is evident for monthly average cloud fraction in July, but there is no negative bias of 10%-30% cloud fraction in the model diagnosis of cloud cover. Correlations between the spatial distributions of model-derived and observed cloud fractions are found to exceed 0.80 for certain geographic regions of the West, and these correlations are largest over mountainous areas during summer. Case studies of a series of daily cloud cover demonstrate the ability of the model to simulate the effects of frontal passage on cloud distribution. The ability of the RegCM1 to simulate daily cloud fraction and diurnal cloud evolution is somewhat weak for the summer convective season. It is anticipated that a more recent version of the regional climate model may improve the simulation of summer season cloud cover, through changes in cloud parameterization and improvements in model resolution.

Wetzel, Melanie A.↗
NASA NTRS1998

Evaluation of Landscape Structure Using AVIRIS Quicklooks and Ancillary Data

Currently the best tool for examining landscape structure is remote sensing, because remotely sensed data provide complete and repeatable coverage over landscapes in many climatic regimes. Many sensors, with a variety of spatial scales and temporal repeat cycles, are available. The Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) has imaged over 4000 scenes from over 100 different sites throughout North America. For each of these scenes, one-band "quicklook" images have been produced for review by AVIRIS investigators. These quicklooks are free, publicly available over the Internet, and provide the most complete set of landscape structure data yet produced. This paper describes the methodologies used to evaluate the landscape structure of quicklooks and generate corresponding datasets for climate, topography and land use. A brief discussion of preliminary results is included at the end. Since quicklooks correspond exactly to their parent AVIRIS scenes, the methods used to derive climate, topography and land use data should be applicable to any AVIRIS analysis.

Sanderson, Eric W.↗
NASA NTRS2000

Using TRMM Field Campaign Data for Assessing GEOS Forecast and Assimilation Products

The wealth of in-situ measurements gathered during Tropical Rain Measuring Mission (TRMM) field campaigns over a wide range of tropical conditions constitute an important data source for evaluating the quality of global model forecasts and assimilated datasets. In this study we use selected observations of cloud microphysics and atmospheric sounding from TEFLUN-1998, SCEMEX-1998, and TRMMLBA-1999 to examine the assimilation and forecast fields produced by the operational GEOS-3 (Goddard Earth Observing System - version 3) global data assimilation system (DAS) and a new finite-volume DAS under development at the Data Assimilation Office. Additionally, TRMM field campaign measurements are used to verify the impact of assimilating rainfall and moisture data derived from TRMM Microwave Imager and Special Sensor Microwave/Imager instruments on the GEOS analysis. We will also explore issues concerning the 'error of representativeness' in using in-situ observations of quantities with large spatial and temporal variability such as precipitation for validating gridded global data products.

Hou, Arthur Y.↗