Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data quality analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56
All sourcesNASA NTRSDOE OSTI
NASA NTRS2019

Satellite Ocean Colour: Current Status and Future Perspective

Spectrally resolved water-leaving radiances (ocean colour) and inferred chlorophyll concentration are key to studying phytoplankton dynamics at seasonal and inter-annual scales, for a better understanding of the role of phytoplankton in marine biogeochemistry; the global carbon cycle; and the response of marine ecosystems to climate variability, change and feedback processes. Ocean colour data also have a critical role in operational observation systems monitoring coastal eutrophication, harmful algal blooms, and sediment plumes. The contiguous ocean-colour record reached 21 years in 2018; however, it is comprised of a number of one-off missions such that creating a consistent time-series of ocean-colour data requires merging of the individual sensors (including MERIS, Aqua-MODIS, SeaWiFS, VIIRS, and OLCI) with differing sensor characteristics, without introducing artefacts. By contrast, the next decade will see consistent observations from operational ocean colour series with sensors of similar design and with a replacement strategy. Also, by 2029 the record will start to be of sufficient duration to discriminate climate change impacts from natural variability, at least in some regions. This paper describes the current status and future prospects in the field of ocean colour focusing on large to medium resolution observations of oceans and coastal seas. It reviews the user requirements in terms of products and uncertainty characteristics and then describes features of current and future satellite ocean-colour sensors, both operational and innovative. The key role of in situ validation and calibration is highlighted as are ground segments that process the data received from the ocean-colour sensors and deliver analysis-ready products to end-users. Example applications of the ocean-colour data are presented, focusing on the climate data record and operational applications including water quality and assimilation into numerical models. Current capacity building and training activities pertinent to ocean colour are described and finally a summary of future perspectives is provided.

ocean colour↗
DOE OSTI2026

Image processing pipeline for AI-driven nanoparticle megalibrary characterization

Recent innovations have made it possible to produce megalibraries, millions of structurally and compositionally distinct nanoparticles on a chip. These megalibraries yield vast volumes of data that are impossible to analyze manually, necessitating the development of automated tools. In previous work, we created a binary classification machine learning model to select quality nanoparticle images for downstream analysis. In this work, we show that adding a custom image processing step before training can produce significantly higher-performing models in a fraction of the time and make them more robust to different image noise levels and microscope acquisition settings. The image processing pipeline proposed here effectively cleans raw nanoparticle images, enhances key features, and allows us to use much lower resolution images and simpler neural network model architectures. These features result in higher performance and significant cost savings. Experiments demonstrate superior performance relative to baseline, including an 18.2% improvement in recall and a 13.1% increase in accuracy. Given the high cost of downstream analysis, it is critical to minimize false positives, and our best-performing model reaches a precision of 95.9% and a weighted F-score of 95.1% on an unseen test set. Additionally, model training time is reduced from hours to less than a minute. We also show that, using this custom image processing pipeline, model performance is significantly improved at lower pixel resolutions compared to downsizing alone. We expect that adopting this pipeline for AI-driven automated nanoparticle characterization will allow researchers to rapidly and accurately analyze much greater volumes of data, thereby accelerating materials discovery.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2020

Robust PCA-Deep Belief Network Surrogate Model for Distribution System Topology Identification with DERs

With the expansion of distribution networks and increased penetration of distributed energy resources (DERs), it is becoming increasingly important to obtain accurate distribution network topology in real-time. In this paper, a robust principal component analysis coupled deep belief network (PCA-DBN) surrogate model is proposed for distribution system topology identification. It integrates the benefits of robust feature extraction from PCA to deal with data quality issues and filter out noise, and the strength of DBN in capturing the nonlinear relationship between voltage amplitudes and the binary states of switchable connections. This also significantly reduces the DBN training complexity without loss of accuracy. It is shown that the widely used standard deviation of voltage drop and the voltage covariance matrix features yield less accuracy as compared to that of the voltage amplitudes in presence of high penetration of DERs and ZIP loads. Comparison results with other alternatives, such as the random forest (RF), multi-output regression (MOR) and the traditional DBN methods demonstrate that the proposed method can achieve a much higher topology identification accuracy while maintaining robustness to missing data and measurement noise under various penetration levels of DERs.

deep belief network↗
DOE OSTI2021

Deep Learning Analysis of Polaritonic Wave Images

Deep learning (DL) is an emerging analysis tool across the sciences and engineering. Encouraged by the successes of DL in revealing quantitative trends in massive imaging data, we applied this approach to nanoscale deeply subdiffractional images of propagating polaritonic waves in complex materials. Utilizing the convolutional neural network (CNN), we developed a practical protocol for the rapid regression of images that quantifies the wavelength and the quality factor of polaritonic waves. Using simulated near-field images as training data, the CNN can be made to simultaneously extract polaritonic characteristics and material parameters in a time scale that is at least 3 orders of magnitude faster than common fitting/processing procedures. The CNN-based analysis was validated by examining the experimental near-field images of charge-transfer plasmon polaritons at graphene/α-RuCl3 interfaces. Our work provides a general framework for extracting quantitative information from images generated with a variety of scanning probe methods.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2011

Validating LES for Jet Aeroacoustics

Engineers charged with making jet aircraft quieter have long dreamed of being able to see exactly how turbulent eddies produce sound and this dream is now coming true with the advent of large eddy simulation (LES). Two obvious challenges remain: validating the LES codes at the resolution required to see the fluid-acoustic coupling, and the interpretation of the massive datasets that are produced. This paper addresses the former, the use of advanced experimental techniques such as particle image velocimetry (PIV) and Raman and Rayleigh scattering, to validate the computer codes and procedures used to create LES solutions. This paper argues that the issue of accuracy of the experimental measurements be addressed by cross-facility and cross-disciplinary examination of modern datasets along with increased reporting of internal quality checks in PIV analysis. Further, it argues that the appropriate validation metrics for aeroacoustic applications are increasingly complicated statistics that have been shown in aeroacoustic theory to be critical to flow-generated sound, such as two-point space-time velocity correlations. A brief review of data sources available is presented along with examples illustrating cross-facility and internal quality checks required of the data before it should be accepted for validation of LES.

Bridges, James↗
DOE OSTI2022

SDSS-IV MaStar: Data-driven Parameter Derivation for the MaStar Stellar Library

The Mapping Nearby Galaxies at Apache Point Observatory (MaNGA) Stellar Library (MaStar) is a large collection of high-quality empirical stellar spectra designed to cover all spectral types and ideal for use in the stellar population analysis of galaxies observed in the MaNGA survey. The library contains 59,266 spectra of 24,130 unique stars with spectral resolution R ~ 1800 and covering a wavelength range of 3622–10,354 Å. In this work, we derive five physical parameters for each spectrum in the library: effective temperature (T eff ), surface gravity ($\mathrm{log} g$), metallicity ([Fe/H]), microturbulent velocity ($\mathrm{log}({v}_{\mathrm{micro}})$), and alpha-element abundance ([α/Fe]). These parameters are derived with a flexible data-driven algorithm that uses a neural network model. We train a neural network using the subset of 1675 MaStar targets that have also been observed in the Apache Point Observatory Galactic Evolution Experiment (APOGEE), adopting the independently-derived APOGEE Stellar Parameter and Chemical Abundance Pipeline parameters for this reference set. For the regions of parameter space not well represented by the APOGEE training set (7000 ≤ T ≤ 30,000 K), we supplement with theoretical model spectra. We present our derived parameters along with an analysis of the uncertainties and comparisons to other analyses from the literature.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1976

Analysis of hydrological features of portions of the Lake Ontario basin using Skylab and aircraft data

The author has identified the following significant results. S190A and S190B photography proved to be useful for mapping large scale geomorophological features, and for assessing water depth and water quality. Available S192 data were affected by low frequency noise caused by diode light. Hydrological features were classified, and upland green herbaceous vegetation was separated into several classes based on percent vegetation cover. A model for estimating surface soil moisture based on red and near infrared reflectance data was developed and subsequently implemented.

Polcyn, F. C.↗
NASA NTRS2000

Atmospheric Boundary Layer Wind Data During the Period January 1, 1998 Through January 31, 1999 at the Dallas-Fort Worth Airport: Quality Assessment - Volume 1

The quality of the Aircraft Vortex Spacing System (AVOSS) is critically dependent on representative wind profiles in the atmospheric boundary layer. These winds observed from a number of sensor systems around the Dallas-Fort Worth airport were combined into single vertical wind profiles by an algorithm developed and implemented by MIT Lincoln Laboratory. This process, called the AVOSS Winds Analysis System (AWAS), is used by AVOSS for wake corridor predictions. During times when AWAS solutions were available, the quality of the resultant wind profiles and variance was judged from a series of plots combining all sensor observations and AWAS profiles during the period 1200 to 0400 UTC daily. First, input data was evaluated for continuity and consistency from criteria established. Next, the degree of agreement among all wind sensor systems was noted and cases of disagreement identified. Finally, the resultant AWAS solution was compared to the quality-assessed input data. When profiles differed by a specified amount from valid sensor consensus winds, times and altitudes were flagged. Volume one documents the process and quality of input sensor data. Volume two documents the data processing/sorting process and provides the resultant flagged files.

Zak, J. Allen↗
DOE OSTI2023

Mammalian Chromosome Analysis and Sorting by Flow Cytometry

The analysis of chromosomes by flow cytometry is termed flow cytogenetics, and it involves the analysis and sorting of single mitotic chromosomes in suspension. The study of flow karyograms provides insight into chromosome number and structure to provide information on chromosomal DNA content and can enable the detection of deletions, translocations, or any forms of aneuploidy. Beyond its clinical applications, flow cytogenetics greatly contributed to the Human Genome Project through the ability to sort pure populations of chromosomes for gene mapping, cloning, and the construction of DNA libraries. Maximizing the potential of these important applications of flow cytogenetics relies on precise instrument setup and optimal sample processing, both of which impact the accuracy and quality of the data that are generated. This article is a compilation of the existing protocols that describe the stepwise methodology of accumulating, isolating, and staining metaphase chromosomes to prepare single-chromosome suspensions for flow cytometric analysis and sorting. Although the chromosome preparation protocols have remained largely unchanged, cytometer technology has advanced dramatically since these protocols were originally developed. Advances in cytometry technologies offer new and exciting approaches for understanding and monitoring chromosomal aberrations, but the hallmark of these protocols remains their simplicity in methodologies and reagent requirements and the accuracy of data resolvable to every chromosome of the cell.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRSDate not supplied

Microbiological Analysis of Mizuna Grown in the Veggie Hardware to Define Critical Control Points and Ensure the Safety of Space Grown Crops

The Veggie facility on the International Space Station has been utilized as a “pick and eat” plant growth system to provide fresh produce for crew consumption. The VEG-04 experiments completed in 2019 examined the effect of red-rich and blue-rich light treatments on the growth of mizuna as well as harvest method and resulting yield. Analysis was performed on plant tissues and associated hardware to evaluate the influence of these experimental variables on the microbial population. VEG-04A plant pillows with pre-planted Mizuna mustard seeds were launched on SpaceX-16 in December 2018. The pillows were initiated, and a single 35-day harvest was performed. Veg 04B pillows were pre-planted with Mizuna seeds and launched on SpaceX-18 in July 2019, initiated and subsequently harvested at days 29, 43 and 58. The crew consumed approximately half of the produce, and the remainder was frozen and returned for analysis. Leaves, swabs, wicking material, substrate, and roots were processed and plated on media for the enumeration and isolation of bacteria and fungi. Isolated bacterial colonies were identified using Biolog Micro ID system or MicroSEQ16S rDNA sequencing technique. Fungal colonies were identified using the MicroSEQ D2 rDNA kit. Sample extracts were plated onto specialized media to identify Escherichia coli/coliforms, Staphylococcus aureus and Salmonella sp. Results indicate that bacterial and fungal counts were higher in plants grown in red-rich lightin VEG-04A, while the opposite was true in the Veg-04B third harvest. Microbial counts increased with the repeated harvest method used in Veg-04B. These data support the understanding of environmental and horticultural practices that can affect the microbiological quality of space-grown produce grown and aid in identification of critical control points for the development of a hazard analysis critical control point plan for ISS-grown crops. This research was co-funded by the NASA’s Human Research Program and Space Biology.

Mary E Hummerick↗
NASA NTRS2010

Evaluating the Utility of Remotely-Sensed Soil Moisture Retrievals for Operational Agricultural Drought Monitoring

Soil moisture is a fundamental data source used by the United States Department of Agriculture (USDA) International Production Assessment Division (IPAD) to monitor crop growth stage and condition and subsequently, globally forecast agricultural yields. Currently, the USDA IPAD estimates surface and root-zone soil moisture using a two-layer modified Palmer soil moisture model forced by global precipitation and temperature measurements. However, this approach suffers from well-known errors arising from uncertainty in model forcing data and highly simplified model physics. Here we attempt to correct for these errors by designing and applying an Ensemble Kalman filter (EnKF) data assimilation system to integrate surface soil moisture retrievals from the NASA Advanced Microwave Scanning Radiometer (AMSR-E) into the USDA modified Palmer soil moisture model. An assessment of soil moisture analysis products produced from this assimilation has been completed for a five-year (2002 to 2007) period over the North American continent between 23degN - 50degN and 128degW - 65degW. In particular, a data denial experimental approach is utilized to isolate the added utility of integrating remotely-sensed soil moisture by comparing EnKF soil moisture results obtained using (relatively) low-quality precipitation products obtained from real-time satellite imagery to baseline Palmer model runs forced with higher quality rainfall. An analysis of root-zone anomalies for each model simulation suggests that the assimilation of AMSR-E surface soil moisture retrievals can add significant value to USDA root-zone predictions derived from real-time satellite precipitation products.

Bolten, John D.↗
NASA NTRS2010

Using Remote Sensing Data to Evaluate Habitat Loss in the Mobile, Galveston, and Tampa Bay Watersheds

The Gulf of Mexico has experienced dramatic wetland habitat area losses over the last two centuries. These losses not only damage species diversity, but contribute to water quality, flood control, and aspects of the Gulf coast economy. Overall wetland losses since the 1950s were examined using land cover/land use (LCLU) change analysis in three Gulf coast watershed regions: Mobile Bay, Galveston Bay, and Tampa Bay. Two primary causes of this loss, LCLU change and climate change, were then assessed using LCLU maps, U.S. census population data, and available current and historical climate data from NOAA. Sea level rise, precipitation, and temperature effects were addressed, with emphasis on analysis of the effects of sea level rise on salt marsh degradation. Ecological impacts of wetland loss, including fishery depletion, eutrophication, and hypoxia were addressed using existing literature and data available from NOAA. These ecological consequences in turn have had an affect on the Gulf coast economy, which was analyzed using fishery data and addressing public health impacts of changes in the environment caused by wetland habitat loss. While recent federal and state efforts to reduce wetland habitat loss have been relatively successful, this study implies a need for more aggressive action in the Gulf coast area, as the effects of wetland loss reach far beyond individual wetland systems themselves to the Gulf of Mexico as a whole.

Steffen, Morgan↗
NASA NTRSDate not supplied

Multidimensional Data Aggregation in the Cloud with Application to Geostationary Satellite-based Air Quality Monitoring

Scientists use satellite data for studying Earth's systems, and the remote sensing data that these satellites collect are typically separated into files of a size small enough for efficient network transfer and storage. However, researchers usually prefer to analyze the data based on real-world dimensions like time, space, or elevation. To help with this, NASA's Atmospheric Science Data Center (ASDC) developed a new cloud-based tool that combines these smaller data chunks into larger, more useful datasets. The tool works on Network Common Data Form (netCDF4) and some HDF5 formatted files, and it is available as a service in NASA's Earthdata Cloud. In this presentation, we showcase this service using data from the Tropospheric Emissions: Monitoring of Pollution (TEMPO) instrument. By combining TEMPO's continuous observations over time, we create longer and more informative analysis-ready time series to facilitate the study of air quality patterns. Insights gained will provide a more comprehensive understanding of pollution sources, transport patterns, and their effects on the environment and human health.

Daniel Kaufman↗
NASA NTRS2014

Link Analysis for Space Communication Links Using ARQ Protocol

In space communications, standard link analysis assumes that messages are sent once. For a communication link that uses an error-correction coding scheme, bit-error-rate (BER) or frame-error-rate (FER), and link margins are common metrics that characterize the quality of a link, and they are used to determine the supportable data rate. With the advent of Automatic Repeat-reQuest (ARQ) protocols, when messages are corrupted during transmission, they can be resent multiple times automatically until they are correctly received and acknowledged. The concept of BER, FER, and link margin cannot be directly applied, and the link analysis approach for ARQ links needs to be re-examined.

Cheung, Kar-Ming↗
DOE OSTI2025

Robust Spectral Anomaly Detection in EELS Spectral Images via 3D Convolutional Variational Autoencoders

Abstract A 3D Convolutional Variational Autoencoder (3D‐CVAE) is introduced for automated anomaly detection in electron energy‐loss spectroscopy spectrum imaging (EELS‐SI) data. This approach leverages the full 3D structure of EELS‐SI data to detect subtle spectral anomalies while preserving both spatial and spectral correlations across the datacube. By employing cross‐entropy loss and training on bulk spectra, the model learns to reconstruct bulk features characteristic of the defect‐free material. In exploring methods for anomaly detection, both the 3D‐CVAE approach and principal component analysis (PCA) are evaluated, testing their performance using FeL‐edge ΔEpeak shifts designed to simulate material defects. These results show that 3D‐CVAE achieves superior anomaly detection and maintains consistent performance across various shift magnitudes. The method demonstrates clear bimodal separation between bulk and anomalous spectra, enabling reliable classification. Further analysis verifies that lower‐dimensional representations are robust to anomalies in the data. While performance advantages over PCA diminish with decreasing anomaly concentration, our method maintains high reconstruction quality even in challenging, noise‐dominated spectral regions. This approach provides a robust framework for unsupervised automated detection of spectral anomalies in EELS‐SI data, particularly valuable for analyzing complex material systems.

Chemistry↗
DOE OSTI2020

Sampling Subjective Polygons for Patch-Based Deep Learning Land-Use Classification in Satellite Images

Model generalization remains a key challenge in the analysis of large amounts of heterogeneous satellite image data. One major limiting factor in developing generalizable models, in the context of supervised learning, is the lack of high quality training datasets. A model's capacity to perform well on new data is often inhibited by imbalance and bias in the data that was used for training. This is especially a problem when using convolutional neural networks to classify urban land-use in satellite images. Notable dataset imbalance issues in this application include land-use type imbalance and image scene imbalance. To begin understanding these dataset imbalance problems in more detail, we develop and test a number of sampling methods for generating training image datasets from subjective training polygons for urban land-use classification. We investigate sampling at different point densities as a means to reduce content repetition and therefore content imbalance and bias in the training image dataset.

Arndt, Jacob↗
NASA NTRS1977

Remote sensing and geographically based information systems

A structure is proposed for a geographically-oriented computer-based information system applicable to the analysis of remote sensing digital data. The structure, intended to answer a wide variety of user needs, would permit multiple views of the data, provide independent management of data security, quality and integrity, and rely on automatic data filing. Problems in geographically-oriented data systems, including those related to line encoding and cell encoding, are considered.

Cicone, R. C.↗
NASA NTRS1981

Landsat - What is operational in water resources

Applications of Landsat data in hydrology and water quality measurement were examined to determine which applications are operational. In hydrology, the principal applications have been surface water inventory, and land cover analysis for (1) runoff modeling and (2) abatement planning for non-point pollution and erosion. In water quality measurement, the principal applications have been: (1) trophic state assessment, and (2) measurement of turbidity and suspended sediment. The following applications were found to be operational: mapping of surface water, snow cover, and land cover (USGS Level 1) for watershed applications; measurement of turbidity, Secchi disk depth, suspended sediment concentration, and water depth.

Middleton, E. M.↗