Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data quality”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Review of Technical Photovoltaic Key Performance Indicators and the Importance of Data Quality Routines

Technical key performance indicators (KPIs) are important metrics used to assess and quantitatively summarize various aspects of photovoltaic (PV) systems, including long-term performance, economic viability, and carbon footprint. Herein, a group of experts of the International Energy Agency's Photovoltaic Power Systems Programme Task 13 collect and describ the most important technical KPIs used in the industry. Thereby, a set of best practices for reliably handling PV system data is presented and the impact of data quality and climatic variability on KPI calculation is investigated. Further, the effective use of technical KPIs allows triggering data-driven and informed decisions to optimize PV systems and providing a comprehensive overview of how PV systems operate across different conditions and climates. With the worldwide growth of the PV industry, more companies operate/own PV systems in different regions, where the climatic and seasonal profiles differ. This requires context-aware evaluation of KPIs, or the judicious application of multiple KPIs, to ensure that each asset is evaluated correctly. Beyond that, there is untapped potential in the utilization of KPIs through geospatial mapping and extrapolation of fleet KPIs. This study demonstrates that the uncertainty in KPI estimation is not well understood and depends on data quality, climatic variability, and system configuration.

14 SOLAR ENERGY↗
NASA NTRS2018

Developing Data Services to Provide Data Quality Information for Global Satellite-Based Precipitation Products

Global satellite-based precipitation products have been widely used in research and applications around the world. Compared to ground-based observations, satellite-based measurements provide data on a global scale, especially in remote continents and over oceans. The NASA Goddard Earth Sciences (GES) Data and Information Services Center (DISC) is home to NASA global precipitation product archives including the Tropical Rainfall Measuring Mission (TRMM), the Global Precipitation Measurement (GPM), as well as other global and regional precipitation products. Precipitation is one of the top downloaded and accessed parameters in the GES DISC data archive. Meanwhile, users want to easily locate and obtain data quality information at regional and global scales to better understand how precipitation products perform and how reliable they are. As a data service provider, it is necessary to provide easy access to data quality information. However, such information normally is not available, and when it is available, it is not in one place and difficult to locate. In this presentation, we will present such challenges and activities at the GES DISC to address precipitation data (other datasets as well) quality issues.

Liu, Z.↗
NASA NTRS1983

LANDSAT 4 image data quality analysis

A comparative analysis of TM and MSS data was completed and the results indicate that there are half as many separable spectral classes in the MSS data than in TM. In addition, the minimum separability between classes was also much less in MSS data. Radiometric data quality was also investigated for the TM by computing power spectrum estimates for dark-level data from Lake Michigan. Two significant coherent noise frequencies were observed, one with a wavelength of 3.12 pixels and the other with a 17 pixel wavelength. The amplitude was small (nominally .6 digital count standard deviation) and the noise appears primarily in Bands 3 and 4. No significant levels were observed in other bands. Scan angle dependent brightness effects were also evaluated.

Anuta, P. E.↗
DOE OSTI2025

Anomaly Detection Based on Machine Learning for the CMS Electromagnetic Calorimeter Online Data Quality Monitoring

Using a semi-supervised machine learning approach we present a real-time anomaly detection system based on an autoencoder used for online data quality monitoring of the CMS electromagnetic calorimeter operating at the CERN LHC. We introduce a novel method that maximizes the anomaly detection performance making use of the time-dependence of anomalies and the spatial variations in the detector response. The autoencoder-based system efficiently detects anomalies in real time and maintains a very low false discovery rate. We validate the performance of this novel system with anomalies from LHC collision data taken in 2018 and 2022. In addition, results are presented after deploying the autoencoder-based system in the CMS online Data Quality Monitoring workflow at the beginning of LHC Run 3 resulting in the system to detect issues that were missed by the existing system.

Harilal, Abhirami [Carnegie Mellon University, Pit↗
NASA NTRS2011

An Investigation Into HPLC Data Quality Problems

This report summarizes the analyses and results produced by a five-member investigative team of Government, university, and industry experts, established by NASA HQ. The team examined data quality problems associated with high performance liquid chromatography (HPLC) analyses of pigment concentrations in seawater samples produced by the San Diego State University (SDSU) Center for Hydro-Optics and Remote Sensing (CHORS). This report shows CHORS did not validate the methods used before placing them into service to analyze field samples for NASA principal investigators (PIs), even though the HPLC literature contained easily accessible method validation procedures, and the importance of implementing them, more than a decade ago. In addition, there were so many sources of significant variance in the CHORS methodologies, that the HPLC system rarely operated within performance criteria capable of producing the requisite data quality. It is the recommendation of the investigative team to a) not correct the data, b) make all the data that was temporarily sequestered available for scientific use, and c) label the affected data with an appropriate warning, e.g., "These data are not validated and should not be used as the sole basis for a scientific result, conclusion, or hypothesis--independent corroborating evidence is required."

Hooker, Stanford B.↗
NASA NTRS1996

Development and status of data quality assurance program at NASA Langley research center: Toward national standards

As part of a continuing effort to re-engineer the wind tunnel testing process, a comprehensive data quality assurance program is being established at NASA Langley Research Center (LaRC). The ultimate goal of the program is routing provision of tunnel-to-tunnel reproducibility with total uncertainty levels acceptable for test and evaluation of civilian transports. The operational elements for reaching such levels of reproducibility are: (1) statistical control, which provides long term measurement uncertainty predictability and a base for continuous improvement, (2) measurement uncertainty prediction, which provides test designs that can meet data quality expectations with the system's predictable variation, and (3) national standards, which provide a means for resolving tunnel-to-tunnel differences. The paper presents the LaRC design for the program and discusses the process of implementation.

Hemsch, Michael J.↗
NASA NTRS1979

A compilation and analysis of helicopter handling qualities data. Volume 2: Data analysis

A compilation and an analysis of helicopter handling qualities data are presented. Multiloop manual control methods are used to analyze the descriptive data, stability derivatives, and transfer functions for a six degrees of freedom, quasi static model. A compensatory loop structure is applied to coupled longitudinal, lateral and directional equations in such a way that key handling qualities features are examined directly.

Heffley, R. K.↗
NASA NTRS1981

Applications notice for participation in the LANDSAT-D image data quality analysis program

The applications notice for the LANDSAT 4 image data quality analysis program is presented. The objectives of the program are to qualify LANDSAT 4 sensor and systems performance from a user applications point of view, and to identify any malfunctions that may impact data applications. Guidelines for preparing proposals and background information are provided.

Source record↗
NASA NTRS1984

Landsat-4 data quality analysis

Landsat-4 satellite Thematic Mapper (TM) and multispectral scanner (MSS) data have been analyzed in order to ascertain data quality and information content. Geometric evaluations have tested band-to-band registration accuracy, and the TM's overall system resolution was evaluated for the case of image objects with high contrast, sharp edge responses. The information content evaluation employed clustering, principal components, and the transformed divergence separability measured on data from Iowa and Chicago, Illinois. The MSS classification analysis compared MSS and TM information contents for a large number of science classes.

Anuta, P.↗
NASA NTRS1984

Landsat-4 MSS and Thematic Mapper data quality and information content analysis

Landsat-4 Thematic Mapper and Multispectral Scanner data were analyzed to obtain information on data quality and information content. Geometric evaluations were performed to test band-to-band registration accuracy. Thematic Mapper overall system resolution was evaluated using scene objects which demonstrated sharp high contrast edge responses. Radiometric evaluation included detector relative calibration, effects of resampling, and coherent noise effects. Information content evaluation was carried out using clustering, principal components, transformed divergence separability measure, and numerous supervised classifiers on data from Iowa and Illinois. A detailed spectral class analysis (multispectral classification) was carried out on data from the Des Moines, IA area to compare the information content of the MSS and TM for a large number of scene classes.

Anuta, P. E.↗
NASA NTRS1974

Statistical summary of air quality data for metropolitian Cleveland, Ohio, 1967 - 1972: Total suspended particulates, nitrogen dioxide, and sulfur dioxide

Air-quality data for metropolitan Cleveland, Ohio, from 1967 through 1972 were collated and statistically analyzed. Total suspended particulates (TSP) departed from lognormal distribution in 1972. Nitrogen dioxide and sulfur dioxide, departed significantly from lognormal distributions in 1972. In Cleveland the Ohio standards were not met. However, the data indicate a general improvement in air quality. Unusually high precipitation (43% above the average in 1972) may be responsible in lowering these values from the 1971 levels. The mean values of TSP, NO2, and SO2 are 104, 191, and 83 microgram/cu m respectively.

King, R. B.↗
DOE OSTI2024

Enhancing Data Quality Monitoring at CMS with Interactive Visualization Tools and Automated Reference Run Selection

Current data quality monitoring (DQM) tools at CMS offer granularity limited to per-run analysis. Consequently, issues manifesting at the per-lumisection level can go unnoticed or, even if detectable, often lead to the classification of the whole run as bad, resulting in unnecessary data loss. Additionally, shifters have to evaluate a large set of monitoring elements during their long shifts, increasing the probability of human errors or overlooked problems. In this contribution, we present ongoing work on the development of tools that will provide shifters with an accessible, granularity-enhanced view of DQM data through interactive and dynamic visualizations. Furthermore, we introduce a reference run selection tool currently under development, which will automate the selection based on data-taking conditions and will offer a curated set of training data for machine learning models that will be used for the partial automation of the offline data certification process. These endeavors will be integrated into the DIALS website, enabling enhancements in data certification accuracy and improving the accessibility of DQM at CMS.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS2011

Use and Limitations of a Climate-Quality Data Record to Study Temperature Trends on the Greenland Ice Sheet

Enhanced melting of the Greenland Ice Sheet has been documented in recent literature along with surface-temperature increases measured using infrared satellite data since 1981. Using a recently-developed climate-quality data record, 11- and 12-year trends in the clear-sky ice-surface temperature (IST) of the Greenland Ice Sheet have been studied using the Moderate-Resolution Imaging Spectroradiometer (MODIS) IST product. Daily and monthly MODIS ISTs of the Greenland Ice Sheet beginning on 1 March 2000 and continuing through 31 December 2010 are now available at 6.25-km spatial resolution on a polar stereographic grid as described in Hall et al. (submitted). This record will be elevated in status to a climate-data record (CDR) when more years of data become available either from the MODIS on the Terra or Aqua satellites, or from the Visible Infrared Imager Radiometer Suite (VIIRS) to be launched in October 2011. Maps showing the maximum extent of melt for the entire ice sheet and for the six major drainage basins have been developed from the MODIS IST dataset. Twelve-year trends of the duration of the melt season on the ice sheet vary in different drainage basins with some basins melting progressively earlier over the course of the study period. Some (but not all) of the basins also show a progressively-longer duration of melt. IST 12-year trends are compared with in-situ data, and climate data from the Modern Era Retrospective-Analysis for Research and Applications (MERRA) Reanalysis.

Hall, Dorothy K.↗
NASA NTRS1972

Statistical summary and trend evaluation of air quality data for Cleveland, Ohio in 1967 to 1971: Total suspended particulate, nitrogen dioxide, and sulfur dioxide

Air quality data for Cleveland, Ohio, for the period of 1967 to 1971 were collated and subjected to statistical analysis. The total suspended particulate component is lognormally distributed; while sulfur dioxide and nitrogen dioxide are reasonably approximated by lognormal distributions. Only sulfur dioxide, in some residential neighborhoods, meets Ohio air quality standards. Air quality has definitely improved in the industrial valley, while in the rest of the city, only sulfur dioxide has shown consistent improvement. A pollution index is introduced which displays directly the degree to which the environmental air conforms to mandated standards.

Neustadter, H. E.↗
DOE OSTI2025

Online and Offline Data Quality Monitoring for the Mu2e Calorimeter

This thesis presents the design, implementation, and validation of a calorimeter Data Quality Monitoring (DQM) toolchain for the Mu2e experiment at Fermilab. Mu2e searches for charged lepton flavor violation via coherent muon-to-electron conversion in the field of an aluminum nucleus, $\mu^- Al \rightarrow e^-Al$, a process whose observation would constitute clear evidence of physics beyond the Standard Model. Achieving target sensitivity requires stringent control of detector performance and data integrity during acquisition, as subtle issues in readout configuration, data formatting, or electronics behavior can compromise reconstruction and bias downstream analyzes. To address these challenges, this work develops a multi-layer DQM approach spanning both raw data validation and reconstructed digi-level diagnostics. At the low level, a fragment analysis component performs word- and bit-field decoding of calorimeter readout blocks, enabling sanity checks of the expected structure and producing detailed error and integrity statistics useful for commissioning and troubleshooting. At the digi level, the CaloDigiDQM analyzer is implemented within the art framework and transforms each CaloDigiCollection into a structured hierarchy of ROOT histograms designed for fast drill-down diagnostics. The module generates coherent monitoring views at global, disk, board, and channel granularity, including occupancy, waveform-derived features (baseline, RMS, peak amplitude and position), and left-right sensor consistency metrics. Detector-aware channel-to-electronics mapping is performed through the conditions system (CaloDAQMap), ensuring that diagnostics remain aligned with hardware identifiers used in operations. For end-to-end testing without reliance on live DAQ data, a synthetic CaloDigi producer is developed to generate realistic waveforms with controlled noise and pulse shapes. The resulting system supports both offline ROOT-file production and online operation, including optional histogram streaming through otsdaq via ots::HistoSender. This toolchain provides a practical and scalable foundation for calorimeter commissioning and stable data collection, enabling early detection of anomalies and reducing operational risk for Mu2e.

Vakulenko, Mark [Drew U.] (ORCID:0009000276197818)↗
DOE OSTI2025

Hydra: An AI-Based Framework for Interpretable and Portable Data Quality Monitoring

Hydra is an advanced framework designed for training and managing AI models for near real time data quality monitoring at Jefferson Lab. Deployed in all four experimental halls, Hydra has analyzed over 2 million images and has extended its capabilities to offline monitoring and validation. Hydra utilizes computer vision to continually analyze sets of images of monitoring plots generated 24/7 during experiments. Generally, these sets of images are produced at a rate and quantity that is exceedingly difficult for shift crews to effectively monitor. Significant effort has been devoted to enhancing Hydra’s user interface, to ensure that it provides clear, actionable insights for shift workers and other users. Gradient Weighted Class Activation Maps (GradCAM) provide added interpretability, allowing users to visualize important regions of the image for classification. Hydra has been containerized to enable the creation of portable demos and seamless integration with container-based technologies such as Kubernetes and Docker. With the user interface enhancements and containerization, Hydra can be rapidly deployed for new use cases and experiments. This talk will describe the Hydra framework, its user interface and experience, and the challenges inherent in its design and deployment.

Britton, Thomas [Thomas Jefferson National Acceler↗
NASA NTRS2011

Effects of Active Sting Damping on Common Research Model Data Quality

Recent tests using the Common Research Model (CRM) at the Langley National Transonic Facility (NTF) and the Ames 11-foot Transonic Wind Tunnel (11' TWT) produced large sets of data that have been used to examine the effects of active damping on transonic tunnel aerodynamic data quality. In particular, large statistically significant sets of repeat data demonstrate that the active damping system had no apparent effect on drag, lift and pitching moment repeatability during warm testing conditions, while simultaneously enabling aerodynamic data to be obtained post stall. A small set of cryogenic (high Reynolds number) repeat data was obtained at the NTF and again showed a negligible effect on data repeatability. However, due to a degradation of control power in the active damping system cryogenically, the ability to obtain test data post-stall was not achieved during cryogenic testing. Additionally, comparisons of data repeatability between NTF and 11-ft TWT CRM data led to further (warm) testing at the NTF which demonstrated that for a modest increase in data sampling time, a 2-3 factor improvement in drag, and pitching moment repeatability was readily achieved not related with the active damping system.

Acheson, Michael J.↗