Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data quality analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

NMF-Based Anomaly Detection in CMS 2D Tracking Occupancy Histograms

The CMS experiment relies on Data Quality Monitoring (DQM) to ensure that recorded collision data are suitable for physics analysis. During LHC Run 3, each run contains many lumisections and tracking monitoring elements, making offline inspection challenging, especially for localized detector effects that may appear only for short periods of time. This poster presents an unsupervised machine-learning approach to identify anomalous lumisections in CMS tracking occupancy histograms using Non-Negative Matrix Factorization (NMF). The workflow uses offline CMS DQMIO tracking histograms retrieved with the CMS DIALS API and organized as two-dimensional occupancy maps for each lumisection. After selecting stable lumisections, the occupancy maps are normalized and arranged into a non-negative data matrix. The NMF model learns a compact set of basis patterns describing normal tracking occupancy. Each lumisection is then reconstructed from these learned components, and the reconstruction error is used as an anomaly score. Large residuals indicate occupancy patterns that deviate from normal detector behavior and are flagged for further inspection. This NMF-based approach provides a fast and interpretable way to flag lumisections whose tracking occupancy patterns differ from normal detector behavior. Preliminary studies show sensitivity to known tracking anomalies, and ongoing work is focused on validating the method across additional Run 3 Pixel and Strip detector issues.

Rodríguez Ramos, Iliomar [Puerto Rico U., Mayaguez↗

Assessment of BQ-9000 Biodiesel Properties for 2022

This is the sixth in a series of reports documenting the quality of biodiesel from U.S. and Canadian-based producers that participate in the BQ-9000 program, the biodiesel industry voluntary quality assurance program. Participants agreed to provide monthly data on critical quality parameters for calendar year 2022. The quality data was provided to a team of experts, who removed any identifying company information and provided anonymized data to the National Renewable Energy Laboratory (NREL) for statistical analysis.

09 BIOMASS FUELS↗

Status Update on the High Precision Isotopic Measurements on High Burnup LWR Fuel in 2020

The US Department of Energy (DOE) Office of Nuclear Energy (NE) is currently investigating the feasibility of directly disposing dual-purpose (storage and transportation) canisters (DPCs) in a spent nuclear fuel (SNF) repository. Criticality during the repository performance period (10,000 years or more) is one of the major concerns related to direct disposal of DPCs, specifically as the system undergoes degradation in the repository environment and timeframe. Oak Ridge National Laboratory (ORNL) is developing an as-loaded criticality analysis methodology using full (actinides + fission products) burnup credit that exploits the inherent criticality margin associated with actual canister-specific loading configuration. Burnup credit criticality analysis requires validation of the depletion/decay codes used to generate the burned isotopic inventory of an assembly by comparing the code-predicted inventory with experimentally determined isotopic data. Currently, isotopic measurement data for boiling water reactors (BWR) SNF are limited, and additional measurements will be highly beneficial for BWR burnup credit analysis, which is essential to demonstrate disposability of BWR DPCs. Moreover, new pressurized water reactor (PWR) samples of isotopic measurements will expand the PWR sample population and consequently will greatly improve sample statistics. In turn, this will reduce uncertainty in the computational determination of the isotopic composition of commercial SNF and eliminate additional penalties currently used for lack of data. As part of the isotopic analysis task, eight diverse, high burnup (HBU) samples from several PWR rods are being dissolved and isotopically analyzed to provide high-quality measurement data for PWR SNF to reducing the uncertainties associated with PWR isotopic depletion validation. The isotopics of interest focus on nuclides important to burnup credit, shielding, and decay heat in PWR SNF. Additionally, eight high burnup BWR fuel samples from the Limerick nuclear power plant have been identified for measurement to support reducing uncertainties associated with BWR isotopic depletion validation.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Next Generation Hydrogen Station Composite Data Products: Retail Stations (Data through Quarter 3 of 2019)

This publication includes 98 composite data products (CDPs) produced for next generation hydrogen stations, with data through the third quarter of 2019. These CDPs include data from retail stations only and represent the following analysis categories: deployment, safety, maintenance and reliability, performance, cost, utilization, hydrogen quality, and component energy.

08 HYDROGEN↗

Optimal dimensionality selection for independent component analysis of transcriptomic data

Independent component analysis is an unsupervised machine learning algorithm that separates a set of mixed signals into a set of statistically independent source signals. Applied to high-quality gene expression datasets, independent component analysis effectively reveals both the source signals of the transcriptome as co-regulated gene sets, and the activity levels of the underlying regulators across diverse experimental conditions. Two major variables that affect the final gene sets are the diversity of the expression profiles contained in the underlying data, and the user-defined number of independent components, or dimensionality, to compute. Availability of high-quality transcriptomic datasets has grown exponentially as high-throughput technologies have advanced; however, optimal dimensionality selection remains an open question. We computed independent components across a range of dimensionalities for four gene expression datasets with varying dimensions (both in terms of number of genes and number of samples). We computed the correlation between independent components across different dimensionalities to understand how the overall structure evolves as the number of user-defined components increases. We then measured how well the resulting gene clusters reflected known regulatory mechanisms, and developed a set of metrics to assess the accuracy of the decomposition at a given dimension. We found that over-decomposition results in many independent components dominated by a single gene, whereas under-decomposition results in independent components that poorly capture the known regulatory structure. From these results, we developed a new method, called OptICA, for finding the optimal dimensionality that controls for both over- and under-decomposition. Specifically, OptICA selects the highest dimension that produces a low number of components that are dominated by a single gene. We show that OptICA outperforms two previously proposed methods for selecting the number of independent components across four transcriptomic databases of varying sizes. OptICA avoids both over-decomposition and under-decomposition of transcriptomic datasets resulting in the best representation of the organism’s underlying transcriptional regulatory network.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

PVAnalytics: A Python Package for Automated Processing of Solar Time Series Data

Multiple publicly available software packages exist that analyze solar time series data, including RdTools and Solar Data Tools, among others. Several of these packages contain their own unique quality assurance (QA) and feature recognition algorithms. The python PVAnalytics package was developed to offer an internally consistent source for these analysis tools, making it easier for the end user to deploy these routines on his or her solar data. The PVAnalytics package currently contains routines for outlier detection, inverter clipping detection, irradiance and temperature checks, orientation checks, and data shift detection, among other functions. These functions have been aggregated from various sources including Solar Forecast Arbiter, RdTools, and the QA process developed by NREL's PV Fleets Initiative. We are continuously adding new functionality to the package, including documentation, examples and algorithms. By bundling QA functionality into a single software package, we hope to make PVAnalytics a comprehensive software library to support analysis of solar metadata and time series data.

data cleaning↗

A comparison of the power potential for surface- and seabed-deployed tidal turbines in the San Juan Archipelago, Salish Sea, WA

The San Juan archipelago lies along the axis of tidal movement between Straits of Juan de Fuca and Strait of Georgia in the Salish Sea. The amplitude of the tidal exchange produces significant tidal currents between the islands, as well as in Rosario and Haro Straits. These are of interest as a future source of electrical power generation, given the islands’ dependence on electricity supply by a subsea cable from the mainland. Here, we evaluate the tidal current energy potential in this region through a re-analysis of measurements collected by the National Ocean Service (NOS) and a high-resolution numerical model. Given the considerable variations in water depth and vertical velocity profiles across candidate tidal energy sites, we consider the trade-offs between tidal turbines deployed from a floating platform and those anchored to the seabed. Measurement re-analysis indicates several locations that could support tidal current power generation by MW-scale turbines with an acceptable balance between turbine size, rated power, and capacity factor. Even for relatively large (30 m) turbine diameters, surface-deployed turbines would be expected to produce up to 30 % more electricity than the same turbine deployed near the seabed due to vertical shear, with this difference increasing for smaller diameter turbines. A benchmark comparison at the measurement locations with the highest power generation potential shows relatively good model fidelity, though, even with relatively high resolution, time-average power density disagreements of ± 50 % persist throughout the water column. Nonetheless, the simulation identifies several locations with power generation potential more than twice as high as locations in the measurement re-analysis. These sites were either not surveyed by NOS or excluded due to data quality issues. Altogether, these results demonstrate the complimentary of models and measurements for tidal energy site assessment and suggest that tidal currents could be an important source of electricity generation in the San Juan archipelago.

16 TIDAL AND WAVE POWER↗

A data compression and optimal galaxy weights scheme for Dark Energy Spectroscopic Instrument and weak lensing data sets

Combining different observational probes, such as galaxy clustering and weak lensing, is a promising technique for unveiling the physics of the Universe with upcoming dark energy experiments. The galaxy redshift sample from the Dark Energy Spectroscopic Instrument (DESI) will have a significant overlap with major ongoing imaging surveys specifically designed for weak lensing measurements: the Kilo-Degree Survey (KiDS), the Dark Energy Survey (DES), and the Hyper Suprime-Cam (HSC) survey. In this work, we analyse simulated redshift and lensing catalogues to establish a new strategy for combining high-quality cosmological imaging and spectroscopic data, in view of the first-year data assembly analysis of DESI. In a test case fitting for a reduced parameter set, we employ an optimal data compression scheme able to identify those aspects of the data that are most sensitive to cosmological information and amplify them with respect to other aspects of the data. We find this optimal compression approach is able to preserve all the information related to the growth of structures.

79 ASTRONOMY AND ASTROPHYSICS↗

The Vera C. Rubin Observatory Data Preview 1

We present Rubin Data Preview 1 (DP1), the first data from the National Science Foundation–Department of Energy Vera C. Rubin Observatory, comprising raw and calibrated single-epoch images, coadds, difference images, detection catalogs, and ancillary data products. DP1 is based on 1792 optical–near-infrared exposures acquired over 48 distinct nights by the Rubin Commissioning Camera (LSSTComCam) on the Simonyi Survey Telescope at the Summit Facility on Cerro Pachón, Chile in late 2024. DP1 covers ∼15 deg 2 distributed across seven roughly equal-sized noncontiguous fields, each independently observed in six broad photometric bands, ugrizy. The median FWHM of the point-spread function across all bands is approximately 1"14, with the sharpest images reaching about 0." 58. The 5σ point-source depths for coadded images in the deepest field, the Extended Chandra Deep Field South, are u = 24.55, g = 26.18, r = 25.96, i = 25.71, z = 25.07, and y = 23.1. Other fields are no more than 2.2 mag shallower in any band, where they have nonzero coverage. DP1 contains approximately 2.3 million distinct astrophysical objects, of which 1.6 million are extended in at least one band in coadds, and 431 solar system objects, of which 93 are new discoveries. DP1 is approximately 3.5 TB in size and is available to Vera C. Rubin Observatory data rights holders via the Rubin Science Platform, a cloud-based environment for the analysis of petascale astronomical data. While small compared to future LSST releases, its high quality and diversity of data support a broad range of early science investigations ahead of full operations in 2026.

Ground-based astronomy↗

Machine Learning-Based Anomaly Detection for PMT Data Quality Monitoring in the SBN and DUNE

Maintaining high-quality detector data is essential for achieving the scientific objectives of the Short-Baseline Neutrino (SBN) Program at Fermilab. Current data quality monitoring (DQM) procedures rely primarily on threshold-based metrics and manual inspection of detector monitoring plots, making the detection of subtle or gradually developing anomalies both time-consuming and dependent on expert interpretation. This project developed and evaluated a machine-learning workflow for automatically identifying anomalous photomultiplier tube (PMT) channels in the Short-Baseline Near Detector (SBND) using optical-hit amplitude data. A Python-based analysis program was developed to process ROOT files, extract statistical features describing individual PMT amplitude distributions, and generate feature vectors for anomaly detection. These features were used to train an Isolation Forest model using data representing normal detector operation. The trained model was subsequently applied to independent detector runs to identify channels exhibiting statistically unusual behavior relative to the learned reference response. To support expert interpretation, the workflow generated complementary diagnostic products, including anomaly score distributions, normalized amplitude comparisons, decision-tree visualizations, and principal component analysis (PCA) projections. This project demonstrated the feasibility of integrating unsupervised machine learning into detector data-quality monitoring and developed a complete workflow for automated PMT performance assessment to aid expert-driven review. Beyond its technical contributions, the VFP appointment fostered a research collaboration between Aurora University and Fermilab and provided direct workforce development benefits by training the visiting faculty member in detector-scale machine-learning methods that are now being incorporated into undergraduate coursework and research. The methodology developed here provides a foundation for future applications to ProtoDUNE and other liquid argon time projection chamber (LArTPC) detectors, contributing to ongoing efforts to improve detector reliability, reduce manual monitoring requirements, and enable scalable data quality monitoring for future large-scale neutrino experiments, including the Deep Underground Neutrino Experiment (DUNE).

Colón Santana, Juan A. [Unlisted, US, IL]↗

Reducing Uncertainty of Fielded Photovoltaic Performance (Final Technical Report)

Improved analysis and reporting of photovoltaic (PV) field performance increases the certainty of owners and financiers that systems will perform as expected. Advanced module technologies (e.g., PERC, HJT, and bifacial) introduce new degradation mechanisms and performance characteristics. The FY19-21 Reducing Uncertainty project leveraged data from the ever-increasing PV fleet to develop models and understanding of the field performance of existing and new technologies. Specifically, we accomplished: report on field performance and degradation rates for high-efficiency silicon (HJT, PERC, IBC) and more conventional technologies; developed automated analysis techniques to quantify system performance (performance ratio, energy yield) and production shortfalls (soiling, degradation, availability); refined the RdTools software toolkit to bring standard, validated analysis techniques to bear on third-party data; analyzed and reported on large datasets including Treasury data and Lawrence Berkeley National Laboratory's Utility-Scale dataset to expand the high-quality degradation-rate histogram published previously; worked with industry partners and the DuraMAT data hub to enable private parties to share and aggregate PV production data anonymously, leveraging cloud-based data analysis infrastructure and publishing on US fleet-scale performance comprising over 7GW of operating systems. (https://www.nrel.gov/pv/fleet-performance-data-initiative.html). Through our industry collaborations we have engaged in NDA-covered data transfer with twelve PV fleet owners as of January 2022, with more agreements in negotiation. Our scalable cloud-based time series database contains over 30 billion rows (20TB) of PV time series data, representing over 1700 commercial and utility-scale systems, and over 7.2 GW of DC capacity (Fig 1). Initial field performance results have been distributed in several public reports. Because our fleet composition and data quality methods are continually improving, annual updates to these results are published to our PV Fleet webpage [ https://www.nrel.gov/pv/fleet-performance-data-initiative.html ] and DuraMAT data hub [DOI: 10.21948/1842958]. Another existing dissemination channel used for observed soiling losses is a map we maintain for soiling losses. Additional products developed include a report detailing fleet-wide performance index, availability, startup loss and snow loss factors, a detailed report on the 1603 grant dataset comprising over 100,000 PV systems with failure and performance details and a utility-scale report coauthored with LBNL on 31 GW of system performance.

14 SOLAR ENERGY↗

Methods for a blind analysis of isobar data collected by the STAR collaboration

In 2018, the STAR collaboration collected data from $_{44}^{96}{\mathrm{Ru}}+_{44}^{96}{\mathrm{Ru}}$ and $_{40}^{96}{\mathrm{Zr}}+_{40}^{96}{\mathrm{Zr}}$ at $\sqrt{s_\text {NN}}=200$ GeV to search for the presence of the chiral magnetic effect in collisions of nuclei. The isobar collision species alternated frequently between $_{44}^{96}{\mathrm{Ru}}+_{44}^{96}{\mathrm{Ru}}$ and $_{40}^{96}{\mathrm{Zr}}+_{40}^{96}{\mathrm{Zr}}$ . In order to conduct blind analyses of studies related to the chiral magnetic effect in these isobar data, STAR developed a three-step blind analysis procedure. Analysts are initially provided a “reference sample” of data, comprised of a mix of events from the two species, the order of which respects time-dependent changes in run conditions. After tuning analysis codes and performing time-dependent quality assurance on the reference sample, analysts are provided a species-blind sample suitable for calculating efficiencies and corrections for individual $\approx 30$-min data-taking runs. For this sample, species-specific information is disguised, but individual output files contain data from a single isobar species. Only run-by-run corrections and code alteration subsequent to these corrections are allowed at this stage. Following these modifications, the “frozen” code is passed over the fully un-blind data, completing the blind analysis. As a check of the feasibility of the blind analysis procedure, analysts completed a “mock data challenge,” analyzing data from Au + Au collisions at $\sqrt{s_\text {NN}}=27$ GeV, collected in 2018. The Au + Au data were prepared in the same manner intended for the isobar blind data. Finally, the details of the blind analysis procedure and results from the mock data challenge are presented.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Anomaly Detection in Seismic Data with Deep Learning: Application for Instrument Failure Detection and Forecasting

Seismic data quality assessment (QA) is the first and one of the most important steps before conducting any further data analysis. Traditional methods involve checking various metrics, such as spike detection and power spectral density, by setting strict thresholds or comparing data against synthetic benchmarks. However, these approaches often rely on pre-existing knowledge and assumptions about data anomalies, leading to potential misclassification of unusual cases. Here, in this study, we propose a deep autoencoder model, an unsupervised learning approach that evaluates data quality without making assumptions about normal and anomalous data, which can be used to identify deviations in recorded data that may indicate nascent instrument failure. We test the model with the U.S. International Monitoring System (IMS) seismic stations and demonstrate the capability of detecting anomalies on a monthly scale. This could prompt station operators to examine potential problems early, allowing sufficient time for instrument maintenance to prevent data outages. In addition, we use a new manually selected testing dataset to compare our model performance against two supervised machine learning (ML) approaches and a standard QA package, as baseline models. When applied to the dataset containing known data anomalies, performance of the supervised and unsupervised ML approaches is similar, with an accuracy of 88.1% for our model compared to ∼90% for the supervised ML approach and 78.2% for the standard QA package. Our model outperforms the baseline models when applied to new stations, where new types of data anomalies can be station-specific and not included in the training dataset. Finally, we show model transferability by training the model with data from the Global Seismograph Network only and applying it to the IMS network data. The results suggest that our model is generalizable and can be applied to new stations with good accuracy.

Lin, Jiun-Ting [Lawrence Livermore National Labora↗

Optimization of Energy Flow through Synthetic Metabolic Modules and Regulatory Networks in a Model Photosynthetic Eukaryotic Microbe

Photosynthetic organisms have recently gained considerable attention for a role in development of renewable energy sources. Genome-enabled systems biology methods, coupled with functional and synthetic genomics, present opportunities to develop sustainable and economical applications such as fuel production within the next 10 to 15 years. However, optimization of light-driven metabolism for biomass or biofuel production will require a detailed systems biology understanding of photosynthetic processes and cellular metabolism. Genome-scale metabolic models (GEMs) are at the core of systems analysis of cellular processes and form a common organizational framework for analyses of data resulting from functional genomics experimental work and computational studies. Therefore, there is a clear demand for high quality photosynthetic model organisms and the appropriate computational tools that enable systems analysis of light-driven metabolism. Through research conducted we expanded the currently available repertoire of photosynthetic GEMs to include the commercially valuable model diatom Phaeoctylum tricornutum. Diatoms have a peculiar and distinct evolutionary footprint and represent a major eukaryotic lineage that is taxonomically and functionally distinct from green and red algae and vascular plants. Therefore, the true potential for light-driven metabolism aimed at biofuel production remains poorly understood at a systems level for a large subset of the global diversity of photosynthetic organisms. The metabolic capabilities of P. tricornutum were comparatively modeled with those from other photosynthetic groups in order to elucidate the occurrence of metabolic traits within and between phototrophs. Additionally, this research resulted in significant extension of the COnstraints Based Reconstruction and Analysis (COBRA) Toolbox to accommodate the crucial need for infrastructure required for ‘omics data integration and analysis in the context of genome-scale models. Therefore, the proposed research achieved two important goals. First, within the broad scope of photosynthetic organisms, we functionally compared and, as a result, identified cellular processes that require optimization in order to enable deployment as biofuel feedstock. Second, the proposed research resulted in development of key computational infrastructure, which can be further extended to other biological systems, that is currently lacking but necessary for multiple ‘omics data integration.

59 BASIC BIOLOGICAL SCIENCES↗

Assessment of BQ-9000 Biodiesel Properties for 2023

This is the seventh in a series of reports documenting the quality of biodiesel from U.S.- and Canadian-based producers that participate in the BQ-9000 program, the biodiesel industry’s voluntary quality assurance program. Participants provided monthly data on critical quality parameters for calendar year 2023 with quality data provided to a team of experts, who removed any identifying company information and provided anonymized data to the National Renewable Energy Laboratory (NREL) for statistical analysis. New for 2023, data on kinematic viscosity, sulfated ash, distillation temperature, carbon residue, and cetane were collected, as well as individual levels of sodium, potassium, calcium, and magnesium. Critical quality parameters analyzed are listed in Table ES-1 with descriptive statistics.

09 BIOMASS FUELS↗

Combining cosmic shear data with correlated photo-z uncertainties: constraints from DESY1 and HSC-DR1

An accurate calibration of the source redshift distribution p(z) is a key aspect in the analysis of cosmic shear data. This, one way or another, requires the use of spectroscopic or high-quality photometric samples. However, the difficulty to obtain colour-complete spectroscopic samples matching the depth of weak lensing catalogs means that the analyses of different cosmic shear datasets often use the same samples for redshift calibration. This introduces a source of statistical and systematic uncertainty that is highly correlated across different weak lensing datasets, and which must be accurately characterised and propagated in order to obtain robust cosmological constraints from their combination. In this paper we introduce a method to quantify and propagate the uncertainties on the source redshift distribution in two different surveys sharing the same calibrating sample. The method is based on an approximate analytical marginalisation of the p(z) statistical uncertainties and the correlated marginalisation of residual systematics. We apply this method to the combined analysis of cosmic shear data from the DESY1 data release and the HSC-DR1 data, using the COSMOS 30-band catalog as a common redshift calibration sample. We find that, although there is significant correlation in the uncertainties on the redshift distributions of both samples, this does not change the final constraints on cosmological parameters significantly. The same is true also for the impact of residual systematic uncertainties from the errors in the COSMOS 30-band photometric redshifts. Additionally, we show that these effects will still be negligible in Stage-IV datasets. Finally, the combination of DESY1 and HSC-DR1 allows us to constrain the "clumpiness" parameter to S 8 = ${0.768}_{-0.017}^{+0.021}$. This corresponds to a ~√(2) improvement in uncertainties with respect to either DES or HSC alone.

79 ASTRONOMY AND ASTROPHYSICS↗

Cinema:Snap: Real-time tools for analysis of dynamic diamond anvil cell experiment data

We report we developed tools and a workflow for real-time analysis of data from dynamic diamond anvil cell experiments performed at user light sources. These tools allow users to determine the phases of matter observed during the compression of materials in order to make decisions during an experiment to improve the quality of experimental results and maximize the use of scarce experimental facility time. The tools fill a gap in dynamic compression data analysis tools that are real-time, are flexible to the needs of high-pressure scientists, connect to automated processing of results, can be easily incorporated into workflows with existing tools and data formats, and support remote experimental data analysis workflows. Specific analytics developed include novel automated two-peak analysis for overlapping peaks and multiple phases, coordinated views of pressure and temperature values, full-compression contour plots, and configurable views of integrated x-ray diffraction. We present an experimental use case to show how the tools produce real-time analytics that help the scientists revise parameters for the next compression

47 OTHER INSTRUMENTATION↗

Carbon fiber classification using raman spectroscopy

Carbon fiber characterization processes are described that include multi-condition Raman spectroscopy-based examination combined with multivariate data analyses. Methods are a nondestructive material characterization approach that can provide predictions as to carbon fiber bulk physical properties, as well as identification of unknown carbon fiber materials for quality control purposes. The framework of the multivariate analysis methods includes a principal component-based identification protocol including comparison of Raman spectral data from an unknown carbon fiber with a data library of multiple principal component spaces.

Houk, Amanda L.↗