Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data quality analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

How Climate and Data Quality Impact Photovoltaic Performance Loss Rate Estimations

Different data pipelines and statistical methods are applied to photovoltaic (PV) performance datasets to quantify the performance loss rate (PLR). Since the real values of PLR are unknown, a variety of unvalidated values are reported. As such, the PV industry commonly assumes PLR based on statistically extracted ranges from the literature. However, the accuracy and uncertainty of PLR depend on several parameters including seasonality, local climatic conditions, and the response of a particular PV technology. In addition, the specific data pipeline and statistical method used affect the accuracy and uncertainty. To provide insights, a framework of (≈200 million) synthetic simulations of PV performance datasets using data from different climates is developed. Time series with known PLR and data quality are synthesized, and large parametric studies are conducted to examine the accuracy and uncertainty of different statistical approaches over the contiguous US, with an emphasis on the publicly available and “standardized” library, RdTools . In the results, it is confirmed that PLRs from RdTools are unbiased on average, but the accuracy and uncertainty of individual PLR estimates vary with climate zone, data quality, PV technology, and choice of analysis workflow. Best practices and improvement recommendations based on the findings of this study are provided.

14 SOLAR ENERGY↗
DOE OSTI2021

Prospects for Galactic and stellar astrophysics with asteroseismology of giant stars in the TESS continuous viewing zones and beyond

ABSTRACT The NASA Transiting Exoplanet Survey Satellite (NASA-TESS) mission presents a treasure trove for understanding the stars it observes and the Milky Way, in which they reside. We present a first look at the prospects for Galactic and stellar astrophysics by performing initial asteroseismic analyses of bright (G < 11) red giant stars in the TESS southern continuous viewing zone (SCVZ). Using three independent pipelines, we detect νmax and Δν in 41 per cent of the 15 405 star parent sample (6388 stars), with consistency at a level of $\sim \! 2{{\ \rm per\ cent}}$ in νmax and $\sim \! 5{{\ \rm per\ cent}}$ in Δν. Based on this, we predict that seismology will be attainable for ∼3 × 105 giants across the whole sky and at least 104 giants with ≥1 yr of observations in the TESS-CVZs, subject to improvements in analysis and data reduction techniques. The best quality TESS-CVZ data, for 5574 stars where pipelines returned consistent results, provide high-quality power spectra across a number of stellar evolutionary states. This makes possible studies of, for example, the asymptotic giant branch bump. Furthermore, we demonstrate that mixed ℓ = 1 modes and rotational splitting are cleanly observed in the 1-yr data set. By combining TESS-CVZ data with TESS-HERMES, SkyMapper, APOGEE, and Gaia, we demonstrate its strong potential for Galactic archaeology studies, providing good age precision and accuracy that reproduces well the age of high [α/Fe] stars and relationships between mass and kinematics from previous studies based on e.g. Kepler. Better quality astrometry and simpler target selection than the Kepler sample makes this data ideal for studies of the local star formation history and evolution of the Galactic disc. These results provide a strong case for detailed spectroscopic follow-up in the CVZs to complement that which has been (or will be) collected by current surveys.

Mackereth, J. Ted↗
DOE OSTI2024

Uncertainty Quantification in GADRAS Inverse Modeling

The Gamma Detector Response and Analysis Software (GADRAS) package includes an inverse modeling tool that is helpful in identifying characteristics of unknown radioactive materials. Traditionally, uncertainties in this analysis were derived solely from measurement data quality and the fit of synthetic spectra. This paper aims to rigorously quantify additional sources of uncertainty, focusing on uncertainties arising from measurements being analyzed, Detector Response Function (DRF) characterization, and DRF extrapolation. Applying these findings to the BeRPBall benchmark data set, we demonstrated the impact of these uncertainties on plutonium and polyethylene estimates. The results underscore the importance of incorporating diverse uncertainty sources to enhance the accuracy and reliability of GADRAS’s inverse modeling capabilities.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Understanding the trade-offs of national municipal solid waste estimation methods for circular economy policy

Policies embracing circular economy concepts have taken hold in national legislation around the world. As the number of governments and organizations adopting circular economy policies increases, so does the need for accurate and timely measurement of material resource flows. Since many countries do not have access to centrally reported municipal solid waste (MSW) data, estimation and modeling are critical in evaluating circular economy policy effectiveness. The purpose of this paper is to examine three modeling approaches estimating national MSW data in the United States, including industry-based material flow analysis, waste-extended input-output modeling, and aggregated regional waste reporting. We establish five criteria to guide the analysis through the context of policy monitoring (data quality, flow totality, update frequency, sensitivity to disruption, and product granularity) and use these criteria to analyze and score each model. We then use a literature search to identify five, internationally-implemented options for circular economy policy and determine the data and modeling components that are most helpful in evaluating policy effectiveness. Finally, we provide a crosswalk of the model scores and policy needs to inform the suitability of model selection by policy type. We found that data quality and update frequency are identified as critical components for evaluating circular economy policies within the models evaluated, and can both be fulfilled by aggregated regional waste reporting. Flow totality, sensitivity to disruption, and product granularity requirements vary by both model and policy types. While none of the evaluated models satisfy the combination of requirements for any of the five policies, industry-based material flow analysis offers flow totality for extended producer responsibility, landfill bans, and recycling rate target policies that typically require it. Here, the waste-extended input-output model can provide disruption sensitivity and product granularity as needed for policies like minimum recycled content and market restrictions. Policy developers in areas where strong centralized data collection is not an option should design policy action(s) with modeling tradeoffs in mind, including the potential hybridization of modeling approaches that may provide the most accurate national MSW estimates.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2025

Distribution Substation Planning Toolkit (dsp-toolkit) v1.0

The Distribution Substation Planning Toolkit (DSP Toolkit) is a software suite designed to streamline the planning and optimization of distribution substations. This toolkit offers a comprehensive set of tools and APIs for data curation, short-term electric load forecasting, and weather-sensitive load adjustment, making it an essential resource for utility companies, engineers, and researchers. Features • Data Preprocessing and Curation: Efficiently manage and preprocess large datasets to ensure high-quality input for analysis. • Short-Term Load Forecasting: Utilize data-driven models to predict short-term electric loads accurately. • Weather-Sensitive Modeling: Automatically adjust load forecasts based on weather data to predict future peak demands more precisely. Uses The DSP Toolkit is ideal for planning and optimizing distribution substations, providing a user-friendly interface and comprehensive documentation. It is suitable for both novice and experienced users, facilitating efficient and accurate planning processes. Advantages • Efficiency: Automates complex planning tasks, reducing manual effort and minimizing errors. • Scalability: Handles large datasets and complex models, making it suitable for large-scale projects. • Community and Support: Open-source with active community contributions, ensuring continuous improvement and support. • Extensibility: Easily extendable with custom modules and plugins, allowing users to tailor the toolkit to their specific needs. The DSP Toolkit stands out by offering a robust, flexible, and user-friendly solution for distribution substation planning. Public Abstract

Li, Han [Lawrence Berkeley National Laboratory (LB↗
DOE OSTI2023

ShadeLab/PAPER_Howe_2023_switchgrass_MetaT

The raw data (metagenomes and metatranscriptomes) for this study are available in the Joint Genomes Institute Genome Portal (https://genome.jgi.doe.gov/portal/ Project ID 503249) with projects designated by year and product type. The MAG genomes analyzed in this paper are available on NCBI, as bioproject PRJNA800073. Plants and microorganisms form beneficial associations. Understanding plant-microbe interactions will inform microbiome management to enhance crop productivity and resilience to stress. Here, we apply a genome-centric approach to identify ecologically important leaf microbiome members on field-grown switchgrass and miscanthus and to quantify their activities for switchgrass over two growing seasons. We integrate metagenome and metatranscriptome sequencing from 192 leaf samples collected over representative time points in crop phenology. We curated 40 medium- and high-quality metagenome-assembled-genomes (MAGs) and focused analysis on seasonal transcript recruitment to them. Classes represented by these focal MAGs (Actinomycetia, Alpha- and Gamma- Proteobacteria, and Bacteroidota) were active and had increases in transcripts for short-chain dehydrogenase, molybdopterin oxidoreductase, and polyketide cyclase in the late season. The majority of MAGs had activated stress-associated pathways, including trehalose metabolism, indole acetic acid degradation, betaine biosynthesis, and reactive oxygen species degradation, suggesting direct engagement with the host environment. We also detected seasonally activated biosynthetic pathways for terpenes (carotenoids and isoprenoids) and for various non-ribosomal peptide pathways that were poorly annotated. Overall, this study overcame laboratory and bioinformatic challenges associated with field-based leaf metatranscriptome analysis to inform both general and likely specialized activities of these phyllosphere populations. These activities collectively support that leaf-associated bacterial populations are seasonally dynamic, responsive to host cues, and interactively engage in feedback with the plant. This analysis represented quality filtering of metagenomes and metatranscriptomes (data-preparation folder), metagenome assemblies (metagenome-assembly folder) and metagenome-assembled genome binning, curation, refinement, annotation (mag-evaluation folder). Abundances of sequencing libraries were calculated based on reads mapped (mapping folder). Additionallly, analysis of our annotated results are also included (analysis folder).

Howe, Adina↗
DOE OSTI2021

KBase Silver Case Study: Determining Media Formulation Requirements for Isolation of Microbiome Constituents

KBase has powerful tools for extracting microbial genomes from metagenomes and performing phylogenomic analysis and metabolic modeling. These tools can be used to predict key media ingredients for isolating uncultured members of microbiomes. Essential to this process are high-quality genomes extracted from metagenomic assemblies, and Kbase has a tool for assessing the quality of genomes too. Below we identify growth factors for a myxobacteria ("slime bacteria") yet to be isolated from the rhizosphere of Miscanthus xgiganteus (hybrid of "Silvergrass"), cultivated at the Kellogg Biological Station in Michigan. Data was transferred with Globus from JGI-IMG. This narrative and the "KBase Gold Case Study: Can you find Delftia?" make up the Silver and Gold Narrative Set for teaching metagenomics concepts to students in the BIT 477/577 course at North Carolina State University. This tutorial will guide the user through the process of extracting and annotating high-quality genomes from a metagenomic data, performing phylogenomic analysis, building a metabolic model, and using these to predict nutrient requirments for growth and isolation of corresponding microbes.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

gcamfaostat: An R package to prepare, process, and synthesize FAOSTAT data for global agroeconomic and multisector dynamic modeling

The gcamfaostat R package is designed for the preparation, processing, and synthesis of the Food and Agriculture Organization (FAO) Statistics (FAOSTAT) agroeconomic data. The primary purpose is to facilitate FAOSTAT data use in global economic and multisector dynamic models while ensuring transparency, traceability, and reproducibility. Here, we provide an overview of the development of gcamfaostat (v1.0) and demonstrate its capabilities in generating and maintaining agroeconomic data required for the Global Change Analysis Model (GCAM). Our initiative seeks to enhance the quality and accessibility of data for the global agroeconomic modeling community, with the aim of fostering more robust and harmonized outcomes in a collaborative, efficient, and open-source framework. One of the important features of the package is the possibility to construct the FAO Food Balance Sheets at the disaggregated commodity level (with over 500 commodities), which provides a comprehensive and detailed data input for a variety of analytical and modeling applications. The processed data and visualizations offered by gcamfaostat can also be valuable to a broader audience interested in gaining insights into the intricacies of global agriculture.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

GTC Follow-up Observations of Very Metal-poor Star Candidates from DESI

Abstract The observations from the Dark Energy Spectroscopic Instrument (DESI) will significantly increase the numbers of known extremely metal-poor stars by a factor of ∼10, improving the sample statistics to study the early chemical evolution of the Milky Way and the nature of the first stars. In this paper we report follow-up observations with high signal-to-noise ratio of nine metal-poor stars identified during the DESI commissioning with the Optical System for Imaging and Low-Resolution Integrated Spectroscopy (OSIRIS) instrument on the 10.4 m Gran Telescopio Canarias. The analysis of the data using a well-vetted methodology confirms the quality of the DESI spectra and the performance of the pipelines developed for the data reduction and analysis of DESI data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Milestone 7714 National Opacity Program: High-Fidelity Iron Data on NIF

Establish the level of reproducibility of NIF iron opacity shots, quantify to-date improvements in data quality through, e.g. background reduction, improved signal, and improved analysis methods, and formalize the approach that will be taken in the future to compare NIF data, Z data, and theory.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2026

Data from: "Towards CONUS-Wide ML-Augmented Conceptually-Interpretable Modeling of Catchment-Scale Precipitation-Storage-Runoff Dynamics"

This data package was generated to support the manuscript “Towards CONUS-Wide Machine Learning-Augmented Conceptually Interpretable Modeling of Catchment-Scale Precipitation-Storage-Runoff Dynamics.” It provides input files, model outputs, plotting data, scripts, notebooks, and documentation used to develop, evaluate, and reproduce Mass-Conserving Perceptron (MCP)-based hydrologic modeling experiments across 513 selected Catchment Attributes and Meteorology for Large-sample Studies in the United States (CAMELS-US) basins. The files are organized by modeling component and analysis purpose, including rainfall–runoff experiments, snow module experiments, coupled hydrologic-snow experiments, Long Short-Term Memory (LSTM) benchmark results, model skill metrics, initialization and epoch records, cell-state normalization files, Akaike Information Criterion (AIC)-based model comparison files, and data used to generate manuscript figures. Tabular files can be opened using standard spreadsheet software or Python/R data-analysis tools. Python scripts, Jupyter notebooks, and selected MATLAB scripts are included for model execution, postprocessing, plotting, and statistical analysis. Quality assurance and quality control were conducted through the source-data selection and modeling workflow. Meteorological forcing, streamflow, and static catchment attributes were derived from the CAMELS-US dataset, and snow water equivalent data were derived from the University of Arizona (UA) Snow Water Equivalent dataset. Selected basins and time periods were screened during the associated research workflow to avoid missing observations or poor-quality cases. Static geospatial features were processed primarily using Quantum Geographic Information System (QGIS) and Geospatial Data Abstraction Library (GDAL) workflows. Additional details are provided in the associated manuscript and documentation.

ESS-DIVE CSV File Formatting Guidelines Reporting ↗
DOE OSTI2024

Sampling and Analysis Plan for LSL2 Underground Storage Tank Evaluation of PFAS in Soil: Protection of Groundwater

This SAP gathers all pertinent information relevant to the sampling and analysis of soil at the removal site for a former 10,000-gallon UST and reference sites in the vicinity of the excavation site. Data Quality Objectives establishes the process and steps for the acquisition of data. Data Flow Diagram establishes the decisions and next steps. Quality Assurance Project Plan establishes the quality requirements for data collection, including planning, implementation, and assessment of sampling, field measurements, and laboratory analysis. The work is being completed for DOE-SC Pacific Northwest Site Office.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

Enhancing nuclear data validation analysis by using machine learning

In this work, we demonstrate how machine learning (ML) techniques can be used as an auxiliary tool for nuclear data validation analysis. The output of the ML analysis can inform evaluators and validators on the quality, or lack thereof, of specific nuclear data and benchmark experiments with respect to simulating these experimental benchmark values. To this end, measured and simulated effective neutron multiplication factors, k eff , of 875 selected ICSBEP critical assemblies and the latter's sensitivities with respect to nuclear data as well as benchmarks' features (e.g., material nuclide, core geometry) are used as input for a random forest (RF) regression model. The RF is used to encode the complex inter-dependencies between thousands of nuclear data features (i.e., sensitivity profiles and aspects describing the measurements) and 875 simulated and experimental k eff values in order to understand what nuclear data features are most informative for predicting bias. The complexity of relationships and high-dimensional space is difficult-to-impossible to search fully using simply expert judgment. As a first proof-of-concept—step, it is demonstrated that this technique is able to correctly trace large discrepancies between simulated and experimental k eff back to fabricated shortcomings in nuclear data that were perturbed to simulated k eff values. In a second, real-case scenario, step, the RF algorithm is used to validate the ENDF/B-VIII.0 library in comparison to ENDF/B-VII.1 nuclear data. One case is showcased where the chosen ML algorithms highlighted nuclear data (the 19 F(n,inl) cross section from 0.4–0.9 MeV) that are shown to be problematic by comparing them to associated differential experimental data and nuclear data from other libraries. In addition to that, it is shown that the RF results point towards poor benchmark experiments and associated underestimated uncertainties (e.g., the PU-SOL-THERM-028 series). However, using the RF algorithm for validating nuclear data with respect to k eff is currently limited to pinpointing groups of questionable nuclear data due to the inherent correlations between features introduced by the nuclear data themselves and how k eff is simulated. Due to this, we recommend that the ML methods presented be used to augment—rather than replace—the expert knowledge of evaluators and validators.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2025

Indoor Air Quality (IAQ) Monitoring for Space Farming Institute [Slides]

Through the U.S. Department of Energy's Energy to Communities (E2C) program, NREL, other national laboratory experts, and select organizations provide Expert Match - free, short-term technical assistance to address near-term energy challenges and questions. Expert Match is for community stakeholders who have decision-making power or influence in their community but need access to additional energy expertise to inform key upcoming decisions. This Expert Match request supported the Space Farming Institute, a nonprofit organization located in Anchorage, AK, with an indoor air quality analysis. The NREL team analyzed indoor air quality data provided by the Space Farming Institute, which experts at PNNL used to design an indoor bioreactor to grow Ulva algae for indoor air quality mitigation purposes.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2025

Data and Code for Understanding Generative AI Content with Embedding Models

This repository contains code for the experiments in the paper "Understanding Generative AI Content with Embedding Models". Constructing high-quality features is critical to any quantitative data analysis. While feature engineering was historically addressed by carefully hand-crafting data representations based on domain expertise, deep neural networks (DNNs) now offer a radically different approach. DNNs implicitly engineer features by transforming their input data into hidden feature vectors called embeddings. For embedding vectors produced by foundation models -- which are trained to be useful across many contexts -- we demonstrate that simple and well-studied dimensionality-reduction techniques such as Principal Component Analysis uncover inherent heterogeneity in input data concordant with human-understandable explanations. Of the many applications for this framework, we find empirical evidence that there is intrinsic separability between real samples and those generated by artificial intelligence (AI).

Vargas, Max [Pacific Northwest National Laboratory↗
DOE OSTI2025

Cleaned 5-Minute Resolution Air Quality and Meteorological Data from Nine TCEQ CAMS Sites in Houston, Texas (Nov 2021 – Oct 2022)

These data encompass 5-minute air monitoring and meteorological observations collected in the greater Houston, Texas metropolitan region, at nine (9) Continuous Ambient Monitoring Stations (CAMS) operated by the Texas Commission on Environmental Quality (TCEQ) between November 1, 2021 and October 31, 2022. The CAMS sites (CAMS 1, 8, 35, 45, 148, 403, 405, 410, and 1052) were chosen because their instrumentation includes measurements of PM2.5. These sites also provide continuous multi-parameter air-quality and meteorological measurements. Particulate matter (PM2.5, PM10) was sampled along with several trace gases, including ozone (O3), nitrogen oxides (NO, NO2, NOx), sulfur dioxide (SO2), and carbon monoxide (CO). The data set also contains standard surface meteorological parameters (temperature, humidity, pressure, wind speed, and wind direction). Several sites also include AutoGC-based measurements of volatile organic compounds (VOCs). Air monitoring instruments deployed at the selected sites comprise the following systems: BAM-1020 or TEOM (PM2.5), Thermo Scientific TEI 49i (O3), TEI 42i (NOx), and AutoGCs (VOCs). This data set is similar to the data included within the houairq5mX1.00 datastream, except for a few additional quality control steps. A systematic data cleaning and verification process was performed on the data set to ensure its quality and preparation for analysis. Removal of non-numeric status flags (e.g., [LIM], [QAS], [SPZ], [CAL], [PMA], [AQI], [SPN], [MAL]) was accomplished by employing rule-based string parsing to extract valid numerical values. Missing entries were set to -9999; however, invalid or anomalous values (e.g., 99999) were retained as originally reported by the TCEQ to preserve data provenance. The time sequence was verified for completeness, removal of duplicates, and uniformity at 5-minute intervals. Column labeling was standardized, and corresponding values were assessed for physical plausibility. All timestamps in the data set were reported in Coordinated Universal Time (UTC) as provided by the TCEQ. Further, the latitude and longitude coordinates were added for each CAMS site. A subset of the data (June 1–September 30, 2022) has been used in the following publication: Subba et al. 2025. “Implications of sea breeze circulations on boundary layer aerosols in the southern coastal Texas region.” EGUsphere 2025: 1–49, https://doi.org/10.5194/egusphere-2025-2659.

latitude↗
DOE OSTI2022

Riverine Plastic Pollution: Sampling and Analysis Methods

Riverine plastic pollution has been found in all major U.S. rivers, but the exact amount of plastic being released to the oceans has not been quantified. Field studies conducted in U.S. rivers have used a range of sampling and analysis techniques and rarely measured the mass of the plastic collected. Measurements of riverine plastic pollution are needed to calibrate and validate models used to estimate the U.S. riverine plastic emissions to the oceans. This report surveys measurement methods used to quantify riverine pollution and current estimates of U.S. riverine plastic pollution from measurements and models. Measurement methods include field sampling and laboratory analysis. Field sampling methods are described for large (macro) and small (micro) plastic particles. Laboratory analysis methods are described for macro and microplastic with an emphasis on the detailed characterization processes of microplastics. Waterborne leachate analysis is also briefly described. Three models are described that estimate plastic pollution based on mismanaged plastic waste in the river catchment basins. The models were validated and calibrated with global data sources. The data sources were predominantly outside of the U.S., where the magnitude and composition of plastic pollution is different than what is found in U.S. rivers. Comprehensive measurements of riverine plastics are needed not only to characterize the riverine plastic pollution, but also parameterize and validate models of plastic fate and transport. This report also describes five key U.S. rivers that span a range of sizes and environmental conditions that could be sampled to obtain data to support characterization and model development of plastic pollution from rivers to oceans. Sampling and analysis protocol recommendations are made to ensure the highest quality of data are collected in the five rivers.

54 ENVIRONMENTAL SCIENCES↗