Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data bias”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

First Cosmology Results using Supernovae Ia from the Dark Energy Survey: Survey Overview, Performance, and Supernova Spectroscopy

We present details on the observing strategy, data-processing techniques, and spectroscopic targeting algorithms for the first three years of operation for the Dark Energy Survey Supernova Program (DES-SN). This five-year program using the Dark Energy Camera mounted on the 4 m Blanco telescope in Chile was designed to discover and follow supernovae (SNe) Ia over a wide redshift range (0.05 < z < 1.2) to measure the equation-of-state parameter of dark energy. We describe the SN program in full: strategy, observations, data reduction, spectroscopic follow-up observations, and classification. From three seasons of data, we have discovered 12,015 likely SNe, 308 of which have been spectroscopically confirmed, including 251 SNe Ia over a redshift range of 0.017 < z < 0.85. We determine the effective spectroscopic selection function for our sample and use it to investigate the redshift-dependent bias on the distance moduli of SNe Ia we have classified. The data presented here are used for the first cosmology analysis by DES-SN (“DES-SN3YR”), the results of which are given in Dark Energy Survey Collaboration et al. The 489 spectra that are used to define the DES-SN3YR sample are publicly available at https://des.ncsa.illinois.edu/releases/sn.

79 ASTRONOMY AND ASTROPHYSICS↗

Evaluation of simulations of near-surface variables using the regional climate model CCLM for the MOSAiC winter period

The ship-based experiment MOSAiC 2019/2020 was carried out during a full year in the Arctic and yielded an excellent data set to test the parameterizations of ocean/sea-ice/atmosphere interaction processes in regional climate models (RCMs). In the present paper, near-surface data during MOSAiC are used for the verification of the RCM COnsortium for Small-scale MOdel–Climate Limited area Mode (COSMO-CLM or CCLM). CCLM is used in a forecast mode (nested in ERA5) for the whole Arctic with 15 km resolution and is run with different configurations of sea ice data. These include the standard sea ice concentration taken from passive microwave data with around 6 km resolution, sea ice concentration from Moderate Resolution Imaging Spectroradiometer (MODIS) thermal infrared data and MODIS sea ice lead fraction data for the winter period. CCLM simulations show a good agreement with the measurements. Relatively large negative biases for temperature occur for November and December, which are likely associated with a too large ice thickness used by CCLM. The consideration of sea ice leads in the sub-grid parameterization in CCLM yields improved results for the near-surface temperature. ERA5 data show a large warm bias of about 2.5°C and an underestimation of the temperature variability.

54 ENVIRONMENTAL SCIENCES↗

Development of a 95-Year Solar Dataset for Resource Adequacy Studies

Long-term high-resolution solar data provides enhanced understanding of variability of solar generation and enhances our ability to develop strategies for a resilient and reliable electric grid under high deployment of solar energy. Therefore, it is important to develop long-term synthetic datasets that can provide multiple occurrences of various severe weather scenarios that are expected to test the limits of resource adequacy under scenarios contain various energy generation sources. Examples of such scenarios could be long periods of high temperatures when demand for electricity is high or periods where high winds could lead to a shut-down of transmission lines for long periods of time to ensure fire safety. NREL has developed the first version of such a dataset covering a 95-year period covering 2006-2100 at a 4km hourly resolution. This dataset contains all variables necessary to calculate solar generation. During development of this dataset, we focused on creating unbiased, high-resolution solar irradiance through statistical downscaling methods, using Regional Climate Model (RCM) simulations from the North American Coordinated Regional Climate Downscaling Experiment (NA-CORDEX) as input. The National Solar Radiation Database (NSRDB) containing over 25 years of observations was used to calibrate the statistical downscaling models. This presentation will outline the primary steps in developing this dataset, including (1) regridding RCM data to a common grid at 20-km resolution, (2) correcting RCM biases with NSRDB, (3) applying temporal and spatial downscaling methods to generate high-resolution (4-km, hourly) solar and ancillary data. Additionally, we will present an evaluation of the downscaled data against the NSRDB across various zones in the CONUS. Lastly, we will present a user guide for accessing the datasets.

14 SOLAR ENERGY↗

Dark Energy Survey Year 3 Results: Measuring the Survey Transfer Function with Balrog

Abstract We describe an updated calibration and diagnostic framework, Balrog , used to directly sample the selection and photometric biases of the Dark Energy Survey (DES) Year 3 (Y3) data set. We systematically inject onto the single-epoch images of a random 20% subset of the DES footprint an ensemble of nearly 30 million realistic galaxy models derived from DES Deep Field observations. These augmented images are analyzed in parallel with the original data to automatically inherit measurement systematics that are often too difficult to capture with generative models. The resulting object catalog is a Monte Carlo sampling of the DES transfer function and is used as a powerful diagnostic and calibration tool for a variety of DES Y3 science, particularly for the calibration of the photometric redshifts of distant “source” galaxies and magnification biases of nearer “lens” galaxies. The recovered Balrog injections are shown to closely match the photometric property distributions of the Y3 GOLD catalog, particularly in color, and capture the number density fluctuations from observing conditions of the real data within 1% for a typical galaxy sample. We find that Y3 colors are extremely well calibrated, typically within ∼1–8 mmag, but for a small subset of objects, we detect significant magnitude biases correlated with large overestimates of the injected object size due to proximity effects and blending. We discuss approaches to extend the current methodology to capture more aspects of the transfer function and reach full coverage of the survey footprint for future analyses.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Misclassification in Workers’ Telecommuting Frequency Choices Using a Generalized Extreme Value Model

Telecommuting frequency is a response variable collected in travel surveys and is, therefore, prone to errors leading to mismeasurements or misclassification. Misclassification of explanatory variables is a common risk when using statistical modeling techniques. We define “misclassification” as a response reported or recorded in the wrong category; for example, a variable is recorded as a 1 when it should be 0. Here, in this context, this study aims to develop a statistical model to analyze telecommuting data which accounts for potential misclassification errors by building on existing literature in econometrics. The empirical analysis was undertaken using the 2017 National Household Travel Survey (NHTS) and the general extreme value (GEV) models available in the literature. Specifically, the frequency of telecommuting days was analyzed using the negative binomial (NB) model recast as the multinomial logit (MNL) model. By nature—and consistent with other studies—NHTS data are prone to errors that can be classified as intentional or unintentional misinformation provided by the person being interviewed. Ignoring these errors while modeling telecommuting frequencies using standard discrete count models can result in biased parameter estimates. The misclassification parameter was calculated for both over-reporting and under-reporting scenarios. The misclassification errors can be as high as 14% over-reported and 10% under-reported, particularly for the neighboring values. Statistical fit comparison between the models shows that models that ignore misclassification have worse data fit and biased parameter estimates with significant policy implications.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

The 4DEnVar-based weakly coupled land data assimilation system for E3SM version 2

Abstract. A new weakly coupled land data assimilation (WCLDA) system based on the four-dimensional ensemble variational (4DEnVar) method is developed and applied to the fully coupled Energy Exascale Earth System Model version 2 (E3SMv2). The dimension-reduced projection four-dimensional variational (DRP-4DVar) method is employed to implement 4DVar using the ensemble technique instead of the adjoint technique. With an interest in providing initial conditions for decadal climate predictions, monthly mean anomalies of soil moisture and temperature from the Global Land Data Assimilation System (GLDAS) reanalysis from 1980 to 2016 are assimilated into the land component of E3SMv2 within the coupled modeling framework with a 1-month assimilation window. The coupled assimilation experiment is evaluated using multiple metrics, including the cost function, assimilation efficiency index, correlation, root-mean-square error (RMSE), and bias, and compared with a control simulation without land data assimilation. The WCLDA system yields improved simulation of soil moisture and temperature compared with the control simulation, with improvements found throughout the soil layers and in many regions of the global land. In terms of both soil moisture and temperature, the assimilation experiment outperforms the control simulation with reduced RMSE and higher temporal correlation in many regions, especially in South America, central Africa, Australia, and large parts of Eurasia. Furthermore, significant improvements are also found in reproducing the time evolution of the 2012 US Midwest drought, highlighting the crucial role of land surface in drought lifecycle. The WCLDA system is intended to be a foundational resource for research to investigate land-derived climate predictability.

58 GEOSCIENCES↗

Neutrino-nucleus cross section impacts on neutrino oscillation measurements

The challenges in neutrino-nucleus cross section modeling and its impact on neutrino oscillation experiments are widely recognized. However, a comprehensive and theoretically robust estimation of cross section uncertainties has been lacking, and few studies have quantitatively examined their impact on oscillation measurements. In this work, we evaluate the effect of cross section uncertainties on oscillation parameters using setups inspired by NOvA and DUNE. To characterize these uncertainties, we adopt multiple neutrino-nucleus event generators and simulate a realistic experimental procedure that incorporates near-detector data and near-to-far-detector extrapolation. Our results confirm that cross section uncertainties cannot significantly bias oscillation results in current statistics-dominated experiments like NOvA. However, they could lead to substantial bias for future systematics-dominated experiments like DUNE, even when near-detector data are employed to mitigate uncertainties. These findings underscore the need for further studies on the quantitative impacts of cross section modeling, improved strategies to utilize near-detector data and the PRISM concept, and more robust cross section models to optimize the success of future experiments.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Simulation and Modeling of Time-Resolved X-Ray Detector for the Saturn Accelerator

In this work, we present the technology-aided computer design (TCAD) device simulation and modeling of a silicon p-i-n diode for detecting time-dependent X-ray radiation. We show that the simulated forward and reverse breakdown current–voltage characteristics agree well with the measured data under nonradiation environment by only calibrating carrier lifetimes for the forward bias case and avalanche model critical fields for the reverse bias condition. Using the calibrated parameters and other nominal material properties, we simulated the radiation responses of the p-i-n diode and compared with experimental data when the diode was exposed to X-ray radiation at Sandia’s Saturn facility and the Idaho State University (ISU) TriMeV facility. For Saturn’s Gaussian dose-rate pulses, we show three findings from TCAD simulations. First, the simulated photocurrents are in excellent agreement with the measured data for two dose-rate pulses with peak values of 1.16×10 10 and 1.88×10 10 rad(Si)/s. Second, the simulation results of high dose-rate pulses predict increased delayed photocurrents with longer time tails in the diode electrical responses due to excess carrier generation. Third, simulated peak values of diode radiation responses versus peak dose rates at different bias conditions provide useful guidance to determine the dose-rate range that the p-i-n diode can reliably detect in experiment. For TriMeV’s non-Gaussian dose-rate pulse, our simulated diode response is in decent agreement with the measured data without further calibration. We also studied the effects of device geometry, recombination process, and dose-rate enhancement via TCAD simulations to understand the higher measured response in the time after the peak dose-rate radiation for the p-i-n diode exposed to TriMeV irradiation.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Using Host Galaxy Photometric Redshifts to Improve Cosmological Constraints with Type Ia Supernovae in the LSST Era

We perform a rigorous cosmology analysis on simulated Type Ia supernovae (SNe Ia) and evaluate the improvement from including photometric host galaxy redshifts compared to using only the "z spec " subset with spectroscopic redshifts from the host or SN. We use the Deep Drilling Fields (~50 deg 2 ) from the Photometric LSST Astronomical Time-Series Classification Challenge (PLAsTiCC) in combination with a low-z sample based on Data Challenge2. The analysis includes light-curve fitting to standardize the SN brightness, a high-statistics simulation to obtain a bias-corrected Hubble diagram, a statistical+systematics covariance matrix including calibration and photo-z uncertainties, and cosmology fitting with a prior from the cosmic microwave background. Compared to using the z spec subset, including events with SN+host photo-z results in (i) more precise distances for z > 0.5, (ii) a Hubble diagram that extends 0.3 further in redshift, and (iii) a 50% increase in the Dark Energy Task Force figure of merit (FoM) based on the w 0 w a CDM model. Analyzing 25 simulated data samples, the average bias on w 0 and w a is consistent with zero. The host photo-z systematic of 0.01 reduces FoM by only 2% because (i) most z < 0.5 events are in the z spec subset, (ii) the combined SN+host photo-z has ×2 smaller bias, and (iii) the anticorrelation between fitted redshift and color self-corrects distance errors. To prepare for analyzing real data, the next SN Ia cosmology analysis with photo-zs should include non–SN Ia contamination and host galaxy misassociations.

79 ASTRONOMY AND ASTROPHYSICS↗

Evaluating the potential of short-term instrument deployment to improve distributed wind resource assessment

Distributed wind projects, which are connected at the distribution level of an electricity system or in off-grid applications to serve specific or local energy needs, often rely solely on wind resource models to establish wind speed and energy generation expectations. Historically, anemometer loan programs have provided an affordable avenue for more accurate onsite wind resource assessment, and the lowering cost of lidar systems has shown similar advantages for more recent assessments. While a full 12 months of onsite wind measurement is the standard for correcting model-based long-term wind speed estimates for utility-scale wind farms, the time and capital investment involved in gathering onsite measurements must be reconciled with the energy needs and funding opportunities that drive expedient deployment of distributed wind projects. Much literature exists to quantify the performance of correcting long-term wind speed estimates with 1 or more years of observational data, but few studies explore the impacts of correcting with months-long observational periods. This study aims to answer the question of how short you can go in terms of the observational time period needed to make impactful improvements to model-based long-term wind speed estimates. Three algorithms, multivariable linear regression, adaptive regression splines, and regression trees, are evaluated for their skill at correcting long-term wind resource estimates from the European Centre for Medium-Range Weather Forecasts Reanalysis version 5 (ERA5) using months-long periods of observational data from 66 locations across the US. On average, correction with even 1 month of observations provides significant improvement over the baseline ERA5 wind speed estimates and produces median bias magnitudes and relative errors within 0.22 m s −1 and 4 percentage points of the median bias magnitudes and relative errors achieved using the standard 12 months of data for correction. However, in cases when the shortest observational periods (1 to 2 months) used for correction are not well correlated with the overlapping ERA5 reference, the resultant long-term wind speed errors are worse than those produced using ERA5 without correction. Summer months, which are characterized by weaker relative wind speeds and standard deviations for most of the evaluation sites, tend to produce the worst results for long-term correction using months-long observations. The three tested algorithms perform similarly for long-term wind speed bias; however, regression trees perform notably worse than multivariable linear regression and adaptive regression splines in terms of correlation when using 6 months or less of observational data for correction. Translating the analysis to wind energy, median relative errors in the capacity factor are on average within 10 % using 1 month of training. If the observation period used for correction is not well correlated with the reference data, however, misrepresentation of the observed capacity factor can be substantial. The risk associated with poor correlation between the observed and reference datasets decreases with increasing training period length. In the worst-correlation scenarios, the median capacity factor relative errors from using 1, 3, and 6 months are within 47 %, 26 %, and 16 %, respectively.

17 WIND ENERGY↗

Lossy compression of statistical data using quantum annealer

Abstract We present a new lossy compression algorithm for statistical floating-point data through a representation learning with binary variables. The algorithm finds a set of basis vectors and their binary coefficients that precisely reconstruct the original data. The optimization for the basis vectors is performed classically, while binary coefficients are retrieved through both simulated and quantum annealing for comparison. A bias correction procedure is also presented to estimate and eliminate the error and bias introduced from the inexact reconstruction of the lossy compression for statistical data analyses. The compression algorithm is demonstrated on two different datasets of lattice quantum chromodynamics simulations. The results obtained using simulated annealing show 3–3.5 times better compression performance than the algorithm based on neural-network autoencoder. Calculations using quantum annealing also show promising results, but performance is limited by the integrated control error of the quantum processing unit, which yields large uncertainties in the biases and coupling parameters. Hardware comparison is further studied between the previous generation D-Wave 2000Q and the current D-Wave Advantage system. Our study shows that the Advantage system is more likely to obtain low-energy solutions for the problems than the 2000Q.

97 MATHEMATICS AND COMPUTING↗

Global constraint on the jet transport coefficient from single-hadron, dihadron, and γ -hadron spectra in high-energy heavy-ion collisions

Modifications of large transverse momentum single-hadron, dihadron, and γ -hadron spectra in relativistic heavy-ion collisions are direct consequences of parton-medium interactions in the quark-gluon plasma (QGP). The interaction strength and underlying dynamics can be quantified by the jet transport coefficient q ̂ . We carry out the first global constraint on q ̂ using a next-to-leading order pQCD parton model with higher-twist parton energy loss and combining world experimental data on single-hadron, dihadron, and γ -hadron suppression at both RHIC and LHC energies with a wide range of centralities. The global Bayesian analysis using the information field (IF) priors provides the most stringent constraint on q ̂ ( T ) . We demonstrate in particular the progressive constraining power of the IF Bayesian analysis on the strong temperature dependence of q ̂ using data from different centralities and colliding energies. We also discuss the advantage of using both inclusive and correlation observables with different geometric biases. As a verification, the obtained q ̂ ( T ) is shown to describe data on single-hadron anisotropy at high transverse momentum well. Predictions for future jet quenching measurements in oxygen-oxygen collisions are also provided. Published by the American Physical Society 2024

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Empirical Validation of UBEM: An Assessment of Bias in Urban Building Energy Modeling for Chicago

Residential and commercial buildings currently account for 30% of total global final energy consumption. Urban-scale building energy modeling (UBEM) can enable scalable investments and unlock building improvements by quantifying energy, demand, emissions, and cost reductions of specific measures or packages for building-specific technologies in large geographic regions. While the sophistication of UBEM data sources and technologies have increased dramatically in the past decade, there remains a knowledge gap for empirical validation and sources of bias between building-specific energy models and measured data at varying geographic scales.As UBEM continues to develop, systemic analysis of accuracy, bias, and limitations of the resulting models is necessary to inform best practices and move toward standardization. These are characterized for the Automatic Building Energy Modeling (AutoBEM) software suite with an initial case study involving metered electricity consumption data from 247,188 buildings in Chicago, Illinois, USA - averaged across years 2019-2021 - compared to the following datasets: (1) the AutoBEM-generated nation-scale Model America version 2 (MAv2) data for 596,064 buildings, (2) tax assessor data for 579,829 buildings, (3) tax assessor data filled with MAv2, and (4) 102 representative dynamic archetypes. The accuracy is reported for every building type and vintage combination, along with multiple sources of bias for unique building descriptors. The AutoBEM simulation workflow produced energy consumption estimates that closely match aggregated metered electricity consumption data for different types of buildings constructed during various time periods at the city scale - with initial normalized mean bias error of 10.9%, and 1.1% after removing outliers. Contribution of statistically significant factors including building type, land use, age, and size to variance in UBEM bias is quantified.

Garg, Ankur↗

APSO-enhanced algebraic derivative estimation approach for real-time traffic flow prediction on critical road sections during wildfire evacuation

In rapid-onset disaster scenarios such as wildfires, evacuation traffic often significantly deviates from historical patterns, rendering conventional data-driven forecasting methods less effective. To address this challenge, we propose an improved algebraic derivative estimation (ADE) incorporating particle swarm optimization (PSO) for real-time traffic flow prediction. Our approach dynamically adjusts the ADE prediction time window at each step by minimizing a cost function based on the mean and variance of accumulated forecasting errors within the window, thereby balancing bias and variability. We evaluate the method using traffic data from the January 2025 California wildfires, focusing on key road segments critical for large-scale evacuations. The results demonstrate that our approach surpasses established machine learning and deep learning models—XGBoost, LSTM, and GRU—in predictive accuracy and maintains high computational efficiency. Notably, the proposed method eliminates the need for offline model training. Moreover, rapid PSO-based tuning enables real-time deployment, which provides a crucial advantage in scenarios where evacuation timings and road closures change dynamically. In conclusion, these findings highlight the benefits of the PSO-enhanced ADE framework for emergency traffic management, where rapid, data-sparse forecasts are essential for effective evacuation planning.

Algebraic derivative estimation↗

Photoelectrochemically Self Improving Si/GaN Photocathode: Figure 1a Raw Data

10 hours chronoamperometry (CA) testing under 1 sun illumination and constant bias at -0.6 V vs RHE, the corresponding Faradaic efficiency reveals a self-improving nature of GaN, inset: CA testing on bare Si for 5 hours under 1 sun illumination and -0.6 V vs RHE, this bare Si photocathode rapidly drops down to < 0.05 mA/cm2 within an hour. Both CA testing performed in 0.5 M H2SO4 (pH=0.4).

photocathode↗

Black-Box Neural System Identification and Differentiable Programming to Improve Earth System Model Predictions February

Focal Area(s): Focal Area 2: AI for predictive modeling, including AI-derived model components, and AI-enabled applications including parameter optimization, data assimilation, and uncertainty quantification. Focal Area 1: AI-assisted data assimilation using numerical Earth system models. Science Challenge: Earth system models have structural biases that lead to uncertain predictions, and their complexity and expense makes it difficult constraining the models with data or improved physical understanding.

54 ENVIRONMENTAL SCIENCES↗

Unsupervised atomic data mining via multi-kernel graph autoencoders for machine learning force fields

Constructing a chemically diverse dataset while avoiding sampling bias is critical to training efficient and generalizable force fields. However, in computational chemistry and materials science, many common dataset generation techniques are prone to oversampling regions of the potential energy surface. Furthermore, these regions can be difficult to identify and isolate from each other or may not align well with human intuition, making it challenging to systematically remove bias in the dataset. While traditional clustering and pruning (down-sampling) approaches can be useful for this, they can often lead to information loss or a failure to properly identify distinct regions of the potential energy surface due to difficulties associated with the high dimensionality of atomic descriptors. In this work, we introduce the Multi-kernel Edge Attention-based Graph Autoencoder (MEAGraph) model, an unsupervised approach for analyzing atomic datasets. MEAGraph combines multiple linear kernel transformations with attention-based message passing to capture geometric sensitivity and enable effective dataset pruning without relying on labels or extensive training. Demonstrated applications on niobium, tantalum, and iron datasets show that MEAGraph efficiently groups similar atomic environments, allowing for the use of basic pruning techniques for removing sampling bias. This approach provides an effective method for representation learning and clustering that can be used for data analysis, outlier detection, and dataset optimization.

Materials science↗

Detect and correct bias in multi-site neuroimaging datasets

The desire to train complex machine learning algorithms and to increase the statistical power in association studies drives neuroimaging research to use ever-larger datasets. The most obvious way to increase sample size is by pooling scans from independent studies. However, simple pooling is often ill-advised as selection, measurement, and confounding biases may creep in and yield spurious correlations. In this work, we combine 35,320 magnetic resonance images of the brain from 17 studies to examine bias in neuroimaging. In the first experiment, Name That Dataset, we provide empirical evidence for the presence of bias by showing that scans can be correctly assigned to their respective dataset with 71.5% accuracy. Given such evidence, we take a closer look at confounding bias, which is often viewed as the main shortcoming in observational studies. In practice, we neither know all potential confounders nor do we have data on them. Hence, we model confounders as unknown, latent variables. Kolmogorov complexity is then used to decide whether the confounded or the causal model provides the simplest factorization of the graphical model. Finally, we present methods for dataset harmonization and study their ability to remove bias in imaging features. In particular, we propose an extension of the recently introduced ComBat algorithm to control for global variation across image features, inspired by adjusting for unknown population stratification in genetics. Overall, our results demonstrate that harmonization can reduce dataset-specific information in image features. Further, confounding bias can be reduced and even turned into a causal relationship. However, harmonization also requires caution as it can easily remove relevant subject-specific information. Code is available at https://github.com/ai-med/Dataset-Bias.

42 ENGINEERING↗