Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,783 records · Page 99

AAPM Truth‐based CT (TrueCT) reconstruction grand challenge

Background: This Special Report summarizes the 2022, AAPM grand challenge on Truth-based CT image reconstruction. Purpose: To provide an objective framework for evaluating CT reconstruction methods using virtual imaging resources consisting of a library of simulated CT projection images of a population of human models with various diseases. Methods: Two hundred unique anthropomorphic, computational models were created with varied diseases consisting of 67 emphysema, 67 lung lesions, and 66 liver lesions. The organs were modeled based on clinical CT images of real patients. The emphysematous regions were modeled using segmentations from patient CT cases in the COPDGene Phase I dataset. For the lung and liver lesion cases, 1–6 malignant lesions were created and inserted into the human models, with lesion diameters ranging from 5.6 to 21.9 mm for lung lesions and 3.9 to 14.9 mm for liver lesions. The contrast defined between the liver lesions and liver parenchyma was 82 ± 12 HU, ranging from 50 to 110 HU. Similarly, the contrast between the lung lesions and the lung parenchyma was defined as 781 ± 11 HU, ranging from 725 to 805 HU. For the emphysematous regions, the defined HU values were −950 ± 17 HU ranging from −918 to −979 HU. The developed human models were imaged with a validated CT simulator. The resulting CT sinograms were shared with the participants. The participants reconstructed CT images from the sinograms and sent back their reconstructed images. Further, the reconstructed images were then scored by comparing the results against the corresponding ground truth values. The scores included both task-generic (root mean square error [RMSE] and structural similarity matrix [SSIM]), and task-specific (detectability index [d’] and lesion volume accuracy) metrics. For the cases with multiple lesions, the measured metric was averaged across all the lesions. To combine the metrics with each other, each metric was normalized to a range of 0 to 1 per disease type, with “0” and “1” being the worst and best measured values across all cases of the disease type for all received reconstructions. Results: The True-CT challenge attracted 52 participants, out of which 5 successfully completed the challenge and submitted the requested 200 reconstructions. Across all participants and disease types, SSIM absolute values ranged from 0.22 to 0.90, RMSE from 77.6 to 490.5 HU, d’ from 0.1 to 64.6, and volume accuracy ranged from 1.2 to 753.1 mm3. The overall scores demonstrated that participant “A” had the best performance in all categories, except for the metrics of d’ for lung lesions and RMSE for liver lesions. Participant “A” had an average normalized score of 0.41 ± 0.22, 0.48 ± 0.32, and 0.42 ± 0.33 for the emphysema, lung lesion, and liver lesion cases, respectively. Conclusions: The True-CT challenge successfully enabled objective assessment of CT reconstructions with the unique advantage of access to a diverse population of diseased human models with known ground truth. This study highlights the significant potential of virtual imaging trials in objective assessment of medical imaging technologies.

60 APPLIED LIFE SCIENCES↗

First Plant Cell Atlas symposium report

The Plant Cell Atlas (PCA) community hosted a virtual symposium on December 9 and 10, 2021 on single cell and spatial omics technologies. The conference gathered almost 500 academic, industry, and government leaders to identify the needs and directions of the PCA community and to explore how establishing a data synthesis center would address these needs and accelerate progress. This report details the presentations and discussions focused on the possibility of a data synthesis center for a PCA and the expected impacts of such a center on advancing science and technology globally. Community discussions focused on topics such as data analysis tools and annotation standards; computational expertise and cyber-infrastructure; modes of community organization and engagement; methods for ensuring a broad reach in the PCA community; recruitment, training, and nurturing of new talent; and the overall impact of the PCA initiative. These targeted discussions facilitated dialogue among the participants to gauge whether PCA might be a vehicle for formulating a data synthesis center. The conversations also explored how online tools can be leveraged to help broaden the reach of the PCA (i.e., online contests, virtual networking, and social media stakeholder engagement) and decrease costs of conducting research (e.g., virtual REU opportunities). Major recommendations for the future of the PCA included establishing standards, creating dashboards for easy and intuitive access to data, and engaging with a broad community of stakeholders. The discussions also identified the following as being essential to the PCA's success: identifying homologous cell-type markers and their biocuration, publishing datasets and computational pipelines, utilizing online tools for communication (such as Slack), and user-friendly data visualization and data sharing. In conclusion, the development of a data synthesis center will help the PCA community achieve these goals by providing a centralized repository for existing and new data, a platform for sharing tools, and new analytical approaches through collaborative, multidisciplinary efforts. A data synthesis center will help the PCA reach milestones, such as community-supported data evaluation metrics, accelerating plant research necessary for human and environmental health.

59 BASIC BIOLOGICAL SCIENCES↗

Evaluation of cooling setpoint setback savings in commercial buildings using electricity and exterior temperature time series data

Commercial buildings account for a significant amount of total energy produced in the US, and the Heating Ventilation and Cooling (HVAC) systems are one of the most significant components of their overall consumption. In this study, we proposed a new data-driven approach to evaluate HVAC cooling systems in commercial buildings and identify savings opportunities. The focus is an investigation of the impact of thermostat setpoint setback but using only whole building, electricity data taken at 15-min intervals for the analysis. We conducted a comparative study of setpoint setback characteristics on 432 commercial buildings with 5 building usage types across the United States. To accomplish this, both piecewise and Random Forest regression algorithms were employed using electricity and exterior temperature datasets to identify operational characteristics and the effective setpoints in the building to determine the corresponding savings opportunities. Both occupied and unoccupied time periods were studied across cooling degree days (CDD), when air conditioning is typically operational. Here the results show that in commercial buildings, on average, cooling systems account for 9.5% of total consumption. When a one degree setback during the cooling season is applied, an average of approximately 1.1% of annual consumption is achieved; retail and office buildings demonstrate the highest potential for savings. Additionally, we identified that the number of cooling degree days and base to peak ratio (BPR) are the most important variables for predicting the magnitude of the consumption of cooling systems.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Stochastic room temperature creep of 316 L stainless steel

The creep behavior of 316 L stainless steel at room temperature was evaluated as a function of time and applied stress using a new high-throughput approach. Several common creep models were evaluated against the observations, leading to deeper analysis of a stress-dependent modified logarithmic creep model. Within this model, multiple sources of uncertainty were compared. Aleatoric stochastic variation between samples under nominally identical conditions was identified as the primary contributor to uncertainty in creep response. Under any particular set of conditions, the sample-to-sample variability in creep strain was as high as a factor of two, highlighting the engineering importance of characterizing large statistical datasets. The model's extrapolation capabilities were assessed by comparing predictions derived from calibration on partial, shorter-duration subsets of the data. In conclusion, these findings underscore the importance of accounting for stochastic effects in predictive modeling of aging phenomena.

High-throughput↗

Understanding the fundamental driver of semiconductor radiation tolerance with experiment and theory

Space is the operating environment of a multitude of systems that our society is heavily reliant on today, however, maintaining operability there necessitates special consideration of the electronic systems' tolerance of space radiation. Electronic systems are critically dependent on the electronic properties of their semiconductor components, which are modified by space radiation with an adverse impact on the space system performance. What innate property allows some semiconductors to sustain little damage while others accumulate defects rapidly with dose is poorly understood, which limits the extent to which radiation tolerance can be implemented as a design criterion. Here, to gain insight into what properties are drivers of semiconductor radiation tolerance, the first step is to generate a dataset of the relative radiation tolerance of a broad sampling of semiconductors. To accomplish this, Rutherford backscatter channeling experiments are used to compare the displaced lattice atom buildup in InAs, InP, GaP, GaN, ZnO, MgO, and Si as a function of stepwise alpha particle dose. With this experimental information on radiation-induced incorporation of interstitial defects in hand, hybrid density functional theory electron densities (and their derived quantities) are calculated and their gradient and Laplacian are evaluated to obtain key fundamental information about the interactions in each material. It is shown that simple, undifferentiated values (which are typically used to describe bond strength) are insufficient to predict radiation tolerance. Instead, the curvature of the electron density at bond critical points provides a measure of radiation tolerance consistent with the experimental results obtained. This curvature and associated forces surrounding bond critical points have the potential to disfavor the localization of displaced lattice atoms at these points, favoring their diffusion toward perfect lattice positions. With this criterion to predict radiation tolerance, simple density functional theory simulations can be conducted on potential new materials to gain insight into how they may operate in demanding high radiation environments.

36 MATERIALS SCIENCE↗

Data Readiness for AI: A 360-Degree Survey

Artificial Intelligence (AI) applications critically depend on data. Poor-quality data produces inaccurate and ineffective AI models that may lead to incorrect or unsafe use. Evaluation of data readiness is a crucial step in improving the quality and appropriateness of data usage for AI. R&D efforts have been spent on improving data quality. However, standardized metrics for evaluating data readiness for use in AI training are still evolving. In this study, we perform a comprehensive survey of metrics used to verify data readiness for AI training. This survey examines more than 140 papers published by ACM Digital Library, IEEE Xplore, journals such as Nature, Springer, and Science Direct, and online articles published by prominent AI experts. This survey aims to propose a taxonomy of data readiness for AI (DRAI) metrics for structured and unstructured datasets. We anticipate that this taxonomy will lead to new standards for DRAI metrics that would be used for enhancing the quality, accuracy, and fairness of AI training and inference.

97 MATHEMATICS AND COMPUTING↗

Modeling the Nucleation and Growth of Lead Sulfate Particles on Lead Electrodes

Lead-acid batteries (LABs) play a pivotal role in the energy storage sector with applications spanning from starting-lighting-ignition batteries to grid energy storage. Passivation of lead negative electrodes by PbSO 4 particles is a fundamental mechanism limiting the performance of LAB. In this regard, an electrochemical model is developed that simulates the nucleation and growth (N&G) dynamics of PbSO 4 particles on a flat lead electrode, responsible for its passivation. The model considers the electrochemical reactions between lead electrode and sulfuric acid, N&G of PbSO 4 particles, passivation of the lead surface, and the ternary transport of PbSO 4 (aq), bisulfate, and protons in H 2 SO 4 electrolyte. The model is validated with a dataset of cyclic voltammetry (CV) responses collected at several scan rates and H 2 SO 4 concentrations. The model shows remarkable qualitative and quantitative agreement with the experimental data including CV peak features, discharge capacity, and particle size. The model was employed to explore key N&G quantities, such as supersaturation, nucleation rate, particle count, growth rate, particle size, and surface coverage, and to examine how their interactions influence electrode utilization. Parametric studies were also conducted to evaluate how scan rates and acid concentrations influence the previously mentioned N&G quantities and, subsequently, the utilization of the electrode.

25 ENERGY STORAGE↗

3D Continuous Forcing Dataset from 3D Constrained Variational Analysis at SGP

The continuous 3D large-scale forcing (VARANAL3D) data set derived from 3D constrained variational analysis (3DCVA) extends the conventional constrained variational analysis method by incorporating multiple sub-columns within the analysis domain. This advancement introduces spatial variability into the large-scale forcing fields, thereby enriching the data set’s applicability. The VARANAL3D data set spans from 2004 to 2018 and covers a region of 5˚×4.5˚ domain around the ARM SGP site. The analysis domain is divided into 10×9 sub-columns with 0.5˚ resolution. The 3D large-scale forcing data provides necessary variables to drive and evaluate single-column models (SCM), cloud-resolving models (CRM) ,and large-eddy simulations (LES), as well as information for testing model sensitivity to spatial variability of the large-scale forcing data, facilitating more rigorous testing and refinement of physical processes in SCM/CRM/LES.

54 ENVIRONMENTAL SCIENCES↗

The Continuous Intercomparison of Radiation Codes (CIRC): Phase I Cases

CIRC aspires to be the successor to ICRCCM (Intercomparison of Radiation Codes in Climate Models). It is envisioned as an evolving and regularly updated reference source for GCM-type radiative transfer (RT) code evaluation with the principle goal to contribute in the improvement of RT parameterizations. CIRC is jointly endorsed by DOE's Atmospheric Radiation Measurement (ARM) program and the GEWEX Radiation Panel (GRP). CIRC's goal is to provide test cases for which GCM RT algorithms should be performing at their best, i.e, well characterized clear-sky and homogeneous, overcast cloudy cases. What distinguishes CIRC from previous intercomparisons is that its pool of cases is based on observed datasets. The bulk of atmospheric and surface input as well as radiative fluxes come from ARM observations as documented in the Broadband Heating Rate Profile (BBHRP) product. BBHRP also provides reference calculations from AER's RRTM RT algorithms that can be used to select the most optimal set of cases and to provide a first-order estimate of our ability to achieve radiative flux closure given the limitations in our knowledge of the atmospheric state.

Oreopoulos, Lazaros↗

Haiti Agriculture II: Evaluating the Success of Reforestation Practices in Haiti

The Caribbean country of Haiti has an extensive history of deforestation and environmental degradation stemming from French colonization. Over the past 33 years, the Haiti Reforestation Partnership (HRP) and their partners, Comprehensive Development Program (CODEP), have planted approximately 15.52 million trees. However, these efforts lacked scientific guidance to ensure successful forest stand survival. The NASA DEVELOP team partnered with the HRP to create a habitat suitability model (HSM) by using PlanetScope and Sentinel-2 Multispectral Instrument (MSI) imagery. The team also incorporated Landsat 8 and 9 OLI surface temperature, Centre National de L’Information Geo-Spatiale (CNIGS) Airborne Lidar, and ancillary datasets to analyze areas suitable for future reforestation efforts. The habitat model suggested locations with higher forest stand survival based on topography, soil health, climate, and feasibility to access suggested locations. We confirmed the HSM through a cross-analyzation of high enhanced vegetation index (EVI) values. Areas with lower EVI values and higher suitability based on the HSM were suggested for future planting as they lack well-established forest stands but have optimal conditions for growth. Through the creation of the HSM, the team provided the HRP with static maps of high suitability, a 3D printed elevation model, and a guidebook for animators. Additionally, the team provided a structured video highlighting the HRP’s efforts. Effective reforestation and better forest stand survival would help to achieve the goal of securing community food security. This would also serve as a guide to expand planting efforts into other locations and communities.

Kelli Roberts↗

Equipping Neural Network Surrogates with Uncertainty for Propagation in Physical Systems

Coarse-grained or filtered models typically rely on closure models to account for unresolved scales. For instance, large eddy simulation for modeling turbulent fluid flows explicitly resolves the largest scales, but requires modeling closure terms to account for the sub-filter scales. With the vast amount of data available from high-fidelity simulations, there are unique opportunities to leverage data-driven modeling techniques to formulate expressive and flexible closure models. Despite their flexibility, data-driven models struggle in domain shift settings, i.e. when deployed in configurations not captured in the training dataset. In particular, the efficacy of neural network surrogates is difficult to assess a priori due to the deterministic, point-estimate nature of predictions. In high-consequence applications, such models require reliable uncertainty estimates in the data-informed and out-of-distribution regimes. To quantify uncertainties in both regimes, we employ Bayesian neural networks which are able to capture both epistemic and aleatoric uncertainties. We will discuss challenges associated with the training and evaluation of these networks. Furthermore, we will discuss uncertainty embedding strategies to enable efficient sampling and propagation of uncertainty through high-fidelity simulations.

Bayesian neural networks↗

Comparative Analysis of Imaging and Measurements of Micrometer-Scale Fracture Aperture Fields Within a Heterogeneous Rock Using PET and X-ray CT

Knowledge of the spatial distribution of fracture apertures is essential for reliable characterization of flow and transport processes in fractured systems and for better understanding of physicochemical matrix–fracture interactions. Here, we propose and test two image-based methods, thereby extending the current experimental capabilities to characterize aperture size distribution in structurally heterogeneous geologic porous media noninvasively. The first approach utilizes an inversion method based on the dataset acquired from positron emission tomography (PET) and the second approach considers an extension of the classic missing attenuation technique that relies on clinical X-ray computed tomography (X-ray CT). Independent sets of imaging experiments are conducted on a fractured basalt core with heterogeneous matrix properties and aperture distributions to compare the two methodologies. A repeat of each experiment is conducted to verify the proposed workflows. The performance of these two imaging techniques is systematically evaluated through the analysis of signal-to-noise ratio, minimum fracture size detectability, and measurement errors. While both approaches provide a reliable estimation of fracture aperture distributions, PET yields a signal-to-noise ratio that is substantially higher than the corresponding X-ray CT measurements. Furthermore, uncertainties of the aperture values for PET are considerably lower ($\bar {\sigma}_{\text {d}} = 15\%$) compared to those obtained from X-ray CT ($\bar {\sigma}_{\text {d}} = 29\%$), allowing for the detection of minimum aperture sizes of 20 $\mu$m with 70% confidence level. Finally, these approaches provide key experimental tools for better understanding dynamic hydromechanical fracture properties in geologic systems.

58 GEOSCIENCES↗

Field validation of data-driven BSDF and peak extraction models for light-scattering fabric shades

Shading and daylighting systems affect cooling, heating, and lighting energy use by modulating solar radiation through the building façade. Characterizing shading systems holistically and accurately helps designers and engineers evaluate shading systems to achieve energy and non-energy performance goals. These complex fenestration systems can be modeled using Bidirectional Scattering Distribution Functions (BSDF), which map incident radiation to hemispherical distributions of outgoing radiation. Data-driven, tabulated BSDFs are derived from interpolated goniophotometer measured data, then sampled during the raytracing calculation. A peak extraction (PE) algorithm was developed to circumvent limits in BSDF angular resolution, where the specular peak is extracted during simulation by evaluating the BSDF in the through direction and surrounding region. The objective of this study was to validate this measurement and modeling workflow using field monitored data from a full scale testbed with eleven installed fabrics of different weaves, openness factors, and colors and assess the accuracy of the workflow under different adaptation and contrast conditions. Test conditions were limited to clear sky conditions with the sun in the field of view. Results showed that, for tensor tree datasets, vertical illuminance, solar luminance (2.5° apex), and daylight glare probability (DGP) were predicted to within a mean bias error (MBE) error of -456 lx (-12.3%), -3.46e5 (-38.4%), and -0.042 (-7.8%) when full PE occurred. With a binary classification of glare/ no glare, DGP was predicted accurately with a true positive rate of 0.98 and true negative rate of 1.0 using tensor tree data and less accurately with Klems BSDF data, particularly for cases of no glare. The workflow may be of insufficient accuracy to distinguish borderline performance between fabrics using the four-point glare scale, particularly under low adaptation, high contrast daylit conditions. Errors were due to reductions in peak shape and intensity across the BSDF interpolation and data reduction workflow. Future work is needed to better preserve measurement fidelity during interpolation and sampling, which in turn will improve PE performance.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Jensen–Shannon divergence based novel loss functions for Bayesian neural networks

Bayesian neural networks (BNNs) are state-of-the-art machine learning methods that can naturally regularize and systematically quantify uncertainties using their stochastic parameters. Kullback–Leibler (KL) divergence-based variational inference used in BNNs suffer from unstable optimization and challenges in approximating light-tailed posteriors due to the unbounded nature of the KL divergence. To resolve these issues, we formulate a novel loss function for BNNs based on a new modification to the generalized Jensen–Shannon (JS) divergence, which is bounded. In addition, we propose a Geometric JS divergence-based loss, which is computationally efficient since it can be evaluated analytically. We found that the JS divergence-based variational inference is intractable, and hence employed a constrained optimization framework to formulate these losses. Our theoretical analysis and empirical experiments on multiple regression and classification data sets suggest that the proposed losses perform better than the KL divergence-based loss, especially when the data sets are noisy or biased. Specifically, there are approximately 5% and 8% improvements in accuracy for a noise-added CIFAR-10 dataset and a regression dataset, respectively. There is about 13% reduction in false negative predictions of a biased histopathology dataset. Additionally, we quantify and compare the uncertainty metrics for the regression and classification tasks.

97 MATHEMATICS AND COMPUTING↗

A unified large language model–based framework for heterogeneous PV image diagnosis

With advances in imaging technologies, modern photovoltaic (PV) systems generate large volumes of heterogeneous image data, including visible, electroluminescence (EL), and infrared (IR) images. Existing PV image analysis models, particularly deep learning approaches, are typically task-specific and lack cross-modality generalization. To address this limitation, this paper proposes an open-source large language model (LLM)–based unified framework for heterogeneous PV image diagnostics. Through task-aware diagnostic prompting, the framework enables analysis of visible, EL, and IR images within a single pipeline, supporting both zero-shot and few-shot inference and binary and multiclass classification. It is compatible with state-of-the-art multimodal LLMs, including ChatGPT, Gemini, Claude, Qwen, and CLIP. The framework is evaluated on PV module condition classification (clean, soiling, snow, hail, and bird droppings) using visible images, cell crack detection using EL images, and hotspot detection using IR images. GPT-5.1 in few-shot mode achieves the best performance, with classification accuracy exceeding 97.3%. Open-source models such as Qwen and CLIP also deliver competitive results on visible images (around 90% accuracy), though their performance is more limited on EL and IR modalities. On the full ELPV dataset, the framework achieves 83.5% zero-shot accuracy, within 2.8% of the supervised CNN baseline, confirming scalability to larger benchmarks. Practical aspects such as reproducibility, response latency, and confidence estimation are systematically analyzed. The framework operates across PV image modalities without modality- or task-specific training, making it well suited as a rapid pre-screening tool to support downstream detailed diagnostics. A benchmark dataset of diverse labeled PV images is also released.

Li, Baojie↗

Citywide indoor air sampling mirrors wastewater and clinical for environmental surveillance of respiratory viruses

Wastewater surveillance of respiratory pathogens can provide timely estimates of viral activity and disease trends in a population. Indoor air surveillance could be used similarly with some advantages but remains largely unvalidated at the community -scale. Here, an indoor air surveillance program was employed as part of public health environmental surveillance in Chicago, Illinois, USA. Ten air samplers were placed in healthcare and congregate living settings across the city. Weekly air samples were evaluated for influenza A, influenza B, respiratory syncytial virus, and SARS -CoV-2 over two respiratory virus seasons (2023 -2025). Citywide, aggregated air sample positivity and viral load were closely correlated with local clinical case and wastewater surveillance data across all respiratory viruses. Virus trends in air data often preceded clinical and wastewater, although this varied across pathogens and respiratory virus seasons. Further, whole -genome sequencing of SARS -CoV-2 showed close correlation of variant proportions across all datasets. At the building -scale, air samples obtained from a single sampling device provided efficient respiratory virus surveillance, with respiratory pathogen levels mirroring citywide clinical surveillance data. These data demonstrate that air surveillance can provide respiratory virus case and variant trend data at a building or community -scale, serving as an alternative or complementary tool for public health environmental surveillance.

Wilton, Rosemarie↗

Global nitrogen deposition inputs to cropland at national scale from 1961 to 2020

Nitrogen (N) deposition is a significant nutrient input to cropland and consequently important for the evaluation of N budgets and N use efficiency (NUE) at different scales and over time. However, the spatiotemporal coverage of N deposition measurements is limited globally, whereas modeled N deposition values carry uncertainties. Here, we reviewed existing methods and related data sources for quantifying N deposition inputs to crop production on a national scale. We utilized different data sources to estimate N deposition input to crop production at national scale and compared our estimates with 14 N budget datasets, as well as measured N deposition data from observation networks in 9 countries. We created four datasets of N deposition inputs on cropland during 1961–2020 for 236 countries. These products showed good agreement for the majority of countries and can be used in the modeling and assessment of NUE at national and global scales. One of the datasets is recommended for general use in regional to global N budget and NUE estimates.

54 ENVIRONMENTAL SCIENCES↗

Photovoltaic (PV) System Levelized Cost of Energy (LCOE) Evaluation with Grid Support Function Valuation and Service Lifetime Estimation

Photovoltaic (PV) systems play a critical role in renewable energy resource grid integration, and levelized cost of energy (LCOE) is commonly used to evaluate PV system feasibility in modern power grids. In this work, a revised PV system LCOE calculation model is derived to quantify the potential of LCOE reduction. Particularly, the grid support functions are valuated to offset the investment and operation costs of PV systems, which thereby reduces the LCOE. Meanwhile, PV system service lifetime is also estimated with the derived PV inverter reliability model, considering the critical components (i.e., semiconductor devices and capacitors). The case studies with field datasets are conducted to validate the effectiveness of the developed LCOE calculation model.

Zhao, Shijia↗